Kafka Consumer Error – AVRO with Field Type as a Union of null and boolean

Recently I ran into some issues in consuming Kafka AVRO messages. Everything went well up to the de-serialization of the record from the Kafka topic. It all just went crazy once it was being serialized into JSON. That’s pretty much where the proverbial buck stopped.

I was scratching my head on why this odd behavior was happening. Looking at the message published on the Kafka topic, it was following the defined AVRO schema as expected. There were no deviations.

Somehow when being converted to JSON it started throwing an exception. It was always at the same spot every time. It would not go any further.

Debugging the flow of the program, it showed me that what should have been this:

{
	"fieldOne": "hello world",
	"fieldTwo": true,
	"foobar": false
}

Became something like this:

{
	"fieldOne": "hello world",
	"fieldTwo": true,
	"foobar": {
		"boolean": false
	}
}

And that is why the program was throwing an error!

ANSWER

The solution I did was to remove null as a field type. Set the field to be strictly boolean. What I wanted was to initially have the state of this field as null. Then further down the line it can be set to either true or false.

After the AVRO schema was updated, the error went away.

This was reported to Apache AVRO Jira. More details about this problem here – https://www.joseyamut.xyz/2020/07/29/de-serializing-kafka-messages-with-union-defined-field/

Link in Java returns HTTP instead of HTTPS

Many times your application can be running behind a reverse proxy or a load balancer, and often in between the protocol is just plain old HTTP. This is a very common setup and one that has been around for several years.

When this happens, when generating URLs through the Java Link class on Spring boot, you might get the wrong scheme or protocol. What should have been HTTPS becomes HTTP. When you’re building APIs that follows HATEOAS/HAL, this becomes an issue. With just a single letter ‘S’ your URLs become invalid.

First off, this should be already handled properly by the underlying framework but if the proxy in front of your app was misconfigured this leads to the problem happening and it is then difficult, maybe impossible, to tell whether the client connecting to the app used HTTP or HTTPS.

Luckily, you can force it somehow to use the proper protocol. This is a solution nonetheless, which works, but I would say having the reverse proxy or load balancer configured correctly is the proper way to do it.

ANSWER

The band-aid solution is to manipulate your Spring Boot app depending on the environment.

For example, if I’m just developing/using it locally HTTP is fine. With profiles setup as it ought to be – dev, test, prod – you can get the correct environment and have your URL use HTTP or HTTPS depending on it. When you deploy it to your server, you know you want it to be using secure HTTP so now you can override the Link to use the right scheme in this case.

Here is a one way on how to do manipulate the protocol or scheme of a Link.

Oh, and getting the profile can be done in 2 ways that I know of in Spring Boot.

First is injecting it via annotation.

@Value("${spring.profiles.active}")
private String activeProfile;

and/or autowiring Environment

@Autowired
private Environment environment;
 
public void getActiveProfiles() {
   for (String profileName : environment.getActiveProfiles()) {
      System.out.println("Active profile: " + profileName);
   }  
}

SSH Time out Error

I am looking for new ideas on how can I do more better.

Home laptop (home Laptop can access only Linux VM)
Linux VM (This machine can access only jump box)
jump box VM

I can ssh fine from Linux VM to jump box keys are setup.

one user is set up in Linux VM called joe and Joe .bash_profile looks like this.

ssh 10.0.0.1 || ssh 10.0.0.2

Note: There is two nic on jump box if one is down we can use the other nic for login.

Let’s say first nic is down and when I do telnet from my home laptop (telnet Linux VM) and provide user name joe it should automatically connect us to 10.0.0.1 but one is down it’s giving message timeout and taking a long time to connect to other nic.

is there any way when I do telnet and give user name check first ssh connection if down automatically connects to other second one like in 2-3 secs?

Right now it will give us a message

Time out
time out
time out
And then it will try to connect the second nic.

I can more clarify if you guys have any more questions for me.

Go to Source
Author: John

ANSWER

Resolving hosts can add up to the connection attempt, so specifying an IP address directly can shave off some of those precious milliseconds. If it’s only in a local network with the IP addresses all laid out that should not be the case. At least that’s what I think.

Perhaps explicitly setting the ConnectTimeout option to a shorter one. Like so,

ssh -o ConnectTimeout=10 user@host

This can help. Adjust the number to a sweet spot that works for your ssh setup as necessary.

How to configure Krusader’s extract context menu to use something other than ark?

I’ve found this context menu in krusader to be very unreliable over the years, it has hardly ever worked for me. Surely it’s possible to configure the context menu to use a custom command, like simply using the zip , ot tar CLI command. I don’t find this in the application settings.

The VFS plugins work great, how come this context menu has to use ARK?

Go to Source
Author: barrymac

ANSWER

Being KDE, it is a very highly configurable desktop environment to the point that it can become overwhelming. It gives you practically a myriad options for customizations for just about everything. I know because I had been a KDE fan/user for several years.

But I think the issue here is on file associations. Krusader, being a KDE application should follow system-wide settings. I don’t see any reason why it should not. As such, it is in my opinion that the file manager by default will use whatever is associated to handle or open a certain file. I’d look for KDE’s file association settings in the Control Panel or some such. Then assign a different application for that file to your liking. That should do the trick.

As for more customizations like using your own script perhaps? It could be possible still under file associations to assign a different application. Don’t have a KDE desktop installed right now so I can’t check this out. Certainly can’t remember too, nor have gone to the extent of doing something like this to handle archives.

Copying a MySQL 5.6 server settings for a new environment?

I have a MySQL 5.6 Server running on a Windows 8 VM on a steel case server. It’s been having issues (The Windows 8, which is then causing the MySQL to have issues). Therefore I was tasked with putting the MySQL 5.6 Server on a Windows 10 VM for stability reasons. The amount of ram the server is allowed and other settings have been tweaked over time. Is there an way I can export these settings, so when I install MySQL on my Windows 10 VM, I can just tell it to use the same?

Go to Source
Author: bjk116

ANSWER

This should be as straightforward as copying the configuration file – my.ini is the default – of the old MySQL server to the new one. It is a regular text file so copying it should not be an issue at all.

If there are other supporting configuration files for the DB server you might have customized, copy those as well.

Restart the MySQL server as necessary. If there are issues, like for example when using InnoDB engine, deleting the datadir should fix that quickly. Of course, I’m assuming this is on a fresh MySQL install. Don’t do this otherwise. You can migrate the data properly through a mysqldump for example.

No sound playback during call in Slack – Ubuntu 20.04

I’m on 20.04 and downloaded Slack using snap. I downloaded Spotify via snap as well as my speakers work. Not sure why Slack isn’t working. I tried every single sound source.

Go to Source
Author: Dan

ANSWER

Is it possible there is another volume control setting for Slack application that was missed out? It is sometimes hidden deep in the Sound Settings under Applications.

I had a similar experience with Zoom for Linux. I could not hear a sound at all. The main system volume was all the way up. The app was connected to the device’s audio correctly. I reconnected a couple of times. Only to find out later on that there was yet another set of input/output controls for Zoom that was not so obvious. I only had to turn it up to hear sound during a call. By default it always seems to be at 0 when I join a call, which is weird.

What firewall should I use?

What is the most comprehensive free firewall solution available for Windows? Personal machine, not work machine. I see a ton of choices online, but not sure which one to pick.

Go to Source
Author: PerpetualLearner

ANSWER

For home use I am fine with the built-in and free Windows Firewall (or Windows Defender Firewall) most of the time. It serves its purpose. Does the job. Not overwhelming to use. Customizable enough.

I would also explore the security options that comes with my home router. Many have it included nowadays, and with a little tweaking, should be able to add an extra layer of protection to one’s home network..

Easy way to bypass password on my computer

I was not able to formule a question so maybe this was already solved, and if that’s the case, sorry.
But I found a huge vulnerability on my computer. It’s a portable computer with a Win10/Ubuntu 20.04 dual boot.
When I’m logged in, and that I lock my session, this sends me to the login screen, okay. But then, if I do Ctrl+Alt+F3 to go to the terminal mode, and then do Ctrl+Alt+F7 to leave terminal mode, this simply unlocks my session !
It doesn’t work at computer start though, the session must be started and locked.
Note that when I close my computer this doesn’t lock my session even if it’s activated in the settings.
This is a critical vulnerability but it’s doesn’t work at all on my friend’s computer which is also a Win10/Ubuntu 20 dual boot.
I don’t know why this happens to me, any idea of solving this ?

Thanks for you time

Go to Source
Author: Double Vé

ANSWER

This does not seem to be a normal behavior from what I’ve experienced at all as far as I can remember. I know before I was fond of having my Linux desktop auto-login on startup. I think I also opted to have it login automatically after sleep/suspend. Perhaps the switch to a different TTY/console constitutes as the same behavior. This looks a lot like buggy behavior to me.

I would go so far as to document on how to replicate it, and report it as a bug. Call the attention of the developers so this can be looked into and fixed if necessary.

Need help with .htaccess redirection for WordPress websites

I am changing the domain of my website.

But, I am struggling with the domain redirection. I need to implement this wildcard redirection.

So that I visit.

https://www.oldomain.com/page1

It should go to

https://newdomain.com/page1

Note: The new domain is without www.

Can anyone help me how to implement this?

Go to Source
Author: Nirmal Kumar

ANSWER

A 301 Redirect, or permanent redirect, is recommended when dealing with something similar to this one.

If you just want to hard redirect anything hitting oldomain.com to newdomain.com then a simple Redirect should do like so:

<VirtualHost *:443>
	ServerName oldomain.com
        ServerAlias www.oldomain.com
	Redirect 301 / https://newdomain.com
</VirtualHost>

The above example is also something that you will want to write in a server config like httpd.conf but not in a .htaccess file somewhere on your site directory. It is also clumsy since your old site’s pages – e.g https://www.domain.com/some/page/123 – will redirect only to https://newdomain.com which is the index page of the site.

So the other option would be Rewrite. It is also in case you don’t have root access to the server, and/or other sites that are not yours are sharing it as well. It would look like the following:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^oldomain.com [NC,OR]
RewriteCond %{HTTP_HOST} ^www.oldomain.com [NC]
RewriteRule ^(.*)$ http://newdomain.com/$1 [L,R=301,NC]

The above considers whether the request is written with or without the www subdomain in it. The old links or pages will also get carried over to the new one.

how to get specific character from a log file in linux

ANSWER

Assuming that (1) the log file looks as it is AND (2) it is multi-line, then a simple Linux grep command like below should work (even with multiple lines of that line you’re looking for in one log file):

INPUT | Tested it on dummy logs from sample

COMMAND | grep -Pn "^Id\s+:\s+(.*)$" *.log

OUTPUT |

log101.log:2:Id : UN123P

log102.log:2:Id : HELLO321

log103.log:2:Id : STUDE23P

log104.log:2:Id : Lorem

log104.log:7:Id : IPsum

log104.log:12:Id : Dolor

have a log file , having multiple contents inside the file like and example below is the content of U123.log . I have n number of logs..

Accepted password for hoover from 10.0.2.2 port 4792 ss

Id : UN123P 

ID_MTCH : UnixProduction

Accepted password for hoover from 10.0.2.2 port 4792 ss

Accepted password for hoover from 10.0.2.2 port 4792 ss

Is there any we can fetch “Id : UN123P” as an output of linux command. We have some 1000’s of logs and from all the logs , trying to fetch the value only starts with ID : and output as “Id : UN123P”

Pls suggest

Go to Source
Author: Arya