Subscribe to Reddit feed
All things Linux and GNU/Linux -- this is neither a community exclusively about the kernel Linux, nor is exclusively about the GNU operating system.Linux, GNU/Linux, free software...
Updated: 14 min 39 sec ago

Linux VM experience bit-shifting for in-memory data

11 hours 39 min ago

I have dozens of c4.xlarge HVM CentOS 6 Linux instances running in AWS. They're part of a distributed measurement network that collects data and fowards it to a central location using rsyslog. All instances are running within the same VPC and it's a flat /24 network. The version of rsyslog that I'm running is 8.36.0-2 and I'm using the syslog protocol, not RELP or TLS or any of the other fancier transport protocols that rsyslog supports. Each measurement node forwards dozens of events per second to the central collector. All of this works great except that once every 3-4 months the measurement data on a small number of instances will become corrupted by having one or more bits shifted by a single character. For example, one of the fields of data that these instances forward to the central collector includes the hostname of the instance. This instance is named "tagserver-prod-us-west-2-i-0d16b7e4c51e85697.internal.domain", but here are the erroneous values that it used for its hostname when this problem occurred during a period of 15 minutes:

tagserver-ppod-us-west-2-i-0d16b7e4c51e85697.internal.domain tagserver-prod-uq-west-2-i-0d16b7e4c51e85697.internal.domain tagserver-prod-us-west-0-i-0d16b7e4c51e85697.internal.domain tagserver-prod-us-west-2-i-0d14b7e4c51e85697.internal.domain tagserver-prod-us-west-2-i-0d16b5e4c51e85697.internal.domain tagserver-prod-us-west-2-i-0d16b7e4a51e85697.internal.domain tagserver-prod-us-west-2-i-0d16b7e4c51e85697.internal.domain tagserver-prod-us-west-2-i-0d16b7e6c51e85697.internal.domain tagserver-prod-us-west-2-i-0f16b7e4c51e85697.internal.domain tagserver-prod-us-wgst-2-i-0d16b7e4c51e85697.internal.domain tagserver-prof-us-west-2-i-0d16b7e4c51e85697.internal.domain tagserver-rrod-us-west-2-i-0d16b7e4c51e85697.internal.domain

Each value is slightly different than the other values by a single character and each character has been shifted by two characters either backwards or forwards. Here are some examples taken from the above list:

r -> p s -> q 2 -> 0 6 -> 4 7 -> 5 a -> c d -> f

This problem was observed on the central rsyslog collector once all of the aggregated data underwent validation before being sent off to another server to be processed, therefore I can not definitely say where the corruption occurred: on the measurement node where rsyslog is running, on the network when rsyslog forwarded the data to the collector, or on the collector itself. There were two measurement nodes that experienced this issue during the same time period.

I'm at a bit of a loss to determine where this bit shifting is occurring. I've looked through my kernel logs and system logs trying to correlate the times when the bit shifting occurred to some issue with the system but haven't been able to find anything, everything else on the system appears to have been working fine. I've read that bit shifting can occur as a result of bad memory, but as I understand it all of the hardware in AWS runs ECC memory which should prevent this from happening due to faulty memory. Is there anything in Linux or rsyslog that I should be looking at to determine the cause of this issue or to prevent its reocurrence? I've read that there's a kernel module called EDAC in Linux that can help detect memory errors but I'm not sure that it would function in a VM, I believe it would need direct access to physical memory. Does anyone have any suggestions on what might be causing this issue or where I should focus my troubleshooting? Is there anything specific to Linux that I can look at to address this issue or to mitigate it?

submitted by /u/campee
[link] [comments]

Looking to create a project using Perl

11 hours 54 min ago

Would like your suggestions. Looking for something that is easy to do and that can I use to build on.


submitted by /u/gainzmaster69
[link] [comments]

Distro Review #2: Tiny Core Linux

17 hours 15 min ago

Summary: Absolutely the smallest distro (besides Damn Small Linux) that is not a terminal.

Storage: Tiny like it’s name. 25 MB.

Preinstalled programs: Terminal emulator, control panel, mounting tool, program runner, app menu, picture editor, turn off button, screenshot app, top, xkill, tc-wbarconfig, time setter, and some weird thing called Services.

Recommended for: Older PCs and people who have storage in mind

Basic preinstalled terminal items: wget, sh, apt or yum or tce-load depending on version (I have tce-load)

Description: Basic operating system with terminal emulator and some system tools. First thing to do is get a web browser from a mirror of your web browser via wget. Then, make sure you like your desktop background. Then, add your favorite apps. If your HDD/SSD/other computer storage object/VM has 1 GB or under of storage, ONLY install mandatory things like a browser and a text operator. Basically, good for light (non-web-gaming) web browsing, light to medium work usage, and anything else that is basic (Especially if you don’t have much storage).

Summary: 6/10 overall, 10/10 lightweight, 4/10 compatibility due to no default web browser, different package managers, and no graphic package manager. 9/10 storage, 1/10 heavy usage.

submitted by /u/WonderedLamb256
[link] [comments]

Make option to automatically detect necessary machine configuration

Wed, 2018-11-14 22:50

Is there an option for kernel building that helps you build the best configuration for your machine?

I have a ThinkPad E470 and I'm having several problems while using Arch Linux apparently because of kernel (the problems include connectivity to performance and I have an AMD desktop which works just fine with the same OS) and I'd like to know if there's a way to automatically detect the best configuration for my laptop.

submitted by /u/victorqueirozg
[link] [comments]

Where to begin with shell scripting?

Wed, 2018-11-14 20:03

Good afternoon r/linux, not a Linux newbie, but totally new at shell scripting. I've used various flavors as my primary OS the last decade or so, but would like to learn shell scripting. Are there any books or websites the people here recommend? I'm completely new to the shell scripting game and would like to learn.

submitted by /u/redmeansdistortion
[link] [comments]

Linux related: if this is in the wrong sub, i'm sorry =(

Wed, 2018-11-14 19:54

OK, so. I have, of late, become very nostalgic for a particular period of my life where I spent a lot of time within the Macintosh System 7 GUI. monochrome graphics and little sounds from doing something wrong, etc.

I would like to re-create this GUI on my Linux system, mostly just the look and feel as much as I am able.

does anybody know where I could get started [ ie someone has already done this or a good tutorial]?

I am looking forward to any suggestions you can offer. I am not afraid of work [ I mean here scripting, compiling stuff etc.]

Thank you for your time -

submitted by /u/FORGOT123456
[link] [comments]