I think ... - Linux

Pimping up My Linux Terminal

2021-03-31T00:00:00+06:00

The purpose of this post is to be my quick, copy-paste source of the commands that I use to set up my terminal on a new *nix system. However, if someone else finds it useful, that’d be some cherries on top.

This command prompt in the below image is the end goal.

Assuming, I’m on a standard pc/server with Ubuntu Linux and I have CLI access with sudo. For other Linux distros or MacOS, some commands might be slightly different.

Step-1: Confirm that Zsh is up-to-date¶

While on most of the Linux systems Zsh is present by default, on others that’s not the case. So, let’s make sure about it.

$ sudo apt install zsh

Confirm the version. Oh-my-zsh recommends Zsh to be 5.0.8 or higher.

$ zsh --version
zsh 5.8 (x86_64-ubuntu-linux-gnu)

Also, you gotta make sure that git (recommended v2.4.11 or higher) is also installed on the system.

Step-2: Install Oh-my-zsh, the fun “configuration” framework¶

Install directly from the source.

$ sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

In the last step of this installation, it will ask to set Zsh as THE shell. Go ahead.

Now we have the default prompt from Oh-my-zsh.

Now, let’s pimp up the prompt. Shall we?

Step-3: Install `powerlevel10k`, a powerful prompt theme¶

I love the powerful Zsh theme powerlevel10k. More on why this theme is awesome.

Let’s install it on top of oh-my-zsh.

$ git clone --depth=1 \
  https://github.com/romkatv/powerlevel10k.git \
  ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k

Now, gotta set ZSH_THEME="powerlevel10k/powerlevel10k" in ~/.zshrc by adding that manually in the file.

Step-4: Make sure the prompt looks like as you want¶

In this step, I’m gonna bring in my already open-sourced Zsh config file aka .zshrc.

# deleting the current one & get my personal one from GitHub
$ rm .zshrc
$
$ wget https://raw.githubusercontent.com/kmonsoor/dot-files/master/.zshrc

I kept the powerlevel10k configs as comments so that Zsh doesn’t complain if I use the config file early. Have to set ZSH_THEME="powerlevel10k/powerlevel10k" in the ~/.zshrc as well.

Otherwise, once the powerlevel10k theme will run for the first time by Zsh, a very friendly step-by-step prompt will run you through towards a desirable prompt for you. Also, whenever you want, you can invoke the config-wizard by executing p10k configure on the shell.

Now is the time to enable the changes by restarting Zsh and enjoy the new config and the powerful prompt.

$ exec zsh

Optional¶

Also, I usually install this very useful, but external plugin zsh-syntax-highlighting for oh-my-zsh.

$ git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting

Don’t forget to activate the plugin by including it in ~/.zshrc. For that, add zsh-syntax-highlighting inside the list of other plugins.

plugins=( plugin_a plugin_b zsh-syntax-highlighting)

Install the latest Zsh on CentOS

2020-10-20T00:00:00+06:00

As the default Zsh on CentOS is usually an older version, many cool things are not possible on this version of Zsh, like installing oh-my-zsh or using awesome powerlevel10k prompt system, it’s understandable if you’d like to have the latest Zsh on your system. Easy peasy !!

Note: Please remember to remove the sudo from the commands if you are already in “root” or sudo-er mode

We’ll be following these steps:

Install the pre-requisites
Download the latest source
Build & Install
Final steps
Related

Install the pre-requisites¶

We need GCC (C++ compiler) and other related stuffs for building Zsh from the source code.

$ sudo yum groupinstall "Development tools"  
$ sudo yum install ncurses-devel

Now, check if GCC is installed properly, by

$ gcc -v

Download the latest source¶

Now, we gonna get the latest code of Zsh.
Please update the link (in the shown command) with the latest by checking this web-folder.
Don’t forget to update the filename as well, if needed.

$ cd /usr/local/src
$ sudo curl -L https://www.zsh.org/pub/zsh-5.8.tar.xz \
-o zsh-5.8.tar.xz

Build & Install¶

Unzip the file, “dig in” to the folder, and build & install from the source.

$ sudo tar -xf zsh-5.8.tar.xz  # the actual version of the downloaded file might be different
$ cd zsh-5.8
$ sudo ./configure && sudo make && sudo make install

Final steps¶

Add Zsh to the login shells by adding ‘/usr/local/bin/zsh’ on the last line of the config file, /etc/shells

$ sudo -e /etc/shells
$ sudo chsh $USER

Update the system’s default symlink to the new Zsh version.

$ sudo ln -sf /usr/local/bin/zsh /bin/zsh
$ zsh --version

It’s always a good habit to clean up after doing stuffs. ;)

$ sudo make clean

That’s it !

Want to have a super, cool-looking command shell? Gotcha, fam. Check out my blog on Pimping up My Linux Terminal.

If you find this post helpful, you can show your support through Patreon or Paypal or by buying me a coffee. Thanks!

Install latest Python 3 on Linux CentOS 7

2018-07-07T00:00:00+06:00

Why¶

Not all distro created equal.
Some are created to join the space race, some are to hold unto the leagcy, some are cutting-edge, some are cutting edge. Some are born to boot-up IoT devices some are to push out heavy graphics.

That’s the fun (albeit, power) of Linux.

CentOS 7 is a powerful and stable distro that runs on thousands (probably, millions) production-grade servers.
In the matter of stability, it’s a beast. However, it doesn’t ship with Python 3, by default. You can install it via EPEL repository, or the below simple steps.

Also, take a note. The Python 2 comes with the system, which is probably 2.7.5, do NOT mess with it. Many system components rely on that specific version. If you need the latest versions of 2, use virtualenv or pipenv.

Prepare your system¶

Start with installing pre-requisite utilities for compilation and development support.

$ sudo yum update && sudo yum groupinstall -y "development tools"  
$ sudo yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel \
       sqlite-devel readline-devel tk-devel gdbm-devel \
       db4-devel libpcap-devel xz-devel expat-devel

Download latest Python source code from Python.org¶

$ wget https://www.python.org/ftp/python/3.6.6/Python-3.6.6.tar.xz
$ tar xf Python-3.6.6.tar.xz
$ cd Python-3.6.6

Enable performance optimizations (optional, but highly recommended)¶

$ ./configure --prefix=/usr/local --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"  
$ ./configure --enable-optimizations

Build and install¶

$ make
$ sudo make altinstall

Now, Python 3.6.6 is ready to be used in your system; located in /usr/local/bin/python3.6

$ which python3.6
/usr/local/bin/python3.6

$ python3.6
Python 3.6.6 (default, Jul 10 2018, 14:04:26)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

For convenience, you can create a symbolic-link with a shorter name. If you had system-installed Python3 (unlikely), don’t do this, as some system-components may depend on that specific older version of Python 3.

$ sudo ln -s /usr/local/bin/python3.6 /usr/local/bin/python3
$ python3
Python 3.6.6 (default, Jul 10 2018, 14:04:26)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Install wheel and pip¶

You don’t need to, because Python 3.6.6 includes these necessary tools included.

$ pip3.6 -V
pip 10.0.1 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)
$ wheel version
wheel 0.29.0

HA(High-Availability) Setup for InfluxDB

2018-01-18T00:00:00+06:00

NOTE Since I have written this article, all the components used in this below architecture have gone through many updates and releases. While the general premise involving influxdb-relay and the multiplexing might still hold, please sync up with the latest release docs before jumping into some serious system design.

Currently, from version 0.9, you cannot create an InfluxDB cluster from the open-sourced free edition. Only commercially available InfluxDB Enterprise can do that for now. That stirred up the early-adopter enthusiast users, especially for their usage in professional setups. They complained that InfluxData, the company behind InfluxDB, is trying to milk the OSS solution for profit.

I can’t blame the InfluxData guys much, as they got to pay their bills too. So far, we — the users of open-source systems — couldn’t show much promise about the financial realities of the projects. Continuing development of OSS products, by only depending on donations, patrons, or enterprise sponsorship, is far too rare and unpredictable, even for the projects that many successful organizations heavily rely on.

Anyways, InfluxDB then promised and later introduced Influx Relay as a complimentary consolation for missing HA parts of InfluxDB. You can get the details here and here about that.

Premise¶

For my needs, I have to try to create a reliable HA(High-Availability) setup from available free options, hence InfluxDB and the relay. It’s quite a bit far from an InfluxDB-cluster in terms of robustness or ease of setup, but it’s got the job done, at least for me.

I needed a setup to receive system-stats from at least 500+ instances and to store them for a while, but without breaking the bank in bills from AWS. Meaning, I could ask for and could use only couple of instances for my solution.

Here were my trade-offs.

Not too many instances for this purpose. Neither, any of the heavyweight lifters e.g. AWS’ m3-xlarge etc. To use only what’s necessary.
To satisfy the budget, hence avoiding pay-per-use solutions as far as it is possible.
Solutions must not be crazy complex, so that handover to the DevOps team be smooth.
Reading the data would be too rarely w.r.t. writing. The related Grafana dashboards will be only used to investigate issues by a handful of people.

Overall Design¶

Write¶

From a birds’ eye view, I decided to use two server instances to run parallelly, hosting InfluxDB on them independently and then sending the same data over to them for storing. This scheme mostly looks like RAID-1 systems.

That brings up a couple of challenges.

None of the agents I used on the sender side could multiplex output. That means, they were able to send data to a single destination, not multiple. On the Windows front, I’ve used Telegraf which is able randomly to switch between pre-listed destinations, but NOT multiple at-once.
In the case of Linux hosts, I used Netdata which is excellent in its own right, but unable to send stats to multiple destinations.
Here comes Influx-relay. It can receive time-series data-stream from hosts on a TCP or UDP port, buffer for a while, and then re-send those received and buffered data to multiple receive ends which can either be an InfluxDB instance or another listening Influx-relay instances.
This chaining can broaden the relaying scheme even further. However, for my purpose, this relay-chaining was not necessary. Rather, from the relay, I am sending data to the separate InfluxDB instances, running on two separate instances.
Now that I partially multiplexed the output, my hosts (senders) still are able to send to one destination. So, I need a proxy as well as a load-balancer. For a while, I was torn between NGINX and HAProxy. Both were new to me.

However, for a couple of reasons, I went for HAProxy. Firstly, I don’t need HTTP session management. Secondly, as I wanted to keep my UDP for later, HAProxy was perfectly capable of that.
NGINX has the support recently, but the maturity was a concern. Also, configuring NGINX seems a little intimidating (which I know might not be so true). Last but not least, and for what it’s worth, out-of-the-box, HAProxy’s stat page carries much more in-depth information than that of free-version of NGINX.
Upon receiving the stats stream, HAProxy was supposed to send that to different Influx-relays in a load-balanced fashion.

So, here’s my rough plan.

collector-agent → HAProxy → (50/50 load-balanced) → Influx-relay → (multiplexed) → 2 InfluxDB instances

Now, each one of the received data is to go to both of the InfluxDB instances, or at least to one in case of failure (or, overload per se) of any the relays or Influx instances. Also, I have chosen to keep Influx-relays deployed as Dockerized and kept HAProxy and InfluxDB instances running as native services. Of course, you can Dockerize HAProxy and InfluxDB, too.

Read¶

As I’ve already noted in the section that reading the data, meaning to fetch data to visualize on Grafana end, will happen rarely and sporadically; only to investigate alarms or any other client-side performance issues.

So, the read requests, reaching the HAProxy end, needed not much routing, other than directly to InfluxDB itself. Still, to better distribute the load I decided to load-balance it 50/50 basis.

Ports¶

As all the READ requests are routed through HAProxy running on each of the instances, to the external world only HAProxy’s port should be opened for this purpose.
On the other hand, for WRITE requests, InfluxDBs are receiving data from relays, one of its own instance and another one on other instance, so InfluxDB should listen on its own port for WRITE requests only. But, this must be accessible only from own VPS zone, but not open to the outside world.
In case of HAProxy as well as InfluxDB, you can use the default ports, obviously, which is 8086 & 8088 respectively. Or, you can choose to go for other ports (security through obfuscation). Your call. In this writing, I’ll go with the defaults.

Authentication, SSL¶

You can configure SSL with your own server certificates through the HAProxy configs. You can even go for SSL from the relays to InfluxDB writes. If your sender hosts are connecting to your HAProxy through public internet, you should at least go for password-based authentication, better to utilize SSL. However, for brevity’s sake, I’ll skip them in this post.

**Note: * Please bear in mind, this is an “in-progress” post; prematurely published to force me to work on it. I have the plan to add all the necessary configurations & commands, that I used, here.

Increase virtual-disk size in VirtualBox on Windows 7

2012-05-26T08:02:00+06:00

Are you stuck with your small virtual disk partition for VirtualBox? You created the partition for some testing & practicing, now it needs more space. I was also stuck with the same problem; so I researched many forums, and here is the gist.

[Note: This procedure won’t work for VM with snapshots. So, please take note the fact about your system beforehand.]

Download Gparted, disk manager for Linux. It’ll come as an ISO file.
Boot into your guest OS, check, with df command, which partition you need to grow. Take note of that specific mounted HD, such as /dev/hdb/sda3
Take backup of your existing data, as I WILL NOT TAKE ANY RESPONSIBILITY IF YOU DAMAGE YOUR DATA.
In host OS, Goto the command prompt by running, cmd.exe
Navigate to the Virtualbox installation folder
Execute the command:

C:\ VBoxManage modifyhd X:\yourVM_DiskPath_InHost\yourVM_Disk.vdi  --resize 40960

you will see progress as below:

If you fail with a message like thid

“VBoxManage.exe: error: Resize hard disk operation for this format is not implemented yet!”

Now, you need this tool CloneVDI.exe from this VirtualBox forum-post. If you’re using this tool, remember to check “Increase virtual drive size to ” with your desired size.

Now, Load the ISO file to the CD/DVD drive of your virtual Linux.

Boot the Guest OS; from the boot menu, select CD/DVD drive, then it will boot in GParted tiny OS. Now the PartitionManager tool will come up automatically, like below:

Now “shrink/Grow” or “Move” your desired partition, but be careful. Don’t rename any of them. Be careful about your data.
Don’t worry, unless you clicked Apply, nothing has actually took place. When you are done with moving and resizing, you have to click Apply to commit the change

Once completed, close Gparted, and then shutdown the OS
Unload the ISO aka virtual CD from virtual OS
Boot into the virtual OS again
Now, check that if the desired partition grew or not by using the df command

Thanks for visiting my blog. If it just helped you, please feel free to “Like” or “share”. Also, your suggestion or comment would be great as well.