Tuesday, November 5, 2019

How to increase Elasticsearch Shard recovery Speed

To Increase Shard recovery speed do this:


Increase node recovery speed


PUT http://es:9200/_cluster/settings

{
"persistent" : {
"indices.recovery.max_bytes_per_sec": "200mb",
"indices.recovery.max_concurrent_file_chunks": 5,
"cluster.routing.allocation.node_concurrent_recoveries" : 5
}
}


Links:

https://www.elastic.co/guide/en/elasticsearch/reference/current/shards-allocation.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/recovery.html


Wednesday, January 23, 2019

Shell Script to SSH into servers

This is a simple script that can ssh into a server by typing the password for you.

Filename: login
#!/usr/bin/expect
eval spawn ssh username@servername
expect "assword:"
send "server_password\r"
interact

# to run the script
# ./login
In this below script you can pass arguments to the shell script
Filename: login
#!/usr/bin/expect
set num [lindex $argv 0];
eval spawn ssh username@servername-$num
expect "assword:"
send "server_password\r"
interact

# make sure to give executable permission to script.

chmod +x login

# to run the script and login to server-01.

./login 01

Now add the script path to your .bashrc file as alias, so that you can access this script from any path.


# vi ~/.bashrc

alias login='/path/to/script/file/login'


After updating the .bashrc file, you will have to source.
source ~/.bashrc


Now call the script from anywhere in your shell.
login 01

Enjoy!!!

Tuesday, July 24, 2018

How to install Ruby 2.X and Rails 5.X using RVM

The best way to install Ruby on Rails is using RVM in Linux.

Advantages of RVM,
1. You can shift between multiple versions of Ruby easily.
2. You don't have to worry about install Ruby dependencies. RVM install will take care of installing them. Some of the ruby dependencies are : gawk, autoconf, automake, bison, libffi-dev, libgdbm-dev, libncurses5-dev, libsqlite3-dev, libtool, libyaml-dev, pkg-config, sqlite3, zlib1g-dev, libgmp-dev, libreadline6-dev, libssl-dev
3. Easily upgrade Ruby version and try out a latest version to make sure your project works with the latest Ruby. If something breaks, changing the version is easy.
4. Delete unwanted Ruby version anytime :)

Installation steps.

Step 1. Install RVM

Source : https://rvm.io/rvm/install
$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

\curl -sSL https://get.rvm.io | bash -s stable
$ source /etc/profile
$ source ~/.rvm/scripts/rvm
$ type rvm | head -n 1  (this command returns 'rvm is a function' )

Step 2. Install Ruby

To List all known Ruby versions by rvm

$ rvm list known
From the list, pick a version and install ruby

$ rvm install ruby-2.3.7

Step 3. Install Rails


Rails is a Ruby gem. I always install bundler gem along with rails installation because it will be used in all Rails application to install dependencies
$ gem install bundler
$ gem install rails

Step 4. check version and start a project

# To check the Ruby version
$ ruby -v

# to check the Rails version
$ rails -v

# to start a new rails project with Mysql database. Rails by default uses sqlite. If you want to use any other database you will have to specify the adapter in config/database.yml
$ rails new project_name -d mysql

# Go inside the new project 
$ cd project_name

# to install the dependencies
$ bundle install

# To start the server
$ rails s

Now in the browser you can access the rails starter web page using,
http://localhost:3000

Wednesday, July 11, 2018

Getting started with Python Flask application with MVC structure

I have added a MVC structure and getting started with Python flask application in my Github. 
I will continue to contribute to this repository with Database connections and JWT Authentication.


Lets get started with Python and Flask :)

Here is the Github link for my project.

Structure your Flask application like a Rails Application.

I'm a Rails developer. I like the way a Rails application is structured and I wanted to implement the similar kind of structure for a Python Flask web Application. Flask by default will not give you this structure. So make use of the getting started with Python flask repository above. The Code structure will look like this:
python-flask-getting-started/
|-- app
| |-- config.py
| |-- controllers
| | |-- hello_controller.py
| | |-- __init__.py
| |-- helpers
| | |-- common_helpers.py
| | |-- __init__.py
| |-- __init__.py
| |-- models
| | |-- __init__.py
| `-- templates
| |-- hello.html
| `-- __init__.py
|-- log
| `-- test.log
|-- README.md
|-- requirements.txt
`-- server.py

Prerequisite

Make sure you have installed python :) To check python version,
python -V

Make sure you have installed python-pip. If you are using Ubuntu, you can apt-get like this
sudo apt-get install python-pip

To use the Python Flask Starter template.

Step 1 : Checkout code

git clone https://github.com/srikanthjeeva/python-flask-getting-started.git cd python-flask-getting-started

Step 2: Install dependencies

(use sudo if required)
pip install -r requirements.txt

Step 3 : Start the server

python server.py
In browser go to,
http://localhost:5000
or
http://localhost:5000/hello
or for UI
http://localhost:5000/ui/hello

Tuesday, May 9, 2017

Install Cpanm module error : No such file or directory opening compressed index

I got this error while installing Rest client perl module
$ HOME=/tmp /home/perl/5.10/bin/cpanm REST::Client
! Finding REST::Client on cpanmetadb failed.
! cannot open file '/tmp/.cpanm/sources/http%www.cpan.org/02packages.details.txt.gz': No such file or directory opening compressed index
! Couldn't find module or a distribution REST::Client

Solution:

The problem is because of "LWP::Protocol::https" module. Removing the directory worked for me.
$ cd /perl_installed_path/perl/5.10/lib/site_perl/5.10.1
$ rm –rf LWP*
or try with option "--no-lwp"
cpanm REST::Client --no-lwp

Error while installing Perl 5.10.1

a -Wdeclaration-after-statement -Wendif-labels -Wc++-compat

cc -fstack-protector -L/usr/local/lib -o miniperl \

gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o xsutils.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o \

miniperlmain.o opmini.o perlmini.o

pp.o: In function `Perl_pp_pow':
pp.c:(.text+0x2d79): undefined reference to `pow'
pp.o: In function `Perl_pp_modulo':
pp.c:(.text+0x3b22): undefined reference to `floor'
pp.c:(.text+0x3b58): undefined reference to `floor'
pp.c:(.text+0x3b90): undefined reference to `fmod'
pp.o: In function `Perl_pp_atan2':
pp.c:(.text+0x8985): undefined reference to `atan2'
pp.o: In function `Perl_pp_sin':
pp.c:(.text+0x8b22): undefined reference to `sin'
pp.o: In function `Perl_pp_int':
pp.c:(.text+0x8fc1): undefined reference to `floor'
pp.c:(.text+0x9031): undefined reference to `ceil'
pp.o:(.rodata+0x120): undefined reference to `cos'
pp.o:(.rodata+0x128): undefined reference to `sin'
pp.o:(.rodata+0x130): undefined reference to `sin'
pp.o:(.rodata+0x138): undefined reference to `exp'
pp.o:(.rodata+0x140): undefined reference to `log'
pp.o:(.rodata+0x148): undefined reference to `sqrt'
pp_pack.o: In function `S_pack_rec':
pp_pack.c:(.text+0x740b): undefined reference to `floor'
pp_pack.c:(.text+0x742e): undefined reference to `floor'
pp_pack.c:(.text+0x7453): undefined reference to `floor'
collect2: error: ld returned 1 exit status
makefile:347: recipe for target 'miniperl' failed
make: *** [miniperl] Error 1
This link helped me to find what i was missing: https://rt.perl.org/Public/Bug/Display.html?id=55980 and finally this worked for me

Step 1: find your lib path which has all libm* files


In most systems it will be under /usr/lib64. Search for the files in your system.
system:/usr/lib64 $ ls libm*
libm.a libmcheck.a libmenuw.so.5 libmpc.so.3.0.0 libmpfr.so.4 libmpx.so.0.0.0 libm.so libmvec.a libmagic.so.1 libmenu.so.5 libmenuw.so.5.9 libmpdec.so.2 libmpfr.so.4.1.4 libmpxwrappers.so.0 libmspack.so.0 libmvec_nonshared.a libmagic.so.1.0.0 libmenu.so.5.9 libmpc.so.3 libmpdec.so.2.4.2 libmpx.so.0 libmpxwrappers.so.0.0.0 libmspack.so.0.1.0 libmvec.so

Step 2: Use the path with glibpth option while doing configure

./Configure -des -Dglibpth='/lib/lib64 /usr/lib64'
or I see some forums suggested plibpth
./Configure -des -Dplibpth='/lib/lib64 /usr/lib64' -Dglibpth='/lib/lib64 /usr/lib64' -Dlibpth='/lib/lib64 /usr/lib64'

I have answered the same here @Stackoverflow: http://stackoverflow.com/a/43878868/453486