Thursday, March 28, 2019

How to install bluejeans Meeting in Ubuntu 18

Bluejeans meeting for desktop is only available for MAC OS, Windows and Linux based on rpm (CentOS, Redhat etc)

For Ubuntu we need to convert the rpm to deb, for this we use the program called Alien(https://wiki.debian.org/Alien)

Step1: sudo apt-get install alien

Step2: Download the rpm from bluejeans website and convert with following command.
sudo alien --to-deb bluejeans-1.37.22.x86_64.rpm
**replace the file with the downloaded rpm file name

Step3: dpkg (debian package) is a low level tool. apt (advanced package tool) is a high level tool, apt is more commonly used than dpkg as it can fetch packages from remote locations and can deal with complex package relations, such as dependency resolution etc.

From previous step deb file with the name "bluejeans_1.37.22-2_amd64.deb" was generated in my case. Now we will unpack the package and install with below command.

sudo dpkg -i bluejeans_1.37.22-2_amd64.deb

Step4: Start the blue jeans from below command

/opt/bluejeans/bluejeans-bin

Done.

**If you get error as below.

/opt/bluejeans/bluejeans-bin: error while loading shared libraries: libudev.so.0: cannot open shared object file: No such file or directory

Then, go to this website http://old-releases.ubuntu.com/ubuntu/pool/main/u/udev/
Download the latest of libudev0 example(libudev-dev_141-1.2_amd64.deb) then install with sudo dpkg -i /path/to/package
(or)
check if there is libdev file in directory
cd /lib/x86_64-linux-gnu/
ls libudev*
Then do soft link for the file from above command, in my case
sudo ln -s libudev.so.1.6.9 libudev.so.0
Done.

***

How to convert rpm to deb?

Converting rpm(centOS/Redhat etc) to deb(ubuntu/debian etc) or deb to rpm

To convert rpm packages to deb you will need a program like Alien (https://wiki.debian.org/Alien)

Alien is a computer program that can convert different Linux package distributions to debian. It supports conversion between Linux Standard Base, RPM, deb, Stampede (.slp) and Slackware (tgz) packages.

To convert a .rpm into a .deb simple run:
alien --to-deb /path/to/file.rpm

And vice-versa:
alien --to-rpm /path/to/file.deb

***

Sunday, March 24, 2019

Message Digest, Cryptography/Cipher, Digital Certificates and Public Key Infrastructure (PKI)

Message Digest:
Message digest is a cryptographic hash function containing a string of digits created by a one way hashing formula. And is a fixed size numeric representation of the contents of a message, computed by a hash function.

It protects the integrity of a piece of data or media to detecct changes and alterations to any part of a message.

**This term is also known as a hash value and sometimes as a checksum.

Message digests can be produced on UNIX systems with MD5 command, and are stored on systems and can reveal if any unauthorized user has accessed a file. But MD5 is unreliable with problems relating to collision and it is no longer used.

Besides MD5, SHA and CRC32 are other message digest algorithms.


Cryptography/Cipher:
Cryptography is the process of converting between readable text, called plain text, and an unreadable form called cipher text.

This occurs in the below phases:
Encryption/Encipherment: Sender Converting plaintext message to ciphertext.

Decryption/Decipherment: Receiver converts the ciphertext message back to its plaintext form.

There are two classes of algorithm:
1) Those that require both parties to use the same secret key. Algorithms that use a  shared key are known as symmetirc algorithms
2) Those that use one key for encryption and a different key for decryption. One of these must be kept secret but the other can be public. Algorithms that use public and private key pairs are known as asymmetric algorithms.

Strength:
Asymmetric algorithms require large keys,
1024 bits Low-Strength asymmetric key
2048 bits Medium-Strength asymmetric key
4096 bits High-Strength asymmetric key

Symmetric keys are smaller, 256 bit keys gives strong encryption.

Block cipher algorithm:
These algorithms encrypt data by blocks. Example RC2 algorithm from RSA Data Security Inc. Uses block of 8 bytes long.

Stream cipher algorithm:
These algorithms operate on each byte of data. Stream algorithms are typically faster.


Digital Certificates:
Digital certificates protect against impersonation, certifying that  public key belongs to a specified entity. They are issued by a certificate Authority.

Digital certificate binds a public key to its owner, whether that owner is an individual, a queue manager, or some other entity. Digital certificates are also known as public key certificates.

1) When the certificate is for an individual entity, the certificate is called a personal certificate or user certificate.

2) When the certificate is for a certificate authority, the certificate is called a CA certificate or signer certificate.


Public Key Infrastructure (PKI):
A PKI is a system of facilities, policies and services that supports the use of public key cryptography for authentication the parties involved in a transaction.

***

Credits:
https://www.techopedia.com/definition/4024/message-digest

https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_7.1.0/com.ibm.mq.doc/sy10490_.htm

***

memtable management in Cassandra

Moving data structures off of the Java heap to native memory, keeps up the datasets to continue to grow.

offheap_buffers moves the cell name and value to DirectBuffer objects. This has the lowest impact on reads -- the values are still "live" Java buffers -- but only reduces heap significantly when you are storing large strings or blobs.

offheap_objects moves the entire cell off heap, leaving only the NativeCell reference containing a pointer to the native (off-heap) data. This makes it effective for small values like ints or uuids as well, at the cost of having to copy it back on-heap temporarily when reading from it.

Tune the memtable_offheap_space_in_mb to reasonable value for your system.

***

Understanding JVM Settings

Usually JVM settings file contains a line-delimited list of JVM arguments.

1) Lines with white spaces are Ingored.

2) # are treated as comments
Example: #Optional G1 Settings

3) Lines starting with - are treated as aJVM option that applies for any version of JVM
Example: -Xms4G

4) Lines starting with a number followed by : and followed by - are treated as JVM option for that specified version of JVM.
Example: 8:-Xms4g

5) Lines starting with a number followed by - and followd by : are treated as JVM option for the version greater than or equal to that number.
Example: 8-:Xms4g

6) Lines starting with number followed by - and then a number followed by : are treated as JVM option that applies for the versions falling in that range.
Example: 8-9:-Xms4g

***

Deciding Heap memory size in Cassandra

32GB or 64GB heap memory to be chosen.

Compressed Oops(Ordinary Object Pointers):

C Language Data Type Models: LP64 AND ILP32

The 32-bit Solaris C language data model, called ILP32, defines int, long, and pointers as 32 bits, short as 16 bits, and char as 8 bits. The C data type model chosen for 64 bit operating system is LP64. This data model defines long and pointers as 64 bits, int as 32 bits, short as 16 bits, and char as 8 bits.

In LP64, only longs and pointers change size; the other C data types stay the same size as in the ILP32 model.

C Type ILP32 LP64
char            8            8
short   16            16
int           32            32
long           32            64
long long           64            64
pointer   32            64

An "oop", or "ordinary object pointer" in Hotspot parlance is a managed pointer to an object.It is normally the same size as a native machine pointer. A managed pointer is carefully tracked by the Java application and GC subsystem, so that storage for unused objects can be reclaimed. This porcess can also involve relocating(copying) objects which are in use, so that storage can be compacted.

On an LP64 system, an oop requires 64 bits, while on ILP32 systems, oops are only 32 bits. But on an ILP32 system there is a maximum heap size of somewhat less than 4Gb, which is not enough for many applications. On an LP54 system, though, the heap for any given run may have to be around 1.5times as large as for the corresponding ILP32 system.

Compressed oops represent managed pointers (in many but not all places in the JVM) as 32-bit values which must be scaled by a factor of 8 and added to a 64 bit base address to find the object they refer to. This allows applications to address up to four billion objects(not bytes), or a heap size of up to about 32Gb.

We use the term decoed to express the operation by which a 32 bit compressed oop is converted into a 64 it native address into the managed heap. The inverse operation is encoding.

Which oops are compressed?
In an ILP32-mode JVM, or if the UseCompressedOops flag is turned off in LP64 mode, all oops are the native machine word size.

If UseCompressedOops is true, the following oops in the heap will be compressed:

1) the klass field of every object
2) every oop instance field
3) every element of an oop array (objArray)

Compressed oops is supported and enabled by default in Java SE 6u23 and later. In java SE 7, use of compressed oops is the default for 64-bit JVM processes when -Xmx isn't specified and for values of -Xmx less that 32 gigabytes.

For JDK 6 before the 6u23 release, use the -XX:+UseCompressedOops flag with the java command to enable the feature.


Heap Size configuring:
If a JVM is started with unequal initial and max heap size, it can be prone to pauses as the JVM heap is resized during system usage. To avoid these resize pauses, it's best to start the JVM with initial heap size equal to maximum heap size.

***
Credits:
https://wiki.openjdk.java.net/display/HotSpot/CompressedOops

https://docs.oracle.com/cd/E19620-01/805-3024/lp64-1/index.html

https://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html

https://www.elastic.co/guide/en/elasticsearch/reference/6.6/heap-size.html

***

Saturday, March 16, 2019

Native-Transport-Requests - All time blocking in Cassandra

Native-Transport-Requests - All time blocking in Cassandra

If you see, there were number of All time blocking under Native-Transport-Request in tpstats.

Two thing need to be adjusted.
max_queued_native_transport_requests=1024
which can be set in cassandra-env.sh file by adding line.
-Dcassandra.max_queued_native_transport_requests=1024

And in cassandra.yaml
native_transport_max_threads: 128

use command to see the changes.
watch -d nodetool tpstats
and
netstat -tupawn | grep 9042 | grep ESTABLISHED | wc -l

***

Attaching/Mounting AWS EBS volume group in CentOS

Follow below steps:

1) List the disk groups existing with in the system

ll /dev/xvd*

2) Verify if the volume groups were attached to any directory.
df -h

3) lsblk

4) use mount to mount and umount to unmount.
Example to remove existing mount, umount /random_directory

5) vgdisplay -c

6) Create a volume group "vgroup1"
vgcreate vgroup1 /dev/xvdc

7) Then create logical volume "lvolume1" from volume group "vgroup1". Here, we are utilizing 100% of disk volume.
lvcreate -l100%FREE -n lvolume1 vgroup1

Now Create file system
mkfs -t xfs /dev/vgroup1/lvolume1

And create required directories and point to them.
mkdir /data_cassandra

mount /vgroup1/lvolume1 /data_cassandra

Then, edit the /etc/fstab file for effective even after reboot. Get the mapper info from df -h or /dev/mapper

8) mount -a (it resets the changes that were modified in /etc/fstab)

9) vgdisplay -c

***

Garbage collect in Cassandra (Removing deleted data from tables)


Follow the following steps for the current session of the terminal or make changes in the yaml configuration.

Set compaction throughput to 0 so that there won’t be any throttling.

nodetool setcompactionthroughput 0

Then set concurrentcompactors to number of CPU’S excluding hyper threading. (lscpu ==> Core(s) per socket * socket(s))

nodetool setconcurrentcompactors 32

To do garage collect run below command.

nodetool garbagecollect -g CELL keyspace_name table_name

Verifying status:
nodetool compactionstats


***