EMC VMAX Storage Automated Performance Report

This blog is being written as a companion to my previous blog on Automated EMC VMAX Capacity Reporting

platform-vmax

In recent times, we’re asked to develop scripts to capture performance metrics from EMC VMAX storage. There is a ‘symstat’ command with many attributes to capture performance metrics information from the array. But this command was not fulfilling all our requirements. While exploring various options and consultation with EMC support / community we decided to try Unisphere / RESTAPI.

So far I was using Perl as THE LANGUAGE to talk to my storage arrays. But I was forced to switch over to Python which works best with REST API / JSON. Additionally, there are lots of code out there on RESTAPI written in Python. So it is easy to ‘get inspired’ by those codes and write customized code for our requirements. So this would make me yet another ‘Pythonistas’ 🙂

This is my first ever Python (version 2.7 on GNU/Debian Linux) script to capture EMC VMAX Performance Metrics retrieved from Unisphere for VMAX (version 8.2) via RESTAPI. I’ve referred this Python script to develop custom script to suit our requirements. Many thanks to Matt Cowger (mcowger) for sharing the script in Github.

There are plenty of metrics that can be captured using this script but I’ve written a simple code for demo purpose to print few metrics in CSV format which can be either imbibed by excel for further reporting / charting or injected to MySQL DB to do many stuffs…

Here is the sample + cropped output for reference. In the below table timestamp (column B) is in epoch format which is converted to MYSQL datetime format via INSERT query

2016-11-04-22_25_58-book1-excel

 

P.S: I’ve changed VMAX serial number for various factors🙂

If interested, please reach out to me to get these Python scripts.

Image Courtesy: https://www.emc.com

References: https://github.com/mcowger/randompython/blob/master/symmREST.py

Thanks for stopping by… Please leave your comments / suggestions.

Advertisements

Let’s use letsencrypt

Few folks told me that while they try to access my website vinaybabu.in they get nasty message from browsers that its unsafe to browse through my website. That’s because I was using self-signed certificate to encrypt the internet traffic. I didn’t want to go with CA vendors who charge more than the cost of my #raspberrypi.

After googling got to know about letsencrypt which provides free trusted SSL certificates. I would like to thank below tech blog / link from techmint for installing letsencrypt on my pi. Only catch is that we need to renew certificate within 90 days.

letsencrypt-logo-horizontal

http://www.tecmint.com/install-free-lets-encrypt-ssl-certificate-for-apache-on-debian-and-ubuntu/

To automate renewal of letencrypt (certbot) certificate we can use below script taken from the link https://www.upcloud.com/support/install-lets-encrypt-apache/

 #!/bin/sh
if ! /usr/local/letsencrypt/letsencrypt-auto certonly -tvv --keep --webroot -w <web root folder> -d vinaybabu.in > /var/log/letsencrypt/renew.log 2>&1 ; then
   echo Automated renewal failed:
   cat /var/log/letsencrypt/renew.log
   exit 1
fi
apachectl graceful

Note: Do change the <web root folder> with actual path

Save the above script in /root folder. I’ve saved this script as auto_renew.sh

Finally, create a new crontab entry as under to run the script every 80 days. Threshold to renew Letsencrypt (certbot) certificate is 30 days.

#crontab -e <run as root>

0 0 */80 * * /root/auto_renew.sh

To check crontab entries use below command

#crontab -l <run as root>

That’s it! folks can access my website from any type of device or browser without any issues. I’ve got free trusted certificate and enabled secure and encrypted traffic while accessing my website.

2016-09-24-19_36_59-welcome-to-vinays-website

Thanks much @letsencrypt and @tecmint.

Disclaimers:

Image’s courtesy: letsencrypt.org

Squirrelmail IMAP 111 : Connection refused

After I started using K-9 Mail on my Oneplus 2, I hardly accessed webmail service hosted on my #RaspberryPi. I wanted to send an urgent note but when I tried to login, got a error message as under.

Error connecting to IMAP server: localhost 111 : Connection refused

As usual, I googled for hours in between my office work 😛 but could not find out the solution. I started rewinding myself to recall the changes made to the Pi, then suddenly remembered that a change was made to /etc/hosts by replacing ‘localhost’ with my server name…

Alas! I ran squirrelmail-configure command and updated server settings which resolved the issue.

Now I realize change management plays an important role in any level be it organization or personal use since the task of managing change is not an easy one!!

Happy Pi-ing!!

 

Personal Cloud Storage on Raspberry Pi

I’m a big fan of Dropbox and using its services from past few years. I’ve referred few of my friends there by increasing my personal cloud storage capacity to about 12 GB! Off late file syncing stopped because of lack of space in my Dropbox account. I wanted to sync photos and other important files from my phone to Dropbox and sync it on my personal laptop.

To avoid buying cloud storage, started googling for setting up personal cloud storage and sync tools. After reading multiple blogs and comparisons, I’ve decided to go with Seafile. Though Owncloud was very close competitor, I did prefer Seafile for client side encryption feature. We do not have apt package for Seafile yet, but installation was simple, thanks to the readily available script which takes care of install and configuration of Seafile. It took roughly 10 mins to setup on my Raspberry Pi.

Prerequisites:
a. Working Raspberry Pi running Raspbian OS
b. Install MySQL  > Follow steps from here https://pimylifeup.com/raspberry-pi-mysql-phpmyadmin/
c. Public IP or Dynamic DNS account to access Cloud from external network / internet

Installation Steps:
1. Download latest version of Seafile for Raspberry Pi from https://github.com/haiwen/seafile-rpi/releases

#wget https://github.com/haiwen/seafile-rpi/releases/seafile-server_xxx.tar.gz [Replace xxx with actual filename]
2. I’m using an external hard drive of size 500GB mounted as /usbstore. Change directory  #cd /usbstore
3. Copy downloaded seafile tarball

#cp seafile-server_xxx.tar.gz /usbstore [Replace xxx with actual filename]
4. Extract file

#tar -xvzf seafile-server_xxx.tar.gz
5. Change directory

#cd seafile-server*
6. Enable execution of script if not already

#chmod +x setup-seafile-mysql.sh
7. Run the script

#./setup-seafile-mysql.sh
8. Installation will start. Script will start collecting information required. Give a good name to your server 🙂 like “limitlesscloud”
9. When it prompts for IP address, Enter the IP address of Raspberry Pi
10. Ports 8000 and 8082 are used by Seafile. Accept the default ports unless you want to change
11. Creating admin account: Provide your email address for account name
12. Enter the password for the account
13. Database configuration: Choose option ‘1’ at the prompt to create new database for Seafile configuration
14. Since MySQL is running locally leave the default option “localhost” as is
15. By default username for the database will be ‘root’. Let’s change it as ‘seafile’
16. Enter password for ‘seafile’ database user account
17. Accept the default database names and continue
18. At the end; it prompts “Enter” to continue and showing the configuration details. Hit enter!
19. Congratulations!!! Seafile installation is completed. Unlike other software, Seafile wont start by itself
20. Start the Cloud Storage services

#cd /usbstore/seafile-server-latest && ./seafile.sh && ./seahub.sh
21. Install Seafile from playstore on Android and use the respecitve methods on other platforms (Sorry Windows ‘Smartphone’ users 😛 ). We do have clients for Linux, Mac and Windows…
22. Enter Public IP details http://10.11.12.13:8000 Provide Username user@somesite.com and Password
23. Create Library / Libraries and enable encryption with password protection
24. Now I’ve about 500GB of personal ‘secure’ cloud storage capacity with encryption enabled. Seafile is flexible to further configure it to suit my requirements.

Enjoy your personal cloud storage. I love my Raspberry Pi ~ Small but orotund!!!

Wire free connect to Android Phone via SSH

I left my USB Type C cable cum charger in office yesterday and couldn’t transfer some files. I am aware of Shareit app but it doesn’t work in Linux. Both my laptop and phone are connected to wireless router.

After googling for while found a native and simplest way to connect to my android phone via SSH.

Installed SSHDroid on my phone and started the SSH service. From my Debian GNU Linux console started SSH session and connected to the IP of my phone and using SCP copied all those required files from phone to PC. BTW copied new songs from PC to phone as well 🙂

Connect external USB HD to Raspberry Pi

 

I wanted to connect the external USB disk to backup logs, configuration files and other critical data which are running on Raspberry Pi internal SD card.

I was trying to connect external USB HD to Raspberry Pi. Though the LED was lit, disk was not detected by Pi. I felt something wrong with the disk so for testing it, connected USB disk to laptop. Disk was detected successfully and was able to read write some files to it. After googling for a while found that the root issue was inadequate power from Pi to USB Disk.

I bought an external powered USB Transcend TS-HUB3K from the local computer shop. It worked like a charm 🙂

Now, I’ve connected two USB disks instead of one and still two more ports left to connect USB stick or any other USB device without worries about lack of power supply from Pi.

Bandwidth throttling using Squid Proxy

Internet is the most important and one of the basic amenities in today’s world. Sharing internet using WiFi router with slow speeds having low bandwidth is a real pain. Plus if you end up working from home and taking Skype for Business (formerly Lync) calls then you’ll be virtually fighting with your kids & wife. They’ll be using the same internet to browse youtube, shopping sites etc which are bandwidth hungry apps.

Squid-cache_logo

 

I was aware of Squid proxy but never explored it to use it for throttling bandwidth for clients. I followed this guide to install squid on my Raspberry Pi and modified configuration file according to my requirements. Below is the configuration file of squid proxy which will throttle the internet speed / bandwidth for the clients like laptop, android phones and tablets.

#squid.conf

acl SSL_ports port 443
acl Safe_ports port 80        # http
acl Safe_ports port 21        # ftp
acl Safe_ports port 443        # https
acl Safe_ports port 70        # gopher
acl Safe_ports port 210        # wais
acl Safe_ports port 1025-65535    # unregistered ports
acl Safe_ports port 280        # http-mgmt
acl Safe_ports port 488        # gss-http
acl Safe_ports port 591        # filemaker
acl Safe_ports port 777        # multiling http
acl CONNECT method CONNECT
#http://www.tecmint.com/configure-squid-server-in-linux/
acl localhost src 127.0.0.1/32
acl localnet src 192.168.2.0/24
#Throttle bandwidth
acl clientpc src 192.168.2.90-192.168.2.100/24
delay_pools    1
delay_class    1 1
delay_parameters    1    64000/64000
delay_access    1    allow    clientpc
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localnet #!clientpc
http_access allow localhost
http_access deny all
http_port 3128
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:        1440    20%    10080
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern -i (/cgi-bin/|\?) 0    0%    0
refresh_pattern .        0    20%    4320

I’ve highlighted important lines in RED color above. I’ve created an acl called “clientpc” for the local network having IP address starting from 192.168.2.90 to .100. Created a default pool ‘1’ and class ‘1’. ‘delay_parameters’ for pool ‘1’ set to 64000 bytes (512KB). We can increase or decrease depending on the internet speed or bandwidth.

Enabled proxy in firefox. If you’re in stone age and still use ‘IE’ then go search for ‘proxy settings for IE’ in search engines 😛

Steps
  1. Click the Firefox menu. Select Options from the submenu.
  2. Open the Advanced category. It is on the right end of the category list. …
  3. Click the Settings button. This will open the Connection Settings window.
  4. Choose “Manual proxy configuration”. …
  5. Enter your proxy information.

Screenshot from 2016-07-24 20-24-38.png

Enabled proxy on the android phones and tablets by referring to this link.

Screenshot_20160724-202759.png

Tested internet speed via speedof.me Below screenshot shows that the download speed was successfully throttled by Squid whereas my internet connection speed is 2 Mbps. Now I can work from home with peace of mind and grabbing a largest piece of “internet” pie.

Screenshot from 2016-07-24 20-31-58

Installed “SARG” to monitor internet usage and resource utilization. I followed this link which is pretty simple and easy to set it up. SARG is optional but good to have!!

It was time consuming and eaten up my Sunday but worth to spend the time as I don’t have to crib in future about internet speed or poor voice / audio issues.

Let me know your queries using below ‘reply’ button…