Investment Price Monitoring Tools For Linux Lite Diamond 4.x
Part Three: Simpler Data Visualization & Storage for Stocks
Introduction:
After completing the first two tutorials on this subject, I realized that the basic script and file structure needed some tuning up, and while working on that project it occurred to me that some Linux Lite users would not want to install ruby full, or the nodejs npm package manager to their system. Since it has always been my intent to set this up as a ruby application, it is not counter-productive for me to go ahead and write a good working shell script for BASH first. Linux is essentially about files: how and where they’re saved and stored; how and where they’re used; and how and where they’re moved. So when it comes to shell scripts, and moving in and out of different shells: basic nix, BASH, DASH, etc.; moving files in and out of directories is the easiest way to utilize different shells and switch from one to the other. What follows here is a clean script for utilizing the AlphaVantage API to obtain stock prices and historical data, and to chart a basic trading candlestick chart, and trading volume chart. I will write a separate tutorial and script to follow this one for crypto-currencies. You must install wget and gnuplot; and optionally the small applications figlet and fim. None of these will bloat your Linux Lite 4.0 system, and when I package this for Debian the first three will be added as dependencies, and when I write this for ruby I will add fim anyway. We will keep the gnuplot exit script from the last tutorial as it is handy for testing plots without a directory structure to move through as it saves escaping shells manually and/or having to delete files over and over when testing. What we will create here, is a sit down at your desk morning coffee, type stockmarket into your Linux Lite terminal, obtain a market pricing overview, and chart a few of your pet stocks, in a matter of seconds application.
1) Configuration
The first thing you will need to do is create the directory structure that the script uses. Open Thunar and right click in your home directory to create a new folder called litemarkets. See image below.
Next open the litemarkets folder and right click and create the folders shown, and then create the empty files (.sh) shown. Use the names shown as they are the names used in the script. The .sh files will be white until we set them to execute. See image below.
Let’s write the two small script files first. Open gpltex.sh with Linux Lite’s simple text editor and type in the lines below.
#!/bin/bash
gnuplot exit
Save the file and close the text editor. Open Linux Lite’s XFCE terminal and type the command below.
chmod +x ~/litemarkets/gpltex.sh
and hit enter.
Next open usefim.sh with the text editor and type the lines below.
#!/bin/bash
fim -R ~/litemarkets/chartstorage/
Save the file and close the text editor. Open the terminal and type the command below.
chmod +x ~/litemarkets/usefim.sh
and hit enter.
To use your new script you will need an AlphaVantage API key. See the first link below to obtain your key. See the second link below for all the free API data services offered by AlphaVantage.
https://www.alphavantage.co/documentation/
What follows below here is the script file from a LibreOffice Writer text document which you will need to copy and paste into your quotes.sh script file. Simply highlight the script as you see it in your browser and select copy, then open up LibreOffice Writer and paste it into a text document. Save it as is, and then copy it by highlighting it again and open up your quotes.sh file with your text editor and paste it in. You cannot paste it directly from your browser to your text editor, as the format will skip a line due to the html code. It doesn’t matter that I wrote the long wget lines in a smaller font so that you could see them as one long line in your browser.
After you have pasted and saved the script as quotes.sh you will need to do some editing. Use your simple text editor in Linux Lite, and wherever you see yourusername you will need to replace that with your actual user name. Wherever you see apikey=XXX you will need to replace the Xs with your new 16 character AlphaVantage API key. Once you have made those changes run the command below from a terminal.
chmod +x ~/litemarkets/quotes.sh
Next in the terminal type sudo nano ~/.bashrc and enter your password. When the file opens move the cursor down to the first line below the last alias line and add the line alias stockmarket=’~/litemarkets/quotes.sh’ and then hit CTRL+x then y then enter to save the file.
#!/bin/bash
#Simple script for Linux Lite 4.0 to use AlphaVantage API & IEX market data
#This script from TC is free to use and modify for Linux Lite users
#This script depends on wget gnuplot figlet fim
figlet -f small "Lite Stock Prices"
figlet -f term "Data Provided By AlphaVantage API & IEX"
wget -q -O - 'https://www.alphavantage.co/query?function=BATCH_QUOTES_US&symbols=FB,GOOG,AMZN,AAPL,MSFT,VMW,AMD,INTC,T,IBM&apikey=XXX&datatype=csv' | column -t -s,
wget -q -P /home/yourusername/litemarkets/temp/ 'https://www.alphavantage.co/query?function=TIME_SERIES_WEEKLY_ADJUSTED&symbol=VMW&apikey=XXX&datatype=csv'
mv /home/yourusername/litemarkets/temp/* /home/yourusername/litemarkets/weekly/vmwweekly.dat
wget -q -P /home/yourusername/litemarkets/temp/ 'https://www.alphavantage.co/query?function=TIME_SERIES_WEEKLY_ADJUSTED&symbol=FB&apikey=XXX&datatype=csv'
mv /home/yourusername/litemarkets/temp/* /home/yourusername/litemarkets/weekly/fbweekly.dat
#The two lines below may be commented out if not using fim
figlet -f small "Lite Stock Charts"
figlet -f term "**see fim window**"
gnuplot <<- EOF
set terminal png
set output "~/litemarkets/chartstorage/vmwprice-$(date '+%FT%T').png"
set title "VMW Dell Price 12 Weeks"
set xdata time
set timefmt"%Y-%m-%d"
set xrange ["2018-08-10":"2018-10-12"]
set yrange [*:*]
set datafile separator ","
plot "~/litemarkets/weekly/vmwweekly.dat" using 1:2:4:3:5 notitle with candlesticks
EOF
gnuplot <<- EOF
set terminal png
set grid
set style line 1 \
linecolor rgb '#9494ff' \
linetype 1 linewidth 2 \
pointtype 7 pointsize 1.0
set output "~/litemarkets/chartstorage/vmwvolume-$(date '+%FT%T').png"
set title "VMW Dell Volume 12 Weeks"
set xdata time
set timefmt"%Y-%m-%d"
set xrange ["2018-08-10":"2018-10-12"]
set yrange ["0.0":"300.000.000"]
set datafile separator ","
plot "~/litemarkets/weekly/vmwweekly.dat" using 1:6 notitle with linespoints linestyle 1
EOF
gnuplot <<- EOF
set terminal png
set output "~/litemarkets/chartstorage/fbprice-$(date '+%FT%T').png"
set title "FaceBook Price 12 Weeks"
set xdata time
set timefmt"%Y-%m-%d"
set xrange ["2018-08-10":"2018-10-12"]
set yrange [*:*]
set datafile separator ","
plot "~/litemarkets/weekly/fbweekly.dat" using 1:2:4:3:5 notitle with candlesticks
EOF
gnuplot <<- EOF
set terminal png
set grid
set style line 1 \
linecolor rgb '#9494ff' \
linetype 1 linewidth 2 \
pointtype 7 pointsize 1.0
set output "~/litemarkets/chartstorage/fbvolume-$(date '+%FT%T').png"
set title "FaceBook Volume 12 Weeks"
set xdata time
set timefmt"%Y-%m-%d"
set xrange ["2018-08-10":"2018-10-12"]
set yrange ["0.0":"300.000.000"]
set datafile separator ","
plot "~/litemarkets/weekly/fbweekly.dat" using 1:6 notitle with linespoints linestyle 1
EOF
#If not using fim the line below may be commented out
~/litemarkets/usefim.sh
mv /home/yourusername/litemarkets/weekly/vmwweekly.dat /home/yourusername/litemarkets/datastorage/vmwweekly-$(date '+%FT%T').dat
mv /home/yourusername/litemarkets/weekly/fbweekly.dat /home/yourusername/litemarkets/datastorage/fbweekly-$(date '+%FT%T').dat
If you are having trouble with the script below is a link to it in plain text on google drive.
https://drive.google.com/open?id=1ExxqweDQdcr9Hp-p6jzxHqi0FtD08zBZ
After completing all the steps above open up your Linux Lite XFCE terminal, toggle it to full screen and type stockmarket at the prompt and hit enter. The results should look like the images below.
This script also saves your charts and data files with a timestamp in the file name so if you don’t want to use fim just comment out the lines in the script that access fim. (Put a # sign at the beginning of the lines) Any image viewer can view your charts. See images below.
Discussion:
Initially because the delimited files from the API download without a file extension, though both LibreOffice Calc and gnuplot could work with them, a shell escape was still necessary in BASH when finished. This is fine, even convenient for initial trial and error testing, but annoying and unnecessary for an application that saves files. So we move and name the files with a .dat extension after download to differentiate data between specific stocks, and name differentiate and timestamp in the file name the gnuplot chart output. Later when we are done with the data we also move and name differentiate and timestamp in the file name the data files. This is useful for using the data with other applications like LibreOffice Calc, or KST, and for sorting and deleting files as you go. In the process this keeps the Temp folder open for adding other wget API downloads, like other stocks you wish to chart, or daily, monthly, moving averages, etc. and by just adding folders for new data types and guiding the files through the script in the same way the charting capacity can be widely expanded. Gnuplot can plot just about anything, so it is also a great place to start, and with stored data files you can test your plots by trial and error without having to download each time. There are many wonderful financial plotting examples online. I think we are on the track of a useful application now.
**This tutorial was prepared entirely in Debian 9 using LibreOffice Web Writer and Linux Lite 4.x Series running in gnome boxes.**