Category: PROGRAMMING

Categories
DATA MINING DATA SCIENCE DATA VISUALIZATION PROGRAMMING PYTHON TUTORIALS

Difference between range vs arange in Python

A very big sum python function
Writing a very big sum function in Python programming language

In this article, we will take a look at range vs arange in Python. Learning the difference between these functions will help us understand when to either of them in our programs. Both range and arange functions of Python have their own set of advantages and disadvantages. This article will help us learn about them in detail.

To better understand the difference between range vs arange functions in Python, let us first understand what each of these functions’ do.

range vs arange in Python

range vs arange in Python: Understanding range function

The range function in Python is a function that lets us generate a sequence of integer values lying between a certain range. The function also lets us generate these values with specific step value as well . It is represented by the equation:

range([start], stop[, step])

So, in the above representation of the range function, we get a sequence of numbers lying between optional start & stop values. Next, each of these values are also getting incremented by the optional step values.

range function example 1

So that was the theory behind the range function. Now, let understand it better by practicing using it. Fire up your Python IDLE interpreter and enter this code:

l = range(1, 10, 2)

When you hit enter on your keyboard, you don’t see anything right? That is because the resulting sequence of values are stored in the list variable “l”.

To be able to see the values stored in it, let us print individual list values. So, by indexing each of the list items, we get the following values printed out.

>>> l[0]
1
>>> l[1]
3
>>> l[2]
5
>>> l[3]
7
>>> l[4]
9

So we see a list of numbers, cool! But when you observe closely, you will realize that this function has generated the numbers in a particular pattern. You can see that the first number it has generated is after taking into consideration our optional start parameter. We had set its value to 1. Next, it is also honoring the stop value by printing the value 9, which is within our defined stop value of 10. If you try to index the list for any further value beyond this point will only return an error:

>>> l[5]
Traceback (most recent call last):
  File "<pyshell#7>", line 1, in <module>
  l[5]
IndexError: range object index out of range

So, this confirms that the last value we get will always be less than the stop value.

But the most important thing to observation we need to make here is the step size between each of these values. We can see that each of these values are incremented exactly with a step size of 2. This is the same step size we had defined in our call to range function (the last parameter)!

range function example 2

Does this mean everytime we want to call range function, we need to define these 3 parameters? Not really. If we take a look at the signature of the arange function again:

range([start], stop[, step])

The parameters start and step (mentioned within the square brackets) are optional. This means that we can call the range function without these values as well like this:

nl = range(4)
>>> nl
range(0, 4)
>>> nl[0]
0
>>> nl[1]
1
>>> nl[2]
2
>>> nl[3]
3
>>> nl[4]

In this case, we have only the stop value of 4. As a result we get our sequence of integers starting with a default value of 0. The step value is also defaulted to a value of 1. So we get the integers in the range between 0 to 3, with a step value of 1.

Alright then, hope everything is clear to you up to this point. If this is the case with Python’s range function, what does the arange function in Python do?

range vs arange in Python: Understanding arange function

Unlike range function, arange function in Python is not a built in function. But instead, it is a function we can find in the Numpy module. So, in order for you to use the arange function, you will need to install Numpy package first!

The signature of the Python Numpy’s arange function is as shown below:

numpy.arange([start, ]stop, [step, ]dtype=None)

Wait a second! Doesn’t this signature look exactly like our range function? Yes, you are right! Python’s arange function is exactly like a range function. It also has an optional start and step parameters and the stop parameter.

But then what is the difference between the two then?

range vs arange in Python – What is the difference?

Where the arange function differs from Python’s range function is in the type of values it can generate.

The built in range function can generate only integer values that can be accessed as list elements. But on the other hand, arange function can generate values that are stored in Numpy arrays. We can observer this in the following code:

import numpy as np
a = np.arange(4)
>>> a
array([0, 1, 2, 3])
>>> a[0]
0

We are clearly seeing here that the resulting values are stored in a Numpy array. Each of the individual values can hence also be accessed by simply indexing the array!

range vs arange in Python – Advantages & Disadvantages

This begs us the next question. When should we use Python’s built-in range function vs Numpy’s arange function? To understand this, lets list out the advantages and disadvantages of each of these functions:

Advantages of range function in Python

  • range function returns a list of integers in Python 2. In case of Python 3, it returns a special “range object” just like a generator.

Disadvantages of range function in Python

  • range function is considerably slower
  • It also occupies more memory space when handling large sized data objects.

Advantages of arange function in Python

  • Numpy’s arange function returns a Numpy array
  • Its performance is wat better than the built-in range function
  • When dealing with large datasets, arange function needs much lesser memory than the built-in range function.

So this is the fundamental difference between range vs arange in Python. We can understand them even better by using them more in our everyday programming.

I hope this gave you some amount of clarity on the subject. If you still have any more doubts, do let me know about it in the comments below. I will be more than happy to help you out.

Having said that, take a look at this article. It gives you a simple explanation on the “Difference between expressions and statements in Python“. I have spent considerable amount of time trying to understand these topics. Since there are not many articles available that explains them clearly, I started this blog to capture these topics. Hope you found them useful! If yes, do share them with your friends so that it can help them as well.

With this, I will conclude this article right here. See you again in my next article, until then, ciao!

Categories
DATA MINING DATA SCIENCE DATA VISUALIZATION PROGRAMMING PYTHON TUTORIALS

Introduction To Matplotlib – Data Visualization In Python

Matplotlib Official Logo

This article will give you an introduction to Matplotlib Data Visualization In Python. Matplotlib is a data visualization library package written specifically for us to be used with Python. So Matplotlib is usually the preferred Python package to visualize data while working on Machine Learning & Data Science. It helps us in visualizing the data by representing them with the help of plots and charts.

Brief Introduction To Matplotlib – Data Visualization In Python

We humans are all highly responsive to images than text messages. Images helps us in better visualizing and understanding a situation over interpreting any raw data. So we always wanted a way to represent data through images. If you look at our history, we have always tried to accomplish this in many ways. While I cant go back in time to explain each and every approach, I can quicky give you an historical introduction to Matplotlib. This should help you understand how this package came to be. Why it matters a lot in Python data visualization.

Historical Introduction To Matplotlib – Data Visualization

In the early days of computer data analysis, data scientists often relied on tools like gnuplot and MATLAB to visualize data. However, the problem here was that they had to do it in two stages. First use programming languages like C or Python scripts to process the data. Then plot the resulting data output using gnuplot or MATLAB.

It was a very cumbersome process to say the least. It also resulted in erroneous calculation at times due to lengthy process. As a result, the scientists were in dire need of a simpler solution to this. This is when Matplotlib – Data Visualization package in Python was born.

Matplotlib Official Logo
Matplotlib Official Logo

This helped scientists to both process the data using Python scripts and also visualize the resulting output using Matplotlib package. Now since the Matplotlib was developed along the lines of MATLAB, it is supporting all the functionalities of MATLAB. Because of this reason, it got embraced by the data scientists over MATLAB pretty quickly.

Now you may be wondering why you should be using Matplotlib over MATLAB. For you to understand and appreciate Matplotlib package, you need to understand some of the benefits it brings over a tool like MATLAB. Discussing the advantages and disadvantages of this library in an article that gives an introduction to Matplotlib is appropirate I believe. This will help you in making appropriate decision while chosing this Python library package.

Advantages Of Matplotlib

  • Matplotlib Is Open Source – One of the primary advantage of Matplotlib is that it is an open source package. Because of this, you can use it in whatever way you want. You don’t need to pay any money for this tool to anyone. You can also use it for both academic and commerical purposes.
  • Written In Python – Yet another benefit of Matplotlib is that it is all written in Python. As you use Python programming language to do data processing, plotting its result in Python again makes it so much more easier.
  • Customizable & Extensible – As its written in Python, you will also be able to customize the package (if required) to suit your requirements. In addition to this, you can always extend its functionalities and contribute it back to the open source community. Since Python also has other useful packages, you can also make use of those packages’ functionalities to extend Matplotlib.
  • Portable & Cross Platform – Since its written in Python it is easily portable to any system which can run Python. It also works smoothly on Windows, Linux & Mac OS.
  • Easy to learn – Because the Python language is much easier to learn, any packages written using this language becomes so much more easier. You will not find Matplotlib any different either when it comes to this.

Matplotlib Output formats

When we plot any data using Matplotlib, we can get the resulting output plots in two different ways.

The first method is to get the resulting output plots in a new window. This is useful if you want an interactive data output. In this case, your output will continue to display for as long as your program is running.

The second method of output plots you can obtain is by saving them permanently on your computers. In this method, your resulting output will be saved in a file in standard image formats such as PNG, JPG, PDFs etc. This will be very useful to you when you want to share your results with others. You can also make use of this method when you want to generate a lot of output charts or plots programmatically. But in this case, you have one disadvantage. You will not be able to interact with the resulting output like the way you could in the first method. But as I mentioned earlier, if you just want to analyze the results in bulk at a later time, this is still one of the best method to make use of.

So you might be wondering now, what other different formats can you use to save the resulting output. Let me help you right there! So this is the list of file formats supported by Matplotlib that you can use to save your resulting plots:

EPS, JPG, PDF, PNG, PS, SVG

So now you understand the different formats that we can store our outputs in. Next, let me introduce you to another important feature of Matplotlib. It is called Backends and this terminology is something that you should be aware of when working with Matplotlib.

Introduction To Matplotlib: What Are Backends?

Backends In Matplotlib is a feature using which you can either visualize the output live or store it to analyze them in the future.

As I mentioned in the previous section, we can store the resulting plots in either files or view it live in a new interactive window. Backends simply represent this factor. So from the previous paragraphs we can already realize that there are two types of backends in Matplotlib:

  • Hardcopy backend – This is the type of backend where we save the images in a file
  • User Interface backend – In this type of backend, we display the resulting output in an interactive output window.

In order to provide us with these two type of backends, Matplotlib makes use of two sub modules called the renderer and the canvas. Let us try to learn more about them to get a better understanding of Matplotlib backends.

What is a renderer in Matplotlib?

A renderer is a module used by Matplotlib to draw its output plot or the graph. So it is this module that does the actual drawing of our Matplotlib’s output. The standard renderer used by Matplotlib to render its output is called the Anti-Grain Geometry(AGG) renderer.

Now are you wondering what this rendere do? Want to know what is so special about this renderer? This AGG renderer is a high performance renderer. It helps us in getting generating a publication level quality output. It also helps us in obtaining our output with sub pixel accuracy and antialiasing. Is this all sounding too alien of a terminology for you? Then simply know that the AGG renderer helps in getting a high production quality graphs and plots that we can adore about!

Now that we understand about the renderer used by Matplotlib, let us turn our focus towards the second module – canvas

What Is A Canvas In Matplotlib?

After getting an understanding about renderer, its time for us to learn about the other module – canvas of Matplotlib.

We mentioned that the renderer is responsible for the drawing. But do you know where exactly is this drawing being done at? That is where the canvas module comes into picture. Canvas is the area where the renderer will draw our out plots and graphs. Ok, so who provides this canvas then? Great question!

Canvas in Matplotlib is usually provided by the GUI libraries. So this can be GTK if you are using a Linux machine. Or it can be WX if you are on a Mac OS. On the other hand if you are using Windows machines, these could be coming in from Windows GUI libraries. But these are not all. It could also be coming in from platform agnostic interfaces like QT if you are developing your visualization tools in QT.

So basically we get the canvas from the GUI library we intend to use in our Visualization app.

But do you really need to worry about it? Do you really need to know about canvases and renderers when using Matplotlib? Well, for the most cases, not really. You can simply use Matplotlib’s functions and get away with generating your visualizations. You dont really need to know the underlying aspects of how Matplotlib works to use it. But if you want to be a good visual data scientist, knowing how Matplotlib works under the hood will help you in mastering it. It will also help you in truly appreciating its versatility.

Conclusion

So that is it. This should give you a good introduction to Matplotlib – data visualization and why you should use it. In the next set of articles, we will learn how to install the library package. We will learn how to make use of it to draw some interesting plots and graphs. So see you there! Until then, have a great learning experience!

If you are interested in using Matplotlib to add text to any image, here is a quick tutorial link describing how you can do this.

Categories
DATA SCIENCE EDITORS PROGRAMMING PYTHON TUTORIALS VISUAL STUDIO CODE

Python Add Text To An Image Using Matplotlib Library

Final image result of Python code adding text to image

We can use Python programming language to add text to an image using its Matplotlib library. In this tutorial, we will take a look at the Matplotlib library and learn how to accomplish this. The process of adding text to an image is often also called as annotating text to an image.

What Is Python Matplotlib library?

Matplotlib is a Python data visualization library that is often used to plot graphs and charts to represent a dataset in a meaningful way. However, we can also use Matlplotlib to do some basic editing on an image file. We can use it to overlay graphical data over an image or plot graphs.

But in this tutorial let us make use of Matlplotlib to add basic text annotation to our image. In other words, we will use Python Matplotlib to add text to our image file.

Adding Text To An Image Using Python Matplotlib Library

In order to get started, we first need to use an image file over which we would like to add text. In this example, I will make use of this picture of a butterfly that I shot in my garden. This is a simple JPG file that I got by capturing the butterfly image using my Android smartphone’s camera.

Picture Of A Butterfly. Shot On Motorola One Action Android smartphone

This is a 278KB JPG file over which I would like to add an annotated text stating “What A Wondeful World!”. I woud like the text to be Orange in color with a font size of 30 and stylized to be bold. I would also like to place this text at the bottom of the image, somewhere over here:

Butterfly image highlighting the area where we would like our text to appear at.
Butterfly image highlighting the area where we would like our text to appear at.

So enough of theory, let us start writing our Python program to add the text to our image. Fire up your favorite text editor to start writing your code. In my case I use VS Code editor, but you can use any text editor you want.

Want to learn more about VS Code editor? Check this article.

Python Program To Add Text To An Image Using Matplotlib

The first line of code we would like to add in our program is:

import matplotlib.pyplot as plt

What this line does is that it would import our matlotlib library’s pyplot module into our program. So this is imported as plt so every time we want to make use of it, we would call it by using the plt name.

Next thing we would like to import is the Python Imaging Library (aka PIL)’s Image module. We need to use this module to import the image file (butterfly.jpg) into our program. So this is achieved by using the following piece of code:

from PIL import Image

We will then go ahead with importing the butterfly.jpg file into our program using the line:

img = Image.open("/home/amar/Pictures/butterfly.jpg")

The Image.open() method will open our file, read its content and store it into the variable img. Notice here again that we have imported the buttefly.jpg file into a new Python variable called img. Because of this from now on wards, we will be able to access the content of this image file by using the img Python variable.

With the image being available, its time to start editing it using Matplotlib library.

The first step in using Matplotlib library is to create a Matplotlib’s figure object, which forms the canvas of our image. All the operations we do from now on will be with respect to this canvas element. We can create Matplotlib’s figure object by using the lines:

plt.figure(figsize=(12,8))

Wait a second! What does this line even mean? What does figsize stand for and what values are we passing here? Let us answer these questions first.

Understanding Figsize Parameter Of Matplotlib’s Figure Object

figsize is a parameter we need to pass to the matplotlib plt module’s figure() method. It allows us to specify the exact width and height of our image.

Therefore when we pass the values (12,8) to this parameter, we are specifying these exact values. These values are expressed in the unit of inches. So when we say (12,8), we are asking the Matplotlib library to draw a figure with a width of 12 inches and a height of 8 inches.

Following this, we will ask Matplotlib library to draw our picture onto the figure‘s canvas by calling imshow fuction as follows:

plt.imshow(img)

As a result of the above code, Matplotlib will draw to image onto our figure object. However, it must be noted that it only draws the image to our figure object. But nothing will be displayed to the end user. If we want to display this image, we need to call another Matplotlib function plt.show(). But we do not want to display it yet. Therefore, we will discuss about this function later in the article when we actually make use of it.

So for now we will further move on with our code. As of now we have created a Matplotlib’s figure canvas object, we have opened our image file and drawn this image onto our figure object’s canvas. It is now time to write our intended “What a wonderful world!” text on top of it.

Adding Text Using Plot’s Text Function

In order for us to write text over an image, Matlplotlib provides us with yet another method, not surprisingly called the text function. So we can access it as part of the plt module. Also, we can find this function’s signature and the parameters it takes in link to its official documentation. With this information, we can finally call the text function to print our string onto the figure object’s canvas like this:

plt.text(200, 700, "What A Wonderful World!", color="orange", fontdict={"fontsize":30,"fontweight":'bold',"ha":"left", "va":"baseline"})

From the above code, we can interpret the parameters of plt.text function as follows:

X & Y Co-Ordinates Parameter

The first two parameters of this function are the x & y co-ordinates of the image. We set its values to be 200 and 700 so that the string starts from there.

Annotation Text Parameter

Following the two co-ordinate parameters is the annotation text paramter. This is the parameter which takes in the text string that we would like to print. We have set it to our string “What A Wonderful World!” that we wanted to add to our image.

Text Color Parameter

Following the the text parameter is the color parameter. We use this parameter to set the color of the string. In our case, we have set this to orange color.

Font Dict Parameters

Finally, we set the parameters for the type of font to use and its properties. In our case, we are asking Matplotlib library to set the following properties to our text fonts:

  • We want to use a font size of 30
  • We want the font to be bold
  • Next, we want the text’s horizontal alignment to be to its left
  • Finally, we also want it to be vertically aligned to its baseline

So with these properties set, we have finally defined the text that we wanted on our canvas. Hence, it is now time to display this image on our screen.

Displaying Image On Screen Using Matplotlib

This is the final stage of our code. At this point, we want our program to simply display the content of our modified image on our screen. So we will achieve this by using the plt.show() function that we had discussed earlier.

Using Matplotlib’s Show Function To Display The Image On Screen

The code to display our image on screen is pretty simple

plt.show()

We just call the plt.show() function as shown above. This should finally display the image in a new window on our screen.

Summary Of Python Program To Add Text To Image

So to summarize, our final program in full looks like this:

import matplotlib.pyplot as plt
from PIL import Image

img = Image.open("/home/amar/Pictures/butterfly.jpg")
plt.figure(figsize=(12,8))

#Finishes drawing a picture of it. Does not display it.
plt.imshow(img)
plt.text(200, 700, "What A Wonderful World!", color="orange", fontdict={"fontsize":30, "fontweight":'bold', "ha":"left", "va":"baseline"})
#Display the image
plt.show()

How To Save Matplotlib Image To A File

So far we edited the image file and added our own text on top of it. We were also able to display the final result to the end user screen in a separate window.

But what if we wanted to save this image? Can we write the program to save this edited image automatically to a new file? How to achieve this? If these are some of the questions you have running your mind then fret not. Because Matplotlib library comes to our rescue, once again!

To save the image onto a new file, Matplotlib provides us with yet another function called savefig().

Using Matplotlib’s Savefig Function To Save An Image

Matplotlib provides us with the function savefig() to save our image in a new file. It takes in many parameters which can be seen in this official documentation link. However, for our case, we simply want to save the image to a simple jpg file. So we need not have to use all the parameters defined in that link. We can simply use the following code to save it as a JPG file:

plt.savefig('/home/amar/Pictures/butterfly_captioned.jpg')

From the above code, we can see that we used plt.savefig() in its simplest form. We just added a path to the new file and saved it as a jpg file. Yes, the format of the file in which we want it to be saved is given by simply using the appropriate file extension. Matplotlib is intelligent enough to understand this and save it accordingly. Hence, if you wanted to save it as a PNG file, you just need to change the extension to .png. Its that really simple!

So with this, we can finally get our captioned image file as shown below:

Final Image file created by the Python code containing added text over the image
Final Image file created by the Python code containing added text over the image

So this was a brief tutorial on how to use Python to add text over an image programatically using Matplotlib library. Hope this was easy for you to follow up and understand. I will continue to write more articles on Python and image/video editing libraries in the future including Matplotlib. If you have any more questions or queries regarding this, do let me know in the comment section below. Until next time, ciao!

Categories
DATA SCIENCE EMBEDDED PROGRAMMING PROGRAMMING PYTHON TUTORIALS WEB DEVELOPMENT

What Are Python Reserved Keywords?

A very big sum python function
Writing a very big sum function in Python programming language

Python reserved keywords are those words in Python programming language that have a specific meaning in a program. These keywords have specific actionable functionalities defined to it in Python. We are not allowed to re-use these reserved keywords. It is also not possible to override the keywords in Python.

Why Do We Have Python Reserved Keywords?

A programming language is defined by a set of keywords that have specific functionalities attach to it. Python programming language is no different from this. There are a set of keywords defined in Python language that performs specific tasks within the program where they are used.

Python Logo For Reserved Keywords In Python

For example, print is a keyword in Python which instructs the Python interpreter (i.e. the Python environment where Python programs run) to print a string to the output terminal. So a Python program line like:

print('Hello, World!')

will print the string:

Hello, World!

to the computer output screen that its user can see. We as a programmer are never allowed to use same keyword “print” for any other purposes like variable name or function name. Thus, we say that it is a Python reserved keyword.

Similarly, the keyword input is used to receive input from the user of a Python program. So a line in the program like:

user_name = input('Enter your name')

will display the string:

Enter your name

on the user screen and wait until the user enters his name. Once he enters the name and hits the “Enter” key, the name gets stored in the variable “user_name“.

So as you can see here, each of these reserved keywords such as print, input etc. each have a very specific functionality attached to it in Python language. We cannot use these same keywords as a variable name or function names. Trying to do so will result in the interpreter throwing error at us!

So now that we understand about reserved keywords in Python, what can we do about them?

For one, we need to know about all the Python reserved keywords to avoid using them in other ways in our program. But in addition to this, knowing about these reserved keywords and their intended functionalities will also help us write useful programs.

Using Reserved Keywords In Python Programs

Python programs are nothing but a bunch of reserved keywords used upon a set of variables to perform certain operations. So we use these set of keywords to write our programs. For example, if we take a look at the below program:

user_name = input('Enter your name')
print('Hello, ' + user_name + '!')

This program simply prompts for an user to enter his name. When he does so, it will just wish Hello to him by addressing his name. So when I run this program, the output I get is something akin to this:

'Enter your name'
> Amar
> Hello, Amar!

Conclusion

So in short, we can say that reserved keywords are a set of words in Python that have pre-defined meaning and functionalities associated with them. We make use of these keywords to write our program and we are not allowed to re-use the same words in our variables or function names. In other words, we are not allowed to alter their pre-defined meaning.

Categories
DATA MINING DATA SCIENCE HTML JAVASCRIPT PROGRAMMING PYTHON STATIC WEBSITES TUTORIALS WEB DEVELOPMENT WEB SCRAPING

How To Extract Data From A Website Using Python

Web scraper Illustrative picture

In this article, we are going to learn how to extract data from a website using Python. The term used for extracting data from a website is called “Web scraping” or “Data scraping”. We can write programs using languages such as Python to perform web scraping automatically.

In order to understand how to write a web scraper using Python, we first need to understand the basic structure of a website. We have already written an article about it here on our website. Take a quick look at it once before proceeding here to get a sense of it.

The way to scrape a webpage is to find specific HTML elements and extract its contents. So, to write a website scraper, you need to have good understanding of HTML elements and its syntax.

Assuming you have good understanding on these per-requisites, we will now proceed to learn how to extract data from website using Python.

Python logo on extracting data from a web page using Python
Python Web Scraper Development

How To Fetch A Web Page Using Python

The first step in writing a web scraper using Python is to fetch the web page from web server to our local computer. One can achieve this by making use of a readily available Python package called urllib.

We can install the Python package urllib using Python package manager pip. We just need to issue the following command to install urllib on our computer:

pip install urllib

Once we have urllib Python package installed, we can start using it to fetch the web page to scrape its data.

For the sake of this tutorial, we are going to extract data from a web page from Wikipedia on comet found here:

https://en.wikipedia.org/wiki/Comet

This wikipedia article contains a variety of HTML elements such as texts, images, tables, headings etc. We can extract each of these elements separately using Python.

How To Fetch A Web Page Using Urllib Python package.

Let us now fetch this web page using Python library urllib by issuing the following command:

import urllib.request
content = urllib.request.urlopen('https://en.wikipedia.org/wiki/Comet')

read_content = content.read()

The first line:

import urllib.request

will import the urllib package’s request function into our Python program. We will make use of this request function send an HTML GET request to Wikipedia server to render us the webpage. The URL of this web page is passed as the parameter to this request. 

content = urllib.request.urlopen('https://en.wikipedia.org/wiki/Comet')

As a result of this, the wikipedia server will respond back with the HTML content of this web page. It is this content that is stored in the Python program’s “content” variable.

The content variable will hold all the HTML content sent back by the Wikipedia server. This also includes certain HTML meta tags that are used as directives to web browser such as <meta> tags. However, as a web scraper we are mostly interested only in human readable content and not so much on meta content. Hence, we need extract only non meta HTML content from the “content” variable. We achieve this in the next line of the program by calling the read() function of urllib package.

read_content = content.read()

The above line of Python code will give us only those HTML elements which contain human readable contents.

At this point in our program we have extracted all the relevant HTML elements that we would be interested in. It is now time to extract individual data elements of the web page.

How To Extract Data From Individual HTML Elements Of The Web Page

In order to extract individual HTML elements from our read_content variable, we need to make use of another Python library called Beautifulsoup. Beautifulsoup is a Python package that can understand HTML syntax and elements. Using this library, we will be able to extract out the exact HTML element we are interested in.

We can install Python Beautifulsoup package into our local development system by issuing the command:

pip install bs4

Once Beautifulsoup Python package is installed, we can start using it to extract HTML elements from our web content. Hope you remember that we had earlier stored our web content in the Python variable “read_content“. We are now going to pass this variable along with the flag ‘html.parser’ to Beautifulsoup to extract html elements as shown below:

from bs4 import BeautifulSoup
soup = BeautifulSoup(read_content,'html.parser')

From this point on wards, our “soup” Python variable holds all the HTML elements of the webpage. So we can start accessing each of these HTML elements by using the find and find_all built-in functions.

How To Extract All The Paragraphs Of A Web Page

For example, if we want to extract the first paragraph of the wikipedia comet article, we can do so using the code:

pAll = soup.find_all('p')

Above code will extract all the paragraphs present in the article and assign it to the variable pAll. Now pAll contains a list of all paragraphs, so each individual paragraphs can be accessed through indexing. So in order to access the first paragraph, we issue the command:

pAll[0].text

The output we obtain is:

\n

So the first paragraph only contained a new line. What if we try the next index?

pAll[1].text
'\n'

We again get a newline! Now what about the third index?

pAll[2].text
"A comet is an icy, small Solar System body that..."

And now we get the text of the first paragraph of the article! If we continue further with indexing, we can see that we continue to get access to every other HTML <p> element of the article. In a similar way, we can extract other HTML elements too as shown in the next section.

How To Extract All The H2 Elements Of A Web Page

Extracting H2 elements of a web page can also be achieved in a similar way as how we did for the paragraphs earlier. By simply issuing the following command:

h2All = soup.find_all('h2')

we can filter and store all H2 elements into our h2All variable.

So with this we can now access each of the h2 element by indexing the h2All variable:

>>> h2All[0].text
'Contents'
>>> h2All[2].text
'Physical characteristics[edit]'

Conclusion

So there you have it. This is how we extract data from website using Python. By making use of the two important libraries – urllib and Beautifulsoup.

We first pull the web page content from the web server using urllib and then we use Beautifulsoup over the content. Beautifulsoup will then provides us with many useful functions (find_all, text etc) to extract individual HTML elements of the web page. By making use of these functions, we can address individual elements of the web page.

So far we have seen how we could extract paragraphs and h2 elements from our web page. But we do not stop there. We can extract any type of HTML elements using similar approach – be it images, links, tables etc. If you want to verify this, checkout this other article where we have taken similar approach to extract table elements from another wikipedia article.

How to scrape HTML tables using Python

Categories
100DaysOfCode EMBEDDED PROGRAMMING PROGRAMMING PYTHON RASPBERRY PI TUTORIALS

Python Program To Add Two Numbers

In this article, we will look at how to write a Python program to add two numbers. This is a very simple Python program that is suitable for even a beginner in Python programming to work on it for getting hands on practice with Python.

In order to add two numbers in Python program, we need to first break down the problem into the following steps:

Breakdown of the problem of Adding two numbers

  1. Receive the first input number from the user and store it in a Python variable.
  2. Receive the second input number from the user and store it in another Python variable.
  3. Add the two numbers by adding the two Python variables using a Python statement (Learn about Python statement here)
  4. Store the final result in another Python variable called “result”
  5. Print the value of the “result” variable.

In the above breakdown of the problem, you will notice that the Python program we will be written in such a way that the two numbers that need to be added will not be hard coded directly into the program itself but instead is written in a generic way such that we prompt the user to enter these two input values every time the Python program to add two numbers is run. This type of programming approach is often called general programming as the program is generic enough to receive any two different values each time it is run.

Pseudo-code To Add Two Numbers Using Python Programming Language

num1 = Receive First Input Number From The User
num1 = Receive Second Input Number From The User
result = (num1) Added to (num2)
Print the (result) on the screen

We can see from the above psuedo code that this is a simple program that receives two numbers from the user, adds them and print their results back on the screen. The above pseudo code gives us a nice little framework on how to write our program. The same pseudo-code can now be used to write a program to add two numbers using any programming language and not just Python!

Now that we have our problem broken down and pseudo code written, it is time for us to replace the pseudo code with actual Python programming code instructions.

Python Program To Add Two Numbers And Print Its Result

Fire up your Python IDLE interpreter and start typing in the code along with me for you to be able to understand this program better:

Python 3.5.2 (default, Oct  8 2019, 13:06:37) 
 [GCC 5.4.0 20160609] on linux
 Type "copyright", "credits" or "license()" for more information.
>>>

Once in our python interpreter, let us start typing in our Python program commands. The first thing we need to do according to the pseudo code written above is the receive the first input number from our user. In Python program, the instruction code to be used to receive a value from its user is by calling the input() function. Input function will accept a string as its parameter that will be displayed to the program user when the program is run. So, with this knowledge, our first line of the Python program to add two numbers will be:

>>> num1 = input('Enter the first number\n')
Enter the first number
10
>>>

As you can see from the above code block, we have used the input() function to prompt our program user with the string “Enter the first number“. We are also saving the value entered by the user to a Python variable called num1.

The input function will then prompt with the above string and wait until the user enters a number. In the above code snippet, I had entered a value of 10 which is now stored in the variable num1.

In a similar way, we will write the next line of code which will prompt the user to enter a second input number that is to be added to the first number. This is achieved with the following piece of code:

>>> num2 = input('Enter the second number\n')
Enter the second number
20
>>>

Again over here, I have entered my second number as 20 when prompted.

Now that we have the two numbers in our Python variables num1 and num2, it is time add them to get the final result that we are going to store in our third Python variable called result. This is achieved using the following Python statement:

>>> result = int(num1) + int(num2)
>>>

So from the above python code, we have now added the numbers num1 and num2 using the Python arithmetic operator “+” and the typecast operator called “int()“. We had to use the typecast operator int because by default all inputs from the user into a python program will be interpreted and stored as a string. We can check this by issuing the following command in the interpreter:

>>> print(num1)
10
>>> type(num1)
<class 'str'>
>>>

So, by calling the typecast operator int, we are converting this string value to an integer value. 

>>> type(int(num1))
<class 'int'>
>>>

finally stored the end result in a new variable called result. Since we have not issued a Python command to print the value of result, nothing gets printed on the IDLE prompt yet. So, the final step is exactly that. To print the value of the result variable onto the screen. This is achieved using another Python function called the print function.

>>> print (result)
30
>>>

Here is the full program that we can store in a file called add2num.py and run it using the command python3 add2num.py everytime we want to add any two numbers!

num1 = input('Enter the first number\n')
num2 = input('Enter the second number\n')
result = int(num1) + int(num2)
print (result)

This concludes our Python program to add two numbers.

Additional Side Notes On Python Programming Language:

Python is an interpreted programming language that gets interpreted and executed on the fly and hence the program written in Python do not need to be compiled like in the case of a C or Java programming language.

Categories
ARM ARM ARCHITECTURE ARM PROGRAMMING COMPUTER HARDWARE ELECTRONICS EMBEDDED EMBEDDED COMPUTERS EMBEDDED PROGRAMMING HARDWARE PROGRAMMING PYTHON RASPBERRY PI RASPBERRY PI PROJECTS TUTORIALS

Blinking An LED Connected To GPIO Pin Of Raspberry Pi Using Python

Introduction

If you are just getting started with Raspberry Pi, connecting a simple LED to one of the GPIO pins of a Raspberry Pi and controlling it using software program that you write will give you a very good grasp of how a computer hardware and its program works internally. You will realize how you can control various aspects of a computer hardware using software, how a computer works at the bit level, how to write Python programs to control hardware and more.

In summary, working on getting an led connected to a GPIO pin of your Raspberry Pi will help you in understanding the fundamentals of a computer architecture and computer science in general.

Raspberry Pi 3B

What You Will Learn From This Project?

Connecting an LED to the GPIO pins of a Raspberry Pi to control it is a simple Beginner Raspberry Pi Project that lets you learn more about:

  • Raspberry Pi hardware internals
  • General Purpose Input/Output (GPIO) pins of a Raspberry Pi
  • Raspberry Pi Register Set
  • Ohm’s Law
  • Python Programming
  • Python Library – Raspberry Pi GPIO library
  • The working of an Light Emitting Diode (LED)

What Hardware Is Required To Set Up A Blinking LED Project?

This a very simple, beginner friendly Raspberry Pi project that can be set up by anyone with minimal hardware or software knowledge. The hardware components required to set up this blinking LED project is also quite minimal. You need the following hardware components available with you to get it going:

  • Raspberry Pi Module
  • Solderless Breadboard
  • Keyboard
  • Monitor
  • Raspberry Pi Power Supply
  • SD Card with working Raspbian OS
  • Jumper wires for rigging up the circuit
  • LED
  • Resistor (1K Ohm)
  • Multimeter

Theory Behind How The Raspberry Pi Blinking LED Project Work

When you look at the Raspberry Pi board, you will see a bunch of pins protruding out. Among these, there is a row of 40 pins located on one side of the board as shown in the image below.

If you look closely enough in the above image, you will notice the label “GPIO” written right under it. These pins are called the GPIO pins or General Purpose Input Output pins. What the name GPIO implies is that these pins do not have any fixed functionality within the board and hence can be used for general purposes. It means that we can connect our LED into one of these pins and can turn it ON or OFF using these pins. But how?

How to control the Raspberry Pi GPIO pins programmatically?

Raspberry Pi 3 board runs on Broadcom’s ARM CPU chipset BCM2837. Among many other things, this processor chipset has a built in GPIO controller aka General Purpose Input Output controller. The 40 GPIO pins header shown in figure 1 is connected to 40 controllable pins of the GPIO controller. Now, we can control each of these pins individually by programming the appropriate registers inside this GPIO controller.

To understand how to program each of these pins using GPIO controller, we need to look into the Technical Reference Manual or datasheet of the Broadcom ARM chipset BCM2837.

In the BCM2837 SOC (System On Chip aka CPU) datasheet linked above, if we jump into page 89 we come across a dedicated chapter talking about General Purpose Input Output (GPIO). If we go through this chapter, we can learn about all the GPIO registers available and figure out the GPIO registers we need to program to turn ON or OFF the LED we are going to connect to the Raspberry Pi 3 GPIO pins.

As the name implies, GPIO pins can be configured as either an Input pin or an Output pin. When we configure a GPIO pin as an input pin, we are sending data bit (either 0 or 1) into the Raspberry Pi BCM2837 SOC i.e. data signal is sent from outside the board to inside the board (hence the name input). On the other hand, if we configure the Raspberry Pi GPIO pin as an output pin, the board will send the data bit signal (either 0 or 1) from inside the board to the outer world where any device connected to it will receive this signal.

So, if we want to control an LED that is connected to one of the Raspberry Pi’s GPIO pin, we need to configure that pin as a GPIO OUT pin (aka output pin) so that we can send an electrical signal from the Raspberry Pi board to the external LED connected to this pin.

The configuration of a GPIO pin to be an INPUT or OUTPUT pin is controlled by programming the GPIO Controller Register called GPIO Function Select Register (GPFSELn) where n is the pin number.

So for example, if we choose to use the GPIO8 pin to control the LED, i.e. we connect our LED to GPIO 8 pin, we need to program the GPFSEL register for the GPIO 8 pin and configure it as an Output pin. When we check the datasheet at page 91 and 92, we notice that GPIO pin is configured by setting the bits 26 to 24 in the GPFSEL register (that is field name FSEL8). And from the datasheet, we also find that to set the pin as an output pin, we need to set its value as 001 i.e. bit 26 is set to 0, bit 25 is set to 0 and bit 24 is set to 1.

So, if we can somehow set these values in the GPFSEL register using a programming language such as Python, we will be able to start controlling the LED connect to this pin!

If this is all overwhelming to you, do not worry. We will not have to scratch our head a lot for now as we can simply make use of Raspberry Pi’s GPIO Python library that helps us in making most of this work for us. But I just wanted to explain to you as to what this GPIO Python library is doing under the hood.

How To Connect An LED To Raspberry Pi GPIO?

Designing The Circuit

In order to connect an LED to GPIO pin 8 of Raspberry Pi, we need to first design and understand how the circuit is going to work.

Can we connect an LED directly to a Raspberry Pi GPIO pin without a resistor?

The answer is No. Raspberry Pi provides 3.3 Volts of power on its GPIO output pin according to Raspberry Pi datasheet specification. However, if we take a look at a standard LED, we notice that it normally operates at a much lower voltage. If we look at an LED specification, we notice that a typical LED usually operates at just 1.7 Volts and draws 20 mA. So, if we need to connect this LED to the GPIO pin of our Raspberry Pi, we need to bring down the voltage delivered by the pin to our LED to operate at or under 1.7V. How to do that? We connect a resistor in series with our LED so that the 3.3 Volts GPIO output of Raspberry Pi gets split between the resistor and our LED. By choosing a right value of the resistor such that it consumes 1.6 Voltage, we can ensure that LED finally gets only 1.7 Volts.

Calculating the resistor value to connect with LED and Raspberry Pi GPIO

In order to calculate the value of resistor that we should be using, we make use of the Ohm’s Law.

Ohm’s Law is defined using the equation:

V = I/R where V is the voltage, I is the current and R is the resistor value.

So, if we want to have V=1.6 Volts consumed by our resistor so that the current coming from GPIO pin is at I=20 mA, we need to connect a resistor whose value is:

1.6 = (20 mA)/R

or R = 80 Ohms (or approx 100 Ohms)

So, we choose a resistor of value 100 Ohms connected in series with our resistor to ensure that we only get 1.7 Volts and 20 mA of current, the optimum operating values as required by our LED.

A 100 Ohm resistor is identified by the color bands: Brown, Black & Brown.

Hook up the led through 100 Ohm resistor to GPIO 8 pin of Raspberry Pi as shown in the figure below:

Note that when you are hooking up the LED, the terminal pin that is longer is positive. Once you have connected as shown in the figure, it is now time to program the Raspberry Pi GPIO controller to start controlling the LED to turn ON or OFF.

We will be using Python to program our Raspberry Pi GPIO controller. Now, the simplest way to program this is by making use of the Python GPIO library.

To install the Python Raspberry Pi GPIO module, open up your linux terminal and type the following command.

sudo apt-get install python-rpi.gpio python3-rpi.gpio

Now the above command will install the required Python GPIO library module onto our Linux development machine. Once successfully installed, It is now time to start programming the Raspberry Pi GPIO controller.

We will be toggling our GPIO pins at 1 second intervals such that our LED will turn ON and OFF forever until the Python program we write will be terminated i.e., we will be running the code to perform infinite loop of toggling the GPIO 8 pin ON and OFF.

Create a new file on your computer by typing the following command in the terminal:

touch blinky.py

This should create our new program file called blinky.py

Open up this file using nano editor by typing the following command in the terminal:

nano blinky.py

Now that the file is opened, it is time to start writing our program to control the GPIO Pin 8 using Python GPIO library module.

First thing first, we will import the Python GPIO library module using command:

import RPi.GPIO as GPIO

Next, we will import python time library to perform 1 sec sleep operation between each GPIO toggle

from time import sleep

Next, we need to configure our GPIO library to use our GPIO physical pin numbering as seen on the Raspberry Pi board physically:

GPIO.setmode(GPIO.BOARD)

This ensures that when we say GPIO pin 8 in the program, it actually maps to the GPIO Pin 8 seen on the Raspberry Pi board.

Next, configure GPIO pin 8 to be a GPIO Out pin and set its initial output value to be low:

GPIO.setup(8, GPIO.OUT, initial=GPIO.LOW)

Finally we will start an infinite loop in Python such that we turn ON the GPIO 8 (by setting it HIGH) or turn it OFF (by setting it LOW) after every 1 second delay. This is achieved using the program below:

while True: # Infinite loop
    GPIO.output(8, GPIO.HIGH) # Turn GPIO 8 pin on
    sleep(1)                  # Delay for 1 second
    GPIO.output(8, GPIO.LOW)  # Turn GPIO 8 pin off
    sleep(1)                  # Delay for 1 second

That’s it, this should be all the program that we need to type in our blinky.py file and run it using the command:

python3 blinky.py

This should start turning your LED ON and OFF every second!

Here is the full code for your reference:

import RPi.GPIO as GPIO
from time import sleep

GPIO.setmode(GPIO.BOARD)
GPIO.setup(8, GPIO.OUT, initial=GPIO.LOW)

while True: # Infinite loop
    GPIO.output(8, GPIO.HIGH) # Turn GPIO 8 pin on
    sleep(1)                  # Delay for 1 second
    GPIO.output(8, GPIO.LOW)  # Turn GPIO 8 pin off
    sleep(1)                  # Delay for 1 second

This should conclude our tutorial on how to get a simple LED connected to a General Purpose Input/Output (GPIO) pin turning ON and OFF using a Python program that makes use of Python Raspberry Pi GPIO library. There can be many variants to this such as using other GPIO pins, connecting more than one LEDs to multiple GPIO pins and controlling them all in different ways to display interesting patterns on the LEDs. If we are even more curious, we can also figure out a way to control the BUILT-IN LEDs that are already present on our Raspberry Pi boards to bypass their current usage and be used for by own programs for our purposes.

We will dwell into these and many other interesting ways to make use of our Raspberry Pis to understand and learn more about the computer hardware, its architecture and much more in our future articles.

Categories
100DaysOfCode PROGRAMMING TUTORIALS

How To Write Code Like A Professional?

Everyday, we programmers write hundreds of lines of programming code. But, have you ever stopped to notice if your program is good enough? How do you make sure your programming skill set is up to the mark? How to make sure you are Programming Like A Pro? In this article, we consider these questions and try to answer them each in greater details.

How To Code Like A Pro?

There are many characteristics of a program written by an experienced professional that sets his code apart from a code newbie. A professional programmer would make use of his past learning experience and implement his code that follows a certain set of guidelines and best practices he has learnt over the time since he began programming.

While each of these set of guidelines and best practices may differ from one Professional Programmer to another Pro, there are many common guidelines which more or less all Professional Programmers follow that can make their code set apart from the rest. Here is a list of some of such common guidelines used by almost all Professional Programmers. If you can practice and incorporate these practices and techniques into your daily programming routine, even your program code will start to look more professional.

Best Practices & Guidelines To Write Code Like A Professional

  • Write code that makes use of the processor efficiently – In other words, write code that runs fast on the processor using lower number of clock cycles. How to achieve this? While there are many things involved in learning to write a highly CPU efficient programs which will be explained in the future articles, some of the most common practices is to use loops sparingly, use efficient looping systems, write programs using instructions that require less clock cycles to accomplish the task at hand etc. You need to have good understanding of the underlying computer system architecture to get a better hold or control on these aspects, so learn more about the computer architecture.
  • Write code that makes use of memory efficiently – Use the memory of a computer system efficiently and sparingly. It usually takes certain amount of computer clock cycles to read and write data to or from the memory. So if you want your code to speed up, it is better to write code using instructions that uses lesser amount of memory. If your code allocates too many variables, it can start gobbling up a lot of RAM space in your user’s computer, there by slowing down the computer all together, thus giving a had user experience.
  • Write code that is more readable – This is one of the most important aspect of a professional code. You can write code in a convoluted way to do something extremely smart on its data, but if in the future you move on in your job role and your code is handed over to your junior developers, they will find it extremely hard to read and understand your code. So always write code that is easily readable by anyone familiar with programming. This can also include adding more comments in your source code, naming the variables appropriately etc. Sometimes, writing readable code could result in writing a slightly inefficient piece of code thus contradicting the above 2 points discussed earlier. But however, in such a situation, it is still advisable to give priority towards readable code than saving a few CPU clock cycles.
  • Write code that is highly modular – What does this mean? It means you need to break down your programming into individual modules such as functions and classes, that does one thing and only one thing but extremely well. Your code need to be split up into several functions, files and libraries that can be made use of in the future for other projects as well.
  • Write code that is well tested and documented – Each of your code modules need to be very well tested to ensure it fulfills and works under all boundary conditions. It should not break down for some unexpected values but gracefully reject the input under such situations. This is achieved by writing robust test cases that can test for all frequently possible conditions even if it occur only one in a million time (boundary conditions). Make sure that the code is well documented using comments and appropriate variable or function names as discussed earlier.
  • Write code that is easy to enhance – Before you sit down to write your code, you need to design to program to be highly modular and easy to enhance of its functionalities in the future. This would mean writing code that is modular, with will defined APIs and interfaces such that a function serving that API can be swapped for another function with a different implementation for the same set of input and output values.

There are still many more aspects of a professional’s code that usually depends on his style and taste of coding, but the above list should give you a good fundamental beginning on how to think and write code like a professional programmer.

In the future articles, we will continue to explore some of the fundamental aspects of computer science such as a computer’s architecture, its basic functionality etc that will help us getting a solid understanding the basics of computer science. Do let me know if you have any comments on this or any other topic you would want me to explore and I will be happy to accommodate your requests.

So until then, happy coding! 🙂