Category: JAVASCRIPT

Categories
DATA SCIENCE JAVASCRIPT PYTHON TUTORIALS WEB DEVELOPMENT

Call Node.js Script From Python – How To Guide

So you have come across a situation where in you want to call a Node.js script from your Python code. But how do we go about doing that? Is there any package that we need to install in Python or Node.js that will help us with this? Or can we do it directly using basic Python and Node.js programming? What is the standard approach to solve this?

Call Node.js script from Python
Call Node.js script from Python

We will answer all the above questions in this article today. So, are you are ready to learn about it now? Cool, let us get started right away!

Setup Required To Call Node.js Script From Python

So to begin with, we will first take a look at what all needs to be installed in our computer to get this going. Alright?

Now as you know, to run any Javascript file on your computer, we need to have Node.js installed. This is the basic thing, right? So, make sure that you have installed Node.js in your computer where you want to call the Node.js script.

There are several tutorials available on the internet that can guide you in how to install Node.js on your computer. But let me give you a gist of the steps usually involved for a Linux computer:

Step 1 – Install Node Package Manager (NPM)

You can do this by running the command:

sudo apt install npm

Step 2 – Check which version of Node.js is installed

You can do this by running the command:

nodejs -v

How To Call Node.js Javascript File From Python

Now that you have your Node.js installed, we can run any Javascript file by issuing the command:

nodejs yourFile.js

That is it! It is as simple as that!

So with Node.js installation out of the way, we can now see how to call a Node.js script from a Python file.

A simple Javascript Hello World File To Run Using Node.js

To begin with, let us create a simple Node.js file that simply prints “Hello, World!” on our screen. So here is how the file looks like:

console.log("Hello, World!");

Save this as hello.js in your computer directory. To test that this file works, simply call the command:

nodejs hello.js

If the above command prints the “Hello, World!” string on your console screen, it means the script is good to run as Node.js script. Alright? So that is our basic Node.js script.

How To Call Node.js Script From Python?

Now, we want to call this hello.js script file from a Python file. But how do we go about doing that? Well, here is the process we are going to follow:

What happens under the hood

  1. We know that our application program – be it Python code or Node.js code runs on top of our Operating System (OS). Right?
  2. But how does an OS run both these programs simultaneously? Well the answer is that they both run as two separate processes in an OS.
  3. But why two separate processes, you ask? It is because it is the job of an OS to make sure both can run paralelly. And it is also the OS’s job to keep these programs separate. Meaning Python program will not be able to access, read or modify the memory of Node.js program and vice versa.
  4. So that is why OS runs them in a sandbox application process where one process is not aware or have access to other process’s resources.
  5. But then how can we now call Node.js script from Python script then? Well for that, we need to take help of the OS itself.
  6. We need to let the OS know that we intend to call Node.js script from our Python script.
  7. So to do that, we will make use of a special Python library module called “subprocess”.
  8. But how do we install this module? Well that’s the best thing about this. It is not an external module that needs to be installed separately. It already comes as part of the Python’s built-in packages.

So with this understanding, we will now write a simple Python script that calls the hello.js script written above using subprocess. Sounds good? Great! Let us check it out then!

A simple script to call Node.js script From Python

So create a file called hello_js.py and add the following code to it:

from subprocess import check_output
p = check_output(['node', '~/hello.js'])
print (p)

And that is it! With just these 3 lines of code shown above, we can call our Node.js script and print out the “Hello, World” string coming from our hello.js Javascript file! Quite simple right? But what is our Python code doing here?

Let me explain you briefly on the above Python code:

Line 1: Here we are importing a function from subprocess called check_output. We will use this function to get the console output coming in from our hello.js file.

Line 2: Here we are calling the imported check_output() and passing in a parameter list to it. This list consists of the console command line arguments we wish to execute. Now if you recall, to call our hello.js file using node, what command we used? So if you noticed, we are passing the same set of commands in this parameter list! Now it all makes sense right?

Line 3: Finally, we print the output we got from our check_output() function.

Conclusion

So this is how we can call a Node.js script from our Python code. We make use of the Python’s subprocess module and call Node.js command to run the javascript file through it. It will all become clear if you practice it further with few more variations around it. But this is the gist of how to run Node.js from Python.

So with this, I will end this article now. But if you have any questions around it, do let me know in the comment section below. I will be more than happy to help!

So until next time, take care!

Categories
JAVASCRIPT PROGRAMMING TUTORIALS WEB DEVELOPMENT

How To Enable Or Disable Console.log For Debugging

How to Enable or Disable console.log

So you are working on your Javascript project and you want to know how to enable or disable console.log messages? Then read on to learn how you can do it quite easily.

Whenever you are working with Javascript, be it on the front-end side with vanilla Javascript or frameworks like React, Vuejs etc. Or on the backend with Node JS, Express JS etc for your servers, you want a way to debug the code. Right?

One of the easiest way we try to debug our code is by using simple print statements. So on the Javascript end, this usually boils down to using console.log( ) messages.

But here is the thing. When you use a programming language that will be compiled – such as C, C++ or Java, we can use compiler flags to enable or disable debug messages.

However, for an interpreted language like Javascript, we dont have a built in method for how to Enable or Disable console.log( ) messages.

So what do we do in that case? Well, if we don’t have a built-in method to enable or disable Javascript console.log( ), then we need to create one!

And that is what we are going to do here. We will be writing a simple Javascript module that we can use to enable or disable Javascript’s console.log( ) messages.

So how do we design this piece of code?

Here is what we are going to do. Instead of using console.log( ) to print our debug messages, we will use our own custom function called debugLog( ). But within this debugLog( ) function, we will have a simple Debug Enable (or) Disable flag.

Now when this flag is set, we will call the console.log( ) in turn and if not set, we will skip printing the log messages. It is as simple as that!

Sounds good? Great! So here is the piece of code that does just that!

How to Enable or Disable console.log
How to Enable or Disable console.log using a switch?

Create a file called “utils.js” and add the following piece of code into it:

DEBUG = true;

function debugLog(msg) {
    if (DEBUG) {
        console.log(msg);
    }
}

//For express.js, we need to export this module
module.exports = debugLog;

So that is it! From now on, call debugLog( ) instead of console.log( ) to write your debug message. An example piece of such a debug message is shown below:

debugLog("[app] Connect to mongoDB...");

So whenever you want to switch on the debug logging, set the flag as below: 

DEBUG = true;

But the moment you want to turn these debug messages OFF, just set it to false:

DEBUG = false;

And that is all there is to it! This is how you can enable or disable console.log( ) messages for debugging!

Categories
DATA MINING DATA SCIENCE HTML JAVASCRIPT PROGRAMMING PYTHON STATIC WEBSITES TUTORIALS WEB DEVELOPMENT WEB SCRAPING

How To Extract Data From A Website Using Python

Web scraper Illustrative picture

In this article, we are going to learn how to extract data from a website using Python. The term used for extracting data from a website is called “Web scraping” or “Data scraping”. We can write programs using languages such as Python to perform web scraping automatically.

In order to understand how to write a web scraper using Python, we first need to understand the basic structure of a website. We have already written an article about it here on our website. Take a quick look at it once before proceeding here to get a sense of it.

The way to scrape a webpage is to find specific HTML elements and extract its contents. So, to write a website scraper, you need to have good understanding of HTML elements and its syntax.

Assuming you have good understanding on these per-requisites, we will now proceed to learn how to extract data from website using Python.

Python logo on extracting data from a web page using Python
Python Web Scraper Development

How To Fetch A Web Page Using Python

The first step in writing a web scraper using Python is to fetch the web page from web server to our local computer. One can achieve this by making use of a readily available Python package called urllib.

We can install the Python package urllib using Python package manager pip. We just need to issue the following command to install urllib on our computer:

pip install urllib

Once we have urllib Python package installed, we can start using it to fetch the web page to scrape its data.

For the sake of this tutorial, we are going to extract data from a web page from Wikipedia on comet found here:

https://en.wikipedia.org/wiki/Comet

This wikipedia article contains a variety of HTML elements such as texts, images, tables, headings etc. We can extract each of these elements separately using Python.

How To Fetch A Web Page Using Urllib Python package.

Let us now fetch this web page using Python library urllib by issuing the following command:

import urllib.request
content = urllib.request.urlopen('https://en.wikipedia.org/wiki/Comet')

read_content = content.read()

The first line:

import urllib.request

will import the urllib package’s request function into our Python program. We will make use of this request function send an HTML GET request to Wikipedia server to render us the webpage. The URL of this web page is passed as the parameter to this request. 

content = urllib.request.urlopen('https://en.wikipedia.org/wiki/Comet')

As a result of this, the wikipedia server will respond back with the HTML content of this web page. It is this content that is stored in the Python program’s “content” variable.

The content variable will hold all the HTML content sent back by the Wikipedia server. This also includes certain HTML meta tags that are used as directives to web browser such as <meta> tags. However, as a web scraper we are mostly interested only in human readable content and not so much on meta content. Hence, we need extract only non meta HTML content from the “content” variable. We achieve this in the next line of the program by calling the read() function of urllib package.

read_content = content.read()

The above line of Python code will give us only those HTML elements which contain human readable contents.

At this point in our program we have extracted all the relevant HTML elements that we would be interested in. It is now time to extract individual data elements of the web page.

How To Extract Data From Individual HTML Elements Of The Web Page

In order to extract individual HTML elements from our read_content variable, we need to make use of another Python library called Beautifulsoup. Beautifulsoup is a Python package that can understand HTML syntax and elements. Using this library, we will be able to extract out the exact HTML element we are interested in.

We can install Python Beautifulsoup package into our local development system by issuing the command:

pip install bs4

Once Beautifulsoup Python package is installed, we can start using it to extract HTML elements from our web content. Hope you remember that we had earlier stored our web content in the Python variable “read_content“. We are now going to pass this variable along with the flag ‘html.parser’ to Beautifulsoup to extract html elements as shown below:

from bs4 import BeautifulSoup
soup = BeautifulSoup(read_content,'html.parser')

From this point on wards, our “soup” Python variable holds all the HTML elements of the webpage. So we can start accessing each of these HTML elements by using the find and find_all built-in functions.

How To Extract All The Paragraphs Of A Web Page

For example, if we want to extract the first paragraph of the wikipedia comet article, we can do so using the code:

pAll = soup.find_all('p')

Above code will extract all the paragraphs present in the article and assign it to the variable pAll. Now pAll contains a list of all paragraphs, so each individual paragraphs can be accessed through indexing. So in order to access the first paragraph, we issue the command:

pAll[0].text

The output we obtain is:

\n

So the first paragraph only contained a new line. What if we try the next index?

pAll[1].text
'\n'

We again get a newline! Now what about the third index?

pAll[2].text
"A comet is an icy, small Solar System body that..."

And now we get the text of the first paragraph of the article! If we continue further with indexing, we can see that we continue to get access to every other HTML <p> element of the article. In a similar way, we can extract other HTML elements too as shown in the next section.

How To Extract All The H2 Elements Of A Web Page

Extracting H2 elements of a web page can also be achieved in a similar way as how we did for the paragraphs earlier. By simply issuing the following command:

h2All = soup.find_all('h2')

we can filter and store all H2 elements into our h2All variable.

So with this we can now access each of the h2 element by indexing the h2All variable:

>>> h2All[0].text
'Contents'
>>> h2All[2].text
'Physical characteristics[edit]'

Conclusion

So there you have it. This is how we extract data from website using Python. By making use of the two important libraries – urllib and Beautifulsoup.

We first pull the web page content from the web server using urllib and then we use Beautifulsoup over the content. Beautifulsoup will then provides us with many useful functions (find_all, text etc) to extract individual HTML elements of the web page. By making use of these functions, we can address individual elements of the web page.

So far we have seen how we could extract paragraphs and h2 elements from our web page. But we do not stop there. We can extract any type of HTML elements using similar approach – be it images, links, tables etc. If you want to verify this, checkout this other article where we have taken similar approach to extract table elements from another wikipedia article.

How to scrape HTML tables using Python

Categories
HTML JAVASCRIPT LAMP PHP STATIC WEBSITES TUTORIALS WEB DEVELOPMENT WEB SERVER

Beginners Tutorial – What Is A Website?

A website example

We have all heard about different websites present on the internet such as Google, Youtube, Facebook, Twitter etc. But what exactly is a website? What is it made up of?

Imagine you just found a 100 year old book in your attic that contains a wealth of information about the world wars. It also has a collection of photographs that revealed some secrets that are lesser known to the public. You wanted to share this information with the world, but how?

One way is to go to the newspapers and get it published, but it can still not reach all the people across the world. What would be an easy and best possible way to share this information to people around the world? Publish it on the internet!

When you publish it on the internet, anyone from around the world having a computer or a smartphone with an access to the internet will be able to consume your content.

You can publish this information on social networking websites such as Facebook, Twitter or Youtube or create your own website to publish it.

A website example

So, a website, simply put, is a collection of information present in different formats such as texts, images, videos, graphs that is published on the internet to helps its users consume them.

But just like you, there are millions of people who are sharing information on the internet, so how do you make your contents accessible from other people’s information? This problem is very similar to having thousands of building in a city and needing to find a particular building. How do we do that? We will discuss about this in the future articles

Categories
HTML JAVASCRIPT LAMP PHP PYTHON TUTORIALS WEB DEVELOPMENT WEB SERVER

Basic Structure Of A Web App/Website

A website or a web app is usually made up of the following 3 web components:

These components are arranged as shown in the following diagram.

Structure Of A Web App
Location of the web app components on the internet

Web App Back-End component

All the core logic of a web app or website is usually implemented in the web app’s back-end component. This includes all the algorithms of the web app, code to perform any storing and retrieval of data from the database, url based route handling etc. All these code forms the back-end of a web app or website and runs on a specialized computer called the web server.

Backend component of the above image

The code for back-end programming can be written using several different programming languages such as PHP, Python, Java, Javascript, Ruby etc. Each of these programming languages comes with their own advantages and disadvantages like for example choosing a language like Python comes with the great benefit of having several readily available libraries useful for data crunching activities, however they can be relatively slow compared to other programming languages. So which backend programming language to choose largely depends on the type of functionalities required for the particular web app that is going to be developed.

Web App Front-End Component

All the visual and interactive elements of a web app or website comes under the Web app’s front-end component. Whenever the user of a web app or website visits the website, he will only see and interact with the web app’s front-end component. So as far as he is concerned, a web app for him is usually just the front-end component. It is what he sees and interacts with in his web browser.

Front end component of the above image

However, these front-end components will not usually store all the relevant data of a web page within itself but instead, will query for them by sending requests to the web app’s back-end component that was discussed earlier. These requests are usually sent using the HTTP protocol.

The programming languages used to write web app’s front end component includes HTML, CSS & Javascript. Among these, HTML is a markup language that a web browser will use to interpret what HTML components needs to be drawn on the browser screen to represent the website. CSS is a styling language used to customize the style (like change the color, size, background color etc.) of these HTML components. Finally Javascript is a programming language that can be used to add interactive functionalities to these HTML components.

With the help of these 3 programming languages, your front end should be highly interactive and user friendly for any non technical person to start using your web app.

Database component

A database is a specialized software used to store and retrieve data efficiently on a computer or a server. The database can sit in a web server along side the server software or can be present in its own seperate dedicated database server.

Database component

Databases are usually used in a web app (or website) to store all relevant data of that web app such as user data, session data, web app specific data etc. There are many different forms of databases available such as relational databases, NoSQL databases, Document Oriented databases, Graph databases etc. Each of these variants of databases have certain unique features that are useful in certain specific situations. The most common database type used in web apps are usually relational databases such as MySQL, PostgreSQL. However Document Oriented databases such as MongoDB are also used frequently.

So these 3 components forms the fundamental elements of a web application. Of course there can be more than these 3 components required as your web app continues to grow. You may need to add multiple servers, load balancers to manage higher traffic, caching mechanisms etc. We will discuss more on these modules further in future articles, but having the knowledge of above three fundamental components of a web app should give you the best start to learn and start working on the development of your first web application.

Do comment below if you liked the article or if you have any questions regarding the above topic and I will be happy to answer your questions. Until then, happy learning! 🙂

Categories
JAVASCRIPT STATIC WEBSITES TUTORIALS WEB BROWSER WEB DEVELOPMENT WEB SERVER

This one value in Javascript is not equal to itself!

We know that Javascript supports all kinds of values such as strings, numbers, constants etc. All these values are deterministic values in that their weights always remains the same. For example, an integer value of 25 is always equal to 25 in Javascript no matter what. Similarly a string value of “Hello” is also always equal to another string “Hello”.

In other words, these values in Javascript can always be compared with another value to determine if they are the same or different. To understand this better with an example, let us open up our browser console. In my case I am using Google Chrome browser console where we will create 3 variables with these values:

We can note from the above Javascript demo video that when the variable a and c are compared, since both their values are same holding a value of 2, they return true when compared with each other. On the other hand when variable a was compared with b, since their values were different, the comparison resulted in a return value of false.

This is true for all type of values present in Javascript – be it string, integers, floats, booleans anything you can think of.

However, there exists one special value in Javascript that is never equal to another variable having the same value. In other words its value is never equal to itself. This value is the NaN value!

NaN in Javascript stands for “Not a Number” and it is that one special value in Javascript which does not return true if it is compared with itself.

Why does NaN not equal to itself in Javascript?

Now you might be wondering why a NaN value does not equal to itself? The answer for this lies in the way Javascript language has been designed.

NaN or Not a Number is a special value in Javascript which is used to represent a nonsensical value – that is it is the value returned whenever a non sensical operation is performed. Now this is where it gets interesting. Why does a non sensical value not be the same all the time or at all the place? In other words, why is this happening here:

Why is it returning false?

The answer is that NaN as mentioned earlier is a value that is used to represent a non sensical values. So if the result of an operation performed is something that cannot be represented by ordinary or normal values, Javascript returns a value of NaN.

Now, if two operations results in non sensical values, they are not necessarily equal. Each of these operations can be returning two non sensical values of different weights. However, they both need to be represented by the value of NaN. Hence, the Javascript treats two NaNs as two different values and never equal to each other.

I learnt about this and many other similar anomalies in the book Eloquent Javascript. This is a very good book to learn and understand such interesting things about Javascript so I will definitely recommend this to anyone interesting in learning Javascript in depth.

If you are also aware of any other similar interesting things about Javascript do let me know in the comments below. Until then, happy coding! 🙂

Categories
BROWSER EXTENSION HTML JAVASCRIPT TUTORIALS WEB DEVELOPMENT

Write A Google Chrome Extension In 5 Minutes

What is a Google Chrome Extension and what does it do?

Google Chrome extension is a Google Chrome browser extension that acts like a plug-in which adds additional functionalities to your default Google Chrome web browser experience.

Google Chrome web browser comes with a default set of functionalities that are applicable to a wide range of use cases as utilized by all the websites out there. But if you have a requirement for additional browser functionalities that are not already supported by default in your browser, you can try to make use of the available Google Chrome browser extension APIs to create a browser extension that would fulfill your such requirement.

In other words, Google Chrome Extensions extends the functionality of the default Chrome browser by introducing new set of functionalities into it.

What programming language is used to write a Google Chrome Extension?

In order to write a Google Chrome browser extension, you just need to have basic web development knowledge. The programming languages used to develop Google Chrome browser extension are HTML, CSS and Javascript. Using just these 3 languages along with the provided Google Chrome Browser Extension Application Programming Interfaces (APIs), you will be able to develop your Google Chrome browser extension in no time!

With this background information in hand, let us now figure out how to write a new Chrome browser extension.

Getting started with developing Google Chrome extension

All Google Chrome browser extensions must have one required file, the manifest.json file. It is using this manifest.json file that the Google Chrome web browser can identify and display all the available information about your Google Chrome extension.

The manifest.json file will have entries related to your Google Chrome browser extension such as its name, description, version number of the extension, author name, permissions and many other metadata, all stored in JSON format.

So let us now go ahead and create a new directory which I called coolExt where we will store all the files related to this Google Chrome extension. Under this newly created directory, create our manifest.json file with the following content:

manifest.json (The only file required to get Chrome extension started!)
{
    "name":"Cool Extension",
    "description":"My new cool extension",
    "manifest_version":2,
    "version":"1.0",
    "author":"@digitallyamar"
}

You can see that this file contains a set of JSON key:value pairs. Here, each of these JSON key values such as “name”, “manifest_version”, “author” etc are already defined by Google’s Chrome API and hence we cannot alter it but use it as is. On the other hand, the values of these are determined by us according to how we want to describe and design our Chrome browser extension!

With the manifest.json file and its contents in place, it is now time to test our Google Chrome extension. In order to do so, open up your Google Chrome browser and in the address bar type: chrome://extensions

This should now open up a new Chrome Extensions window listing any Google Chrome extensions you might have already installed. At the top of this window, you should see an button called “Load unpacked“, click on it and browse to the “coolExt” directory where we have the above manifest.json file saved in and open it. This should now result in our Google Chrome extension getting installed into our Google Chrome browser and displayed as an installed extension in chrome://extensions

Google Chrome Extension Development In Action

There we have it! A basic version of our simple Google Chrome extension is created and installed into our Google Chrome browser. Although at the moment the Google Chrome extension is not doing much other than just describing itself and some other meta information, we have successfully created the basic framework for our Chrome extension.

We will in the next article continue to develop on this basic Chrome extension to bring in some interesting functionalities to our browser and to some of the websites we will visit. But that will be in the next article, so for now hope you got a basic understanding of how Chrome extensions are built under the hood. Checkout for the next article and if you have any comments on queries with respect to this article, do let me know in the comments below.

Until next time, happy coding! 🙂

Categories
JAVASCRIPT TUTORIALS WORDPRESS

How To Enable Javascript Code In WordPress Page

WordPress CMS by default blocks any Javascript code from running as part of the website’s Page or Post. This is done to to prevent any malware attacks or website hijacking through vulnerabilities created by enabling Javascript on the WordPress Page and Post level.

However, it is still possible to enable running of Javascript if the WordPress website owner knows what he is doing. The website owner can enable Javascript on WordPress using two methods:

Method 1: Installing a third party WordPress Plugin

By installing an external WordPress Plugin that allows adding Javascript per post, any non technical WordPress site owner can also enable Javascript on the WordPress Posts and Pages level.

Method 2: By editing function.php file of the active WordPress Theme

In this method, you would need to ftp to your installed and currently active WordPress theme directory and look for the file called functions.php. You will then need to hook a new PHP function that gets loaded right after the WordPress Post’s <head> section gets loaded. Below is an example of how to write such a hook function in your functions.php file.

function wp_hook_js() {
 ?>
    <script type="text/javascript">
        //Add your Javascript code here
    <script>
 <?php
}

add_action('wp_head', 'wp_hook_js');

In the above code you can see our new function wp_hook_js() that contains our Javascript code. However, in order for this function to get called, we use the add_action() function. From the above code, you can see that we are hooking up our wp_hook_js() function to ‘wp_head’. This is a hook that gets called whenever a WordPress Post or Page gets loaded.

Thus by using either of these two methods i.e. by using a WordPress Plugin or by editing our theme’s function.php file and adding a hook function, we can ensure our Javascript code gets executed each time a WordPress Post or Page gets loaded.

In addition to this, you can add custom CSS to your post using “Additional CSS” option under Theme Customization option you get when you go to the “Appearance” option in the WordPress Admin dashboard.

This is all that is required to be done in order to get custom Javascript codes running at the WordPress Page or WordPress Post level. Hope this article was helpful in giving an insight on how to get it all working.

Until next time, happy coding 🙂 !!