Categories
PROGRAMMING TOOLS TUTORIALS

Curl Command To Access URL Of A Website

In this article, we will learn how to use the Curl command to access the URL of a website. So by doing this, we will be able to fetch a web page from the website. Or learn how to download an image from a given URL.

Prerequisite

But before we take a look at that, we want to make sure you are aware of what a Curl command line tool is and how it works. So if you do not already know about of it, then take a look at our earlier tutorial:

What Is A Curl Command & How To Use Curl?

But if you are already aware of how curl works, then it is about time we take a look at how to use it to access an URL.

How To Use Curl Command To Access An URL

Curl, as we had discussed in our earlier article linked above, can understand a lot of protocols. These includes FTP, SMTP, POP3, MQTT etc.

But the most used protocol among them all are the HTTP and HTTPS protocols. But why, you may ask, right?

Curl Command To Access URL Of A Website
Curl Command To Access URL Of A Website

It is because you can use these protocols to talk to a web server. So, you can use HTTP from curl to request for a web page or an image file or anything else. So any resource that has an URL, can be accessed using the curl command!

Is that clear? Great! So then we will now take a look at how to do that in the next section.

Example Curl Command To Access URL

So let us say that you want to access Google’s home page from command line using curl. How do you do that? Take a look at the code below:

curl -X GET https://www.google.com

This is the curl command you need to use to fetch the Google’s home page. Does it look a bit daunting? Well, do not worry. Because I will explain to you what each of these part of the command is actually doing.

So let us take a look at the command, one part at a time, alright?

“curl” – This is the first part of the command, which as you know will call the curl command line tool.

“- X GET” – What is this, you may wonder right? Well, curl as you know supports a lot of protocols. And each of these protocols have their own set of options built into it. So in case of HTTP, you know that we can send many types of requests to a web server. Some of the prominent ones include GET, POST, PUT, DELETE etc. So in case of curl, we can send a GET request to a web server using the -X GET switch option!

“https://www.google.com” – Finally, this is the web server URL to which we want to send the GET request.

So that is it! That is all you need to enter as part of the curl command to access an URL of a website.

We suggest you practice using this curl command a few more times to get used to it. Once you do that, you will get a clear understanding of how Curl works under the hood.

If you have any questions about it, do let me know in the comment below and I will be happy to answer, alright?

So see you until next time! 🙂

Categories
PROGRAMMING TOOLS TUTORIALS

What Is A Curl Command & How To Use Curl?

Did you just hear about curl and want to know what a curl command is? What it does and how to use it? Then you have come to right place. Because in this article, we will go through each of these questions and answer it in an easy way.

So if you are ready to learn more about it, let us get started!

What Is A Curl Command?

Curl is an open source project that mainly has two modules in it. These modules are:

Curl Command Line Interface Tool

Curl CLI tool is the tool that you have most probably heard of and want to use. This is a command line interface tool. So you need to type in the curl commands in your computer’s command prompt. But while this might seem tedious to you, it has one great advantage for being a CLI tool.

Can you guess what it is?

Curl logo
Curl logo

It is the fact that you can then use it in your scripts as well! Now that is always welcome, right? But you may now be wondering what this curl CLI tool even do, Right? Well, you will have to wait a bit to get answer for that. As we will learn about it only in the next section.

But for now just know that this is one of the main module of Curl. Alright? So then what is the other module of Curl? Let us take a look at that now!

libcurl C library

So along with the Curl CLI tool, we also have a C library called libcurl. In fact, the CLI tool uses libcurl under the hood. But why have two separate modules – curl CLI and libcurl, you ask?

Well, here is the thing. While you can use the curl tool, wouldn’t it be great if you could also build a curl GUI application if you want? Or want to call it programmatically from other programming languages like Python, Java etc.?

libcurl is created for exactly that reason. It gives us a bunch of C language based APIs that you can use to create applications on top of that! Or call it from other programming languages if you want to!

Alright! So now that you know what a curl tool is, it is now time to learn what it does.

What Does A Curl Command Do?

The curl command is used to transfer data between a computer and a server. That’s all it does at it’s core! It is used to exchange information between a client (your computer) and a server.

Now, that server can be of any type such as a web server, ftp server, email server, ssh server anything. It doesn’t matter. You can always use the curl command to download or upload data from or to these servers.

Wait a second! So how is that possible for a single tool like curl to talk to these many types of servers, you ask? Well, it is possible because the libcurl C library that the curl CLI tool uses under the hood can talk to any of these servers. Because libcurl library has full understanding of all the different protocols that these servers use to talk!

So because of this, if you want to talk to a web server, libcurl can make use of HTTP protocol. But on the other hand, if you want to talk to an email server, it can use SMTP, POP3 and other email protocols.

So you may then be wondering as to what all protocols does this libcurl C library support, right? Well, here is a list of protocols that the curl tool supports.

List Of Protocols Supported By Curl

  • HTTP
  • HTTPS
  • SMTP
  • SMTPS
  • POP3
  • POP3S
  • DICT
  • FILE
  • GOPHER
  • FTP
  • FTPS
  • IMAP
  • IMAPS
  • LDAP
  • LDAPS
  • MQTT
  • RTMP
  • RTSP
  • SCP
  • SFTP
  • SMB
  • SMBS
  • TELNET
  • TFTP

So as you can see, libcurl has support for a lot of protocols and this list is still growing! Because, the maintainers of curl keep adding new protocols as and when the need arises!

Alright! Now that we know what a curl command is and what it is used for, it is time to learn how to use it.

But as we saw that curl is a versatile tool that can talk with many different protocols, it’s use cases are also many. So trying to list all the ways you can use curl in this article will not be possible. So we will be creating a separate post for that.

But we do not want to leave you hanging without showing you how to use it at least at a basic level. So here is a simple example of how you can use curl to fetch a web page from a web server.

A Simple Curl Command Example

If you have curl installed on your computer, just issue the following command:

curl -X GET https://www.google.com

When you enter this command on your computer’s command prompt, you should see a bunch of text running along your console. But it is not some gibberish text. But rather it is the HTML code for the google.com home page!

So what exactly has happened here? When you issued the above curl command, the curl CLI asked google.com web server to send it’s HTML home page. You just issued a HTTP GET request to Google’s server and as a result it sent back the home page!

Now how did Google know if it was a GET or a POST HTTP request? How did it know it had to send the home page back to your computer? These are all the questions that has answer’s in the command that we issued above.

But we will discuss about this curl GET command and many other related commands in our next article. So you will have to wait until you read the next article to get an answer for that. Alright?

So see you in the next article. Until then, take care!