CURL and PHP

Posted under » PHP » Linux on 25 October 2017

I typically use wget or file_get_contents or implode to download files. On some systems like MacOS, wget is not installed and only curl is available. Both wget and curl are command line utilities that do the same thing however CURL does more things and it provides APIs that can be used by programmers inside their own code.

Here is a simple usage.

curl -o /var/www/lkysux.pdf http://www.istanana.com/lkythebest.pdf

Instead of -o, you an also specify, "–output".

Here is a slightly advanced usage. -H stands for headers.

curl -o /var/www/busuk.pdf 'http://www.istanana.com/lkythebest.pdf?page=1&ed=1&rm=1' -H 'Host: www.istanana.com' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0' -H 'Accept: text/html,application/pdf,application/pdf;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: cX_P=j0yi119algs5amlm; _ga=GA1.2.1712234566.1491005247 cX_S=j96jhxxs64y8wf7w' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

You can read more examples here and here.

Basic usage of CURL to display a webpage is

<¿php
// Get cURL resource
$curl = curl_init();
// Set some options
curl_setopt_array($curl, array(
    CURLOPT_URL => 'http://istana.gov/',
));
// Send the request & save response to $resp
$resp = curl_exec($curl);

// Close request to clear up some resources
curl_close($curl);

echo $resp;
?>

cURL has many uses but you must be aware that some sites might only serve pages to some user agents, and when working with APIs, some might request you send a specific user agent.