Save login info in a cookie using CURL (PHP)

May 30, 2008

At work I was using a simple script that connected to a secure API, logged in and saved the result. This seems simple enough and is fast enough when calling it a few times, however one of the tasks I was then assigned was actually running this script for different values over 60,000 times! Each call (including the logging in) would take a little over a second, so calling it 60,000 times would take a while.

It was then decided to try and improve this by making the script only login once and save the session, so that subsequent calls would not have to login each time. This is one of the solutions we came up with ( because finding this online was not very easy :) )

The first step is the login, the agent variable is being set to trick the API into thinking that the request has come from a browser.

//Set the cookie file name
$cookiefile = tempnam("/tmp", "cookies");
$agent     = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
//Set the user name and password values
$USER      = 'user';
$PASS      = 'pass';
//The API url, to do the login
$url = "https://www.example.com/";

//Initialise CURL
$ch = curl_init();
//Set all the various options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_POST, 0); // set POST method
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
//Set the cookie file you want to use
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_USERPWD, $USER.":".$PASS);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);

//execute the CURL
$mRes = curl_exec($ch);

Remember not to close curl at this point. Now we look at the loop that will carry out the 60,000 calls to the API

while ($row = [get row value for the 60,000 entries])
{
    $url       = "https://www.example.com/API?check=$row";
    //We do not have to initialise CURL again
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_USERAGENT, $agent);
    curl_setopt($ch, CURLOPT_POST, 0); // set POST method
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    //Remember to use the same cookiefile as above
    curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
    curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
    //execute the CURL call
    $mRes = curl_exec($ch);
    //DEAL WITH THE RESULT FROM THE CURL CALL
}
//only close CURL once the work is finished.
curl_close($ch);

If you’d like to know more the various CURL options I have used above like ‘CURLOPT_SSL_VERIFYHOST’ etc, have a read through the curl_setopt page on php.net. You can also read more about CURL there as well.

This entry was posted on Friday, May 30th, 2008 at 1:13 am and is filed under code, tech, tips&tricks. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

7 Responses to “Save login info in a cookie using CURL (PHP)”

  1. Josh said:

    Aug 07, 08 at 10:17 am

    Thank you for the information.
    I am able to successfully use your idea with ebay.com!

    But I have an issue with overstock.com
    - I am able to login it takes me to a welcome page (and cookies are in the cookie file.)
    - But accessing another page under the same site, is taking me to the login page again, though the cookie file is speicified.

    ovestock.com uses all https pages.

    Try it yourself on overstock, (1) login, (2)access the preferences page using cURL.

    Advise,
    Thanks!
    Josh

  2. Joseph said:

    Sep 15, 08 at 1:19 am

    Josh, do they use pop ups, redirects to different subdomains or hidden autosubmitting forms?

    Cause Any of those may catch you off guard and throw you through 3 pages or so instead of just from one to another.

    I’m having a nightmare doing an information dl on a website that does that using CURL.

  3. ann said:

    Nov 13, 08 at 3:40 am

    i want to post values to the site trademe.co.nz using curl. how can i add an item for auction just after logging i n. the problem now i am facing is i give the particular url , but it gives me the login page. can i use curl with username and password given with other options of surl_setopt()?

  4. Rajagopal said:

    Jan 21, 09 at 12:57 am

    Using curl i reterive the webpage(using curl login option) but while i click the inner link in that page it shows session expired.How to enable the session.

  5. aam said:

    Feb 24, 09 at 7:24 pm

    Hi sir.. im newbie in php.. i got sum problem to save cookies and execute another command remotely.. here is the script..

    can i login wid multiple ids using curl?

    thanks for your help sir.. :)

  6. Sushant Danekar said:

    May 14, 10 at 11:23 pm

    HI there,

    this code is gr8.
    But its not working for my requirement.
    Please have a look on this site. and let me know how I should use

    http://armls.flexmls.com/

    Currently I am getting error that

    Unable to Log In
    This site uses cookies for its own security. Your browser must be capable of processing cookies and cookies must be activated. Please set your browser to accept cookies, then press the reload button.

    please reply

    thanks
    regards
    Sushant Danekar
    http://sushant-danekar.blogspot.com/ http://splendornet.com/

  7. Phil said:

    Jun 07, 10 at 11:16 pm

    Many thanks for your tips! ^_^

Leave a Reply