Pinterest bot - How to auto-login with php-webdriver + Selenium

Previous article : How to avoid Selenium webdriver from being detected as bot or web spider

Only part of my code will be displayed and discussed. No disclosure on full code. I created a simple wrapper class for Selenium.

class SeleniumChrome
{	
	private $_host, $_driver, $_session, $_selenium, $_pipes;
	private $_chrome_options = array();
	private $_cookies = array();
				
	// Initialize varialbes.
    public function __construct( $java_path, $selenium_path, $chromedriver_path, $selenium_port, $chrome_options ) {
		$chromedriver_path_envvar = 'webdriver.chrome.driver';
		putenv( $chromedriver_path_envvar .'='. $chromedriver_path );
		
		$descriptorspec = array(
			0 => array('pipe', 'r'), 
			1 => array('file', __DIR__ . '\\selenium_log_file\\selenium_log-' . date('Ymd-His').'_'. $selenium_port . '-stdout.txt', 'a'),  
			2 => array('file', __DIR__ . '\\selenium_log_file\\selenium_log-' . date('Ymd-His').'_'. $selenium_port . '-stderr.txt', 'a')   
		);

		$selenium_cmd = '"' . $java_path .'" -D'. $chromedriver_path_envvar .'="'. $chromedriver_path .'" -jar "'. $selenium_path .'" -port '. $selenium_port; 
		$this->_selenium = proc_open( $selenium_cmd, $descriptorspec, $this->_pipes, null, null, array( 'bypass_shell' => true ) );		
		$this->_host = 'http://localhost:'. $selenium_port . '/wd/hub';
		$this->_chrome_options = $chrome_options;
		$ops = new ChromeOptions();
		$ops->addArguments( $this->_chrome_options );
		$ops->setExperimentalOption("excludeSwitches", array("enable-automation"));		
		$capabilities = DesiredCapabilities::chrome();
		$capabilities->setCapability( ChromeOptions::CAPABILITY, $ops );
		$this->_driver = RemoteWebDriver::create( $this->_host, $capabilities );
		$this->_session = $this->_driver->getSessionID();
	}
// ... and other methods...

 From the pinterest main file, first we instantiate SeleniumChrome object and passing full path of Java, Selenium and Chromedriver binary files. We also passing Selenium port number (which is 4444) as well as Chrome options (some discussed in previous article) to the constructor.

$driver = new \SeleniumChrome( $java_path, $selenium_path, $chromedriver_path, $selenium_port, $chrome_options );

 The $selenium_cmd is to construct long string command as below. 

$selenium_cmd = '"' . $java_path .'" -D'. $chromedriver_path_envvar .'="'. $chromedriver_path .'" -jar "'. $selenium_path .'" -port '. $selenium_port; 

"C:\Program Files\Common Files\Oracle\Java\javapath\java.exe" -Dwebdriver.chrome.driver="C:\xampp\htdocs\phpwebdriver\webdriver\chromedriver.exe" -jar "C:\xampp\htdocs\phpwebdriver\webdriver\selenium-server-standalone-3.141.59.jar" -port 4444

 Then we use proc_open to run the command.

$this->_selenium = proc_open( $selenium_cmd, $descriptorspec, $this->_pipes, null, null, array( 'bypass_shell' => true ) );		

 The rest of code will call up Chrome browser with the options that we set.

		$this->_host = 'http://localhost:'. $selenium_port . '/wd/hub';
		$this->_chrome_options = $chrome_options;
		$ops = new ChromeOptions();
		$ops->addArguments( $this->_chrome_options );
		$ops->setExperimentalOption("excludeSwitches", array("enable-automation"));		
		$capabilities = DesiredCapabilities::chrome();
		$capabilities->setCapability( ChromeOptions::CAPABILITY, $ops );
		$this->_driver = RemoteWebDriver::create( $this->_host, $capabilities );

 I store the current value of Chrome session id. This is because when Selenium throw an exception, session will be restarted with new id. Further actions can not be performed on Pinterest. Restore the old session id can resolve this problem.

$this->_session = $this->_driver->getSessionID();

 At the pinterest automation page, we call the get() method from SeleniumChrome wrapper.

Two parameters passed to the get() method.

First is the Pinterest login url, https://www.pinterest.com/login/.

Second is the condition to tell Selenium that the page is loaded. In this case, we use xpath to detect the Login button. 

In the try/catch block, webdriver will load Pinterest login page, then wait until Login button detected. I set the waiting time between 20 to 4000 seconds due to internet speed in my area. You can cut to shorter time.

If an exception was thrown during this action, either not able to detect the expected  Loin button, or due to webdriver timeout, or cURL exception, webdriver will retry 5 attempts before give up completely.

const LOGIN_PAGE = 'https://www.pinterest.com/login/';
$driver->get( LOGIN_PAGE, WebDriverExpectedCondition::visibilityOfElementLocated(WebDriverBy::xpath("//div[contains(text(),'Log in')]")) );

// from SeleniumChrome wrapper
	public function get( $page, WebDriverExpectedCondition $condition, $attempts = 1 ) {
		try {
			$this->_driver->get( $page );
			$this->_driver->wait( 20, 4000 )->until( $condition  );
			return TRUE;
		} 
		catch ( Facebook\WebDriver\Exception\ExpectedException |  Facebook\WebDriver\Exception\TimeOutException | Facebook\WebDriver\Exception\WebDriverCurlException $e ) {
			if ( $attempts < 5 ) {
				echo "\nException caught get page. Retry for " . ($attempts + 1) . " times.";	
				$this->_driver = RemoteWebDriver::createBySessionID($this->_session, $this->_host);				
				$this->get( $page, $condition,  ($attempts + 1) );
			} else {
				echo "\nDrive get page error. Return false.";
				echo "\n" . $e->getMessage();
				return FALSE;
			}
		}		
	}

 After confirm Pinterest login page loaded, we call login() method and passing five parameteres to it.

First $cookie_file is the cookie file with full path.

Second s the condition to tell Selenium that login is successful. In this case, we use xpath to detect the username at the user icon after login redirection. 

Follow by login email and password.

Lastly the button to click if webdriver try to login with email and password.

$driver->login( $cookie_file, WebDriverExpectedCondition::presenceOfElementLocated(WebDriverBy::xpath("//a[@href='/" . $username . "/']")) , $email, $password, 'SignupButton' );

// from SeleniumChrome wrapper
	public function login( $cookie, WebDriverExpectedCondition $condition, $email, $password, $log_in_button ) {
		if ( ! $this->_loginWithCookie( $cookie, $condition) ) {
			$this->_loginWithNoCookie( $cookie, $condition, $email, $password, $log_in_button );
		}
	}
	

	private function _loginWithCookie( $cookie, WebDriverExpectedCondition $condition) {
		if ( file_exists( $cookie ) )  {	
			$this->_readCookie( $cookie );	
			$this->_refreshPage();	
			return $this->waitUntil( $condition );
		} 		
	}

	private function _loginWithNoCookie( $cookie, WebDriverExpectedCondition $condition, $email, $password, $log_in_button ) {
		$this->sendKeysToElement( WebDriverBy::id('email'), $email );
		$this->sendKeysToElement( WebDriverBy::id('password'), $password );
		sleep(rand(4,7));	
		$this->clickElement( WebDriverBy::className( $log_in_button ) );	
		if ( $this->waitUntil( $condition ) ) {
			$this->_saveCookie( $cookie );	
		}	
	}
	
	private function _refreshPage() {
		$this->_driver->navigate()->refresh();
		sleep(10);
	}
	
	public function waitUntil( WebDriverExpectedCondition $condition, $attempts = 1 ) {
		try {		
			$this->_driver->wait( 20, 5000 )->until( $condition  );	
			return TRUE;
		}
		catch ( Facebook\WebDriver\Exception\TimeOutException | Facebook\WebDriver\Exception\WebDriverCurlException $e ) {
			if ( $attempts < 5 ) {
				echo "\nException caught wait until. Retry for " . ($attempts + 1) . " times.";	
				$this->_driver = RemoteWebDriver::createBySessionID($this->_session, $this->_host);					
				$this->waitUntil( $condition, ($attempts + 1)  );
			} else {
				echo "\nDrive wait-until error. Return false.";
				echo "\n" . $e->getMessage();				
				return FALSE;
			}		
		}				
	}	

 The login() method will first try to login with cookie file. Note that webdrive need to refresh the page after reading cookie file, else nothing will happen to login page.

If login with cookie successful, you will be redirected to either Pinterest business page or personal page. You will not see anything happen to login form.

If login with cookie not successful, webdriver will fill up email and password text box, then click Log in button. Then you will be redicted to either Pinterest business page or personal page. Our script read and save cookie in a file. Webdriver will use this cookie file for next login.

 Currently Pinterest allow two types of account, business account and personal account.

You can upload up to five images or one video for business account, together with analytic tool. Only one image allowed for personal account.

In this case, we need to detect the redirected url and determine the account type. Once we have the account type info, we can decide how many images or video to upload.

const BUSINESS_PAGE = 'https://www.pinterest.com/business/hub/';
$account_type = ( $driver->isURL( BUSINESS_PAGE ) ) ? TRUE : FALSE;

// from SeleniumChrome wrapper
	public function isURL( $url ) {
		$current_url = $this->_driver->getCurrentURL();		
		$url_parts = array_flip(['host', 'port', 'path']);		
		$defaults = ['path' => '/']; 
		$url = array_intersect_key(parse_url($url), $url_parts) + $defaults;
		$current_url = array_intersect_key(parse_url($current_url), $url_parts) + $defaults;
		return $url === $current_url;		
	}

If business account, url https://www.pinterest.com/business/hub/ will be detected.

If personal account, a different url will be detected.

 In next article, i will discuss how to auto post to personal account.