Email Extractor Script with PHP cURL and Regular Expression (1)

First, we look at a very simple email extractor. We are going to use the HttpCurl class created earlier.

<?php

define('EMAIL_PATTERN', '/([\s]*)([_a-zA-Z0-9-]+(\.[_a-zA-Z0-9-]+)*([ ]+|)@([ ]+|)([a-zA-Z0-9-]+\.)+([a-zA-Z]{2,}))([\s]*)/i');

interface HttpScraper
{
	public function parse($body, $head);
}

class Scraper implements HttpScraper
{
	
	public function parse($body, $head)	{
	   if ($head == 200) {
		$p = preg_match_all(EMAIL_PATTERN, $body, $matches);
			if ($p) {
				foreach($matches[0] as $emails) {
					echo "<pre>";
					print_r($emails);	
					echo "<pre>";
				}
			}
		}
	}
}

class HttpCurl {
	protected $_cookie, $_parser, $_timeout;
	private $_ch, $_info, $_body, $_error;
	
	public function __construct($p = null) {
        if (!function_exists('curl_init')) {
            throw new Exception('cURL not enabled!');
        }	
		$this->setParser($p);
	}

	public function get($url) {	
		return $this->request($url);
	}

	protected function request($url) {
        $ch = curl_init($url);
		curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
		curl_setopt($ch, CURLOPT_MAXREDIRS, 5);		
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
		curl_setopt($ch, CURLOPT_URL, $url);
		$this->_body = curl_exec($ch);
        $this->_info  = curl_getinfo($ch);
        $this->_error = curl_error($ch);
        curl_close($ch);

		$this->runParser($this->_body, $this->getStatus());				
    }

	public function getStatus() {
		return $this->_info[http_code];
	}
	
	public function getHeader() {
		return $this->_info;
	}

	public function getBody() {
		return $this->_body;
	}
	
	public function __destruct() {
	}	
	
	public function setParser($p)	{
		if ($p === null || $p instanceof HttpScraper || is_callable($p))	
			$this->_parser = $p;
	}

	public function runParser($content, $header)	{
		if ($this->_parser !== null)
		{
			if ($this->_parser instanceof HttpScraper)
				$this->_parser->parse($content, $header);	
			else
				call_user_func($this->_parser, $content, $header);
		}
	}	
}

?>

How it works:

1. First define a regular expression to match email pattern. The pattern will be used in this example only. We will change to working patterns in next article.

define('EMAIL_PATTERN', '/([\s]*)([_a-zA-Z0-9-]+(\.[_a-zA-Z0-9-]+)*([ ]+|)@([ ]+|)([a-zA-Z0-9-]+\.)+([a-zA-Z]{2,}))([\s]*)/i');

 2. Then define Interface called HttpScraper with a public method parse().

interface HttpScraper
{
public function parse($body, $head);
}

3.  Next we implement the above interface in a class called Scraper. Two information will be passed to function parse(), one is the content of the webpage ($body) and another is http_code of that web page ($head). If the value to $head is 200, we proceed to match any email pattern (the defined EMAIL_PATTERN) in the web page content using function preg_match_all. For this tutorial, we print the results on the screen.

class Scraper implements HttpScraper
{
	
	public function parse($body, $head)	{
	   if ($head == 200) {
		$p = preg_match_all(EMAIL_PATTERN, $body, $matches);
			if ($p) {
				foreach($matches[0] as $emails) {
					echo "<pre>";
					print_r($emails);	
					echo "<pre>";
				}
			}
		}
	}
}

 4. Here you can see some minor changes to the HttpCurl class that we created earlier.

class HttpCurl {
	protected $_cookie, $_parser, $_timeout;
	private $_ch, $_info, $_body, $_error;
	
	public function __construct($p = null) {
        if (!function_exists('curl_init')) {
            throw new Exception('cURL not enabled!');
        }	
		$this->setParser($p);
	}

	public function get($url) {	
		return $this->request($url);
	}

	protected function request($url) {
        $ch = curl_init($url);
		curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
		curl_setopt($ch, CURLOPT_MAXREDIRS, 5);		
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
		curl_setopt($ch, CURLOPT_URL, $url);
		$this->_body = curl_exec($ch);
        $this->_info  = curl_getinfo($ch);
        $this->_error = curl_error($ch);
        curl_close($ch);

		$this->runParser($this->_body, $this->getStatus());				
    }

	public function getStatus() {
		return $this->_info[http_code];
	}
	
	public function getHeader() {
		return $this->_info;
	}

	public function getBody() {
		return $this->_body;
	}
	
	public function __destruct() {
	}	
	
	public function setParser($p)	{
		if ($p === null || $p instanceof HttpScraper || is_callable($p))	
			$this->_parser = $p;
	}

	public function runParser($content, $header)	{
		if ($this->_parser !== null)
		{
			if ($this->_parser instanceof HttpScraper)
				$this->_parser->parse($content, $header);	
			else
				call_user_func($this->_parser, $content, $header);
		}
	}	
}

5. The constructor of HttpCurl - we added a function call to setParser(). setParser() stores the object or call back function passed to $p into private variable $_parser.

	public function setParser($p)	{
		if ($p === null || $p instanceof HttpScraper || is_callable($p))	
			$this->_parser = $p;
	}

 6. Function runParser() is added into HttpCurl. When called, it will execute the call back function or object in $_parser. In this case, I am using object.

	public function runParser($content, $header)	{
		if ($this->_parser !== null)
		{
			if ($this->_parser instanceof HttpScraper)
				$this->_parser->parse($content, $header);	
			else
				call_user_func($this->_parser, $content, $header);
		}
	}

 7. The runParser() function is called in request() after cURL requested web page source file.

	protected function request($url) {
        $ch = curl_init($url);
		curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
		curl_setopt($ch, CURLOPT_MAXREDIRS, 5);		
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
		curl_setopt($ch, CURLOPT_URL, $url);
		$this->_body = curl_exec($ch);
        $this->_info  = curl_getinfo($ch);
        $this->_error = curl_error($ch);
        curl_close($ch);

		$this->runParser($this->_body, $this->getStatus());				
    }

 Now that we have our code ready, we can test out the script.

Assuming you have completed development of a property listing website and released to live server. You need property agents to register as members and upload their properties to your website. Getting property agents to find your website through search engine will take forever. We can go to other property websites and find their contacts. For example, this website has more than one thousand agents.

If you right click on browser and select "View Source", you can see the web page source file.

To test out our script, here is our new test.php.

<?php
include 'httpcurl.php';
  
$target = "http://<domain name>";
 
$up = new Scraper;
$test = new HttpCurl($up);
 
$test->get($target);
 
?>

Here is the output of test.php.

So now we are able to extract email addresses from a web page. However, we should not send out email without ability to address the receivers. Starts your email with "Dear Sir", "Hello Madam" will made our email looks like spam.

In the next article, we will modify the script a bit to grab additional information, such as name, and telephone from the example. Click here to go to next article.