在centos上无界面运行浏览器

安装

python环境可跳过

1)python环境, 检查python的版本,是否为2.7.x,如果不是,安装2.7.6。

如果是centos 6.x,升级python2.6到python2.7,参考教程 http://ruiaylin.github.io/2014/12/12/python%20update/

如果是centos 7.x,默认就是python2.7,不用升级

如果是mac osx,可以使用virtualenv,安装python2.7

安装浏览器环境 selenium依赖.(如果是mac环境,仅需安装firefox, 但确保版本是 firefox 36.0,使用最新的版本会报错)

yum install xorg-x11-server-Xvfb
yum upgrade glib2 # 确保glib2版本大于2.42.2,否则firefox启动会报错 
yum install firefox # centos下安装最新的firefox版本

PHP版

http://selenium-release.storage.googleapis.com/index.html?path=3.0/

wget http://selenium-release.storage.googleapis.com/3.0/selenium-server-standalone-3.0.1.jar


安装虚拟现实器

yum install -y Xvfb

启动Xvfb

Xvfb -ac :7 -screen 0 1280x1024x8

启动 firefox or chrome

export  DISPLAY=:7 (和上一步的number号相同)
firefox http://www.baidu.com  //firefox 浏览http://www.baidu.com


如果运行完后,出现:

Xlib: extension "RANDR"  missing on display ":7"

我想说,you made it. 这是个无关紧要的信息,之前我试图解决掉它,没有成功。最后我在运行selenium脚本后,完全没有出现这个信息,脚本执行很正常,所以现在我把它当做是安装成功的信息

当然运行selenium 脚本前总不能老是敲一遍这些命令,太麻烦了。

弄成一个服务,先 touch /etc/init.d/xvfb

脚本如下:

#! /bin/bash
if [ -z "$1" ]; then 
echo "`basename $0` {start|stop}"
exit
fi

case "$1" in
start)
      /usr/bin/Xvfb :7 -ac -screen 0 1024x768x8 &
;;
stop)
      killall Xvfb
;;
esac


修改脚本权限,启动服务:

chmod +x /etc/init.d/xvfb
chkconfig xvfb on
service xvfb start


停止服务的话就是: service xvfb stop

完毕了。

参见 http://www.tools138.com/create/article/20150313/020006784.html

PHPWebderiver

composer require facebook/webdriver:1.2.0

Eg:

<?php
// An example of using php-webdriver.
namespace Facebook\WebDriver;
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
require_once('vendor/autoload.php');
// start Firefox with 5 second timeout
$host = 'http://localhost:4444/wd/hub'; // this is the default
$capabilities = DesiredCapabilities::firefox();
$driver = RemoteWebDriver::create($host, $capabilities, 5000);
// navigate to 'http://docs.seleniumhq.org/'
$driver->get('http://docs.seleniumhq.org/');
// adding cookie
$driver->manage()->deleteAllCookies();
$driver->manage()->addCookie([
    'name' => 'cookie_name',
    'value' => 'cookie_value',
]);
$cookies = $driver->manage()->getCookies();
print_r($cookies);
// click the link 'About'
$link = $driver->findElement(
    WebDriverBy::id('menu_about')
);
$link->click();
// print the title of the current page
echo "The title is '" . $driver->getTitle() . "'\n";
// print the URI of the current page
echo "The current URI is '" . $driver->getCurrentURL() . "'\n";
// Search 'php' in the search box
$input = $driver->findElement(
    WebDriverBy::id('q')
);
$input->sendKeys('php')->submit();
// wait at most 10 seconds until at least one result is shown
$driver->wait(10)->until(
    WebDriverExpectedCondition::presenceOfAllElementsLocatedBy(
        WebDriverBy::className('gsc-result')
    )
);
// close the Firefox
$driver->quit();

3.x的selenium Firefox 需要额外的驱动

https://github.com/mozilla/geckodriver/releases/tag/v0.9.0

wget https://github.com/mozilla/geckodriver/releases/download/v0.9.0/geckodriver-v0.9.0-linux64.tar.gz

tar -zxvf geckodriver-v0.9.0-linux64.tar.gz

mv geckodriver /usr/bin/


java -jar selenium-server-standalone-x.x.x.jar -Dwebdriver.firfox.driver="/usr/bin/geckodriver"

参见 http://stackoverflow.com/questions/18272468/error-the-path-to-the-driver-executable-must-be-set-by-the-webdriver-chrome-dri

phantomjs安装

wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2
tar xjvf phantomjs-2.1.1-linux-x86_64.tar.bz2 -C /usr/local/
mv /usr/local/phantomjs-2.1.1-linux-x86_64/ /usr/local/phantomjs
ln -s /usr/local/phantomjs/bin/phantomjs /usr/bin/

此处建议安装phantomjs 1.9.8 经测试2.1.1 版本cookie设置有bug

个版本build好的安装包下载页面

https://bitbucket.org/ariya/phantomjs/downloads/

参见 http://www.cnblogs.com/yestreenstars/p/5511212.html

java安装

yum -y install java-1.8.0-openjdk*

参见 http://jingyan.baidu.com/article/4853e1e51d0c101909f72607.html

相关连接:
参考 https://github.com/bowenpay/wechat-spider
参考 https://github.com/ariya/phantomjs/tree/2.1.1
参考 http://phantomjs.org/download.html
参考 https://github.com/facebook/php-webdriver/tree/1.2.0
参考 https://github.com/facebook/php-webdriver/blob/1.2.0/example.php
参考 https://www.v2ex.com/t/324309

添加新评论