安装
python环境可跳过
1)python环境, 检查python的版本,是否为2.7.x,如果不是,安装2.7.6。
如果是centos 6.x,升级python2.6到python2.7,参考教程 http://ruiaylin.github.io/2014/12/12/python%20update/
如果是centos 7.x,默认就是python2.7,不用升级
如果是mac osx,可以使用virtualenv,安装python2.7
安装浏览器环境 selenium依赖.(如果是mac环境,仅需安装firefox, 但确保版本是 firefox 36.0,使用最新的版本会报错)
yum install xorg-x11-server-Xvfb
yum upgrade glib2 # 确保glib2版本大于2.42.2,否则firefox启动会报错
yum install firefox # centos下安装最新的firefox版本
PHP版
http://selenium-release.storage.googleapis.com/index.html?path=3.0/
wget http://selenium-release.storage.googleapis.com/3.0/selenium-server-standalone-3.0.1.jar
安装虚拟现实器
yum install -y Xvfb
启动Xvfb
Xvfb -ac :7 -screen 0 1280x1024x8
启动 firefox or chrome
export DISPLAY=:7 (和上一步的number号相同)
firefox http://www.baidu.com //firefox 浏览http://www.baidu.com
如果运行完后,出现:
Xlib: extension "RANDR" missing on display ":7"
我想说,you made it. 这是个无关紧要的信息,之前我试图解决掉它,没有成功。最后我在运行selenium脚本后,完全没有出现这个信息,脚本执行很正常,所以现在我把它当做是安装成功的信息
当然运行selenium 脚本前总不能老是敲一遍这些命令,太麻烦了。
弄成一个服务,先 touch /etc/init.d/xvfb
脚本如下:
#! /bin/bash
if [ -z "$1" ]; then
echo "`basename $0` {start|stop}"
exit
fi
case "$1" in
start)
/usr/bin/Xvfb :7 -ac -screen 0 1024x768x8 &
;;
stop)
killall Xvfb
;;
esac
修改脚本权限,启动服务:
chmod +x /etc/init.d/xvfb
chkconfig xvfb on
service xvfb start
停止服务的话就是: service xvfb stop
完毕了。
参见 http://www.tools138.com/create/article/20150313/020006784.html
PHPWebderiver
composer require facebook/webdriver:1.2.0
Eg:
<?php
// An example of using php-webdriver.
namespace Facebook\WebDriver;
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
require_once('vendor/autoload.php');
// start Firefox with 5 second timeout
$host = 'http://localhost:4444/wd/hub'; // this is the default
$capabilities = DesiredCapabilities::firefox();
$driver = RemoteWebDriver::create($host, $capabilities, 5000);
// navigate to 'http://docs.seleniumhq.org/'
$driver->get('http://docs.seleniumhq.org/');
// adding cookie
$driver->manage()->deleteAllCookies();
$driver->manage()->addCookie([
'name' => 'cookie_name',
'value' => 'cookie_value',
]);
$cookies = $driver->manage()->getCookies();
print_r($cookies);
// click the link 'About'
$link = $driver->findElement(
WebDriverBy::id('menu_about')
);
$link->click();
// print the title of the current page
echo "The title is '" . $driver->getTitle() . "'\n";
// print the URI of the current page
echo "The current URI is '" . $driver->getCurrentURL() . "'\n";
// Search 'php' in the search box
$input = $driver->findElement(
WebDriverBy::id('q')
);
$input->sendKeys('php')->submit();
// wait at most 10 seconds until at least one result is shown
$driver->wait(10)->until(
WebDriverExpectedCondition::presenceOfAllElementsLocatedBy(
WebDriverBy::className('gsc-result')
)
);
// close the Firefox
$driver->quit();
3.x的selenium Firefox 需要额外的驱动
https://github.com/mozilla/geckodriver/releases/tag/v0.9.0
wget https://github.com/mozilla/geckodriver/releases/download/v0.9.0/geckodriver-v0.9.0-linux64.tar.gz
tar -zxvf geckodriver-v0.9.0-linux64.tar.gz
mv geckodriver /usr/bin/
java -jar selenium-server-standalone-x.x.x.jar -Dwebdriver.firfox.driver="/usr/bin/geckodriver"
参见 http://stackoverflow.com/questions/18272468/error-the-path-to-the-driver-executable-must-be-set-by-the-webdriver-chrome-dri
phantomjs安装
wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2
tar xjvf phantomjs-2.1.1-linux-x86_64.tar.bz2 -C /usr/local/
mv /usr/local/phantomjs-2.1.1-linux-x86_64/ /usr/local/phantomjs
ln -s /usr/local/phantomjs/bin/phantomjs /usr/bin/
此处建议安装phantomjs 1.9.8 经测试2.1.1 版本cookie设置有bug
个版本build好的安装包下载页面
https://bitbucket.org/ariya/phantomjs/downloads/
参见 http://www.cnblogs.com/yestreenstars/p/5511212.html
java安装
yum -y install java-1.8.0-openjdk*
参见 http://jingyan.baidu.com/article/4853e1e51d0c101909f72607.html
相关连接:
参考 https://github.com/bowenpay/wechat-spider
参考 https://github.com/ariya/phantomjs/tree/2.1.1
参考 http://phantomjs.org/download.html
参考 https://github.com/facebook/php-webdriver/tree/1.2.0
参考 https://github.com/facebook/php-webdriver/blob/1.2.0/example.php
参考 https://www.v2ex.com/t/324309