The GoogleApps Script Bot is a useragent / bot that Google’s Javascript app uses to fetch pages. Example and JS Code found below.
Here’s the standard apache log when this bot is accessing your site
64.233.172.162 - - [02/Jun/2014:15:12:59 -0400] "GET / HTTP/1.1" 200 749 "-" "Mozilla/5.0 (compatible; GoogleApps script; +http://script.google.com/bot.html)"
This bot can be used by any Google Docs user in order to scrape or otherwise access content on a website.
The particular IP accessing my site resolved to google-proxy-64-233-172-162.google.com
I’ve setup a test doc for anyone that would like to checkout the script in action. The code powering it can be found under the script editor.
You’re welcome to punch in your own url to have the crawler fetch your site.
See it in action: https://docs.google.com/a/rehmann.co/spreadsheets/d/1junWawm5kNziFJAZHdUP9wMpW9o-HHu3DGgQ0bLTyY4/edit#gid=0
Unfortunately, it looks like this script doesn’t follow rules set fourth in your robots.txt file, so if your website is being abused by a user using the Google Docs script bot, I would block the IP or setup your site to serve a 404 error to any user-agent matching “GoogleApps script”
I’d love to hear of your experience with this bot. Are people abusing it to scrape your content?
Cheers,
Luke
Functioning Example:
https://docs.google.com/a/rehmann.co/spreadsheets/d/1junWawm5kNziFJAZHdUP9wMpW9o-HHu3DGgQ0bLTyY4/edit#gid=0
(feel free to make a copy or test with your own URL)
Code behind it:
function readRows() {
var sheet = SpreadsheetApp.getActiveSheet();
var rows = sheet.getDataRange();
var numRows = rows.getNumRows();
var values = rows.getValues();
for (var i = 0; i <= numRows - 1; i++) { var row = values[i]; Logger.log(row); } }; function GetPage(url) { var response = UrlFetchApp.fetch(url); return response.getContentText(); } function encodeURIC( r ) { if( r.constructor == Array ) { var out = r.slice(); for( i=0; i< r.length; i++){ for( j=0; j< r[i].length; j++){ out[i][j] = encodeURIComponent(r[i][j].toString() ) ; } } return out ; } else{ return encodeURIComponent(r.toString() ) } }