aws s3 – cp / mv Skip Existing Files

When moving files between buckets or “folders” with Amazon’s command line interface, aws s3, it would be great to be able to skip files already at the destination preventing overwrites and wasted bandwidth. Unfortunately, there is no documented feature for aws s3, but s3cmd does have this feature.

You cannot “skip-existing” files with aws s3 command line interface. 

 

Jan-22 2017: Search Console performed an infrastructure update that may cause a change in your data.

Google Webmaster Tools performed an update on January 22nd which may cause anomalies in search data.

For my web properties, this resulted in a 50% drop in reported “Structured Data” elements in webmaster tools.

According to Google’s Data Anomalies reporting page, an event for January 22nd has not yet been reported.

 

Email Delivery Problems [Returned From Office 365]

The message undelivered returned email from Office 365 [onmicrosoft.com] is a mailer-daemon equivalent mailing system which notifies the sender of the delivery issues. These issues can range from user-not-found to mailbox full, or attachment rejections. If you are having trouble with this message, leave a detailed comment at the bottom!

Here is an example rejection email from Office365 for an invalid user on an enterprise domain name.

Your message to [email protected] couldn’t be delivered.
feedbackloop wasn’t found at kinesissurvey.com.
office-365-email-returned
How to Fix It
The address may be misspelled or may not exist. Try one or more of the following:
  • Send the message again following these steps: In Outlook, open this non-delivery report (NDR) and choose Send Again from the Report ribbon. In Outlook on the web, select this NDR, then select the link “To send this message again, click here.” Then delete and retype the entire recipient address. If prompted with an Auto-Complete List suggestion don’t select it. After typing the complete address, click Send.
  • Contact the recipient (by phone, for example) to check that the address exists and is correct.
  • The recipient may have set up email forwarding to an incorrect address. Ask them to check that any forwarding they’ve set up is working correctly.
  • Clear the recipient Auto-Complete List in Outlook or Outlook on the web by following the steps in this article: Fix email delivery issues for error code 5.1.10 in Office 365, and then send the message again. Retype the entire recipient address before selecting Send.
If the problem continues, forward this message to your email admin. If you’re an email admin, refer to the More Info for Email Admins section below.
Was this helpful? Send feedback to Microsoft.

More Info for Email Admins
Status code: 550 5.1.10

This error occurs because the sender sent a message to an email address hosted by Office 365 but the address is incorrect or doesn’t exist at the destination domain. The error is reported by the recipient domain’s email server, but most often it must be fixed by the person who sent the message. If the steps in the How to Fix It section above don’t fix the problem, and you’re the email admin for the recipient, try one or more of the following:

The email address exists and is correct – Confirm that the recipient address exists, is correct, and is accepting messages.

Synchronize your directories – If you have a hybrid environment and are using directory synchronization make sure the recipient’s email address is synced correctly in both Office 365 and in your on-premises directory.

Errant forwarding rule – Check for forwarding rules that aren’t behaving as expected. Forwarding can be set up by an admin via mail flow rules or mailbox forwarding address settings, or by the recipient via the Inbox Rules feature.

Recipient has a valid license – Make sure the recipient has an Office 365 license assigned to them. The recipient’s email admin can use the Office 365 admin center to assign a license (Users > Active Users > select the recipient > Assigned License > Edit).

Mail flow settings and MX records are not correct – Misconfigured mail flow or MX record settings can cause this error. Check your Office 365 mail flow settings to make sure your domain and any mail flow connectors are set up correctly. Also, work with your domain registrar to make sure the MX records for your domain are configured correctly.

For more information and additional tips to fix this issue, see Fix email delivery issues for error code 5.1.10 in Office 365.

Original Message Details
Created Date: 1/30/2017 3:06:55 PM
Sender Address: [email protected]
Recipient Address: [email protected]
Subject: collector182804
Error Details
Reported error: 550 5.1.10 RESOLVER.ADR.RecipientNotFound; Recipient not found by SMTP address lookup
DSN generated by: BLUPR17MB0130.namprd17.prod.outlook.com
Message Hops
HOP TIME (UTC) FROM TO WITH RELAY TIME
1 1/30/2017
3:06:56 PM
mail-ot0-f177.google.com SMTP 1 sec
2 1/30/2017
3:06:56 PM
mail-ot0-f177.google.com us-smtp-1.mimecast.com ESMTP *
3 1/30/2017
3:07:00 PM
us-smtp-1.mimecast.com CO1NAM03FT055.mail.protection.outlook.com Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) 4 sec
4 1/30/2017
3:07:01 PM
CO1NAM03FT055.eop-NAM03.prod.protection.outlook.com SN2PR17CA0028.outlook.office365.com Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) 1 sec
5 1/30/2017
3:07:01 PM
SN2PR17CA0028.namprd17.prod.outlook.com BLUPR17MB0130.namprd17.prod.outlook.com Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) *

Original Message Headers

Received: from SN2PR17CA0028.namprd17.prod.outlook.com (10.169.188.166) by
 BLUPR17MB0130.namprd17.prod.outlook.com (10.160.190.26) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id
 15.1.860.13; Mon, 30 Jan 2017 15:07:01 +0000
Received: from CO1NAM03FT055.eop-NAM03.prod.protection.outlook.com
 (2a01:111:f400:7e48::206) by SN2PR17CA0028.outlook.office365.com
 (2603:10b6:804:1a::38) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.874.12 via
 Frontend Transport; Mon, 30 Jan 2017 15:07:01 +0000
Authentication-Results: spf=softfail (sender IP is 207.211.31.81)
 smtp.mailfrom=rehmann.co; kinesissurvey.com; dkim=fail (signature did not
 verify) header.d=rehmann-co.20150623.gappssmtp.com;kinesissurvey.com;
 dmarc=none action=none header.from=rehmann.co;
Received-SPF: SoftFail (protection.outlook.com: domain of transitioning
 rehmann.co discourages use of 207.211.31.81 as permitted sender)
Received: from us-smtp-1.mimecast.com (207.211.31.81) by
 CO1NAM03FT055.mail.protection.outlook.com (10.152.81.162) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id
 15.1.888.7 via Frontend Transport; Mon, 30 Jan 2017 15:07:00 +0000
Received: from mail-ot0-f177.google.com (mail-ot0-f177.google.com
 [74.125.82.177]) (Using TLS) by us-smtp-1.mimecast.com with ESMTP id
 us-mta-133-_r7Wi_YoNH2ngwqioJp7JQ-1; Mon, 30 Jan 2017 10:06:56 -0500
Received: by mail-ot0-f177.google.com with SMTP id f9so244090514otd.1
        for <[email protected]com>; Mon, 30 Jan 2017 07:06:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rehmann-co.20150623.gappssmtp.com; s=20150623;
        h=to:from:date:message-id:subject:mime-version
         :content-transfer-encoding:content-disposition;
        bh=DPge1nLYaUwmdiz4JO5/3OjjZ9jtA1AF9mdHTUBFVJc=;
        b=gj4diNNzhOrx08HalTtdmwCgPmmyeuQecFlHV7eYjHz0Gcg+xNQkrHkn9eoXYfo7qS
         X/0yrf+JvdRx/pPbwp2M2L71oQ0/KI84uYjuv1I19abthPn8KNxSf6j7xM7uTBSx4EXg
         XU/IzSzx6OhRNJTtyloy75ihQF2eCJYjrcLgWpmejAzbMZG1eqtB2yhnK6+H/Ed7zCF6
         gFwAg6c4wPycTi7+mRYerM6jccm/drS+OVm1RDDJqY34HcPPvJdlme7sZG0KCxKbLRbZ
         GlpAPIb74ZZEt2q6PRJsYRpnKl4a8CoWlGUiNPwBzDN14TU70SRHIymaMzB8dzFWLrjP
         8isQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:to:from:date:message-id:subject:mime-version
         :content-transfer-encoding:content-disposition;
        bh=DPge1nLYaUwmdiz4JO5/3OjjZ9jtA1AF9mdHTUBFVJc=;
        b=enAkiOq/4B9U3DPETCKIMIBskTvcptAxxh0nAxSP1gTL3IqcYHfPJypXdNx58JT3Lc
         RETMTGTTjMZMFJPZ0eCX1ZzbfGUQHmB3+zX5/7NYtsxP86GQZzAHP81njVeAJCNbC2aO
         F5ka+su42Atz6gTX/QnEVo2j7N19xGO8cl9gLDTYLJF9NPLmk3xN+hG3oLGJLNi+DyT8
         c//e1/40+tBsCuXwlM/khJNAwn5PsWgLLNYoiVI1yITxR3NzraqSujaRPdsePHeIP2Om
         QzUNe4JKD+p1BZos6G6hXlwZPgt8M0O1PGelFlpMIfqUSl6f+KmmOOSnUUl2W5o8vqgN
         yQXA==
X-Gm-Message-State: AIkVDXKdJ19KQ3HRgnNMku1ksDes4P+h28zn8o70c3/ZBSaHUo3Lwc9SHwijHZk1vdyC5GA+XJbv7HAhF3GldaohIht7cbNo4Q==
X-Received: by 10.157.45.73 with SMTP id v67mr10051697ota.202.1485788815621;
        Mon, 30 Jan 2017 07:06:55 -0800 (PST)
To: <[email protected]com>
From: Luke R <[email protected]>
Date: Mon, 30 Jan 2017 07:06:55 -0800
Message-ID: <CALub9Fd7Cb=TLa_7ZxHCndhA211hg7qK6w4aVeUjBZrYn[email protected]>
Subject: collector182804
MIME-Version: 1.0
X-MC-Unique: _r7Wi_YoNH2ngwqioJp7JQ-1
Authentication-Results: mimecast.com;
	spf=softfail (spfCheck: transitioning domain of rehmann.co does not designate 74.125.82.177 as permitted sender) client-ip=74.125.82.177; envelope-from=[email protected]; helo=mail-ot0-f177.google.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Return-Path: [email protected]
X-EOPAttributedMessage: 0
X-EOPTenantAttributedMessage: c22acb6e-38c1-494b-90f8-7fd2f61bcef3:0
X-Forefront-Antispam-Report: CIP:207.211.31.81;IPV:NLI;CTRY:US;EFV:NLI;
X-Microsoft-Exchange-Diagnostics: 1;CO1NAM03FT055;1:MF3KS23g5Lg9TQGU39WogKxb1K+G1YGQfmj6R69DfA8jjUO2q3iPESEupR78OQl5btfNniBCKSG931TyfoaHCZZ2KOZs/dqpeGWwjMYM/qT4ek4ifAPOaVRQTGqa0NfUlMq/HDZzkDepuG8vAGufusmcijuhABvxHVz5cGeRsScaJqKesxH6SbApx2dXEHfUzH2gXelSxTYra2qzoeXFID6TMWxRnzUyJTwFrvz1OP4ZtAmtzhnpL3jPQSWbMOtmqtxx+7BG6uC9GNDoBahdy1nSVOxYaxgVjd858+h2CeXErnvsZ63+elmLVMDj5cbZXk7KT2TiTKXHN24fNjiZTaDRxaO8Kv+5fdyuHhYt28HPDWXAyYDCaLIuU1En6uBKInqZbKh8tpPyarcU/rVohNk7T1Y1UfjyQtvRwyY2RWNoNqn7c4e2XMpfJHawbWH5o6thdeior00C8G10ESgAUaaJSCqU3udkxox+ufAoo1n78AiBzBbVhjDHQDpkPvO4qyF01BiXTPZSdELNsBh5BluaaeXi+ptBqm67AyLIsjq8NvrLkzDA4OrnQPkMx2gV
X-MS-Office365-Filtering-Correlation-Id: 80ea1e89-0b21-49ac-ba84-08d44921a4a2
X-DkimResult-Test: Failed
X-Microsoft-Antispam:
	UriScan:;BCL:0;PCL:0;RULEID:(22001)(81800161)(3002016)(71702078);SRVR:BLUPR17MB0130;
X-Microsoft-Exchange-Diagnostics:
	1;BLUPR17MB0130;3:MWOkqTGSsk2KEorKXL28tP4QZFAEIznHWhcx2DWtmosz8DKwBm3GdkXA9V4+tIw0DPpnjWTPQ1JrLdLYMeEwD+OyEfz0ymbOuBPWUD9piF5AWUpHTURdaak7NeiSKHbwSu1IVBLmjPhm0LMDSCGM56dw0HtgEv2wYFyKvDGsscJqZZLMK45H1SFb9j8EWGuZf19P56g5/WI4EH/6N/7xXkiw66Qg/UcSBS1At57i+N6kfBVwytBbItL42Lvy1frbBCqPaOHS8GgAIrxi2J62b7d1kQMsZxHjwf6+GDMDQ+CB9XZ2PfGcUoJrP3FRvlLiF5Fj7ZSY9U70DplLCdvjic8gxOUJ1kCXEbZNRyWkr0Pl56qKZ16QHwHGwnUxGJg6ey8KMwO1zq51psdFFawnhXYGbc9YviRqA5Vnx0xwYPN6eX9b0En8DvKFOf0MiDFX;25:woyxyv8NGR9WzfTpi5x2N+vBOtRoOtQA5u/JB73bS6Mi4/HAKB5uMWtWy0HmpAIP7r5O54G10VHsQiPJjcEQgwWoOXCIfzCovLygBPpJyNEYFaQbkwYhJVsP8Jh6T/r36Jd+wezskoFcWVbg4hrtWolt3qwJRqXqFloeZY540o8KCoRt7E/xrOyIJt2ZwwD1IaRWqBw+i6snyV76c2jc3i3qB8esS6hgJFrvrvNfIzYt6wnQYHBeO0c7ywQqXj4QkJKInfKJEllcKwlALbq43Cx7sU201VyGWPUUCIKAs0OQXwqcYWe4s8kQ0pvk4o7Dhse1PDMVkYdw4RUMe5qxXd2OCwLmLV8uRwKij6tH1d2j3FXab3yNAIL6pvRRmbQx9qTSIULXt/67zGL52PfFa9RSz6aeOwvEwKQvNOSGGKouk8nNZLDehqGuJyjB8erec5fe7aSZ/epUR2Gj04F2NA==

Final-Recipient: rfc822;[email protected]kinesissurvey.com
Action: failed
Status: 5.1.10
Diagnostic-Code: smtp;550 5.1.10 RESOLVER.ADR.RecipientNotFound; Recipient not found by SMTP address lookup

———- Forwarded message ———-
From: Luke R <[email protected]>
To: <[email protected]>
Cc:
Date: Mon, 30 Jan 2017 07:06:55 -0800
Subject: collector182804
This message was automatically generated by Gmail.

Netflix Hidden Categories : Full List of Categories

Select A Netflix Hidden Genre

Hit Go to be taken to Netflix

How Does GoogleWebLight Work?

GoogleWebLight, as you may see in your referral traffic as “googleweblight.com” optimizes web traffic for slow 2G / 3G connections. When Google detects a user is browsing the web with a slow connection and they are about to access a content-heavy page, the user will then be redirected to https://googleweblight.com/?lite_url=http://example.com where example.com is the original URL. Google’s support page for this product is here.

See The GoogleWebLight Version Of Any URL [WebLight Tester]:



 

A Quick Way To Shorten Links: lite.website/http://example.com

What is GoogleWebLight? 

“Google shows faster, lighter pages to people searching on slow mobile connections in selected countries. To do this, we transcode (convert) web pages on the fly into a version optimized for slow networks, so that these pages load faster while saving data.”

In a similar way to AMP (Accelerated Mobile Pages), Google is stripping out what it deems as extra data and unnecessary javascript elements to make pages load faster. They also often reduce image size and quality to afford faster loading.

How to prevent your site from being shown in GoogleWebLight Version?
If you do not want your pages to be transcoded, set the HTTP header “Cache-Control: no-transform” in your page response. If Googlebot sees this header, your page will not be transcoded.

Please note that pages that opt-out of being transcoded will be labeled in search results to indicate to users that they may take longer to load and may use more data.

 

Preparing Your Kalm with Kava Concentrate

To prepare your Kalm with Kava Concentrate is quite simple. Just add half the bottle to 8 or so ounces of juice, water, or smoothie. Each bottle is 2 ounces. A serving size is 1 ounce.

For those who don’t have superhuman vision, here are the ingredients in the January 2017 batch of Kalm with Kava Guava Concentrate

Serving Size 1 oz (30 ml) – add to 7oz of water

Amount per Serving % of Daily Value
Servings per container: 2
Calories: 11
Total Carbohydrates: 3 grams / 1% DV
Sugars: 3 grams
Vitamin C: 103 mg / 171% DV
Proprietary Blend: 753mg*

*Daily Value Not Established
*Contains 15% Alcohol from Extraction
Other Ingredients: Natural Flavors, Citric Acid, Sodium Benzoate, Potassium Sorbate, Sucralose, Yellow 6, Red 40

Ingredients List - Kalm with Kava Concentrate
Kalm with Kava (Guava Flavor) Concentrate Ingredients

Manufactured by Monsoon Beverates LLC, Billings, MT
www.KalmwithKava.com

FDA Disclaimer: This product is not intended to diagnose, prevent, treat, or cure any disease and has not been evaluated by the FDA. Excessive Consumption may impair one’s ability to drive or operate heavy equipment. Not recommended for pregnant or nursing women. Ask a healthcare professional before use if you have or have had liver problems, frequently use alcohol, or are taking any medication.

 

On a personal note, I’m quite disappointed this concentrate includes Sodium Benzoate, Potassium Sorbate, Sucralose, Yellow 6 and Red 40. Vitamin E and Vitamin C make great alternative preservatives, sucralose should be replaced by Stevia Extract, and the colorings are completely unnecessary (how about annatto extract for the same color but naturally?)

After raising this concern with Monsoon Beverages, here is their reply:

Hi Luke,
Thanks for your business and feedback! Yes, the current formulation is not ideal and our only product that is not 100% natural. Unfortunately it has proved very difficult to produce the concentrate as 100% natural but I assure you we are working hard at it! Natural colors notoriously have settling issues that affect the appearance of the drink. We’ve tried all the usual options like turmeric, beet root, etc to no avail but have recently made a break-through with some newer natural ingredients. Kava is very unique being a botanical that is hard to work with for shelf stability as well. Citric acid actually changes the effectiveness and stability of certain kavalactones but we have some food scientists looking into other options.
In short, this is an issue that is very important to us and we’re very close to making some changes to get rid of the ingredients you’ve mentioned. Hoping to roll out a new formula in the next couple of months.
Thanks again for your message and please let us know if you have any other comments/questions!
Thank you,
Mike
Kalm with Kava

Great to know they’re working on it!

Weird Character Modifier – ” ͈̮” / A͈̮ B͈̮ C͈̮ D͈̮ E͈̮ F͈̮ G͈̮ H͈̮ I͈̮ J͈̮ K͈̮ L͈̮ M͈̮ N͈̮ O͈̮ P͈̮ Q͈̮ R͈̮ S͈̮ T͈̮ U͈̮ V͈̮ W͈̮ X͈̮ Y͈̮ Z͈̮

The ” ͈̮” character acts as a character modifier inserting itself below the character that immediately precedes it. URL encoded, it’s the 8 byte string “CD88CCAE”

When put together, it looks like a smiley face – as in below the title of this page. CD88 acts as the eyes and CCAE acts as the mouth.

A͈̮ B͈̮ C͈̮ D͈̮ E͈̮ F͈̮ G͈̮ H͈̮ I͈̮ J͈̮ K͈̮ L͈̮ M͈̮ N͈̮ O͈̮ P͈̮ Q͈̮ R͈̮ S͈̮ T͈̮ U͈̮ V͈̮ W͈̮ X͈̮ Y͈̮ Z͈̮

Many different modifiers act in a very similar way, see:

I̳̙͇̜̟̻͇̟̤̘̲̟͚͐ͯ̔̓́ͅ ̯̟͈͎̹̜̫̹̠̣̠̝̙̖̾̆̈̆͂ͪͭͥͪ͆͐h̺̯̤̪͖͉͖̠̮̺̫̙̩͓̟̼̑̎ͩ̍ͬ͋̓̂̎̈ͦ̇̓ͯ̓͒̚̚ͅͅaͭ̓̿͗͐ͪ͑̊͂ͩͣ̚ ̹͚̰̤̮̿v̟̹̗̺͚̗̺̭̠͉̠ͫ͋̅ͭ̈ͤ͐̏̓̔͗̇ͭͫ̀͊̚ĕ͙̟̟̲̎͐̌̓̾ͩ́͛ͪ̄̂͊ͤͪ̚̚ ͕͈̻͛́ͫͪ͌ͭͧ́n̲̯̩̝̦̮̻̦͕̳͉̬̙͖̯̩̘͍͚͐͂̎͌ͦͯ̿ͣ͂͌̍̃͗o̪̩͖͖̱̩̦͎͓̫̲̖̦̩̙̐ͯ͐̈́͐͂ͣ͐̍̒͌ ͔͇ ̖̫̝͓͇͚̣̬̙ͧ̎͌͑ͥ͗̎ḭ̯̳̭̗̜̠͉̻͚̩̘͈̜̰͓̿ͫͪ̋̓̓͐̀̑̈̽̽ḏ̝͖͛̓̆͌̆ͦͮ̽ͩ̐̾̓̚e͓̪͇͐ͮͣ͒͋̓͊͋̌ ̦͓͖̗̻a̼̭͕̠͕̫͚͓̗̻͈͍̜̙̥͔̞ͩ̅̏ͥ̅̓ͮ͒ͯͤ́͌̇̉͗ͦͤ̚ ͔̫̱͔͉̳̪̻̼͖̝͎̩͓̖ͥ͋̂͐̾̂̎͗ͥ͗͂̚h̭̝̯̗̰̜̫̬̦͓͕̜̳͚̓̽͋̏͆̉͌͂ͬͦͮ̄ͧ̒͋́ͅo̗͉͕̦̹͈͉̗̝̠͐̿̋ͅͅ ̩̞̪̼̯̼̹w̺̟̞͇̖̪͓̟̫ͣ̓͋͑͋͒ͭ̾̚ ̠͖͎̹ͥ͂͋͆͂͑͐̀̂͗͆ͤ͋͋ͣ̾͋̓ͅt͙̳̳͓͈́ͯ͒̓͛͐̈ͤo̯̖̼̥̺̙̟̣͔̹̘͌͒̉̀̈́ͬ̿̑ ̻͙̪̥͓͇̞̞͉̥͍̥̮̞̣̻̆̎͌ͫ̾̆͒̆͆ͪͅͅd̯͎̳͙̝̮̝̝͈͈̲͐̎̊̃̏̍ͬ̓ͭ͂o͖͖̖̲̟̾̈́ͮ̏̈̔͑̌́̒̋̿̈́͐̚ ͈̮ ̥̭̙̠̻̘͎̳̣ͯ̉ͤͮ͆̓ͣ̂͌̈́̎ͪ̃͗ͤ͒͋͂̈ẻ̯̯̟̟̥̭͍̳̩̭͔͍̱̗̠͍̠ͪ̿̈́͆͂̆ͥ̓ͣ̊̆ͪ̿̎ͅmͣͤ̍̓̒͐ͨͤ̍̒̽̏ ̙̗͖͍͚̩͖̬̬̹̪͉͖̉ͅō̤͍͔̦̙͗̐̂ͣ̈ͩͪͧͭ͂ͤ̋j͇͇̮͎̟̭́̐ͩͣi̥̜̗̜̅̃̾ͥ̌́͊ͩͤ̄ͣ̾̚s̘̯̥̭̟̥̉ͥ̋̆ͧ ̗͚͍̲̣̭̬̦

Google’s reCAPTCHA Introduces Street Sign Tagging

Rather than selecting all the boxes that contain street signs, Google is introducing a new reCAPTCHA feature to have user’s outline the border of street signs.

boundingbox2

This new captcha system will certainly be difficult for bots to defeat and it will help add to Google’s understanding of what defines a street sign.

Google Street Sign Selection- reCAPTCHA
Google Street Sign Selection- reCAPTCHA

Archive of STORET Data

On 25 Jan 2017, I requested a copy of the entire STORET Data Station Data and Station Results

Request was submitted to https://ofmpub.epa.gov/storpubl/dw_pages.querycriteria

Used default generic search and “Station Download” to download all stations.

Used a search for each state and “Result Download” to download all station information for each state historically.

Response:

Station Information:

Your request for STORET Station Description download is completed via Overnight batch processing.  The Request_ID is 950317.  You can download your file (size : 57302.3 KB) using the hyperlink https://www3.epa.gov/storet/modern/downloads/LSR_20170125_094524.zip [rehmann.co mirror]

Part 2: Making Sense of your STORET Data [pdf]

The data files will remain here for archival purposes.

The zip file contains 3 files:

StationAliases.txt 4MB | 64,138 Lines

Header: Organization ID Station ID Station Name Station Alias Station Alias Type

StationWeights.txt 44.2 MB |  702,574 Lines

Organization ID Station ID Station Name Project ID Location Weighting Factor Location Weighting Factor Unit Location Statistical Stratum Text Location Category Location Status Reference Location Type Reference Location Start Date Reference Location End Date Citation ID

Stations.txt 358.5 MB | 798,964 Lines

Header: Org ID Beach ID/Project ID Station ID Station Name Org Name Primary Type Secondary Type S/G/O Indicator Well Number Well Name Pipe Number NAICS Code Spring Type Improvement Permanence USGS Geologic Unit Code-Name Spring Other Name USGS Lithologic Unit Code-Name Location Point Type Point Sequence Number Point Name Latitude Longitude Horizontal Datum Converted Latitude Converted Longitude Converted Horizontal Datum Geopositioning Method Horizontal Accuracy Map Scale Elevation Elevation Unit Elevation Datum Elevation Method Country Name State County Hydrologic Unit Code Hydrologic Unit Name Generated Hydrologic Unit Code Generated Hydrologic Unit Name RF1 Segment Code RF1 Segment Name RF1 Mileage On Reach Ind NRCS Watershed ID Primary Estuary Secondary Estuary Other Estuary Name Great Lake Name Ocean Name Tribal Land Indicator Natv American Land Name FRS Key Identifier Description Text Well Type Aquifer Name Formation Type Well Hole Depth Measure Well Hole Depth Measure Unit Station Document/Graphic Name Station Document/Graphic URL HUC Twelve Digit Code Generated HUC Twelve Digit Code Last Change Date User ID Last Change Last Trans

 

Station Data:

There were 61 Unique requests sent for over  220,000,000 lines of data. Here are the links to the original and mirror data files. [check back later, still downloading]