Introducing: Private SmartScan on Amazon Mechanical Turk

David Barrett —  November 25, 2017 — Leave a comment

I had hoped to keep this feature quiet until next year, but it seems some enterprising sleuthers have beaten us to the punch. Alas! So with the cat out of the bag, let me proudly announce a new privacy-enhancing feature we’ve been focused on for some time: Private SmartScan! This puts you in control of exactly which humans step in when technology alone is insufficient.

As background, SmartScan is the patented, award-winning technology that underpins our “fire and forget” design for expense management. When you get a receipt, rather than stuffing it into your pocket to dread for later, just:

  1. Take your phone out of your pocket
  2. SmartScan the receipt
  3. Put your phone back in your pocket

SmartScan is the multi-layered technology that reads all the information off your receipt, verifies that it is compliant with your company’s rules, and then puts it on the right expense report for you — automatically — without any manual effort on your part. It’s one of the things that makes Expensify so easy to use, and so much faster than doing it yourself by hand.

As for how that system works, it’s a combination of many different technologies that all attempt to extract the information from the receipt automatically. But the key to making it accurate enough to enable this “realtime expense report” flow is what happens when the technology fails: rather than kicking it back to you to manually enter, SmartScan has a team of human transcription agents standing by to do the typing for you. Because even if the technology fails some of the time, the Expensify experience is automated for you all of the time — for every receipt, under every weird lighting condition.

As you might imagine, doing this is easier said than done. Given the enormous scale and 24/7 nature of this task, we have agents positioned around the world to hand off this volume from timezone to timezone. Most of the US team is located in Ironwood, MI or Portland, OR (where we have offices and can train in person). Most of the international team is in Nepal or Honduras (where we work with a third-party provider to manage the on-site logistics). But regardless of the location, every single agent is bound by a confidentiality agreement, and subject to severe repercussions if that agreement is broken.

That said, there are companies and industries (especially in defense) that require an even stronger level of control. Accordingly, we’re very happy to reveal a new feature that enables organizations to take direct control of the privacy and security concerns of human transcription. We call it “Private SmartScan”, where we open up our proprietary tools to your company to staff your own team of human transcription agents anywhere in the world.

Though our own agents work on a different set of internal tools, we have built an interface atop Amazon Mechanical Turk to enable your private team to execute the human portion of SmartScan for your company’s receipts. It works like this:

  1. You hire a 24/7 team of human transcription agents.
    • For the fastest processing we suggest staffing three separate shifts — or daytime shifts in three different offices around the world. Otherwise your receipts might lag for many hours before getting processed.
  2. They apply to Amazon Mechanical Turk for an account. Be aware that this is a surprisingly involved process, including:
    • The agent must sign up using their actual personal Amazon account. If your account doesn’t have an adequate history of purchases (each of which implies a successful credit card billing transaction and package delivery) or other activity, you will be rejected.
    • The agent must provide their full name, address, and bank account information for reimbursement. Amazon verifies this with a variety of techniques (eg, confirm that your IP is in the country you say you are, verify the bank account is owned by the name and address provided, full criminal background check), and if anything doesn’t add up, you will be rejected.
    • Rejection is final. It requires such an abundance of verifiable documentation (most notably being an active Amazon account with a long history) that you can’t just create a new account and try again.
    • There is no apparent appeals process. Accordingly, I would recommend confirming before hiring that the candidate can pass Amazon Mechanical Turk’s many strict controls because we have no ability to override their judgement.
  3. You notify us of the “workerID” of each of your authorized agents.
    • Though you are not obligated to share your staff’s identity with us directly, your staff will still be obligated to follow the Expensify terms of services. Failure to comply with our terms will result in an appropriate response, starting with immediate banning by our automated systems, ranging up to our legal team subpoenaing you (or failing that, Amazon) for the identity of the agent to press charges directly.
  4. We will create a “Qualification” for your “Human Intelligence Tasks” (HITs) that ensures only your agents will see your receipts.
  5. Your staff will use the Amazon Mechanical Turk interface to discover and process your employee’s receipts.

This feature is only in the early stages of testing right now, and will not be available until next year. (We hadn’t intended to reveal this feature quite yet, so… surprise!) Additionally, given the high overhead of this feature, it will be reserved for our Enterprise Plan customers. If you are interested in learning more, please speak with your sales rep to get on the waitlist to receive additional information about functionality and pricing as it becomes available.

In the meantime, Expensify will continue to provide our own staff of transcription agents included in the base charge, and we expect the overwhelming majority of customers will continue finding this the fastest, most accurate, and most cost-effective method of scanning receipts on the planet. But privacy is a complex topic, and if for any reason you have doubts regarding our agents or your own Private SmartScan staff, always remember that SmartScan is an entirely optional feature that you can disable in your personal settings.

That’s all for now, and I can’t wait to share more details soon!

David Barrett


Founder of Expensify, destroyer of expense reports, and savior to frustrated employees worldwide.

No Comments

Be the first to start the conversation!

Have something to say? Share your thoughts with us!

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s