Nice write up! Congrats on getting this far with software.

I’m impressed that Intuit has put engineer behind this complex piece of work. As an engineer myself I appreciate the work required to do this. Most companies will opt in for using offshore humans in the loop/labor (Expensify & Bill.com using CloudFactory) which puts US customer PII at risk of social engineering. CCPA 2020 data privacy regulations will impact those using offshore labor. So nice work there. Hats off to Intuit leadership and engineering for not following the crowd.

Now onto the tech detail in your article. The tech details tell me that you are using off-the-shelf NLP approaches which unfortunately don’t work well for these type of docs.

For example; you are using BERT (presumably) pre-trained on lot’s of normal texts (books, internet dump etc). It knows a lot about all but very little about financial docs. This means you will need to rely on rules to correct predictions like what constitutes a date or a zip.

You report high per token accuracy (~93% overall accuracy across all token classes), but that’s a bit misleading since a field is often multiword.

There are few other areas including your use of CNN but that’s probably best left for an offline chat ;-)