Form Recognizer API (v1.0-preview)

Form Recognizer extracts key value pairs and tables from documents and includes the following options:

  • Custom - Extracts information from forms (PDFs and images) into structured data based on a model created by a set of representative training forms. Form Recognizer learns the structure of your forms to intelligently extract text and data. It ingests text from forms, applies machine learning technology to identify keys and tables, and then outputs structured data that includes the relationships within the original file.
  • Prebuilt Receipt - Detects and extracts data from receipts using optical character recognition (OCR) and our receipt model, enabling you to easily extract structured data from receipts such as merchant name, merchant phone number, transaction date, transaction total, and more.


This API is currently available in:

  • Australia East - australiaeast.api.cognitive.microsoft.com
  • Canada Central - canadacentral.api.cognitive.microsoft.com
  • Central US - centralus.api.cognitive.microsoft.com
  • East Asia - eastasia.api.cognitive.microsoft.com
  • East US - eastus.api.cognitive.microsoft.com
  • East US 2 - eastus2.api.cognitive.microsoft.com
  • North Europe - northeurope.api.cognitive.microsoft.com
  • South Central US - southcentralus.api.cognitive.microsoft.com
  • Southeast Asia - southeastasia.api.cognitive.microsoft.com
  • UK South - uksouth.api.cognitive.microsoft.com
  • West Europe - westeurope.api.cognitive.microsoft.com
  • West US 2 - westus2.api.cognitive.microsoft.com

Analyze Form

Extract key-value pairs from a given document. The input document must be of one of the supported content types - 'application/pdf', 'image/jpeg' or 'image/png'. A success response is returned in JSON.

Select the testing console in the region where you created your resource:

Request URL

Request parameters

string

Format - uuid. Model Identifier to analyze the document with.

(optional)
string

An optional list of known keys to extract the values for.

Request headers

string
Media type of the body sent to the API.
string
Subscription key which provides access to this API. Found in your Cognitive Services accounts.

Request body

Response 200

Success

{
  "status": "success",
  "pages": [
    {
      "number": 0,
      "height": 0,
      "width": 0,
      "clusterId": 0,
      "keyValuePairs": [
        {
          "key": [
            {
              "text": "string",
              "boundingBox": [
                0.0
              ],
              "confidence": 0.0
            }
          ],
          "value": [
            {
              "text": "string",
              "boundingBox": [
                0.0
              ],
              "confidence": 0.0
            }
          ]
        }
      ],
      "tables": [
        {
          "id": "string",
          "columns": [
            {
              "header": [
                {
                  "text": "string",
                  "boundingBox": [
                    0.0
                  ],
                  "confidence": 0.0
                }
              ],
              "entries": [
                [
                  {
                    "text": "string",
                    "boundingBox": [
                      0.0
                    ],
                    "confidence": 0.0
                  }
                ]
              ]
            }
          ]
        }
      ]
    }
  ],
  "errors": [
    {
      "errorMessage": "string"
    }
  ]
}
{
  "description": "Analyze API call result.",
  "type": "object",
  "properties": {
    "status": {
      "description": "Status of the analyze operation.",
      "enum": [
        "success",
        "partialSuccess",
        "failure"
      ],
      "type": "string"
    },
    "pages": {
      "description": "Page level information extracted in the analyzed\r\ndocument.",
      "uniqueItems": false,
      "type": "array",
      "items": {
        "description": "Extraction information of a single page in a\r\nwith a document.",
        "type": "object",
        "properties": {
          "number": {
            "format": "int32",
            "description": "Page number.",
            "type": "integer"
          },
          "height": {
            "format": "int32",
            "description": "Height of the page (in pixels).",
            "type": "integer"
          },
          "width": {
            "format": "int32",
            "description": "Width of the page (in pixels).",
            "type": "integer"
          },
          "clusterId": {
            "format": "int32",
            "description": "Cluster identifier.",
            "type": "integer"
          },
          "keyValuePairs": {
            "description": "List of Key-Value pairs extracted from the page.",
            "uniqueItems": false,
            "type": "array",
            "items": {
              "description": "Representation of a key-value pair as a list\r\nof key and value tokens.",
              "type": "object",
              "properties": {
                "key": {
                  "description": "List of tokens for the extracted key in a key-value pair.",
                  "uniqueItems": false,
                  "type": "array",
                  "items": {
                    "description": "Canonical representation of single extracted text.",
                    "type": "object",
                    "properties": {
                      "text": {
                        "description": "String value of the extracted text.",
                        "type": "string"
                      },
                      "boundingBox": {
                        "description": "Bounding box of the extracted text. Represents the\r\nlocation of the extracted text as a pair of\r\ncartesian co-ordinates. The co-ordinate pairs are arranged by\r\ntop-left, top-right, bottom-right and bottom-left endpoints box\r\nwith origin reference from the bottom-left of the page.",
                        "uniqueItems": false,
                        "type": "array",
                        "items": {
                          "format": "double",
                          "type": "number"
                        }
                      },
                      "confidence": {
                        "format": "double",
                        "description": "A measure of accuracy of the extracted text.",
                        "type": "number"
                      }
                    }
                  }
                },
                "value": {
                  "description": "List of tokens for the extracted value in a key-value pair.",
                  "uniqueItems": false,
                  "type": "array",
                  "items": {
                    "description": "Canonical representation of single extracted text.",
                    "type": "object",
                    "properties": {
                      "text": {
                        "description": "String value of the extracted text.",
                        "type": "string"
                      },
                      "boundingBox": {
                        "description": "Bounding box of the extracted text. Represents the\r\nlocation of the extracted text as a pair of\r\ncartesian co-ordinates. The co-ordinate pairs are arranged by\r\ntop-left, top-right, bottom-right and bottom-left endpoints box\r\nwith origin reference from the bottom-left of the page.",
                        "uniqueItems": false,
                        "type": "array",
                        "items": {
                          "format": "double",
                          "type": "number"
                        }
                      },
                      "confidence": {
                        "format": "double",
                        "description": "A measure of accuracy of the extracted text.",
                        "type": "number"
                      }
                    }
                  }
                }
              }
            }
          },
          "tables": {
            "description": "List of Tables and their information extracted from the page.",
            "uniqueItems": false,
            "type": "array",
            "items": {
              "description": "Extraction information about a table\r\ncontained in a page.",
              "type": "object",
              "properties": {
                "id": {
                  "description": "Table identifier.",
                  "type": "string"
                },
                "columns": {
                  "description": "List of columns contained in the table.",
                  "uniqueItems": false,
                  "type": "array",
                  "items": {
                    "description": "Extraction information of a column in\r\na table.",
                    "type": "object",
                    "properties": {
                      "header": {
                        "description": "List of extracted tokens for the column header.",
                        "uniqueItems": false,
                        "type": "array",
                        "items": {
                          "description": "Canonical representation of single extracted text.",
                          "type": "object",
                          "properties": {
                            "text": {
                              "description": "String value of the extracted text.",
                              "type": "string"
                            },
                            "boundingBox": {
                              "description": "Bounding box of the extracted text. Represents the\r\nlocation of the extracted text as a pair of\r\ncartesian co-ordinates. The co-ordinate pairs are arranged by\r\ntop-left, top-right, bottom-right and bottom-left endpoints box\r\nwith origin reference from the bottom-left of the page.",
                              "uniqueItems": false,
                              "type": "array",
                              "items": {
                                "format": "double",
                                "type": "number"
                              }
                            },
                            "confidence": {
                              "format": "double",
                              "description": "A measure of accuracy of the extracted text.",
                              "type": "number"
                            }
                          }
                        }
                      },
                      "entries": {
                        "description": "Extracted text for each cell of a column. Each cell\r\nin the column can have a list of one or more tokens.",
                        "uniqueItems": false,
                        "type": "array",
                        "items": {
                          "uniqueItems": false,
                          "type": "array",
                          "items": {
                            "description": "Canonical representation of single extracted text.",
                            "type": "object",
                            "properties": {
                              "text": {
                                "description": "String value of the extracted text.",
                                "type": "string"
                              },
                              "boundingBox": {
                                "description": "Bounding box of the extracted text. Represents the\r\nlocation of the extracted text as a pair of\r\ncartesian co-ordinates. The co-ordinate pairs are arranged by\r\ntop-left, top-right, bottom-right and bottom-left endpoints box\r\nwith origin reference from the bottom-left of the page.",
                                "uniqueItems": false,
                                "type": "array",
                                "items": {
                                  "format": "double",
                                  "type": "number"
                                }
                              },
                              "confidence": {
                                "format": "double",
                                "description": "A measure of accuracy of the extracted text.",
                                "type": "number"
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    },
    "errors": {
      "description": "List of errors reported during the analyze\r\noperation.",
      "uniqueItems": false,
      "type": "array",
      "items": {
        "description": "Error reported during an operation.",
        "type": "object",
        "properties": {
          "errorMessage": {
            "description": "Message reported during the train operation.",
            "type": "string"
          }
        }
      }
    }
  }
}

Response 500

Response entity accompanying non-successful responses containing additional details about the error.

{
  "error": {
    "code": "string",
    "innerError": {
      "requestId": "string"
    },
    "message": "string"
  }
}
{
  "type": "object",
  "properties": {
    "error": {
      "type": "object",
      "properties": {
        "code": {
          "type": "string"
        },
        "innerError": {
          "type": "object",
          "properties": {
            "requestId": {
              "type": "string"
            }
          }
        },
        "message": {
          "type": "string"
        }
      }
    }
  }
}

Code samples

@ECHO OFF

curl -v -X POST "https://switzerlandwest.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/custom/models/{id}/analyze?keys={string}"
-H "Content-Type: multipart/form-data"
-H "Ocp-Apim-Subscription-Key: {subscription key}"

--data-ascii "{body}" 

using System;
using System.Net.Http.Headers;
using System.Text;
using System.Net.Http;
using System.Web;

namespace CSHttpClientSample
{
    static class Program
    {
        static void Main()
        {
            MakeRequest();
            Console.WriteLine("Hit ENTER to exit...");
            Console.ReadLine();
        }
        
        static async void MakeRequest()
        {
            var client = new HttpClient();
            var queryString = HttpUtility.ParseQueryString(string.Empty);

            // Request headers
            client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "{subscription key}");

            // Request parameters
            queryString["keys"] = "{string}";
            var uri = "https://switzerlandwest.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/custom/models/{id}/analyze?" + queryString;

            HttpResponseMessage response;

            // Request body
            byte[] byteData = Encoding.UTF8.GetBytes("{body}");

            using (var content = new ByteArrayContent(byteData))
            {
               content.Headers.ContentType = new MediaTypeHeaderValue("< your content type, i.e. application/json >");
               response = await client.PostAsync(uri, content);
            }

        }
    }
}	
// // This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
import java.net.URI;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

public class JavaSample 
{
    public static void main(String[] args) 
    {
        HttpClient httpclient = HttpClients.createDefault();

        try
        {
            URIBuilder builder = new URIBuilder("https://switzerlandwest.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/custom/models/{id}/analyze");

            builder.setParameter("keys", "{string}");

            URI uri = builder.build();
            HttpPost request = new HttpPost(uri);
            request.setHeader("Content-Type", "multipart/form-data");
            request.setHeader("Ocp-Apim-Subscription-Key", "{subscription key}");


            // Request body
            StringEntity reqEntity = new StringEntity("{body}");
            request.setEntity(reqEntity);

            HttpResponse response = httpclient.execute(request);
            HttpEntity entity = response.getEntity();

            if (entity != null) 
            {
                System.out.println(EntityUtils.toString(entity));
            }
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }
}


<!DOCTYPE html>
<html>
<head>
    <title>JSSample</title>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
</head>
<body>

<script type="text/javascript">
    $(function() {
        var params = {
            // Request parameters
            "keys": "{string}",
        };
      
        $.ajax({
            url: "https://switzerlandwest.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/custom/models/{id}/analyze?" + $.param(params),
            beforeSend: function(xhrObj){
                // Request headers
                xhrObj.setRequestHeader("Content-Type","multipart/form-data");
                xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key","{subscription key}");
            },
            type: "POST",
            // Request body
            data: "{body}",
        })
        .done(function(data) {
            alert("success");
        })
        .fail(function() {
            alert("error");
        });
    });
</script>
</body>
</html>

#import <Foundation/Foundation.h>

int main(int argc, const char * argv[])
{
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    
    NSString* path = @"https://switzerlandwest.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/custom/models/{id}/analyze";
    NSArray* array = @[
                         // Request parameters
                         @"entities=true",
                         @"keys={string}",
                      ];
    
    NSString* string = [array componentsJoinedByString:@"&"];
    path = [path stringByAppendingFormat:@"?%@", string];

    NSLog(@"%@", path);

    NSMutableURLRequest* _request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:path]];
    [_request setHTTPMethod:@"POST"];
    // Request headers
    [_request setValue:@"multipart/form-data" forHTTPHeaderField:@"Content-Type"];
    [_request setValue:@"{subscription key}" forHTTPHeaderField:@"Ocp-Apim-Subscription-Key"];
    // Request body
    [_request setHTTPBody:[@"{body}" dataUsingEncoding:NSUTF8StringEncoding]];
    
    NSURLResponse *response = nil;
    NSError *error = nil;
    NSData* _connectionData = [NSURLConnection sendSynchronousRequest:_request returningResponse:&response error:&error];

    if (nil != error)
    {
        NSLog(@"Error: %@", error);
    }
    else
    {
        NSError* error = nil;
        NSMutableDictionary* json = nil;
        NSString* dataString = [[NSString alloc] initWithData:_connectionData encoding:NSUTF8StringEncoding];
        NSLog(@"%@", dataString);
        
        if (nil != _connectionData)
        {
            json = [NSJSONSerialization JSONObjectWithData:_connectionData options:NSJSONReadingMutableContainers error:&error];
        }
        
        if (error || !json)
        {
            NSLog(@"Could not parse loaded json with error:%@", error);
        }
        
        NSLog(@"%@", json);
        _connectionData = nil;
    }
    
    [pool drain];

    return 0;
}

<?php
// This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
require_once 'HTTP/Request2.php';

$request = new Http_Request2('https://switzerlandwest.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/custom/models/{id}/analyze');
$url = $request->getUrl();

$headers = array(
    // Request headers
    'Content-Type' => 'multipart/form-data',
    'Ocp-Apim-Subscription-Key' => '{subscription key}',
);

$request->setHeader($headers);

$parameters = array(
    // Request parameters
    'keys' => '{string}',
);

$url->setQueryVariables($parameters);

$request->setMethod(HTTP_Request2::METHOD_POST);

// Request body
$request->setBody("{body}");

try
{
    $response = $request->send();
    echo $response->getBody();
}
catch (HttpException $ex)
{
    echo $ex;
}

?>
########### Python 2.7 #############
import httplib, urllib, base64

headers = {
    # Request headers
    'Content-Type': 'multipart/form-data',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.urlencode({
    # Request parameters
    'keys': '{string}',
})

try:
    conn = httplib.HTTPSConnection('switzerlandwest.api.cognitive.microsoft.com')
    conn.request("POST", "/formrecognizer/v1.0-preview/custom/models/{id}/analyze?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################

########### Python 3.2 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64

headers = {
    # Request headers
    'Content-Type': 'multipart/form-data',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.parse.urlencode({
    # Request parameters
    'keys': '{string}',
})

try:
    conn = http.client.HTTPSConnection('switzerlandwest.api.cognitive.microsoft.com')
    conn.request("POST", "/formrecognizer/v1.0-preview/custom/models/{id}/analyze?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################
require 'net/http'

uri = URI('https://switzerlandwest.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/custom/models/{id}/analyze')

query = URI.encode_www_form({
    # Request parameters
    'keys' => '{string}'
})
if query.length > 0
  if uri.query && uri.query.length > 0
    uri.query += '&' + query
  else
    uri.query = query
  end
end

request = Net::HTTP::Post.new(uri.request_uri)
# Request headers
request['Content-Type'] = 'multipart/form-data'
# Request headers
request['Ocp-Apim-Subscription-Key'] = '{subscription key}'
# Request body
request.body = "{body}"

response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
    http.request(request)
end

puts response.body