Block reference / UI Interaction

Read text (OCR)

Read the text in an area of the phone's screen, by OCR.

Only inWarpX Phone

Read text (OCR) turns part of the screen's picture into text: a game's coin balance, a code shown in an image, a total drawn by a page the elements do not describe. The text goes to a variable or the table. Text the app writes as text is read faster and exactly by Read element.

Read text (OCR) reads the balance in the top corner as 12480.
The Read text (OCR) block's settings, filled in as in the example below.

Settings

Area

The part of the screen to read, as [left, top, right, bottom] shares of the screen between 0 and 1; empty reads the whole screen. Pick opens the phone's screen so you can draw the box, and the Inspector's Image & text mode has Copy area for the same thing.

Example[0.62, 0.04, 0.95, 0.09]
Language

What the text is written in: Vietnamese and English (the default), Vietnamese or English. Naming only the language on screen reads it a little better.

ExampleEnglish
Only these characters

Limits what can be read to these characters, such as 0123456789 for a number. Empty allows anything.

Example0123456789
Assign to variable

Also stores the text read in a workflow variable.

ExampleChecked
Variable namewhen Assign to variable is on

The variable to write. Later blocks read it as {{variables.coins}}; start the name with $push: to add to a list instead of replacing it.

Examplecoins
Insert to table

Writes the text read into the workflow's table, one row per run of the block. Offered once the workflow's table has columns.

ExampleChecked
Select columnwhen Insert to table is on

The table column to write to. Later blocks read the latest row as {{table.$last.coins}}.

Examplecoins
Delay (ms)

A pause after the block finishes, before the next block starts. Write one number such as 1000, or a range such as 1000,3000 for a random pause in that range on each run.

Example0

Every block also has the shared block settings — a description, Disable block and what to do on an error. Fields that take text accept data from earlier blocks.

What it hands on

Hands on the text read, trimmed, as {{prevBlockData}}, and writes it to the variable and the table column when asked. Lines stay on their own lines. When nothing is read, the text is empty; the block does not fail for that.

Example: Record a game's coin balance

  1. Start
  2. Start app
  3. Read text (OCR)
Where Read text (OCR) sits in the example workflow.
  1. Pick the phone in the editor's toolbar and open the game on the screen that shows the balance.
  2. Add a Read text (OCR) block, click Pick next to Area and draw a box round the number only.
  3. Set Language to English and Only these characters to 0123456789.
  4. Tick Insert to table and pick a coins column.
  5. Press Run, then open Table: the row holds the balance, such as 12480.

Good to know

  • The text is read on this computer with Tesseract, from a picture of the screen, so no agent is needed. The first read loads the language data and takes longer.
  • A tight Area round just the text reads best. Around icons or busy pictures, OCR can add stray characters, which Only these characters helps to keep out.
  • A thousands separator such as the comma in 12,480 is dropped when Only these characters leaves it out.

See also

  • Find text (OCR) — Look for text in the picture of the screen, and say whether it is there or tap it.
  • Read element — Read an element's text, description, id, position or state, or count how many match.
  • Screenshot — Save the phone's screen as a PNG file on this computer.