Block reference / UI Interaction

Find text (OCR)

Look for text in the picture of the screen, and say whether it is there or tap it.

Only inWarpX Phone

Find text (OCR) reads the screen as a picture, so it finds text that has no element behind it: a game's buttons, a banner, words inside an image. It reads again until the text shows or the time runs out, and can tap it. It is slower than Find text (dump), which should be the first choice whenever the app writes the text as text.

Settings

Text

The text to look for. Capitals and extra spaces do not matter, and it may run over several words.

ExampleContinue

Area

The part of the screen to read, as [left, top, right, bottom] shares of the screen between 0 and 1; empty reads the whole screen. Pick opens the phone's screen so you can draw the box, and the Inspector's Image & text mode has Copy area for the same thing.

Example[0.1, 0.6, 0.9, 0.85]

Language

What the text is written in: Vietnamese and English (the default), Vietnamese or English. Naming only the language on screen reads it a little better.

ExampleEnglish

Then

Only say whether it is there answers true or false. Tap it also taps the middle of where the text was read.

ExampleTap it

Wait up to (ms)

How long to keep reading, from 0 (read once) to 120000. The default is 5000.

Example8000

Fail when it is not found

On, the block fails with "Continue" was not read on the screen when the time runs out. Off (the default), it answers false and goes on.

ExampleChecked

Save the answer (true / false) to a variable

Also stores the answer, true or false, in a workflow variable.

ExampleChecked

Variable namewhen Save the answer (true / false) to a variable is on

The variable to write. Later blocks read it as {{variables.levelDone}}, for example in a Conditions path.

ExamplelevelDone

Delay (ms)

A pause after the block finishes, before the next block starts. Write one number such as 1000, or a range such as 1000,3000 for a random pause in that range on each run.

Example1000

Every block also has the shared block settings — a description, Disable block and what to do on an error. Fields that take text accept data from earlier blocks.

What it hands on

Hands on true when the text was read (and tapped, with Tap it), false when it was not, as {{prevBlockData}} and in the variable when asked. Leaves by its single output either way.

Example: Press a game's Continue button after a level

1. Pick the phone in the editor's toolbar and open the game.
2. Add a Find text (OCR) block with Text Continue and Then Tap it.
3. Click Pick next to Area and draw a box round the lower part of the screen, where the button appears.
4. Set Language to English and Wait up to (ms) to 8000, and tick Fail when it is not found.
5. Press Run at the end of a level: the button is read and tapped.

Good to know

• The text is read on this computer with Tesseract, from a picture of the screen, so no agent is needed. Each read takes a moment, and the first one loads the language data.
• It reads again every half second or so until Wait up to (ms) runs out. A smaller Area reads faster and is less likely to find the words somewhere else on screen.
• The words are matched in reading order, up to twelve in a row, so Level complete is found even when OCR reads the two words as separate pieces.
• Stylised fonts, text over busy pictures and very small text read less reliably. When Tap it misses, try a tighter Area, or Image search with a picture of the button.
• When the block fails, it leaves by its fallback output if one is connected: a phone block comes with On error set to Execute fallback. With nothing connected there, the workflow's On workflow error setting decides.

See also

• Find text (dump) — Look for text among the screen's elements, and say whether it is there or tap it.
• Read text (OCR) — Read the text in an area of the phone's screen, by OCR.
• Image search — Find a picture on the phone's screen, then tap it, wait for it or say whether it is there.