AutoTools Detect x y coordinates of specific text

Discussion in 'AutoApps' started by Taylan, Dec 20, 2018.

  1. Taylan

    Taylan New Member

    Joined:
    Dec 20, 2018
    Messages:
    5
    Likes Received:
    0
    I tried write an automation for my games in Android with Tasker and AutoTools plugins. Somethings ok at this point but i need capture the screenshot and need interpret it for my needs.

    That's exactly what I need;
    Some texts are important on screen and i want to click on it wherever they are in the screen. So i need OCR for this task i think. I follow some solutions but fail or stuck everytime. Let me explain which solutions i tried.

    Following Solution 1:
    • I tried AutoInput UIQuery method but fail. Because UIQuery of AutoInput just works on android UI i think. Cant get any information from 3D App like games.
    Following Solution 2:
    • I search OCR solution and find AutoTools.
    • Create a task and take screenshot and interpret it with AutoTools OCR method. Thats ok. AutoTools OCR succesfully read a text from Image file.
    • But i stuck again. Because i succesfully read a text from image file but i dont know x y coordinate of important text.
    Question is; Is there any way get the coordinates of each words captured via OCR

    I am open to all advice.
     
  2. joaomgcd

    joaomgcd Administrator Staff Member

    Joined:
    Feb 3, 2015
    Messages:
    9,479
    Likes Received:
    806
    Ok, there was no way to do it, but I've added it to the OCR action in AutoTools! :)
    Can you please try this version? :)
     
    AndreasNYC likes this.
  3. Taylan

    Taylan New Member

    Joined:
    Dec 20, 2018
    Messages:
    5
    Likes Received:
    0
    Hi,

    Thats what exactly i want! Thanks for your help. Do you plan add this function to global market version? Or should i keep this apk file for future use?

    Regards.
     
  4. joaomgcd

    joaomgcd Administrator Staff Member

    Joined:
    Feb 3, 2015
    Messages:
    9,479
    Likes Received:
    806
    Whenever I update the app on Google Play it'll also have this :) So you can update safely. Glad it worked for you!!
     
  5. Taylan

    Taylan New Member

    Joined:
    Dec 20, 2018
    Messages:
    5
    Likes Received:
    0
    Actually i have an another idea for this OCR method. If you like this one you should add it too. Let me explain it;

    * Its a configuration option,
    * Center of scan point: Users can enter x and y position in text box "120 1400" for example.
    * Scanner wide: Users can enter how much pixels should scan horizontally. "100" for example. (50 to left 50 to right of course)
    * Scanner height: Users can enter how much pixels should scan vertically. "100" for example. (50 to up 50 to down of course)
    * So OCR can scan x:70-170 y:1350-1450 space instead of scan whole image.

    Totally up to you. How much options, so much freedom ;)
     
    Last edited: Dec 21, 2018
  6. joaomgcd

    joaomgcd Administrator Staff Member

    Joined:
    Feb 3, 2015
    Messages:
    9,479
    Likes Received:
    806
    Ok! :) Added!

    Check out this version.
    You can define the area you want to scan.

    Let me know if it helps!
     
  7. Taylan

    Taylan New Member

    Joined:
    Dec 20, 2018
    Messages:
    5
    Likes Received:
    0
    Super but little calculating problem i think.

    First scenario: I enter 19,60,147,103 for Area. "Tasker" words at this point. atocrvalue() returns "Tasker" word its ok. But atocrcenterxy() returns "60,23" Android drawer at this point. Actually Tasker words centerxy is "70,83"

    Second scenario: When i remove Area from the configuration words atocrcenterxy() returns correct. For exampe; atocrvalue(1) is "Tasker" and atocrcenterxy(1) is "70,83"

    Also i have added my second scenario task xml.
     

    Attached Files:

    Last edited: Dec 21, 2018
  8. Taylan

    Taylan New Member

    Joined:
    Dec 20, 2018
    Messages:
    5
    Likes Received:
    0
    Also i noticed another strange thing (not releated about Area problem)

    When i interpret with OCR attached screenshot atocrvalue gives me attached txt file results.

    "LBMacro" "textdeneme" "textdeneme2" are looks one array item as you can see.

    Is it problem or wanted thing i dont know. (I think you design this app for Camera capturing)
     

    Attached Files:

Share This Page