Início

  • Top 8 Python Frameworks to develop RPA

    Top 8 Python Frameworks to develop RPA

    Python is the best solution for tech teams to develop RPAs. Python is an open technology, cross-platform, free to use, and supported by major technology companies (lots of SDKs and libraries). We have Python libraries for almost all automation use cases. Tech teams go faster with Python, delivering more robust and reliable solutions.

    This post lists the Top 8 Python frameworks to automate desktop, web, legacy, and mobile applications.

    1. Selenium (Web, webdriver)

    Selenium was initially released in 2004 and is probably the most used framework for web automation. The project supports multiple browsers and all major programming languages, including Python.

    License: Apache 2.0
    Site: https://selenium-python.readthedocs.io/

    2. Pyppeteer (Web, chromium)

    Python port of Puppeteer Javascript chrome/chromium browser automation library. It is a framework created specifically for web automation on Chrome browser. It uses Chrome DevTools Protocol under the covers.

    License: MIT License
    Site: https://github.com/pyppeteer/pyppeteer

    3. BotCity Core (Desktop, Mobile and Legacy)

    BotCity Core is an open-source automation framework that combines multiple technologies, such as UI manipulations via Ids, computer vision (Open CV), and other strategies. BotCity project provides tools to generate codes automatically as the developer specifies the actions in the target application UI. It is cross-platform, can operate any operating system and browser.

    License: Apache 2.0
    Site: https://github.com/botcity-dev/botcity-framework-core-python

    4. BotCity Web (Web, supports Computer Vision)

    BotCity Web is a web automation framework that combines Selenium with Computer Vision for interacting with web UI. Developers can choose the best approach to develop UI automation, combining component selection and manipulation by Id, XPath, or its visual appearance. The automations run in headless mode even when using computer vision.

    License: Apache 2.0
    Site: https://github.com/botcity-dev/botcity-framework-web-python

    5. PyWinAuto (Windows applications)

    PyWinAuto combines Python modules to allow developers to send mouse and keyboard actions to windows applications. It uses Microsoft Windows API to access and manipulate dialogs and controls.

    License: BSD 3-Clause
    Site: https://pywinauto.readthedocs.io/en/latest/

    6. PyAutoGui (Mouse and keyboard control)

    Python library to programmatically control the mouse and keyboard. It works on Windows, macOS and Linux and runs on Pyhton 2 and 3.

    License: BSD 3-Clause
    Site: https://pyautogui.readthedocs.io/en/latest/

    7. AutoPy

    AutoPy is another library for controlling the keyboard and mouse. It also allows finding colors and bitmaps on-screen to choose UI components for interaction. Currently supported on macOS, Windows, and X11 with the XTest extension.

    License: Apache 2.0
    Site: https://github.com/autopilot-rs/autopy

    8. Appium (Mobile)

    Appium Python Client can be used to automate mobile applications using WebDriver protocol and Appium commands. Like many other automation frameworks, its primary purpose is software testing, but it can also be used to develop RPA modules.

    License: Apache 2.0
    Site: https://github.com/appium/python-client

    PLUS – RPA Orchestration

    Developing your bots is just one step in delivering and maintaining RPA in your organization. RPA Orchestration is crucial to monitor, improve and scale your operation.

    BotCity Maestro is a cloud-based orchestrator for your RPAs. Easy deployment (VMs or containers on AWS, Azure, GCP, or on-premise), task queue, error handling, alerts, logs, result files, and full integration and control over API.

  • Python RPA → Our message at Web Summit 2022

    Python RPA → Our message at Web Summit 2022

    RPA projects are closer to ordinary software than anything else. It has requirements, data validation, business logic, testing, integrations, etc.

    Business users should be stakeholders, not developers!

    No more shadow it, prototypes that do not scale, and solutions stuck in proprietary formats and proprietary runtime environments.

    Python RPA community is trying to solve this mess, proposing best practices and guidelines to deliver robust, reliable, scalable automation across business units.

    Let’s embrace open technologies, orchestration, governance, elastic computing and the best practices in software engineering.

    Get involved in this movement.
    We’re BotCity, and we’ll deliver this message at Web Summit.

    Find us at our Booth on AI & Machine Learning Alley on November 2nd.

  • BotCity Maestro Updates

    BotCity Maestro Updates

    👋 Hello!

    We are excited to announce a new set of developments that our team has been working on over the last two weeks.

    Our cloud orchestrator BotCity Maestro received major user experience improvements and new features!

    Over the next sections we will cover what changed and guide you through the new features! I hope you enjoy it!

    Simplified Sign Up

    Do you have an e-mail and a password in mind? Great! That’s all you need!

    New sign up form

    New Layouts

    From the new login and password recovery screens all the way to a major product redesign we refreshed BotCity Maestro design and worked to make it more functional and pleasant to work with.

    Login screen

    Your door into BotCity Maestro now not only looks better but it is also more functional and bring you easy access to our community channels.

    New login screen

    The password recovery screen also received a visual update and bug fixes.

    Password recovery screen

    New Home Screen

    The redesigned home screen now features four cards allowing you quick access to download the BotCity Studio SDK, our Documentation Portal, BotCity Academy and the BotCity Maestro.

    BotCity Maestro Home screen

    Log Management

    The log management was only available via the BotCity Command Line Interface (CLI) or via SDK. Logs visualization and download were the only available options via the BotCity Maestro portal.

    Today we are glad to announce the availability to not only visualize and download but also create and exclude logs directly from the BotCity Maestro interface.

    Clicking on the New Log button will take you to the Log creation screen where you can define the Log label, which is the unique ID to be used when interacting with this Log via the BotCity Maestro SDK or API as well as the columns.

    Log creation screen

    Another major change in regards to logs is that they are no longer limited or tied to a single Automation. Now you can create as many logs as you desire to better organize your data and automation management.

    Try it out now!

    Runner Management

    We recently updated our terminology with the intent to simplify the understanding of what Machines are. Recent updates on the BotCity Maestro replaced all occurrences of the term Machine with the more appropriate term Runner.

    To remember, a Runner is a service created by BotCity which runs on a host (computer, virtual machine, cloud-based host, container or some other form). A host can have one or more BotCity Runners being executed in parallel depending on the use case and hardware capabilities.

    In the past we had to create Runners via the BotCity CLI using a command like the one describe below:

    BotCLI machine new -machineId vm01

    Now you can create Runner instances directly from the BotCity Maestro interface by clicking on the New Runner button.

    New runner screen

    The Runner card contains the other operations that you can perform:

    • View instance information
    • View Runner instance log file
    • Capture Screenshot
    • Edit the instance
    • Remove the instance
    Runner card with latest screenshot and action menus

    Bot Management

    Another great change that we are proud to present is the new feature to manage Bots via the BotCity Maestro interface.

    Before this feature was added, the only way to handle the lifecycle of a Bot (Deploy, Release and Update) was via our command line interface and exclusion of bots was not allowed.

    Starting now, you will be able to perform all those operations directly from the BotCity Maestro user interface with a couple of clicks.

    From the new My Bots menu you can now deploy new bots and versions, mark versions of bots as released, update the bot code and exclude them.

    Here is how to do it.

    Deploy

    Deploy is the process used to add new bots or new versions of existing bots on the BotCity Maestro platform.

    You can deploy a bot by simply clicking on the Deploy button and filling the form with the Bot ID, version, file and technology used. Very similar to the syntax of the command line which was:

    BotCLI bot deploy -botId demoWeb -version 1.0 -file demoWeb-1.0.tar.gz -python
    Deploying a new Bot using the BotCity Maestro interface

    Update

    Update is the process used to overwrite or replace the code or executable that a given bot is associated with. Please note that this process is irreversible and will overwrite an existing version even if it is marked as release so please be careful.

    You can update a bot by simply clicking on the pencil icon on the right side of the line associated with your desired bot version and selecting a new file to replace the current one. That is much simpler than the command line which was:

    BotCLI bot update -botId demoWeb -version 1.0 -file demoWeb-1.0.tar.gz -python
    Updating an existing Bot using the BotCity Maestro interface

    Release

    Release is the process used to mark which of the bot available versions is the one currently being used when executing the tasks. This version will be downloaded by the BotCity Runner and executed on your host.

    You can define the released version by simply clicking on the ribbon icon (highlighted by the red arrow below).

    Marking a version as the Release version

    The equivalent procedure via the command line interface was:

    BotCLI bot release -botId demoWeb -version 2.0

    Delete

    A new feature! To remove a deployed version, you can simply click on the trash can icon (highlighted by the red arrow) and confirm on the confirmation dialog that will be displayed.

    Deleting a deployed bot version

    Create your Account

    Don’t have a BotCity Account? Sign up for Free

    More on the Way

    We hope you are enjoying the new features as much as we are while developing and using them.

    As always, if you have questions or find any sort of issues or would like to suggest a feature, feel free to reach out to us via one of our community channels.

    I won’t give you spoilers but there are many cool features coming up over the next weeks! 🚀

    Keep an eye out and make sure to follow us on the social media channels so you can be notified when we release them!

    Have fun automating! 🦾🤖

  • Python RPA + Orchestration – Here it comes.

    Python RPA + Orchestration – Here it comes.

    There is a growing trend for developing RPA solutions as ordinary software using traditional programming languages like Python. Citizen developers are primarily automating their basic day-to-day tasks. When automating complex processes in the core of the company’s operations that involve multiple employees, a technical squad with experts in automation and integrations is mandatory. As technical squads are involved in such a process, choosing powerful programming languages like Python instead of a low-code solution has become more common.

    Benefits of using Python to Develop RPAs

    There are many benefits of using programming languages like Python to develop your RPAs:

    • Use thousands of open source frameworks for automation-related tasks.
    • Easy to reuse your solutions through modularization (i.e., creating libraries)
    • Automations developed in open technology instead of proprietary format.
    • Adopting software engineering best practices like Design Patterns, Refactoring, Automated Tests, CI/CD.
    • Customize your technology stack based on your needs.
    • Best use of computational resources through software optimization.

    Orchestrating Python RPAs

    However, the development of RPA automation is just one step in delivering a solution to production since you must be able to:

    • Deploy your automations into runtime environments.
    • Schedule the executions of your automations.
    • Manage tasks in Queues
    • Monitor executions.
    • Trigger alerts and notifications.
    • Handle errors immediately.

    In order to have all these features in your RPA operation, you have to use an Automation Orchestrator. BotCity Maestro is the first RPA Orchestrator created for RPAs developed as ordinary software. Therefore, it’s easy to use via SDK or API directly in your RPA code.

    BotCity Maestro Orchestrator

    BotCity developed a Cloud Orchestrator that addresses all the issues described in the previous section. Now you can deploy and orchestrate your Python RPAs using Command Line Interface (CLI), APIs and web platform.

    Home Screen

    On the Home Screen, a dashboard is shown displaying basic information, so now you have big picture of your entire operation.

    BotCity Maestro Home Screen

    Tasks

    The Task Queue shows tasks in execution, ready to be executed and finished. Each card represents a single task. The color in the bottom bar indicates the state of the task.

    BotCity Maestro Task Queue

    Using the New Task feature, you can create a new task for a specific activity directly from the portal:

    BotCity Maestro New Task Module.

    You might also create tasks through API or CLI. Just a single HTTP Post or CLI command and a new task is added to the Queue. Runtime environments get the tasks from the Orchestrator and execute them automatically.

    Clicking on the task card you can see the task details including its properties, execution errors, alerts or files uploaded during the execution, as shown below:

    Task Details

    Easy Deployment

    Maestro Orchestrator provides a simple 3-step deployment process:

    1. Automation Name

    2. Bot Deployment File and Properties

    3. Runtime Environment

    Error Handling

    The development of RPAs mainly involves third-party systems that we do not have any control over. These systems might be updated at any time, changing UI behavior and breaking our automations. Time response to changes in systems is crucial in RPA maintenance.

    Maestro Orchestrator provides a simple and powerful method to monitor and react faster to errors in automation execution. A single line of Python code sends errors to Maestro while handling exceptions in Python. Developers are notified, and they can check error messages, stack trace, a screenshot, and other information in a single screen:

    Error Form

    Logs

    Logs are a compelling way to track the execution of your automation and collect metrics. You can set different columns for each log table and log information in real-time from a single Python command.

    BotCity Maestro Logs Module.

    Alerts

    When we have multiple automations running at the same time, it is a challenge to visualize the entire operation. Alerts are used to provide small messages that can describe some specific aspect of given automation. Just a single line of Python code and it is done.

    BotCity Maestro Alerts Module.

    Runtime Environments

    Machines are runtime environments used to execute automations. It can be a virtual machine, a container or even a physical machine. It is a computing resource for execution. This module lets you visualize and manage such environments.

    BotCity Maestro Runtimes Module.

    Dashboards

    When it comes to monitoring and managing a complex operation with multiple automations, it is often necessary to have a dashboard that shows the status and critical KPIs. Using BotCity Connectors you can to bring your RPA statistics into Google Data Studio and Power BI.

    BotCity Orchestrator Dashboard Module

    Free Version

    BotCity Maestro Community lets you explore the orchestration of Python RPAs for free. Just signup for free and start using it today.

  • Free Python Automation Course

    Free Python Automation Course

    BotCity Academy is Online 🎓

    We’re pleased to announce that BotCity Academy is online. 🎉

    It is a platform to learn how to develop and manage automations using open source and BotCity. It is totally free.

    The courses are going to provide video lessons, articles, examples, challenges and other references to teach concepts related to automation ops.

    Get Started with BotCity Python

    In our first course, we provide an introduction to BotCity’s technological stack, explain how to setup your development environment and create your first automation through the following lessons:

    • Welcome
    • BotCity
    • Download BotCity Studio SDK
    • Development Environment Setup
    • Hello Bot
    • Desktop Automations
    • Web Automations
    • PDF Processing

    If you don’t have a BotCity account yet, just sign up for free now.

    After login, simply click on the BotCity Academy card to start your learning journey.

  • BotCity Maestro and Documentation Portal

    BotCity Maestro and Documentation Portal

    👋 Hello!

    We are excited to announce two major developments that our team has been working on over the last two months.

    Our cloud orchestrator BotCity Maestro received a major architectural, performance upgrades and a handful of new features.

    Our Developers Portal was merged with our various API reference websites giving origin to our new unified Documentation Portal as to create a place for everything BotCity!

    BotCity Maestro

    From the signup to orchestration of your tasks and dynamic dashboards, everything was rebuilt as to bring you a more modern, performant and robust solution. All that with 100% retro compatibility with existing SDK and API code.

    The experience continue the same, but we have some important new features:

    Login using your e-mail and password

    Here the major change was the replacement of the Login field with the E-mail. Now instead of your login just simply type your e-mail and BotCity Maestro password to access your organization workspace.

    New BotCity Login panel

    Task Queue Performance and Live Update

    Fast access to information is paramount when orchestrating automations and other processes.

    We realized that our Task Queue page started to present some delays for intense operations where thousands of tasks were created every day. With that in mind, our team refactored the UI and the backend to make sure the information about your tasks is available almost instantaneously.

    Moreover, with the previous version users were required to constantly reload the page to obtain newer information about the Task Queue and verify if a task that was queued had start processing, if it was finished and which finishing status it had. Now the BotCity Maestro’s task queue page will automatically update in a reactive format and update your task cards as soon as the status of the task changes. 🚀

    Give it a try and let us know what you think.

    Task Queue

    New Tasks – Directly from the UI

    Up to now, most users created tasks with either the BotCity Command-line Interface (CLI) or via the BotCity Maestro SDK. The “New Task” menu was not generally available.

    We are glad to introduce now the exciting dynamic “New Task” feature.

    Using the New Task feature, you can create a new task for a specific activity directly from the portal. New Task

    Creating a new task is as simple as clicking on the New Task button and confirming the action on the subsequent screen.

    For activities where parameters are involved, the New Task screen will be dynamically construct the proper form with the best components for each field and data type associated. 

    New Task

    Now operators can simply login into BotCity Maestro and create tasks with or without parameters with ease.

    This is the first version of this feature and it will soon receive updates.

    Give it a try now and let us know if you have suggestions. We are excited to hear what you think.

    Alerts Live Update

    For enhanced experience while monitoring your operation, now the Alerts page will automatically load new alerts as they are generated on the platform. This means that you no longer need to reload the page to receive new notifications.

    Other Improvements

    Many other improvements were introduced with this new version, mainly architectural changes for enhanced performance, availability and robustness.

    The Log and Result Files features received a new pagination feature and performance improvements.

    Documentation Portal

    Now you no longer need to bookmark a bunch of links or browse around to find related information about BotCity’s command line interface, a feature at BotCity Maestro or even the Java and Python open-source frameworks.

    We got you covered with our new unified documentation portal.

    Documentation Portal

    This new Documentation Portal is a living organism and it will receive constant updates for APIs, Plugins, Features, Tutorials and much more!

    No need to bookmark it as we made the URL super easy to remember… it is:

    https://documentation.botcity.dev

    Hello, Hola, Olá, こんにちは, Ciao, Hallo, 你好, i18N 🌐

    We are adding Internationalization support to our documentation portal!

    You can select your preferred language via the top menu icon near the search bar:

    As of now, the core language of our documentation is English but our team is working hard to add support to more languages starting with Portuguese.

    As a rule of thumb, if a content is not yet available at your language of choice the system will automatically fallback to English.

    More on the Way

    We hope you are enjoying the new features as much as we are while developing and using them.

    As always, if you have questions or find any sort of issues or would like to suggest a feature, feel free to reach out to us via one of our community channels.

    Have fun automating! 🦾🤖

  • RPA Iceberg – The challenges go beyond automation

    RPA Iceberg – The challenges go beyond automation

    ➡ RPA priority is growing in companies, now what? The challenges are different 👇🤖

    From the results of the “State of RPA” survey, conducted by BotCity with more than 200 companies, we saw that RPA is gaining scale and priority in companies’ strategies:

    ✅ 90% of the companies had an increase of more than 25% in the amount of automation compared to last year;

    ✅ 88% of the companies have exited the pilot and are in the process of scaling up the automations;

    ✅ 66% of companies plan to increase investments in APR in the next 2 years;

    📌 As RPA initiatives gain scale, different areas start to have more automation in critical processes, with more complexity and bringing results; the challenges become other, and the focus turns to productivity, optimization, and governance of the operation.

    📌 The development of RPA automations is just one step in delivering a solution into production. Python RPA projects tend to scale quickly. It is paramount to obtain governance over shadow IT, preserving the agility of the operation with the help of guidelines and tools that facilitate the orchestration of the automations.

    📌 BotCity Maestro is an Enterprise Orchestrator that is designed to serve Python RPA initiatives in complex environments. The platform facilitates the governance of the RPA operation by being able to manage deploys (VMs, containers, serverless), schedule and chain triggers, execution queues, monitor logs and dashboards, alerts, user management, bot versioning, among other features. The Community version is free 😀

    💡 If you or someone on your team needs to master Python Automations with Orchestration, check out our free training at BotCity Academy 🤖😉

  • 3 key advantages of a multi-platform architecture in RPA

    3 key advantages of a multi-platform architecture in RPA

    More than half of RPA leaders (56%, “RPA in Latin America” survey conducted by BotCity) use more than one automation platform, segmenting their architecture into modules with the most suitable options for each specialty.

    This type of multi-vendor architecture provides 3 key advantages:

    ✅ Avoids lock-in with the same supplier;

    ✅ Increases bargaining power in the acquisition of licenses;

    ✅ Provides better solutions for each use case;

    With the majority (56%) of companies adopting a multi-platform architecture and using Python (79%) in their automations, we observe a trend toward the adoption of non-proprietary technologies and open APIs in RPA stacks, to facilitate the integration with the diversity of projects present in the companies and favor the cooperation between the automation, data science/data engineering and integration teams, and compose a technological stack capable of supporting these different initiatives.

  • How to prevent your automations from breaking in RPA

    How to prevent your automations from breaking in RPA

    ➡️ In RPA, we know that automations will stop working. It will happen; it is a fact. Learn about the leading causes, how to avoid them, and make your bots more resilient 🤖👇

    In companies, we see that essential and complex processes are 100% automated. If they stop working, they can significantly impact the business areas, which are no longer prepared to meet these demands manually.

    📌 Learn about the leading causes of breakage in robots:

    1. Application Updates: RPA bots interact with systems that, if they undergo changes in their interfaces, may fail;

    2. Environment Problems: Bots can be sensitive to server and network configurations, resolution, fonts, positioning, limits on systems capacity, and so on;

    3. Exceptions and Errors: When encountering exceptions or errors, bots can fail if not handled correctly;

    4. Data Variability: If the input or output data format changes, the bot logic or processing may fail;

    5. Process Changes: If processes undergo significant changes, the bot logic may become incompatible;

    6. Licensing and Expiration: RPA software often has licenses or subscriptions, which, if they become invalid, the bot will stop working;

    7. Human Errors and Attacks: Errors in configuration, logic or development, or security attacks can cause failures;

    ✅ To avoid problems and make your automations more resilient, follow these practices:

    Error and exception handling: Implement comprehensive mechanisms, including potential scenarios and actions, such as retries or alternate paths;

    2. Automate by IDs / XPath and use APIs (if any): interfaces change frequently, and computer vision is sensitive to changes in resolution, fonts, and so on;

    3. Monitoring: use a tool that provides real-time indicators, complete logs, and preventive notifications to anticipate problems, interface failures, bottlenecks, and resource escalation;

    4. Scalability: Make sure the infrastructure can handle the increased workload and can scale quickly to accommodate more extensive processes or transaction volumes;

    5. Maintenance, Updates, and Security: Perform regular maintenance and updates to your RPAs, ensuring compatibility, license and certificate renewals, more secure versions, and application of best security practices;

    6. Testing: validate changes before they go into production, do automated, unit, and integration testing, and release incrementally;

    7. Training and documentation: Train your teams regularly and ensure complete and accessible documentation;

    8. Stability of the environment: Avoid unnecessary changes and identify dependencies that may impact it. Coordinate with teams to minimize unavailability;

    9. Version control and Backup: Implement a backup and recovery strategy and make sure it works. Use versioning to control changes and rollback;

    10. Sustainment and Improvement: have people responsible for monitoring, preventing, acting on problems with the bots, and making revisions;

  • How to optimize performance in automations with Python RPA

    How to optimize performance in automations with Python RPA

    ✅ Robots often run processes 24/7, for months and years. Although Python provides up to 20x faster performance than low-code platforms because it is pure code and easier for the machine to process, applying small improvements can still bring big results over time;

    To test the performance of your code in more detail, you can use libraries/profilers to identify bottlenecks:
    – Timeit
    – line_profiler
    – memory_profiler
    – cProfile
    – Stackfy

    ✅ Every project has its requirements, but it is worth considering a few items to further optimize your robot in Python:

    – Don’t import unnecessary libraries. Only load them when you need them

    – Reduce data traffic. Download only what you use

    – Evaluate other processes running on the same machine and avoid peak times

    – Monitor the performance of your bots to identify degradations

    – Don’t do repetitive and unnecessary calculations. Use caching

    – Use appropriate algorithms and structures for each case (lists/tuples, sets/dictionaries)

    – Use built-in functions and libraries

    – Use generators (prefer xrange to range) and keys for sorting

    – Free memory (close, exit…) and avoid creating unnecessary objects

    – Optimize database queries and processing

    – Check how best to access some of the data with APIs

    – If possible, use asynchronous communication

    – Evaluate whether it makes sense to deploy with container or serverless

    – Consider using background and parallel automations when possible

    – Evaluate loops and commands within loops, prefer List Comprehensions

    – It can be more performant to use IDs and keyboard shortcuts than image recognition and mouse movement

    – Use set operations

    – Avoid global variables

    – Use vector/NumPy operations and optimized libraries like Pandas and Polars

    – Use multiple assignments

    – Use in when possible

    – Optimize string operations (e.g. join instead of + for concatenation) and interning strings

    – Also learn about Multiprocessing, Itertools, CPython/Cython, Pypy, Mojo…

    – Avoid optimizing performance and hindering robot code reading and maintenance

    ✅ There are also several tools to help optimize and secure code in Python:
    – Deepsource
    – Codacy
    – SonarQube
    – Veracode
    – Checkmarx

  • Python RPA Security – Check out best practices

    Python RPA Security – Check out best practices

    ➡️ What about security in Python RPA? Check out good practices to make your robots more secure 🤖👇

    📌 In RPA, bots often have direct communication with critical systems that, without proper measures, can leave sensitive data vulnerable and facilitate unauthorized access;

    💡 The most prominent tech companies use Python massively in their stacks. With Python, your RPA operation can guarantee point-to-point the highest level of security (with encryption, vulnerability testing, and more);

    💡 See BotCity ‘s recommendations for further protecting your bots and mitigating key risks (according to OWASP):

    ✅ Protect the environment in which the bot is developed and run. Including the operating system, database, dependencies, and network, with the latest security updates;

    ✅ Separate development, test/homologation, and production environments;

    ✅ Make communication secure with VPNs, encryptions, SSL/HTTPS certificates, allow-lists, and specific ports on the firewall;

    ✅ Ensure that actions are assigned with secure and unique identification credentials for each robot;

    ✅ Separate credentials from the code and store them in an encrypted vault, and change them periodically;

    ✅ Grant strictly necessary permissions to the bot and collaborators, with multi-factor authentication where applicable;

    ✅ When deactivating a bot, don’t forget to revoke the credentials;

    ✅ Monitor the activities of bots and create alerts for unexpected behavior;

    ✅ Manage sessions with screenshots or videos;

    ✅ Protect the integrity of logs by storing them separately and ensuring that they are complete and auditable;

    ✅ Do not use the default Python interpreter. It is likely outdated. Install and make sure you have the latest version;

    ✅ Use virtual environments;

    Avoid making relative imports;

    ✅ Beware of unknown, unofficial, or out-of-date libraries (check the Snyk Advisor). Be careful not to misspell the library name and use a malicious one by mistake;

    ✅ Avoid loading unnecessary data/libraries;

    ✅ Keep the code clean;

    ✅ Always validate/treat data entries to protect against injections, use encryption for sensitive data, and handle errors and exceptions;

    ✅ Avoid leaving unlimited things and free up memory for objects you are no longer going to use;

    Set DEBUG = FALSE in production;

    ✅ Deserialize with caution when using Pickle. Evaluate using PyYAML;

    ✅ Establish an adaptive governance framework;

    ✅ Perform unit tests, integration tests, canary releases and periodically evaluate the implementation as a whole;

    Do code review, versioning control, and backups;

    ✅ Scan your code for vulnerabilities (get to know Bandit, Snyk Code, SonarQube…);

    ✅ Reflect on scenarios where things can go wrong and have a plan for incidents;

    ✅ Have a proactive dialog with the security team from the beginning of the project;

  • Creating templates for reading and processing text PDFs with Python RPA – Meet BotCity Documents – Part 1

    Creating templates for reading and processing text PDFs with Python RPA – Meet BotCity Documents – Part 1

    We have to deal with different types of PDF documents in many areas, be they text or images. And there is a much easier way to solve the processing of these documents using Python RPA instead of regex, XML, or other more complex procedures. And all this with support from other OCR tools or frameworks as well.

    In this article, we will show you one of our frameworks: BotCity Documents, which, in addition to being integrated with BotCity Studio, facilitates the PDF documents’ reading, individually or in batches, using computer vision to convert the reading and analysis relationship between fields and values in the documents for Python code that you can add to your RPA automation project.

    In this part 1 of the BotCity Documents series, you will see how to process text PDF documents. In part 2 we will demonstrate how to process PDF documents with images.

    💡 Important: BotCity Documents is currently not available in the community license. But expect news soon. 👀

    How to use BotCity Documents

    Let’s take a step-by-step to understand how to use the framework. For this model we will use a desktop type bot, but you can also use web type bots. To create a similar template, you can follow the instructions in this link from our documentation.

    However, we emphasize that our BotCity Documents framework is not for exclusive use in automation projects built with our Desktop or Web framework. It is a standalone package, and although you can use it with other tools, it is possible to use it in other projects without any dependencies.

    Prerequisites

    1. You must have a license that is not Community;
    2. Install BotCity Studio SDK. You can check how to do this by clicking on this link in our documentation;
    3. Install the BotCity Documents package. To do this, you must contact us for guidance on how to install the package according to your license (which, for now, cannot be the community version).
    4. Have an IDE installed. For example: Visual Studio Code, PyCharm, and others.

    Example: reading PDF text files

    In your IDE:

    Open your favorite IDE that you already use in your daily life. In your project, identify the .py file in which you will develop or create a new one according to the organization and your project patterns. In the example, we created the file “bills.py“.

    Import the installed package as indicated in the third item of the prerequisites.

    Then instantiate the PDF reader and read your file, as per the example below in Python code.

    # Creating a reader
    reader = PDFReader()
    
    # Reading the pdf file
    parser = reader.read_file("document.pdf")

    You can also follow the documentation and see how to do the same in Java, if you are using that language. Check this link.

    In BotCity Studio:

    Open BotCity Studio and create or upload your project. Click on the “File” option and then “New Project” for a new project. Or “Load Project” to load an existing project.

    Identify your project folder and, within it, find the file with the extension “.botproj”. By doing this we will have the same code in both tools: IDE (which in the example we used Visual Studio Code, but it can be other IDEs) and BotCity Studio.

    From now on, it is important to leave the cursor positioned in BotCity Studio after the last line of code. Because the codes that are automatically generated by the tool, will be included from it.

    At this point we need to load the PDF that we will use to create the template. Click on “Documents” in the top menu. After that, click “Load new document” and identify the PDF file to be read.

    After loading the PDF, you will have the “Code” tab, where the code will be; the “UI” tab, which we are not using in this project; and the “PDF” tab, where the file you have loaded will be.

    Click on the “PDF” tab. You must identify the fields and respective values the files will have as default to select them. The selections will take place with two pieces of information at a time. The first refers to the field, which will be a red highlight on the BotCity Studio interface, and the second relates to the value of this field, which will be a blue highlight on the BotCity Studio interface.

    Analyzing the generated code for its use:

    _account_no = parser.get_first_entry("Account No:")
    value = parser.read(_account_no, 1.078947, -2.25, 1.513158, 3.5)
    _statement_date = parser.get_first_entry("Statement Date:")
    value = parser.read(_statement_date, 1.06, -2, 1.18, 3.5)
    _due_date = parser.get_first_entry("Due Date:")
    value = parser.read(_due_date, 1.079365, -1.75, 1.920635, 3.5)
    

    BotCity Studio starts taking important readings to generate the code. For example, the field in the PDF called “Account no”, it already understands that it can be the variable name and therefore names it “_account_no”. Just as happened with the “Statement Date” and “Due Date” fields.

    Let’s validate that the reading is being taken with the right value? Let’s add the print command for each field and compare the result to make sure it is reading the data correctly.

    The code could look like this:

    _account_no = parser.get_first_entry("Account No:")
    value = parser.read(_account_no, 1.078947, -2.25, 1.513158, 3.5)
    print(f"Account no: {value}")
    
    _statement_date = parser.get_first_entry("Statement Date:")
    value = parser.read(_statement_date, 1.06, -2, 1.18, 3.5)
    print(f"Statement Date: {value}")
    
    _due_date = parser.get_first_entry("Due Date:")
    value = parser.read(_due_date, 1.079365, -1.75, 1.920635, 3.5)
    print(f"Due Date: {value}")

    And by running the code from our “bills.py” file, the result can be compared with the PDF:

    💡 Important tips:

    • After making the selections you need in the PDF, go to the “Code” tab and press “Ctrl + S” on your keyboard to save the newly generated code. This will make the code appear automatically in your IDE.
    • If you change something in the code from your IDE, when you save the change and click again in BotCity Studio, it will receive the change you made.
    • Even if the mapped field and its value change location within the document, the tool can still read it, as it is mapped with computer vision, as long as the relationship between the anchor and the reading area is maintained.
    • BotCity Studio automatically generates the code considering the variable name created for reading the PDF as parser. If you used another name, simply refactor your code.
    • If you do not want to see the outlines of what you are selecting, you can click the checkbox in the “Show Selections” field. If you have ✅, the selections will be shown. If ✅ is missing, the selections will not be shown.
    • If you make a mistake in the first selection (from the field highlighted in red), press the ESC key on your keyboard, that single selection will be erased, and you will be able to do it again.
    • If you make many mistakes, you can delete the code snippet generated in the “Code” tab and reselect what you need.
    • Note that in the demo GIF, we show that the field selection is snapped to the word, while the value selection is more extensive in the PDF. This is because the field is fixed, but the value can be larger than the model we checked. With this wider selection, we can guarantee that we will get the full result.
    • You can also follow this quick video on our YouTube channel demonstrating the use. Subscribe now and receive news there.

    And for batch execution?

    There are some libraries in Python itself that can be a support for PDF file discovery. For example: pathlib or glob. In addition to creating structures, using features of the language itself, for execution in a loop, of several files, according to the needs of your project.

    What did you think?

    Did you like this feature? If you have any questions, don’t hesitate to contact us, and don’t forget to join our community, to enjoy the exchange of knowledge and experiences.

  • New features: Runner and task visualization, error alert information by e-mail and Office 365 plugin

    New features: Runner and task visualization, error alert information by e-mail and Office 365 plugin

    Hello, world! 👋

    We have fantastic news to share with you about our Python RPA tools. Follow this article.

    [0] Runners visualization

    From now on, you can have a new way to view runners in the BotCity Maestro interface.

    By clicking on “Runners” in the menu on the left side of BotCity Maestro, you have the distribution of the runners in “cards” as small squares, each with a runner, with the status, name in link format, an icon to copy the runner’s name, the screen, and the three dots icon to access the options menu.

    BotCity Maestro's runner screen with the card-view.

    But now, next to the runners status filter, you can click on the icon to choose one of the available screen view formats. In one format you will have the runners in the organization informed above, in another you will have the list format.

    BotCity Maestro runners screen with the list view with columns.

    This list has identified columns:

    • screenshot: if it is a desktop runner, this column will have the screen print; if it is a background runner, there will only be a generic image;
    • name: to inform the name of the runner and this name is a link that, by clicking on it, you will access the runner’s information screen;
    • machine id: is the runner’s identification;
    • type: tells you the runner’s type;
    • status: when it is red, it means that the runner is offline; when it is green, it means that the runner is online;
    • In the last column is the three dots icon, with the same menu options as the screen laid out in card format.

    You can still continue to use the filters as usual to view only what you need at that moment. For example, bring only the runners that are online.

    BotCity Maestro runner screen with the list view with columns, showing a filter applied to return only runners that are online.

    [1] Task queue visualization

    The same change that occurred for runners has also been applied to the task queue view.

    Next to the filters is the icon for changing the view mode. In one of the formats, we can identify the tasks in cards, as is already possible today.

    BotCity Maestro's task queue screen with the card-format view.

    The second form of visualization is organized in a list.

    BotCity Maestro's task queue screen with the view in list format with columns.

    And in this list, we have some columns with the following identifications:

    • id: which informs the task’s ID;
    • priority: which tells you the priority of the task (from 0 to 10);
    • state: line with the color that indicates the status of the task. To find out what each color means, check the legend located above the search filters;
    • automation label: identification of the automation by which the task was created;
    • runner: informs the runner in which the task was executed;
    • user name: indicates the name of the user who created the task;
    • date creation: field in which is the date and time that the task was created;
    • date last modified: field containing the date and time at which the task was last modified;
    • finish message: informs the message that the task was finished.

    [2] New information in the e-mail error alert

    Until now, it was already possible to set up error alerts to be received by e-mail. However, there has been an enhancement to these alerts, in which we have included more information to better understand what has happened and whether any action is required.

    Screenshot of the error alert received via e-mail.

    In the alert we have the information:

    • organization: which indicates which organization the automation was created as a link to access, if necessary;
    • task id: contains the id of the task that had an error as a link to access it, if necessary;
    • automation label: which informs the identification of the automation as a link for access, if needed;
    • bot id: which informs the bot’s identification as a link to access, if necessary;
    • bot version: which indicates the version of the bot that was executed;
    • error type: which contains the type of error that occurred during execution;
    • error message: which tells you the error message.

    It is important to remember that you can configure and customize both errors and alerts by using the BotCity Maestro SDK in your project, as oriented in our documentation. Access the link about the errors here and the link about the alerts here.

    [3] Office 365 Plugin

    We have several plugins that can be very useful during the development of your automations. All the documentation about these plugins and how to use them is on this link, and you can easily follow the step-by-step process.

    And now we have the new plugin to make it easier to use the Office 365 (or Microsoft 365) API if you need it. To better understand how to use this plugin in your project, either to integrate with Onedrive or Excel, see our documentation with all the guidelines.

    Important: Before any step, you need to create your credentials. Without this, you will not be able to use the plugin to integrate with services correctly. So start by creating the credentials and understand that part first with the guidelines in this link in our documentation.

    Keep following to know the next news

    We hope you have enjoyed these new features. And soon we will bring you more content. Be a part of our community by joining slack, following the news in our documentation and here on the blog, and learn a lot from our forum.

  • Migrating from Low-code to Python RPA

    Migrating from Low-code to Python RPA

    ✅ For you considering migrating low-code RPA to an open technology platform, know that this transition is simpler than it sounds. We have seen companies completely migrating dozens of robots per month.

    ✅ It is also possible that it will be done gradually, coexisting low-code and open-technology platforms where necessary. Still, for companies that need to scale their automation, the migration from Low-code to RPA in Python makes a lot of sense, mainly because of the technical advantages and the savings on licensing.

    💡 To make this transition, some companies miss having Python developers with RPA familiarity. In this case, these companies can:

    1️⃣ Train your existing RPA Low-code developers → Python is an easy language to learn and will empower your developers much more. Tip: BotCity Academy offers free courses in Python and RPA;

    2️⃣ Hire Python developers who have some experience in RPA → Python is the 2nd most widely adopted programming language by devs and the most widely used in RPA;

    3️⃣ Request support from specialized partners → BotCity has more than 40 partners approved and trained in Python RPA, which perform from mapping and development to automation support and training;

    💡 Another important aspect of migrating from Low-code RPA to Python is selecting the right platform that makes it possible to achieve more productivity in the development and orchestration of the automations. BotCity is a specialized Python RPA platform, flexible and fully orchestrated, capable of handling complex automation scenarios.

    ✅ With these skills on the team, partners, and the right platform, organizations can unleash the full potential of RPA with Python and achieve maximum efficiency and productivity in their automations.

    💡 Want to know about unleashing the full potential of automations in smaller projects and companies of all sizes? BotCity Academy has free training on Python RPA → https://botcity.dev/academy

  • Python RPA vs Low-Code comparison

    Python RPA vs Low-Code comparison

    Advantages of Python RPA vs Low-code RPA:

    1. Licensing model

    ⛔ Low-code: Enterprise licenses ($$$) per bot/machine.

    ✅ Python RPA: No commercial licenses are involved in development. Orchestration platform in SaaS model, pay as you scale with up to 80% savings.

    2. Lock-in

    Low-code: High lock-in. Proprietary technology. If you don’t pay the license, the bots stop working. To migrate, you need to rewrite the automations.

    Python RPA: No lock-in. Open technology, pure Python code. You can rotate the robots and use them as you wish.

    3. Compatibility

    Low-code: Windows.

    ✅ Python RPA: Windows / Linux / Mac, containers, serverless

    4. Time-to-success

    Low-code: Requires multiple sessions of specific training.

    ✅ Python RPA: Dev JR masters the technology in a few days and can get coding right away.

    5. Performance of the bots

    ⛔ Low-code: Slow because it requires an intermediate parser.

    ✅ Python RPA: 3x-15x faster. It’s pure code.

    5. Scalability and Support

    Low-code: Low. Because of licensing and technical restrictions. Support needs to open tickets (n1, n2, n3) or go to specific communities.

    Python RPA: High. Elastic Computing, Ultraparallelism, affordable licensing, easy learning, and numerous open communities.

    6. Orchestration / Governance

    Low-code: For automations of specific types and flows.

    Python RPA: Complete orchestration, for any automation in code, in any framework.

    7. Knowledge Requirement

    Low-code: Tool-specific. Only 100,000 professionals in the world.

    Python RPA: Python only. 14 million devs. People of all professions program in Python: Finance, Marketing, HR, designers, lawyers, etc.

    8. Productivity

    ⛔ Low-code: Always need to recreate the streams, depending on available connectors.

    Python RPA: Easy code reuse, 440,000 available packages, numerous open APIs and SDKs for easy integrations, versioning control, and simplified rollback.



    💡 Want to know about unleashing the full potential of automations in smaller projects and companies of all sizes? BotCity Academy has free training on Python RPA → https://botcity.dev/academy

  • Advantages of Elastic Computing for Automations with Python RPA

    Advantages of Elastic Computing for Automations with Python RPA

    1) ECONOMY: Pay only for the infrastructure actually used, eliminate idleness, implement parallelism, and have significant savings;

    2) AGILITY: Quickly scale up or down resources (even automatically) as needed to meet variations in demand, faster execution of automations, and easier portability;

    3) INNOVATION: Testing new ideas without requiring significant infrastructure investments;



    👉 As RPA grows in enterprises, the dynamism of needs requires modern computing architectures that enable dynamic resource allocation for scale gain and more agility in the execution of operations.

    ✅ Elastic computing is the ability to rapidly (or even automatically) expand or shrink computing resources of storage, memory, and processing to meet peak demand, eliminate idleness, and optimize costs.

    ✅ In elastic computing, you pay only for what you use. Thus, you can achieve significant savings and provide a better user experience by immediately making infrastructure and resources available as needed.

    ✅️ Furthermore, elastic computing favors experimentation and innovation by providing access to computational resources without requiring significant infrastructure expenditures.

    ✅️ Python automations can run in any environment: desktops/VMs (Win/Linux/Mac), containers, and even serverless, and can run multiple automations in the background simultaneously (ultra-parallelism).

    ✅️ With the BotCity platform, you can easily make deploys, manage the deploys on the dashboard, and orchestrate the automations;

    ✅️ Containers are lighter than VMs. They enable multiple tasks in the same instance, reduce implementation time, and facilitate portability between environments. Serverless is lighter than containers, scales faster, and further abstracts infrastructure provisioning.

    ✅️ VMs, containers, and serverless have different maintenance and limitations. It is up to you to evaluate which model is most suitable for the automation scenario.