The most important takeaway from this post is the fact that my new VIM config is pretty tight, I challenge you to find and use it yourself! Now that the fun part is over, let’s begin by watching a video that was recently shown to me:
Puzzled? You might’ve figured that the video had nothing to do with the topic of this post, again? Maybe, but isn’t your immune system just epic? What if the Neutrophil was the agent with a task of finding and destroying the poor bacteria in the environment of your body. We have many algorithms/models to use as brains for our agents and tons of problems to solve. Proper environments, however, tend to be scarce. I guess, something between those lines might’ve sparked the idea of creating Gym by OpenAI. Like that wasn’t enough the same team introduced the granddaddy of all - Universe.
More recently, even web browsers got some attention when Mini World of Bits (Mini WOB) was introduced. While the other environments were focused primarily on games and some physics simulations, WOB tries to present an environment that has more instantly applicable tasks. However, Mini WOB has tasks that are constrained within 210x160 pixels. Furthermore, web sites tend to contain lots of elements per page.
Are mobile apps the final frontier for RL agents? Highly doubtful. They might be an exciting next step, though. While the resolution of modern smartphones are close (or even greater) to those of desktop monitors, their physical size seems to hover around 5”. Due to our humanly fat fingers (don’t look at yours now!) the touchable elements per screen can’t be that many. Some design guidelines even suggest that there should be one primary action per screen. So why not give it a go?
Let’s create an environment that is as fun as games and probably more useful. How hard can it be? After all, Android development is so “easy” these days. No coding required (yeah, right). So, what do you need?
Let’s try to stick the Env interface (just in case we might integrate into it someday). Let’s make an environment:
1env = AndroidEnv(app_package, dict(width=1080, height=1920))
Nothing magical so far. Just specifying the package of the app we want to run with and the resolution of the device (hopefully that will not be necessary in the future). Couple of additional things are happening under the hood:
Next up - resetting the state of the environment.
1state, actions = env.reset()
Here’s what is done:
Now for the grand finale - the step function:
1action = choose_action(state, actions)2next_state, actions, reward, done = env.step(action)
The code is at GitHub here and here. Feel free to poke around. It is not ready for the big league, yet. You can expect updated README and a pip package (hopefully around the corner). In some of the next posts we will explore how AppGym can be used for agents that are trying to take over QA people jobs! Is this even possible?
You'll never get spam from me
This book brings the fundamentals of Machine Learning to you, using tools and techniques used to solve real-world problems in Computer Vision, Natural Language Processing, and Time Series analysis. The skills taught in this book will lay the foundation for you to advance your journey to Machine Learning Mastery!