Deep Learning summary for 2017: Reinforced Learning and Miscellaneous apps
In the third part of our Deep Learning summary for 2017, we discuss the new discoveries and breakthroughs in reinforced learning and other fields.
Reinforced Learning or RL is also on the rise. This term refers to training an agent in an interactive and rewarding environment, essentially giving them experience, just like humans gain knowledge in the real life. The famous from Google DeepMind, a company we mentioned quite a lot in the is a bright example of such training: the bot analyzed a huge number of games and played versus itself to advance. 2017 brought even more impressive results.
Reinforced learning with unsupervised auxiliary tasks
Another example of RL using a game is. However, uninterrupted RL takes quite a lot of time, so accelerating it using various auxiliary tasks is a highly lucrative and hot topic. Google DeepMind has published an showcasing the consequences of adding some additional losses (auxiliary tasks like pixel control) to make the training rewarding. This speeds up the process significantly.
Here is what the results look like:
Robot training using Deep Learning virtual environments
OpenAI team has published a research on one-shot learning, which proves: even one virtual reality simulation is enough to train the robot to perform a task. People are not so smart…
RL using human feedback
Yet anotherfrom Google DeepMind and OpenAI describes the results of reinforced learning imbued with human feedback. The algorithm had a task and presented two variants of the solution to a human, who chose the optimal variant. The algorithm was trained to solve the task quite rapidly.
Robot movement programming
We reckon you all were following the progress of the adorable Atlas from Boston Dynamics? The robot is rapidly mastering even the most complicated forms of movement as is now even capable of!
This progress is based on the software that teaches the robot to walk based on a simple function that uses the rewards for teaching the robot to make the correct movements. Google Deepmind researchers published a paper on, describing the way the agent (a humanoid figure) chose the correct route through the course of obstacles. This is what the training process should have looked like from the algorithm’s point of view:
OpenAI has kindly shared these algorithms on their, so you can train your agent too!
There are quite a few impressive practical applications of Deep Learning algorithms in real-world businesses. We will list some of them below.
Google also used some of DeepMind algorithms to significantlyby leveraging data from thousands of sensors in their data centers to optimize the cooling process.
A universal DL model
Until recently every ML model had to be trained/retrained to solve a specific task. Google Brain team made a significant progress in creating a universal! The network uses multiple decoders and encoders and switches them depending on the task at hand:
Due to using such approach, the models become quite well suited for solving multiple tasks, the blocks to not interfere and sometimes might even help each other, and the performance is great for both parsing small and huge datasets.
Get the code fromand play with it yourself!
Imagenet training in 1 hour
FB research group has published the, allowing to increase the efficiency of Imagenet training by 90%. They did have to use a HUGE cluster of 256 CPUs for the task, yet this model now allows Facebook to experiment with even better conditions!
Self-driving cars are ever closer
, and their self-driving cars have completed more than 3,000,000 kilometers across the US. As , the future is ever closer.
As we have already told in the first article of the series on, Google Deepmind is actively working to introduce their DL algorithms into the healthcare industry, for . They have already created a separate department, for this purpose.
Remember — in quite a near future knowledge an experience with Machine Learning systems will be as expected of an IT specialist as the ability to handle databases today. Be prepared — this brave new world of self-driving cars, image-recognizing robots, fake TV interviews with the President and computer-synthesized music is near!
2017 was a very fruitful year for Deep Learning. Multiple projects in the fields of text and speech recognition, decoding and generating audio,and improving reinforced learning mechanisms — all of this happened within one year, and much more is sure to come!
We are thrilled to witness this rapid progress and have engaged in quite a few of Deep Learning projects ourselves. Stay put, more great news are surely on the way, and should you have any inquiries – feel free toat once! If you found this summary useful — share it with your friends!