The Parallax View

Joel Gethin Lewis

Lecture 5: Machine Learning 1

What I'm going to talk about today:

1. What Machine Learning, Deep Learning and ml5.js are.
2. Classification of images.
3. Transferring the style of an image to another.
4. Live tracking of body pose.
5. Brief: letting your computer learn.
6. Discussion: Kyle McDonald.

1. What Machine Learning, Deep Learning and ml5.js are.

• What is Machine Learning? What is Deep Learning?
• Let's watch Introduction to Deep Learning by Teubi at the FANTASTIC (I watch live every year) Chaos Communication Congress - the 2018 wiki is a great resource.
• Also see Daniel Shiffman's "A Beginner's Guide to Machine Learning with ml5.js" - particularly his discussions on algorithms, models and data sets - particularly what we are doing - using existing models, rather than training our own - and criticality - we should be using machine learning for art and design so we can critcise goverments (and others) use of it.

1. What Machine Learning, Deep Learning and ml5.js are (continued).

• It's important to remember that there are limitations to Deep Learning.
• Interesting article on the difference between supervised and unsupervised learning.
• Computerphile has some great videos on neural networks, deep learning and other interesting things.
• Also see the "Machine Learning for the Web" class at ITP, NYU for some great notes, exercises and examples.
• Lets look at some ml5.js examples from the ml5.js website.

2. Classification of images.

• This example, Video Classification, is based on Daniel Shiffman's tutorial "ml5.js: Webcam Image Classification" and Kyle McDonalds CV Examples.
• Remember - the model we are using is only trained on a few image classes. It can't see anything it hasn't been trained for! This is something to bare in mind whenever you use someone elses model - what has it been trained for? What biases might it have? Via Daniel Shiffman's tutorial "ml5.js: Image Classification with MobileNet".
• Also try this YOLO (You Only Look Once) (Chrome Only!) demo again from Kyle McDonalds CV Examples.

3. Transferring the style of an image to another.

• Unfortunately, there is currently a bug with getting ml5.js working with the p5.js editor.
• So we'll have to run our code locally, and run a local server so that we can see it.
• Download this zip file containing a live video style transfer example to your desktop. It will only work if you run your own local web server and view the demo in Chrome.
• If you want to learn how to train your own models (advanced!), you can follow these two tutorials from The Coding Train: "Style Transfer Part 1: Training a model with on Spell with Yining Shi" and "Style Transfer Part 2: Real-Time Style Transfer with ml5.js with Yining Shi".

4. Live tracking of body pose.

• Let's try this example of live tracking of body pose. Based on Kyle McDonalds CV Examples.
• What could you use this for? This was only previously possible using a desktop computer and a 3D sensor, such as the Microsoft Kinect. A newer version of the Kinect runs the just announced Hololens 2.
• An example of the cutting edge of this kind of work is OpenPose.

5. Brief: letting your computer learn.

• In your pairs, what do you want your computer to learn?
• About what is inside images?
• About the style of images?
• About the movement of humans?
• See this great playlist by Daniel Shiffman: "A Beginner's Guide to Machine Learning with ml5.js".
• See this list of AI Experiments from Google.
• Also see the Machine Learning 4 Artists (ML4A) website by Gene Kogan.
• All works should be published on the brand new p5.js Web Editor if possible, but local copies are ok - make sure you email me the complete zip if you just work locally!

6. Discussion: Kyle McDonald.

• In session 3 we drew on Kyle McDonald's cv-examples. Today, we'll be connecting with him live.
• What questions do you have for Kyle?
• Let's connect with him now.

Thanks!

• More about me: @joelgethinlewis
• Get in contact via: www.joelgethinlewis.com
• Website:
  https://jgl.github.io/TheParallaxView/
• Today's slides:
  https://jgl.github.io/TheParallaxView/lecture5.html