AR experiment uses AI technology to cut and paste real objects into Photoshop

According to foreign media reports, augmented display (AR) or mixed reality (MR) experiences usually refer to making virtual objects appear in the same frame as things in the real world, such as through glasses or mobile phone screens. On the other hand, it is to place real-world objects on a digital canvas and achieve real-time operation, but this operation is not common and not so easy.

And this is exactly what AI programmer and UX designer Cyril Diagne is doing, he can "cut and paste" real-world things onto the Photoshop canvas.

Perhaps you may find this idea a bit ridiculous, as taking a photo of a vase and then copying and pasting it onto a computer is a simple task. Perhaps this may sound reasonable, but in reality, cutting and pasting AR is much more complex.

It is understood that its uniqueness lies in taking photos of objects only when there is no surrounding environment, and then adding the photos to the Photoshop canvas when it is dragged onto the computer. This is almost like cutting objects from the background and dragging and pasting them into a document.

The secret of this experiment, of course, is AI, especially in terms of salient object detection and background removal. Many mobile phones now use similar but simpler methods to separate foreground and blurred background, such as out of focus blur simulation or portrait mode. In this case, AI is still too complex, it must run on a computer and cannot be completed on a mobile phone.

Considering the significant delay in the action, this experiment is far from perfect, but it can still be considered an impressive demonstration.

Prev: Engineers create ultra-thin and stretchable electronic materials with gas permeability