Pinterest Outlines AI Background Generation Process for Product Shots

Pinterest is developing its own AI text-to-image generation technology, which takes a slightly different approach than other apps.

According to a recent overview from the Pinterest Engineering team, Pinterest’s “Canvas” model is designed to offer generated background options for product shots while keeping the product itself as the primary focus.

This process requires additional training. While most large language models generate images based on textual descriptions by correlating text with visual elements from other images, product shots often lack detailed descriptions of their backgrounds. To address this, Pinterest’s team has developed a new method to separate the background from the foreground, enabling users to guide the tool with straightforward commands.

According to Pinterest:

“Training Pinterest Canvas gives us a strong base model that understands what objects look like, what their names are, and how they are typically composed into scenes. However, as previously stated, our goal is training models that can visualize or reimagine real ideas or products in new contexts.”

Conceptually, Pinterest aims to leverage its existing database of product images to identify common framing, placement, and background types. This will help improve the AI’s ability to generate backgrounds by providing more accurate and relevant options based on these established patterns.

It’s a complex approach, but Pinterest has successfully developed a system that achieves this with a high degree of accuracy.

“[We] use a segmentation model to generate product masks by separating the foreground and background. Existing text captions typically describe only the product while neglecting the background, which is critical to guide the background in the painting process, so we incorporate more complete and detailed captions from a visual LLM. In this stage, we train a LoRA on all UNet layers to enable rapid, parameter-efficient fine-tuning. Finally, we briefly fine-tune a curated set of highly-engaged promoted product images, to steer the model toward aesthetics that resonate with Pinners.”

In summary, the system is tailored to generate backgrounds based on existing Pin images. Additionally, Pinterest has aimed to align the model with specific visual styles to streamline the creation process further.

Ultimately, this will allow brands to specify their preferred style using common descriptors, and Pinterest’s system will provide background options for product shots that match that aesthetic.

It’s an intriguing concept that Pinterest is currently testing with select advertising partners.

This could be an effective way to generate more variations of your Pin images and enhance your product’s appeal across different design styles.

You can read more about Pinterest’s approach to AI background generation here

Source: Social Media Today

Find more information here: http://bit.ly/2BPQn38

For more information contact us at: [email protected]