Fashion CUT: Boosting Clothing Pattern Classification with Synthetic Data

Fashion CUT is an innovative unsupervised domain adaptation technique that harnesses the power of AI and computer graphics to achieve accurate clothing pattern classification.
April 2023
Read paper
When it comes to online shopping, accurate product information is crucial to ensure a seamless customer experience. However, providing high-quality product data can be challenging, especially in the fashion industry, where training image classification models requires large amounts of annotated data.
Synthetic data generation

We generate synthetic fashion images using computer-graphics techniques. The generated images do not require manual human validation. For each render we start with a provided 3D object, add lighting effects, apply a procedural material and then randomize its properties (e.g. colors, scale). This setup allows an arbitrary amount of different images for each 3D object to be generated programmatically.
Approach

Unsupervised domain adaptation has shown excellent results when translating images to other domains. Nevertheless, translated images can’t be readily used to train classification models because image features, such as patterns, are distorted during the translation step since the translation model doesn’t have information about the features. Specifically, when complex patterns are shifted to a different domain, they can be distorted to a level that they no longer adhere to the original pattern label for the synthetic image. For example, stripes may no longer look 'stripey' when shifted to the new domain.
Fashion CUT architecture
The proposed architecture includes an image translation model and a classifier model, which are optimized together via a common loss that ensures realistic images with reliable annotations. Pseudo-labeled real images are included in each mini-batch to improve the classifier generalization.
Results

We compare the performance of domain adaptation algorithms trained only on our 31,840 synthetically generated dataset and evaluated on real fashion images. Our approach outperforms the other algorithms for the pattern classification task. Finally, we found that using pseudo-labels improves the results with minor changes in the training.
Method Accuracy
No adaptation 0.441
BSP [1] 0.499
MDD [2] 0.540
AFN [3] 0.578
fCUT (ours) 0.613
fCUT + PL (ours) 0.628
Conclusions

Combining synthetic data generation with unsupervised domain adaptation can successfully classify patterns in clothes without real-world annotations. We also found that attaching a classifier to an image translation model can enforce label stability, thus improving performance. Furthermore, our experiments confirm that Fashion CUT outperforms other domain adaptation algorithms in the fashion domain. In addition, pseudo-labels proved to be beneficial for domain adaptation in the advanced stages of the training.