Many people reported that the lora training sucks for z-image base. Less than 12 hours ago, someone on Bilibili claimed that he/she found the cause – unit 8 used by AdamW8bit optimizer. According to the author, you have to use FP8 optimizer for z-image base. The author pasted some comparisons in his/her post. One can check check https://b23.tv/g7gUFIZ for more info.
submitted by /u/Recent-Source-7777
[link] [comments]
Prerequisites You'll need Python 3.
Fine-tuning is the process of continuing to train a pretrained large language model (LLM) on…
The speed of development in large open-source projects has always been limited by code review,…
Some of the best deals on my favorite laptops of the year have already dropped…
From electric vehicles to renewable energy systems and data centers, modern technologies depend on efficient…
Overview This post is divided into three parts; they are: • Why Activations Are Hard…