Many people reported that the lora training sucks for z-image base. Less than 12 hours ago, someone on Bilibili claimed that he/she found the cause – unit 8 used by AdamW8bit optimizer. According to the author, you have to use FP8 optimizer for z-image base. The author pasted some comparisons in his/her post. One can check check https://b23.tv/g7gUFIZ for more info.
submitted by /u/Recent-Source-7777
[link] [comments]
Your monthly "Anzhc's Posts" issue have arrived. Today im introducing - Mugen - continuation of…
Your monthly "Anzhc's Posts" issue have arrived. Today im introducing - Mugen - continuation of…
This article is divided into three parts; they are: • How Attention Works During Prefill…
This article is divided into three parts; they are: • How Attention Works During Prefill…
Feature engineering is where most of the real work in machine learning happens.
Feature engineering is where most of the real work in machine learning happens.