Great job!
@Chickaboo Hey Lucas!
Great job!
Definitely looks like an improvement in terms of loss and architecture :)
Please upload/post output samples and/or inference code! I would love to check it out! :)
Sincerely,
Alex.
PS. Thank you very much for listening to my samples and using my MIDI assets and software! :) It means a whole lot to me :)
Hey Alex!
Thanks for taking a look at the model.
I Appreciate you taking the time to give me feedback.
Before the next model I will have to do some more ablation testing.
I'm not really sure how much hybrid heads benefited this model, but we'll see.
The biggest factor in quality I think with it being a little under trained. The bottom line is there's still a more room for improvement architecturally, once I think I've hit the ceiling, I'll scale it up to a bit larger of a model I think.
These are a few sample generations made with this notebook
Let me know what you think!
Best regards.
Lucas.
@Chickaboo Hey Lucas,
I've listened to your samples (thanks for uploading them). They are definitely an improvement from a previous model. "Hungarian Rhapsody.wav" sounds great although "Hungarian Rhapsody (2).wav" loses the key at the end. Other samples are also nice but could use some work.
However, all samples sounded a bit repetitive. I think maybe increasing the temperature, reducing top_k, top_p or whatever constraints you may be using may help with that.
Otherwise, its certainly a step in the right direction! Keep working on it!
I will try to eval the model directly myself with your colab soon and when I do, I will let you know the results.
Sincerely,
Alex.