When you flatten the ids, now the level3 of item1 is followed by level1 of the item2, is that what we want the transformer to learn? For pure next item prediction shouldnt we keep the item ids in array and not flatten it?
This reminds me... In the past I worked with Multiplying Matrices Without Multiplying (MMWM, https://arxiv.org/abs/2106.10860), where quantization was used to efficiently multiply large matrices with a lookup table. It's fascinating to see this in Generative Recommenders too.
Thanks for covering my work :)
Thanks for the amazing work. Apologies if we misinterpreted it!
If you are interested I’d love to collaborate on a future post with you :)
That's a vid id like to see
Oh, Hey Mahesh :)
When you flatten the ids, now the level3 of item1 is followed by level1 of the item2, is that what we want the transformer to learn? For pure next item prediction shouldnt we keep the item ids in array and not flatten it?
Great talk! Thanks for sharing. This was in my videos to watch backlog :)
This reminds me... In the past I worked with Multiplying Matrices Without Multiplying (MMWM, https://arxiv.org/abs/2106.10860), where quantization was used to efficiently multiply large matrices with a lookup table. It's fascinating to see this in Generative Recommenders too.