Meta Introduced Chameleon as ChatGPT Competitor

Those changes include introducing techniques including query-key normalization and revised placement of layer norms.

They also utilized two tokenizers, which process input data, using one for text and one for visuals. The data is then used to form the entire input. The same process occurs in Chameleon’s outputs, enabling the model to focus on the data coming in and out.

Through their changes, the researchers were able to train the model on five times the tokens used to train Llama 2 despite Chameleon being under half the size, standing at 34 billion parameter model.