Since January, the world has been excited and gone mad with ChatGPT by OpenAI. Formerly an open-source company, but once Microsoft pumped $10B into OpenAI, it became a commercial company like all the other tech players. We have all been delighted with ChatGPT’s potential across the world.  In parallel, Meta’s LLM Llama 2 and Google’s PaLM 2 (Bard) have gained a ton of visibility. Always great to see competition among the known tech giants that we know.  Nothing new there.

But what is super cool is when a new player, from an unlikely place in the world, enters the game, so rapidly. Technology Innovation Institute’s (TII) release of Falcon 180B – A 180-billion-parameter Large Language Model (LLM) – was such a surprise entry a few days ago. Falcon 180B sets a new state-of-the-art for open models. It is the largest openly available language model, with 180 billion parameters, and was trained on a massive 3.5 trillion tokens using TII’s Refined Web dataset. And it is completely Open Source – definitely piques one’s interest.

Disclaimer: I am just learning about Falcon 180B and have a ton of questions regarding the quality of the 3.5 trillion tokens and 180B parameters.  I am curious and have a lot of questions about the integrity/accuracy and details of the parameters and tokens. More on my investigations in the next few weeks. So, no conclusive final judgement, but for now, let us assume what I am reading is correct.

Falcon 180B has comparable performance to Google’s PaLM 2 (Bard) and is not far behind OpenAI’s GPT-4. It is touted as the Meta “Llama 2 killer” (serious claim) due to its higher performance as a pretrained-only model. Falcon’s 180B represents the longest single-epoch pretraining for an open model. Clearly, in being open source it does beat the competition, but is it all we think it can be? This is a wonderful problem to explore and solve.

I think the fact that it is open-access may offer a unique advantage (competitive advantage) for a variety of stakeholders, from AI researchers to professionals looking to harness AI tools. Perhaps this is as critical as the 180B and 3.5T tokens and Parameters. Lots of questions, and it makes you wonder… As it learns, at what rate will we see the parameter and token size and accuracy increase? Any drawbacks? Clearly, as it is open source, it may grow at a much higher rate and speed.

As many of you know, I am a strong advocate of Global Innovation. It is thrilling for me to see, perhaps for the first time, such a potential emerging so quickly (see disclaimer above), pushing to get in the front of this fierce line with competitors such as Meta, Google, OpenAI, and others. TII is based in Abu Dhabi, UAE. This is fantastic. When is the last time we heard of such innovation from the Middle East? We need to see much more competition arising from ALL locations in the world. Talent is massive and pervasive around the globe – so should innovation be. So, when we think AI, we need to stop thinking: US, China, and Canada, and start thinking all over the globe.

My tagline for my 2010 book ProVoke is:

Why the Global Culture of Disruption is the Only Hope for Innovation

To see a player from the Middle East rise and become a competitor in the fiercest race in technology (AI) today is hugely inspiring, and a huge boost for the potential from around the globe. Clearly, technology advances require huge funding. UAE is a region very capable of infusing unusually large funds to pursue advanced technology and science, while at the same time inspiring students across their universities and internationally, that anyone can compete in technology, from anywhere in the world. The impossible is only bounded by your imagination!

If (big if) Falcon 180B can deliver (and clearly will be evolving rapidly) and become a true LLM alternative, the possibilities are immense and the global ramifications are outstanding! More on this as I learn and explore more.

 

Imagine | Disrupt | Innovate | Lead