Is Strawberry the next step on our road to AGI ?

Author

Hamza Tahir
July 19, 2024

Science offers the boldest metaphysics of the age. It is a thoroughly human construct, driven by the faith that if we dream, press to discover, explain, and dream gaian, thereby plunging repeatedly into new terrain, the world will some-how come clearer and we will grasp the true strangeness of the universe. And the strangeness will all prove to be connected, and make sense.
-Edward O. WIlson

Today’s edition includes :

  • OpenAI lays out their 5 step plan to reach AGI and “Strawberry” might take them to the next step.

  • Meta to release their largest Llama model.

  • Google is set to acquire Wiz.

  • and more from the world of AI and Security.

Read time: 7 min.

AI and OpenAI

OpenAI is taking a new approach to developing it’s artificial intelligence models in a project code-named “Strawberry”. It seeks to solve some key challanges that current AI models struggle with.

Key Challenges:

  1. Enabling AI to plan ahead and navigate the internet autonomously

  2. Improving AI's reasoning capabilities

  3. Overcoming "hallucinations" or generation of bogus information

  4. Performing long-horizon tasks (LHT) that require planning and multiple actions over time

  5. Conducting "deep research" by autonomously browsing the web

How Strawberry Aims to Overcome These Challenges:

  1. Post-training specialization: Strawberry involves a specialized way of processing AI models after they've been pre-trained on large datasets.

  2. Possible similarity to Stanford's "Self-Taught Reasoner" (STaR):

    • This method allows AI models to "bootstrap" themselves into higher intelligence levels.

    • It works by iteratively creating their own training data.

    • In theory, this could enable language models to surpass human-level intelligence.

  3. Use of a "deep-research" dataset: OpenAI is creating, training, and evaluating models on this specialized dataset.

  4. Computer-using agent (CUA): Strawberry aims to use a CUA that can take actions based on its findings while browsing the web autonomously.

  5. Targeted capabilities: OpenAI plans to test Strawberry's abilities in performing tasks typically done by software and machine learning engineers.

It's important to note that while source familiar with OpenAI describe these goals and methods, they do not provide specific technical details on how Strawberry will achieve these capabilities. The project seems to be an evolution of the previously reported Q* project, which was said to show promise in answering difficult science and math questions.

There is disagreement among researchers about whether large language models can incorporate the kind of reasoning and planning abilities that OpenAI is aiming for with Strawberry.

OpenAI has recently introduced a 5 level system to track the progress towards AGI.

  1. Level 1: Chatbots - AI with conversational language (current level)

  2. Level 2: Reasoners - human-level problem solving (approaching with "Strawberry" technology)

  3. Level 3: Agents - systems that can take actions

  4. Level 4: Innovators - AI that can aid in invention

  5. Level 5: Organizations - AI that can do the work of an organization

Definitions of two key concepts:

  1. AGI (Artificial General Intelligence):

    • AI that can understand, learn, and apply knowledge across various tasks at a human-comparable level

    • Capable of performing any intellectual task a human can

    • Goal is to create machines that think and reason like humans

  2. Singularity:

    • Hypothetical future point where technological growth becomes uncontrollable and irreversible

    • Often linked to superintelligent AI surpassing human intelligence

    • Could lead to profound, unpredictable changes in society and raise significant ethical questions

Expert Opinions:

"Near Future AGI Is a Myth" Camp:

  1. Yann LeCun (Meta): Skeptical about near-term AGI, believes we're missing essential components.

  2. Gary Marcus: Argues current AI approaches are insufficient for AGI, advocates for interdisciplinary methods.

  3. Andriy Burkov (Co-founder of OpenAI): Emphasizes that true AGI is still a distant goal despite recent advancements.

"AGI Is Taking Over the World" Camp:

  1. Sam Altman (OpenAI): Previously optimistic about near-future AGI, but recent statements suggest a more cautious stance.

  2. Ilya Sutskever (OpenAI): Expressed concerns about rapid AI development and potential consequences.

  3. Demis Hassabis (DeepMind): Optimistic about AGI development, focusing on AI systems that can generalize across tasks.

  4. Nick Bostrom: Discusses potential of AGI and the need to prepare for its implications.

These developments highlight the evolving nature of the AGI debate and the complexity of AI development. They emphasize the need for continued innovation while maintaining realistic expectations about the current state and future potential of AI technology.

AI and Meta

Here are some key highlights regarding this release:

  1. Model Sizes and Performance:

    • Initially released in 8B and 70B parameter sizes

    • Outperformed Llama 2, Google's Gemma, and initially Anthropic's Claude Sonnet

    • A 400B parameter model is in development

  2. Llama 3 400B Model:

    • Scored 86.1 on the MMLU benchmark, comparable to GPT-4

    • Achieves this with less than half the parameters of GPT-4

    • Suggests significant advancements in model architecture and training

  3. Efficiency and Resources:

    • Likely to be more efficient than ChatGPT 4 in computational resources, energy consumption, and cost

    • Meta is utilizing hundreds of thousands of Nvidia H100 GPUs for development

  4. Open License:

    • Llama 3 is released under an open license for research and commercial use

    • Unclear if the 400B version will have the same open license

  5. Potential Impact:

    • Could accelerate innovation in AI applications

    • May enable more researchers and developers to access state-of-the-art language capabilities

  6. Future Developments:

    • Meta hints at upcoming models with multimodality, multilingual abilities, longer context windows, and stronger capabilities

    • Beta testing of Llama 3-405B spotted in WhatsApp Android beta

  7. Release Timeline:

    • Next week!

AI and DeepMind

Here are the key highlights about DeepMind's PEER (Parameter Efficient Expert Retrieval) research:

  • PEER is a novel architecture that scales Mixture-of-Experts (MoE) models to millions of experts.

  • It addresses limitations of current MoE techniques, which are restricted to a small number of experts.

  • PEER replaces fixed routers with a learned index for efficient routing to a vast pool of experts.

  • It uses tiny experts with a single neuron in the hidden layer, sharing hidden neurons among experts.

  • PEER employs a multi-head retrieval approach, similar to multi-head attention in transformers.

  • It can be added to existing transformer models or replace feedforward (FFW) layers.

  • PEER is related to parameter-efficient fine-tuning (PEFT) techniques and could potentially select PEFT adapters at runtime.

  • Experiments show PEER models achieve better performance-compute tradeoffs than dense FFW layers and other MoE architectures.

  • Increasing the number of experts in PEER leads to further perplexity reduction.

  • PEER challenges the belief that MoE models reach peak efficiency with a limited number of experts.

  • This approach could help reduce the cost and complexity of training and serving very large language models.

  • PEER might be used in DeepMind's Gemini 1.5 models, which use a new MoE architecture.

    For more detail read their research paper.

AI and the Environment

GEMS Air Pollution Monitoring Platform, is an initiative co-founded by the United Nations Environmental Program (UNEP) to monitor and mitigate air quality. Here some key highlights:

  1. Platform Overview:

    • Aggregates data from about 25,000 air quality monitoring stations

    • Covers 140 countries

    • Uses AI to provide insights based on the collected data

  2. Purpose and Importance:

    • Provides real-time insights to inform health protection measures

    • Addresses a critical need for high-quality, credible air quality data

    • Utilizes AI as a key tool in air quality management

  3. Global Air Pollution Context:

    • UNEP reported in 2022 that 9 out of 10 people globally are exposed to air pollution

    • Air pollution is considered one of the most significant environmental health issues

    • Impacts public health, agricultural efforts, and biodiversity

  4. Potential Impact:

    • Allows both private and public sectors to use data and digital technologies

    • Aims to accelerate global environmental action

    • Has the potential to disrupt business as usual and contribute to systemic change

  5. UNEP's Perspective:

    • David Jensen, coordinator of UNEP's digital transformation task force, emphasized the platform's potential for unprecedented speed and scale in environmental action

This initiative represents a significant effort to leverage technology and data for addressing global air pollution, a critical environmental and health concern.

Robotics

The Human Touch

Clone Robotics, a company in Poland, has created what they claim is the world's first biomimetic hand. This robotic hand can reportedly:

  • Mimic human hand movements and functions

  • Grasp a wide variety of objects, from tennis balls to power tools

  • Function almost identically to a real human hand, including fingers, thumb, and internal muscles

While specialized robotic designs might be more efficient for specific tasks, there's a strong argument for human-like robotic hands. This is because our world and tools are designed around human physiology, particularly our hands.

Variou companies are working in the development of humanoid Robotics

These developments suggest a growing trend in robotics towards creating more human-like machines, potentially for use in environments designed for human interaction. The advancements in hand technology, in particular, could have significant implications for tasks requiring dexterity and human-like manipulation of objects.

Cyber Security

  • Kaspersky Lab, a Russian cybersecurity company, will begin shutting down its U.S. operations on July 20, 2024.

  • The company will lay off its U.S.-based employees, affecting less than 50 people.

  • This decision follows sanctions imposed by the U.S. Treasury Department on twelve Kaspersky Lab executives on June 21.

  • The U.S. Department of Commerce added Kaspersky Lab and its associated entities to its Entity List, prohibiting U.S. businesses from conducting business with them.

  • The Department of Commerce's Bureau of Industry and Security (BIS) banned Kaspersky from selling software and delivering antivirus updates in the U.S., effective September 29, citing potential cybersecurity risks to national security.

  • The U.S. government's actions were based on concerns about the Russian government's potential influence over Kaspersky's operations and the associated national security risks.

  • Kaspersky stated that these decisions have made their U.S. operations "no longer viable," leading to the gradual wind-down of their U.S. business.

  • The company described this as a "sad and difficult decision" resulting from the U.S. legal requirements and their impact on business opportunities in the country.

This situation highlights the growing tensions between the U.S. and Russia in the cybersecurity sector and the impact of geopolitical issues on technology companies.

Cloud Security

Google is set to acquire Wiz

Alphabet's potential acquisition of Wiz will strengthen it’s grip on cloud security. Here are some key points regarding this development:

  • Alphabet (Google's parent company) is in advanced talks to acquire Wiz, a cybersecurity startup, for approximately $23 billion.

  • If completed, this would be Alphabet's largest acquisition ever, surpassing its $12.5 billion Motorola purchase.

  • The deal is reportedly being funded mostly in cash and could be finalized soon.

Wiz's Products and Services:

  • Wiz provides cloud-based cybersecurity solutions.

  • Their services include real-time threat detection and responses powered by artificial intelligence.

  • Wiz works with 40% of Fortune 100 companies.

  • Notable clients include BMW, Slack, and Salesforce.

  • Wiz collaborates with major cloud providers, including Amazon, Microsoft, and Google.

How Google could benefit:

  • Strengthening Cloud Security: The acquisition would significantly boost Google's cloud security offerings, an increasingly important area as companies move data to cloud platforms.

  • Competing with Rivals: This move is seen as a "shot across the bow" at Microsoft and Amazon, potentially helping Google Cloud compete more effectively with these market leaders.

  • Diversifying Revenue: By enhancing its cloud and cybersecurity capabilities, Google could further diversify its revenue beyond its core search advertising business.

  • Acquiring Expertise: Wiz's founders have significant experience in cybersecurity, having met in Unit 8200, the cyber intelligence division of the Israel Defense Forces.

  • Rapid Growth Potential: Wiz has shown explosive growth since its founding in 2020, suggesting potential for continued expansion under Alphabet.

  • Complementing Previous Acquisitions: This follows Alphabet's 2022 purchase of Mandiant for $5.4 billion, further solidifying its position in the cybersecurity market.

The deal, however, faces potential scrutiny due to ongoing antitrust concerns surrounding major tech acquisitions. Despite this, if completed, it would mark a significant move in the cybersecurity and cloud computing sectors.

Tools to make you cool

 

AI Voices

  • Vocol AI: A voice collaboration platform powered by GPT technology, Vocol AI can quickly convert speech into text with multi-language translation.

  • Audioscribe: Turn scrambled, messy thoughts into a well-structured note with Audioscribe.

Business

  • B2B Rocket AI Agents: This tool automates B2B sales processes and lead generation, aiming to boost revenue by creating a digital workforce that operates on autopilot.

  •  PipeLime: Automatically collect real-time leads and drive them through the sales funnel.

Productivity

  • MyMemo: Gather articles, links, screenshots, and videos into a single, accessible platform, then ask questions about the content you’ve collected.

  •  Ariglad: Use AI to analyze support tickets, then auto-maintain your company’s help center

  • Learnbase: is an AI-driven learning platform using machine learning to personalize curriculum paths, optimize outcomes, and tailor content based on user progress

Super links ( Click and you will not regret )

AI Generated Images

Leonardo.ai prompt: Winter Norwegian forest Painting,Vintage Landscape Print, Edvard Munch
--style raw --ar 4:3 --stylize 750