“Young man, in mathematics you don’t understand things. You get used to them”.
John von Neumann

Today’s edition includes :

• Why is AI so vulnerable

• Andrej Karpathy launches Eureka labs

• Taiwan basks in the bounties of the AI boom

• OpenAI continues to deliver

• and more…

Read time: 7 min.

AI and Cybersecurity

Why is AI so vulnerable

A report by cybersecurity company HiddenLayer found that 77% of companies have reported breaches to their AI in the past year. The rest were "uncertain whether their AI models had seen an attack".

"Artificial intelligence is, by a wide margin, the most vulnerable technology ever to be deployed in production systems, " HiddenLayer CEO Chris Sestito said in the report. "It is vulnerable at a code level, during training and development, post-deployment, over networks, via generative outputs and more."

The key issue that makes AI so vulnerable to cyber attacks at this current stage of development is its overwhelming and rapid adoption without adequate security measures in place. AI models are being developed and improved at an ever increasing rate and companies worldwide see an increase in productivity with the latest products by OpenAI, Stable Diffusion and others. However the ability of cybersecurity professionals to enact appropriate security measures around these products is yet to catch up.

On the other hand cybercriminals have added generative AI as another weapon in their arsenal. Examples of how AI is being used in cybercrimes are:

• Phishing attacks: Generative AI has made it easier to create more targeted and sophisticated phishing campaigns.

• Voice cloning: Cybercriminals can now clone voices with just a few seconds of audio, enabling various scams.

• Deepfakes: AI-generated images and videos of celebrities are being used to trick people into scams.

• Enhanced password cracking: AI can generate more relevant password lists based on publicly available data.

• CAPTCHA bypass: AI-powered bots are becoming faster and more accurate at solving CAPTCHAs.

• Prompt injection: Attackers can manipulate AI models to override developer instructions and subvert safeguards.

• Code generation: AI makes it easier for individuals with little coding knowledge to develop and launch attacks.

So how can companies safeguard their assets and indeed their AI models against these vulnerabilities:

• Implement AI-powered security solutions: Ultimately "the best defense against AI is AI." Companies should invest in advanced AI-powered security tools that can detect and respond to AI-generated threats more effectively than traditional security measures.

• Educate employees: Provide comprehensive training on AI-related security risks, best practices for using AI tools, and how to identify potential AI-generated threats like deepfakes or sophisticated phishing attempts.

• Establish clear AI usage policies: Develop and enforce company-wide guidelines for the safe and responsible use of AI tools, including data handling and prompt engineering practices.

• Adopt a layered security approach: Don't rely solely on AI-specific security measures. Maintain and strengthen traditional security protocols like firewalls, endpoint detection, and network monitoring.

• Regular security audits and penetration testing: Conduct frequent assessments of AI systems and applications to identify vulnerabilities and address them promptly.

• Implement strong authentication methods: As voice cloning becomes more sophisticated, companies should move beyond voice authentication and implement multi-factor authentication systems.

• Monitor and restrict AI model access: Limit access to AI models and closely monitor their usage to prevent unauthorized manipulations or prompt injections.

• Invest in AI model security research: Support or conduct research into methods for securing AI models, such as watermarking or improved prompt sanitization techniques.

• Stay informed and adaptable: Given the rapidly evolving nature of AI technology and associated threats, companies must stay informed about the latest developments and be prepared to quickly adapt their security strategies.

It's crucial to recognize that this is an ongoing process that requires continuous vigilance and adaptation as both AI capabilities and potential threats evolve.

AI and Education

Andrej Karpathy launches Eureka Labs

Andrej Karpathy, a founding member of OpenAI, is announcing the launch of Eureka Labs, an AI+Education company he's starting. The company aims to create an AI-native school, with their first product being an undergraduate-level AI course that would enable students to build a simpler version of the AI teaching assistant! This venture combines Andrej's longstanding passions for both AI and education, which he has pursued for about two decades. In his tweet he explains how it would feel like to be tutored in physics by Feynman, a feat which is now possible thanks to the developments in generative AI. The ultimate goal is to enable anyone to learn anything they want. The announcement reflects Andrej's excitement about building Eureka Labs publicly and his vision for using AI to increase human potential through education.

AI and Economics

Taiwan basks in the bounties of the AI boom

Taiwan's tech industry, especially companies involved in AI-related hardware, is experiencing rapid growth and increased profits.

• Companies like Quanta Computer, which makes high-end servers for AI, are seeing jumps in share prices and profits, leading to increased employee bonuses and wealth creation.

• Taiwan is expected to see a 47% increase in millionaires by 2028, largely driven by growth in the semiconductor industry.

• The wealth effect is spreading beyond just senior managers and engineers, reaching a broader range of employees and sectors.

• Taiwan Semiconductor Manufacturing Company (TSMC), the world's largest chipmaker, is a major driver of this wealth creation.

• Many tech companies are giving out large bonuses and increasing salaries significantly.

• The wealth is also positively impacting other sectors of the economy, such as real estate and high-end consumer goods.

• There's a growing labor shortage in Taiwan, which is driving up wages even in traditionally lower-paying sectors like hospitality and tourism.

• Despite these challenges, some officials hope that the AI boom will continue to spread wealth more broadly throughout the economy.

  

AI and OpenAI

OpenAI continues to deliver

OpenAI announced GPT-4o mini, their most cost-efficient small language model.

GPT-4o mini’s key features:

• OpenAI's most cost-efficient small language model

• Outperforms GPT-3.5 Turbo and other small models on various benchmarks

• Scores 82% on MMLU (textual intelligence and reasoning benchmark)

• Supports text and vision inputs, with a 128K token context window

• Designed for applications requiring multiple model calls, large context, or real-time responses

Built-in Safety Measures:

• Incorporates the same safety mitigations as GPT-4o

• Uses pre-training filtering to exclude undesirable content

• Employs post-training alignment techniques like RLHF

• Assessed using automated and human evaluations

• Implements a new "instruction hierarchy" method to resist jailbreaks and prompt injections

• Continuously monitored and improved for safety

Availability and Pricing:

• Available in the Assistants API, Chat Completions API, and Batch API

• Accessible to ChatGPT Free, Plus, and Team users

• Will be available to Enterprise users starting next week

• Fine-tuning capabilities coming soon

• Priced at 15 cents per 1M input tokens and 60 cents per 1M output tokens

OpenAI researchers employ Prover-Verfier games to make LLMs more legible

This research focuses on improving the legibility of AI-generated solutions, particularly for complex tasks like math problems. Here's a summary of the key points:

1. Problem: When language models are optimized solely for correctness, their solutions can become difficult for humans to understand and verify.

2. Approach: The researchers developed a method inspired by "prover-verifier games" to improve the legibility of AI-generated solutions.

3. Method:

   • A strong "prover" model generates solutions.

   • A weaker "verifier" model checks these solutions for accuracy.

   • The prover is trained to create solutions that the verifier can easily understand and verify.

4. Training Process:

   • Alternates between training the verifier to assess solutions and training the prover to generate solutions.

   • The prover plays both "helpful" and "sneaky" roles to improve the verifier's robustness.

5. Results:

   • This method achieved a balance between performance (correctness) and legibility.

   • Human evaluators made fewer errors when assessing solutions generated by this method compared to solutions optimized only for correctness.

6. Benefits:

   • Improves trust in AI-generated outputs.

   • Reduces reliance on human demonstrations for defining legibility.

   • Potentially applicable to aligning future AI systems with human values.

7. Limitations:

   • Tested on only one dataset.

   • Still requires ground truth labels.

8. Conclusion: This approach is seen as a step towards developing AI systems that produce not only correct but also transparently verifiable outputs, enhancing trust and safety in real-world applications.

Amongst other things

• France’s Mistral released a pair of AI models:
  Codestral Mamba specialises in code generation
  Mathstral designed for advanced mathematical problem solving
  

• Apple and Meta withhold future AI models for EU countries.

• Wild Gaussians: New AI method enables 3D reconstruction from user-captured web photos

• Fei-Fei Li build a $1 billion AI startup focusing on spatial awareness

• Is AI already sentient ? 74% of Americans either do think that or are not sure

• Nvidia completes its fourth acquisition of the year

• AMD unveiled just its latest AI PC chips

Tools to make you cool

• Hidden Layer: A security platform for AI that provides real-time awareness of your models health and security

• Brev: Easy to use GPUs on the cloud

• TypeflowAI: Build AI tools without coding

• Veedo: AI powered insights into your video content

• Archie: Design and plan software applications with AI

Super links

AI Generated Images

prompt: a woman made out of ice, in the style of balanced symmetry, creamy white and purple/gold, detailed facial features, organic forms, meticulous portraiture, complex patterns
--chaos 2 --ar 1:2 --stylize 350

Thanks for reading.

Hamza from AI and Security.