WordPress Ad Banner

Meta Unveils Llama 3: The Next Leap in Open Generative AI Models

Meta has launched the latest iteration of its renowned Llama series of open generative AI models: Llama 3. With two models already released and more to follow, Meta promises significant advancements in performance and capabilities compared to its predecessors, Llama 2 8B and Llama 2 70B.

Meta Llama 3

Meta introduces two models in the Llama 3 family: Llama 3 8B, boasting 8 billion parameters, and Llama 3 70B, with a staggering 70 billion parameters. These models represent a major leap forward in performance and are among the best-performing generative AI models available today.

Performance Benchmarks: Meta highlights Llama 3’s impressive performance on popular AI benchmarks such as MMLU, ARC, and DROP. The company claims superiority over comparable models like Mistral 7B and Gemma 7B, showcasing dominance in multiple benchmarks.

Meta Llama 3

Enhanced Capabilities: Llama 3 offers users more “steerability,” lower refusal rates, and higher accuracy across various tasks, including trivia, history, STEM fields, and coding recommendations. Llama 3’s larger dataset, comprising 15 trillion tokens, and advanced training techniques contribute to these improvements.

Meta Llama 3

Data Diversity and Safety Measures

Meta emphasizes the diversity of Llama 3’s training data, sourced from publicly available sources and including synthetic data to enhance performance across different languages and domains. The company also introduces new safety measures, including data filtering pipelines and generative AI safety suites, to address toxicity and bias concerns.

Meta Llama 3

Availability and Future Plans

Llama 3 models are available for download and will soon be hosted on various cloud platforms. Meta plans to expand Llama 3’s capabilities, aiming for multilingual and multimodal capabilities, longer context understanding, and improved performance in core areas like reasoning and coding.

Conclusion: With the release of Llama 3, Meta continues to push the boundaries of open generative AI models, offering researchers and developers powerful tools for innovation. While not entirely open source, Llama 3 promises groundbreaking advancements and sets the stage for future developments in AI technology.

Llama 2 Long: Redefining AI for Handling Complex User Queries

Meta Platforms has unveiled a groundbreaking AI model that may have slipped under the radar during its annual Meta Connect event in California. While the tech giant showcased numerous AI-powered features for its popular apps like Facebook, Instagram, and WhatsApp, the real standout innovation is Llama 2 Long, an extraordinary AI model designed to provide coherent and relevant responses to extensive user queries, surpassing some of the leading competitors in the field.

Llama 2 Long is an extension of the previously introduced Llama 2, an open-source AI model from Meta known for its versatility in tasks ranging from coding and mathematics to language comprehension, common-sense reasoning, and conversational abilities. What sets Llama 2 Long apart is its capacity to handle more substantial and complex inputs, making it a formidable rival to models like OpenAI’s GPT-3.5 Turbo and Claude 2, which struggle with extended contextual information.

The inner workings of Llama 2 Long are a testament to Meta’s dedication to pushing the boundaries of AI technology. Meta’s research team used varying versions of Llama 2, spanning from 7 billion to 70 billion parameters, which are the adjustable values that govern how the AI model learns from data. They augmented the model with an additional 400 billion tokens of data containing longer texts compared to the original Llama 2 dataset.

Furthermore, the architecture of Llama 2 underwent subtle alterations, primarily in how it encodes the position of each token within a sequence. The introduction of Rotary Positional Embedding (RoPE) proved pivotal, as it allowed each token to be mapped onto a 3D graph that reflects its relationship with other tokens, even when rotated. This innovation enhances the model’s accuracy and efficiency, reducing its reliance on extensive information and memory, which sets it apart from other techniques.

The researchers took the innovative step of reducing the rotation angle of the RoPE encoding from Llama 2 to Llama 2 Long, enabling the model to accommodate more distant or less frequent tokens in its knowledge base. Additionally, they employed reinforcement learning from human feedback (RLHF) and synthetic data generated by Llama 2 itself to fine-tune the model’s performance across various tasks.

The paper detailing Llama 2 Long’s capabilities asserts that the model can generate high-quality responses to user queries containing up to 200,000 characters, equivalent to approximately 40 pages of text. The paper provides illustrative examples of Llama 2 Long’s responses across a range of subjects, including history, science, literature, and sports.

Meta’s researchers regard Llama 2 Long as a significant stride towards the development of more versatile and general AI models capable of addressing diverse and intricate user needs. They also acknowledge the ethical and societal implications of such models, emphasizing the need for further research and dialogue to ensure their responsible and beneficial utilization.

In conclusion, Meta’s introduction of Llama 2 Long represents a remarkable advancement in the realm of AI, with the potential to revolutionize how AI models handle complex and extensive user queries while also underlining the importance of ethical considerations in their deployment.

Meta Denies Plans for WhatsApp Ads, Focusing on Alternative Revenue Streams

Meta has refuted recent reports suggesting that it is considering introducing advertisements to the popular messaging platform WhatsApp, dispelling the notion that ads might soon invade the user experience.

An article in the Financial Times had claimed that some teams within Meta were contemplating the idea of displaying ads within lists of conversations on WhatsApp’s home screen. However, in a formal statement, WhatsApp made it clear that they were neither testing this concept nor actively working on it, and it was not part of their foreseeable plans.

The notion of integrating ads into WhatsApp has been a topic of speculation among industry analysts for some time. As Meta explores various avenues to monetize WhatsApp, a platform used daily by over 2 billion individuals worldwide, the question of whether advertising will eventually make its way to the app remains unanswered. In contrast, Instagram, another Meta-owned platform, has made significant strides in terms of monetization.

However, Meta has thus far resisted the idea of incorporating ads into the WhatsApp experience. Instead, they have relied on generating revenue through WhatsApp Business, a specialized offering designed for merchants who pay for specific services. WhatsApp Business has already garnered more than 200 million monthly active users.

Meta announced changes to the pricing structure and messaging categories of WhatsApp Business in February, aimed at boosting revenue. These new categories encompass utility, authentication (for sending one-time passcodes), marketing, and user-initiated service conversations.

During Meta’s Q3 earnings call last year, Mark Zuckerberg highlighted the success of “click-to-WhatsApp” ads, which had achieved an annual revenue run rate of $1.5 billion, with an impressive 80% year-on-year growth. The company has plans to introduce personalized messaging models for merchants in the near future.

Furthermore, Meta is actively facilitating peer-to-peer and customer-to-merchant payments in countries like India, Brazil, and Singapore. They have also announced their intention to integrate payments into the recently launched global feature, Channels, demonstrating their ongoing commitment to enhancing the WhatsApp experience.

In light of these developments, industry analysts at AllianceBernstein noted, “With WhatsApp Business users reaching 200 million monthly active users and the introduction of GenAI customer service tools, Meta seems to be taking steps towards monetizing its vast user base of over 2 billion active users. ‘Click to Message’ ads are already performing strongly, providing a substantial revenue stream and fostering interaction between users and businesses within the WhatsApp platform.”

Meta Blocks News Content in Canada Amidst Controversial Online News Act

Facebook and Instagram users in Canada will notice a significant change in their feeds as Meta, the parent company of these platforms, begins blocking access to links and stories from news publishers. This move is a response to the recently passed Canadian Online News Act, which aims to address the declining news industry by forcing tech platforms to negotiate fair revenue sharing with publishers for their content. When negotiations fail, the law allows for mandatory arbitration, putting tech companies in a difficult position.

Meta’s policy communications director, Andy Stone, expressed their disagreement with the law on Twitter, stating that it is based on a flawed premise. Consequently, the only way Meta can comply is by ending news availability in Canada, and this process will be rolled out gradually over the next few weeks.

Google also plans to follow suit by implementing a news blackout in its search results in response to the same law.

For over a decade, tech platforms have benefited from publishers’ original content without compensating them, leading to a decline in the news industry. Despite some attempts to fund news initiatives, Meta has now taken an adversarial stance and distanced itself from the news content. They argue that news has little economic value for them, which has sparked criticism given the significant engagement news and political content generate on their platforms.

Critics of the Canadian Online News Act point out that the news industry is already heavily reliant on social networks, and this legislation might deepen their dependence, necessitating new and more sustainable solutions. Additionally, some argue that the forced negotiation frameworks could favor large media groups over smaller, independent publishers.

While controversial, these laws could pave the way for future regulations worldwide, with California also considering a proposal to make social platforms pay for content, on hold until 2024. As the situation unfolds in Canada, it will likely be a significant indicator of how such laws impact the relationships between social platforms and news publishers.

Meta to Introduce AI-Powered Chatbots ‘Personas’ in September

Financial Times (FT) recently reported (2 Aug 2023) that Meta, the owner of the tech giant company, is gearing up to launch a series of AI-backed chatbots. These innovative chatbots, set to debut next month, aim to enhance engagement with Meta’s massive user base of nearly four billion users by offering human-like discussions.

The company has been developing persona-based chatbot prototypes, each exhibiting distinct personalities, including an AI impersonating Abraham Lincoln and another providing travel advice with a laid-back surfer style. The primary purpose of these chatbots will be to offer a novel search function, provide recommendations, and entertain users with interactive experiences.

With several tech giants like Microsoft, Google, and Elon Musk’s ventures entering the AI space, Meta’s move into the AI industry is driven by the goal of attracting and retaining users amidst fierce competition from other social media platforms like TikTok and Twitter.

While the AI-backed chatbots present exciting possibilities, experts have raised concerns regarding user data privacy, manipulation, and potential data exploitation. Ravit Dotan, an AI ethics adviser and researcher, pointed out that interacting with chatbots exposes more user data to companies, leading to concerns about privacy and potential manipulation of users’ preferences.

Meta’s CEO, Mark Zuckerberg, envisions these AI agents serving as assistants, coaches, and facilitators for user interactions with businesses and creators. The company also plans to develop AI-powered productivity assistants for internal use and an avatar chatbot in the metaverse in the future.

To address ethical concerns, Meta intends to employ technology to screen users questions and automate checks on chatbot outputs, ensuring appropriate speech and preventing hate speech or rule-breaking vocabulary.

Meta is expected to provide further details about its AI product roadmap during the Connect developer event in September. As the tech industry rapidly advances in the AI race, Meta’s AI-powered chatbots will undoubtedly play a significant role in shaping the future of user interactions and content delivery.

Meta Unveils Llama 2, a New AI Competitor to ChatGPT and Bard

Meta Platforms, the parent company of Facebook, has recently introduced its latest AI system, Llama 2, which is set to rival OpenAI’s ChatGPT and Google’s Bard. The unique aspect of Llama 2 is that it will be available for free, strategically leveling the playing field for startups and businesses. This move allows smaller companies to compete with industry giants like OpenAI and Google, who offer expensive AI systems.

To distribute Llama 2, Meta has partnered with Microsoft, making it their “preferred partner.” Microsoft will provide access to the AI system through its widely-used Azure cloud service, expanding Meta’s reach across a vast network of businesses and developers already utilizing Azure’s services.

Llama 2 is part of Meta’s Large Language Model (LLM) series, which includes the backbone of generative AI products like ChatGPT. Building on advancements from previous language models, Meta aims to push the boundaries of AI capabilities, positioning itself as a strong contender in the AI landscape.

Mark Zuckerberg, Meta’s CEO, emphasizes the company’s commitment to openness and innovation with the release of Llama 2. By offering the technology for both research and commercial use, Meta aims to foster a collaborative environment for the development of new AI applications.

In line with their transparent approach, Meta has open-sourced Llama 2, promoting accessibility and idea exchange within the AI community. However, some concerns have been raised regarding the level of detail provided in the research paper introducing Llama 2. While the model was trained on publicly available data and excluded Meta’s proprietary products and services, information from websites containing personal data was removed to protect user privacy.

Users can access Llama 2 through various avenues, including direct download from Meta and access through Microsoft’s Azure cloud platform. Llama 2 will also be available on other platforms like Amazon Web Services and Hugging Face, ensuring widespread accessibility for developers and businesses worldwide.

During Microsoft’s Inspire event, the company also announced its new AI tool, Microsoft 365 Copilot, available to businesses for a monthly fee of $30 per user. This offering further diversifies Microsoft’s AI portfolio, providing businesses with an alternative AI solution to Meta’s free Llama2.

With the release of Llama2, Meta Platforms aims to drive innovation in the AI industry while promoting transparency and accessibility. By offering Llama2 for free, Meta enables startups and businesses to leverage AI without significant costs. As Llama2 enters the competitive landscape alongside ChatGPT and Bard, it will be intriguing to see how businesses and developers embrace this new AI rival and the impact it will have on the future of AI applications.

Meta Unveils CM3leon: Advancing Text-to-Image Generation

Meta is making significant progress in its research on generative AI models, unveiling its latest project called CM3leon (pronounced “chameleon”). CM3leon is a multimodal foundation model designed for text-to-image and image-to-text creation, specifically for generating automatic captions for images.

While AI-generated images are not new, with popular tools like Stable Diffusion, DALL-E, and Midjourney already available, Meta’s techniques and claimed performance for CM3leon set it apart.

Text-to-image generation typically relies on diffusion models, as seen in Stable Diffusion, for image creation. However, CM3leon takes a different approach by utilizing a token-based autoregressive model.

Meta researchers explain in their paper titled “Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning” that diffusion models have been dominant due to their strong performance and computational efficiency. On the other hand, token-based autoregressive models offer improved global image coherence but are more computationally expensive to train and use for inference.

Surprisingly, CM3leon demonstrates that the token-based autoregressive model can be more efficient than the diffusion model approach. Meta researcher stated in a blog post, “CM3leon achieves state-of-the-art performance for text-to-image generation, despite being trained with five times less compute than previous transformer-based methods.”

Meta’s approach to image training with CM3leon is centered on ethical considerations. Instead of scraping images from the internet, which has raised legal concerns for diffusion-based models, Meta sourced licensed images from Shutterstock. This decision allows them to sidestep issues related to image ownership and attribution without compromising performance.

CM3leon follows a two-stage process: retrieval-augmented pre-training and supervised fine-tuning (SFT). The pre-training stage enhances the model’s capabilities, while SFT optimizes resource utilization and image quality. Meta draws parallels between SFT and OpenAI’s use of the approach in training ChatGPT, emphasizing its effectiveness for generative tasks that require understanding complex prompts.

The research paper highlights that instruction tuning significantly enhances multi-modal model performance across various tasks, such as image caption generation, visual question answering, text-based editing, and conditional image generation.

Meta

Examining the sample sets of generated images shared by Meta in a blog post, CM3leon’s ability to understand complex, multi-stage prompts and produce high-resolution images is evident and impressive.

As of now, CM3leon is a research project, and it remains unclear when or if Meta will make this technology available to the public on its platforms. However, considering its impressive capabilities and increased efficiency in image generation, it seems highly likely that CM3leon and its approach to generative AI will eventually extend beyond research.

Meta Introduces Threads Beta Program for Android Users

In an exciting development, Meta has announced the launch of a beta program for Threads, specifically tailored for Android users. This program, revealed by a company engineer on Friday, aims to provide early access to new features and bug fixes. However, as with any beta program, users should be aware of the potential risks associated with downloading an unstable build.

Threads, which was introduced just two days prior, has already garnered an impressive user base of 70 million individuals. Despite its rapid success, the platform currently lacks some notable features, including direct messages, a “Following” feed, a full web version, and a chronological feed. Given the platform’s current minimalist structure, the allure of the new beta program may be strong for users seeking to explore forthcoming enhancements.

Invitations for the beta program highlight that certain data regarding users’ app usage will be collected and shared with the developers to aid in the app’s improvement. The invitations also caution that testing versions may exhibit instability.

Interested individuals can readily sign up for beta access today, as there is no waitlist. This means that anyone with an Android device can enter the program and try out future builds.

Threads, a platform closely integrated with Instagram, enables users to authenticate with their existing credentials, facilitating the posting of short updates comprising text (up to 500 characters), links, photos, and videos (up to five minutes long). Upon its launch, Threads became available on iOS and Android devices in 100 countries, excluding the EU due to concerns related to adhering to local data privacy regulations.

Remarkably, within its mere two-day existence, Threads has already captured the attention of Twitter, which is owned by Elon Musk. Twitter has threatened legal action against Threads, accusing Meta of poaching former Twitter employees to create the new platform. Meta, on the other hand, has firmly denied these allegations.

With the launch of the Threads beta program for Android users, Meta is undoubtedly providing an opportunity for enthusiastic individuals to actively participate in shaping the future of the platform. As users eagerly sign up for the beta program, Meta remains focused on refining and expanding Threads to meet the expectations and demands of its growing user base.

Twitter Threatens to Sue Meta Over Its New Threads App

In a swift response to Meta’s launch of its Threads app, Twitter has issued a threat to sue the company. The move comes amidst allegations that Meta has recruited former Twitter employees to develop the new platform.

Threads, a text-based platform similar to Twitter, has garnered over 30 million sign-ups within 24 hours of its release. Shortly after the launch, Twitter’s lawyer, Alex Spiro, sent a letter to Meta CEO Mark Zuckerberg, accusing the social media giant of unlawfully misappropriating Twitter’s trade secrets and intellectual property.

The letter, shared online by Semafor, stated, “Twitter intends to strictly enforce its intellectual property rights, and demands that Meta take immediate steps to stop using any Twitter trade secrets or other highly confidential information. Twitter reserves all rights, including legal remedies and injunctive relief, to prevent any further use or disclosure of its intellectual property by Meta.”

Spiro further alleged that Meta hired several former Twitter employees who had access to Twitter’s trade secrets and confidential information. These employees were supposedly tasked with developing a copycat app, utilizing Twitter’s intellectual property, in violation of state and federal laws and their ongoing obligations to Twitter.

In response, Meta’s communications director, Andy Stone, addressed Twitter’s claims in a Threads post, stating, “To be clear: No one on the Threads engineering team is a former Twitter employee—that’s just not a thing.”

Threads joins the ranks of Twitter competitors that gained momentum under Elon Musk’s leadership at Twitter. However, Twitter’s swift action against Threads signifies it as the most prominent rival.

While Musk has remained silent on Threads’ launch thus far, he tweeted support for Spiro’s claims, asserting that “Competition is fine, cheating is not.”

Twitter CEO Linda Yaccarino also took a jab at the new platform, tweeting, “Twitter is often imitated—but the Twitter community can never be duplicated.”

Coinciding with Threads’ launch, Zuckerberg made his first tweet in a decade, sharing a Spider-Man meme that humorously highlighted the platform’s resemblance to Twitter.

Amidst Twitter’s controversial decisions, the demand for alternative platforms has surged. Mastodon and Bluesky have gained attention, but none have matched the rapid adoption rate of Threads.

Threads is currently available on iOS and Android in 100 countries, excluding the EU due to concerns about complying with local data privacy regulations. To access Threads, users must authenticate using their existing Instagram login credentials, after which the app populates with their account details, including name, username, photo, and followers.

Meta Enhances Transparency and User Control in Content Recommendations Across Instagram and Facebook

Meta, the parent company of Instagram and Facebook, is taking steps to improve transparency and user control over its recommendation engine. In a recent blog post, Nick Clegg, Meta’s president of Global Affairs, expressed the company’s commitment to being more transparent about its AI systems and providing users with greater control over the content they see.

As part of this effort, Meta is introducing a new feature called “Why am I seeing this?” for Instagram and Facebook Reels, as well as Instagram’s Explore page. This option allows users to understand why a particular post is being shown to them, empowering them with insights into the content displayed on their feeds.

To further enhance user control, Meta is testing a system for Instagram that marks posts with “Interested,” enabling users to see more similar content. Additionally, the company is working on making Facebook’s “Show more, Show less” controls more prominent. While the exact location of these controls is not specified, Meta aims to make them easily accessible to users.

Meta has also released 22 “System Cards” that provide detailed explanations of various AI systems, including Facebook Feed recommendations, Facebook Group timelines, suggested people and groups, Instagram Reels recommendations, notifications system, and Stories AI. Although these cards offer valuable information, they may require some time for an average user to read and comprehend.

Clegg emphasized that Meta is also disclosing certain signals, such as liking or sharing a post, that influence user recommendations. However, the company has chosen not to reveal signals that could potentially be exploited to bypass their safeguards.

In parallel, Meta’s AI blog has discussed the possibility of developing AI models with tens of trillions of parameters, surpassing the current models like ChatGPT and GPT-4. Such colossal models can provide significant insights into user behavior, raising concerns about privacy and content recommendations.

To encourage further research, Meta is inviting academics and researchers to study its algorithms through a new Content Library and API. This initiative grants access to public posts, pages, groups, and events from Facebook, as well as public posts and data from creators and business accounts on Instagram.

Meta’s push for transparency and user control comes in response to increased scrutiny over the company’s content recommendation practices. Previous revelations by a Facebook whistleblower and a settlement with the U.S. Justice Department have highlighted the potential negative impact of algorithms on public content and user experiences.

While Meta aims to fill 30% of users feeds with algorithmic recommendations, the company’s recent release of documents and controls signifies an effort to address regulatory concerns and privacy advocates’ demands. Moreover, the rise of decentralized platforms like Bluesky and Mastodon has sparked conversations about granting users more control over the algorithms that shape their social media feeds. Centrally-operated platforms such as Instagram and Facebook could draw inspiration from these alternative networks to provide better algorithmic choices and control to their users.