The advances of large foundation models necessitate wide-coverage, low-cost, and zero-contamination benchmarks. Despite continuous exploration of language model evaluations, comprehensive studies on the evaluation of Large Multi-modal Models (LMMs) remain limited. In this work, we introduce LMMS-EVAL, a unified and standardized multimodal benchmark framework with over 50 tasks and more than 10 mod
Discover how prover-verifier games improve the legibility of language model outputs, making AI solutions clearer, easier to verify, and more trustworthy for both humans and machines.
The department is pleased to announce the appointment of Stefano Germano and Ulrik Lyngs as Research Community Coordinators, as they aim to enhance collaboration and inclusivity within our community of researchers
Official technical announcement and publication from Hugging Face covering How we leveraged distilabel to create an Argilla 2.0 Chatbot.
Official technical announcement and publication from Hugging Face covering SmolLM - blazingly fast and remarkably powerful.
Official Mistral AI technical update and publication covering Codestral Mamba.
Official Mistral AI technical update and publication covering MathΣtral.
A research team led by Professor Oege de Moor developed an innovative program analysis solution that allows for automated application to complex problems and code
Research led by three professors at the Department of Computer Science has had significant impact on the development and deployment of semantic technology across the world through state-of-the-art reasoning systems
This research project uncovered critical flaws in Bluetooth systems, leading to changes to its core technology and urgent modifications by industry leaders such as Intel, Microsoft, Apple, Cisco, Google, and Huawei
Today’s aviation industry relies on the security of these systems, and this research project revealed a range of security and privacy challenges, resulting in major adjustments across the industry
Official technical announcement and publication from Hugging Face covering How NuminaMath Won the 1st AIMO Progress Prize.
OpenAI and Los Alamos National Laboratory are working to develop safety evaluations to assess and measure biological capabilities and risks associated with frontier models.
Official technical announcement and publication from Hugging Face covering Announcing New Hugging Face and KerasHub integration.
Official technical announcement and publication from Hugging Face covering Experimenting with Automatic PII Detection on the Hub using Presidio.
Official technical announcement and publication from Hugging Face covering Preference Optimization for Vision Language Models.
Official technical announcement and publication from Hugging Face covering Banque des Territoires (CDC Group) x Polyconseil x Hugging Face: Enhancing a Major French Environmental Program with a Sovereign Data Solution.
Official technical announcement and publication from Hugging Face covering Google Cloud TPUs made available to Hugging Face users.
Official technical announcement and publication from Hugging Face covering Announcing New Dataset Search Features.
The department is pleased to announce the appointment of five Associate Professors with Tutorial Fellowships (APTFs), aligning with its strategic priority of increasing the capacity and enhancing the world-leading quality of its teaching.
Official technical announcement and publication from Hugging Face covering Accelerating Protein Language Model ProtST on Intel Gaudi 2.
Official technical announcement and publication from Hugging Face covering Our Transformers Code Agent beats the GAIA benchmark 🏅.
Official LMSYS Chatbot Arena release and benchmark update covering RouteLLM: An Open-Source Framework for Cost-Effective LLM Routing.
Test set contamination, wherein test data from a benchmark ends up in a newer model's training set, is a well-documented obstacle for fair LLM evaluation and can quickly render benchmarks obsolete. To mitigate this, many recent benchmarks crowdsource new prompts and evaluations from human or LLM judges; however, these can introduce significant biases, and break down when scoring hard questions. In
CriticGPT, a model based on GPT-4, writes critiques of ChatGPT responses to help human trainers spot mistakes during RLHF
We’re partnering with TIME and its 101 years of archival content to enhance responses and provide links to stories on Time.com
Official technical announcement and publication from Hugging Face covering Welcome Gemma 2 - Google’s new open LLM.
Official LMSYS Chatbot Arena release and benchmark update covering The Multimodal Arena is Here!.
Official technical announcement and publication from Hugging Face covering XLSCOUT Unveils ParaEmbed 2.0: a Powerful Embedding Model Tailored for Patents and IP with Expert Support from Hugging Face.
Official technical announcement and publication from Hugging Face covering Ethics and Society Newsletter #6: Building Better AI: The Importance of Data Quality.
Official technical announcement and publication from Hugging Face covering Fine-tuning Florence-2 - Microsoft's Cutting-edge Vision Language Models.
By Ian Gormely The third and final day of this year’s Collision Conference saw talks that outlined AI’s potential to aid in some of the world’s most wicked problems while […] The post Climate change and AI-compute cap off the third and final day of Collision 2024 appeared first on Vector Institute for Artificial Intelligence .
Research Associate Ulrik Lyngs has won an MPLS Early-Career Social Impact Award for his ‘Reduce Digital Distraction’ (ReDD) Workshop
OpenAI Acquires Rockset
By Natalie Richard AI was front and centre at the Collision Conference this week. “You’re probably done hearing about it. Stop talking about AI!” joked Shingai Manjengwa, Head of AI […] The post ChainML, Private AI, and Geoffrey Hinton underscore the importance of responsible AI development and governance at Collision 2024 appeared first on Vector Institute for Artificial Intelligence .
Highlighting innovative research and AI integration in cybersecurity
Official technical announcement and publication from Hugging Face covering Data Is Better Together: A Look Back and Forward.
Consistency models are a nascent family of generative models that can sample high quality data in one step without the need for adversarial training.
We present a holistic approach to building a robust and useful natural language classification system for real-world content moderation.
Diffusion models have significantly advanced the fields of image, audio, and video generation, but they depend on an iterative sampling process that causes slow generation.
By Natalie Richard Self-driving trucks could be a reality on our roads as soon as next year, says Vector Institute Co-Founder and Faculty Member Raquel Urtasun. Speaking during her centre […] The post Self-driving trucks will be on the road next year says Vector co-founder Raquel Urtasun at Collision 2024 appeared first on Vector Institute for Artificial Intelligence .
Researchers from the Department of Computer Science have made a significant advance towards ensuring that information produced by generative artificial intelligence (AI) is robust and reliable
Official technical announcement and publication from Hugging Face covering Going multimodal: How Prezi is leveraging the Hub and the Expert Support Program to accelerate their ML roadmap.
Paf adopted ChatGPT Enterprise across its entire company, with engineers using custom GPTs on a daily basis to speed up routine development tasks. Paf also integrated ChatGPT Enterprise into the grit:lab coding academy (gritlab.ax), training the next generation of software developers using an AI-augmented, systems-architecture mindset from day one. In addition to the wide range of use cases for de
Official technical announcement and publication from OpenAI covering Achieving 10x growth with agentic sales prospecting.
Official technical announcement and publication from Hugging Face covering BigCodeBench: The Next Generation of HumanEval.
Vector researchers are presenting more than 12 papers at this year’s IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 (CVPR 2024). The conference is being held in Seattle, WA […] The post Vector researchers are presenting over a dozen papers at CVPR 2024 appeared first on Vector Institute for Artificial Intelligence .
DPhil student Edd Salkield was lead author of a paper that took top honours in two categories at the European Space Agency's inaugural Security for Space Systems (3S) conference, winning both the Best Paper and Best Student Paper awards
Color Health is working with OpenAI to pioneer a new way of accelerating cancer patients’ access to treatment. Their new Cancer Copilot application uses GPT-4o to identify missing diagnostics and create tailored workup plans, enabling healthcare providers to make evidence-based decisions about cancer screening and treatment.
Former DPhil student Ruiwen Dong has received a Distinguished Dissertation Award from the European Association for Theoretical Computer Science (EATCS) for his thesis
Nakasone brings cybersecurity experience to growing Board of Directors; will join the Board’s Safety and Security Committee
A paper co-authored by Professor Alessandro Abate has been awarded the Test-of-Time Award at this year’s Hybrid Systems: Computation and Control (HSCC) conference
Official technical announcement and publication from Hugging Face covering From DeepSpeed to FSDP and Back Again with Hugging Face Accelerate.
Official technical announcement and publication from Hugging Face covering Diffusers welcomes Stable Diffusion 3.
Official technical announcement and publication from Hugging Face covering Putting RL back in RLHF.
OpenAI and Apple announce partnership to integrate ChatGPT into Apple experiences.
We are delighted to announce the winners of our departmental Teaching Awards 2023/24. These annual awards recognise the remarkable commitment and teaching demonstrated by colleagues
OpenAI welcomes Sarah Friar (CFO) and Kevin Weil (CPO)
Exploring the technology behind our text-to-speech model.
Official technical announcement and publication from Hugging Face covering Introducing the Hugging Face Embedding Container for Amazon SageMaker.
Official technical announcement and publication from Hugging Face covering Making sense of this mess.
Researchers from the Department of Computer Science and EY have published a White Paper on responsible quantum computing, offering insights into the future of the technology
Official technical announcement and publication from OpenAI covering Improving India’s critical care infrastructure.
Official technical announcement and publication from Hugging Face covering Launching the Artificial Analysis Text to Image Leaderboard & Arena.
Using new techniques for scaling sparse autoencoders, we automatically identified 16 million patterns in GPT-4's computations.
We’ve been working closely with a diverse range of companies and developers to make Devin a more collaborative, knowledgeable, and productive teammate. We’re excited to share some recent improvements here.
Official technical announcement and publication from Hugging Face covering Introducing NPC-Playground, a 3D playground to interact with LLM-powered NPCs.
Official Mistral AI technical update and publication covering My Tailor is Mistral.
Official Mistral AI technical update and publication covering Mistral AI Fine-tuning Hackathon.
Official technical announcement and publication from Hugging Face covering Faster assisted generation support for Intel Gaudi.
Official technical announcement and publication from Hugging Face covering Space secrets security update.
Announcing Sonic: a low‑latency voice model for lifelike speech
We’ve terminated accounts linked to covert influence operations; no significant audience increase due to our services.
An affordable offering for universities to responsibly bring AI to campus.
We’re launching a new initiative to enhance the accessibility of our tools for nonprofit organizations, including discounted rates for ChatGPT Team and Enterprise.
By Arber Kacollja The Vector Institute’s recent Computer Vision (CV) workshop brought together members of the Vector research community to showcase and discuss new work in the field. Recent years […] The post Vector Institute Computer Vision Workshop showcases the field’s current capabilities and future potential appeared first on Vector Institute for Artificial Intelligence .
MavenAGI is a new software company for the AI era. They recently launched an AI customer service agent, built on the flexibility of GPT-4, which a number of companies like Tripadvisor, Clickup and Rho are already using to save time and better serve their customers.
Official technical announcement and publication from OpenAI covering The Newsroom AI Catalyst: a global program with WAN-IFRA.
The Atlantic is announcing a strategic content and product partnership with OpenAI, which positions The Atlantic as a premium news source within OpenAI. The Atlantic’s articles will be discoverable within OpenAI’s products, including ChatGPT, and as a partner, The Atlantic will help to shape how news is surfaced and presented in future real-time discovery products.
In a multi-faceted agreement, Vox Media’s content will enhance the output of OpenAI’s ChatGPT, and the company will build on OpenAI’s technology to develop products to better serve its audiences and advertisers.
Official technical announcement and publication from Hugging Face covering Benchmarking Text Generation Inference.
Official Mistral AI technical update and publication covering Codestral.
Official Mistral AI technical update and publication covering The Mistral AI Non-Production License.
By Gautam Kamath As statistics and machine learning are applied in increasingly broad settings, we need to prepare our methods to face wide-ranging challenges. Some — like gathering data from […] The post Vector researcher Gautam Kamath breaks down the latest developments in robustness and privacy appeared first on Vector Institute for Artificial Intelligence .
Official technical announcement and publication from OpenAI covering OpenAI Board Forms Safety and Security Committee.
Official technical announcement and publication from Hugging Face covering Training and Finetuning Embedding Models with Sentence Transformers.
Official technical announcement and publication from Hugging Face covering CyberSecEval 2 - A Comprehensive Evaluation Framework for Cybersecurity Risks and Capabilities of Large Language Models.
Official technical announcement and publication from Hugging Face covering Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens and 11 languages.
Companies Join Forces to Enrich OpenAI’s Generative AI Products and Platforms with Premium Journalism
Official technical announcement and publication from Hugging Face covering Deploy models on AWS Inferentia2 from Hugging Face.
– Showcases Unparalleled AI Trust and Safety Expertise in Canada, including five experts from the Vector Institute – Provides important and timely recommendations as world policy and business leaders gather […] The post World-leading AI Trust and Safety Experts Publish Major Paper on Managing AI Risks in the journal Science appeared first on Vector Institute for Artificial Intelligence .
Artificial general intelligence has the potential to benefit nearly every aspect of our lives—so it must be developed and deployed responsibly.
Official technical announcement and publication from Hugging Face covering From cloud to developers: Hugging Face and Microsoft Deepen Collaboration.
Official technical announcement and publication from Hugging Face covering Build AI on premise with Dell Enterprise Hub.
Official technical announcement and publication from Hugging Face covering Introducing Spaces Dev Mode for a seamless developer experience.
Official technical announcement and publication from Hugging Face covering Hugging Face on AMD Instinct MI300 GPU.
How the voices for ChatGPT were chosen We worked with industry-leading casting and directing professionals to narrow down over 400 submissions before selecting the 5 voices.
Official LMSYS Chatbot Arena release and benchmark update covering Introducing Hard Prompts Category in Chatbot Arena.
DPhil student Lia Yeh talks about her experience attending a Responsible Quantum Technologies workshop and explains why it is important quantum is made accessible to all
Improvements to data analysis in ChatGPT Interact with tables and charts and add files directly from Google Drive and Microsoft OneDrive.
6584 articles sourced historically · 100 per page