Evaluating LLMs with Chatbot Arena and Joseph E. Gonzalez

Evaluating LLMs with Chatbot Arena and Joseph E. Gonzalez

In this episode of Gradient Dissent, Joseph E. Gonzalez, EECS Professor at UC Berkeley and Co-Founder at RunLLM, joins host Lukas Biewald to explore innovative approaches to evaluating LLMs.

They discuss the concept of vibes-based evaluation, which examines not just accuracy but also the style and tone of model responses, and how Chatbot Arena has become a community-driven benchmark for open-source and commercial LLMs. Joseph shares insights on democratizing model evaluation, refining AI-human interactions, and leveraging human preferences to improve model performance. This episode provides a deep dive into the evolving landscape of LLM evaluation and its impact on AI development.

🎙 Get our podcasts on these platforms:

Apple Podcasts: http://wandb.me/apple-podcasts

Spotify: http://wandb.me/spotify

Google: http://wandb.me/gd_google

YouTube: http://wandb.me/youtube

Follow Weights & Biases:

https://twitter.com/weights_biases

https://www.linkedin.com/company/wandb

Join the Weights & Biases Discord Server:

https://discord.gg/CkZKRNnaf3

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(140)

Elon's Former Battery Chief: AI Data Centers Will Make Electricity Cheaper

Elon's Former Battery Chief: AI Data Centers Will Make Electricity Cheaper

Can today's power grid handle the explosive growth of AI data centers? Tesla Alum Drew Baglino, now founder & CEO of Heron Power, doesn't think so.In this episode, he joins Lukas Biewald to talk about...

18 Aug 1h 34min

40 Trillion Tokens a Day (Yes, More Than OpenAI) | Lin Qiao, CEO of Fireworks

40 Trillion Tokens a Day (Yes, More Than OpenAI) | Lin Qiao, CEO of Fireworks

Should American companies be worried about Chinese open-source AI models?Lin Qiao, CEO of Fireworks, doesn't think so.In this episode, she joins Lukas Biewald to talk about why she believes the indust...

3 Aug 1h 19min

He's Building an AI That Can't Lie | Dan Klein

He's Building an AI That Can't Lie | Dan Klein

"We are going to switch from the problem in AI being that nothing works to the problem being that everything works."Dan Klein has been studying language models for over two decades and is now a profes...

16 Juni 1h 14min

He Raised $70M to Cure Every Disease With AI | Samuel G. Rodriques, Edison Scientific

He Raised $70M to Cure Every Disease With AI | Samuel G. Rodriques, Edison Scientific

Samuel Rodriques left physics because there were no unsolved problems left.Instead, he built an AI scientist named Kosmos to cure every disease, solve aging, and map the human brain.In this episode:Th...

26 Maj 1h 14min

Uber, Nissan, and Mercedes Chose This Self-Driving Startup | Alex Kendall, Wayve

Uber, Nissan, and Mercedes Chose This Self-Driving Startup | Alex Kendall, Wayve

"Every vehicle is capable of driverless operation. That's clearly the steady state of where we're going."Wayve started in a rented house in Cambridge with $1.5M, a car in the garage, and an aim to int...

15 Apr 45min

Why Netflix, Uber, and Spotify Never Lag: The Database Nobody Talks About | Aaron Katz

Why Netflix, Uber, and Spotify Never Lag: The Database Nobody Talks About | Aaron Katz

"Companies designing for agents, not humans, are going to get a lot of lift."ClickHouse started as an internal tool at Yandex. Today it's the database Anthropic, OpenAI, Meta and Tesla all run on.In t...

31 Mars 43min

The $64M Bet on an AI That Has to Be Right | Carina Hong, CEO of Axiom

The $64M Bet on an AI That Has to Be Right | Carina Hong, CEO of Axiom

Formal verification already consumes years of human effort.In this episode, Lukas Biewald talks with Carina Hong, Founder & CEO of Axiom, about why verification is becoming the real bottleneck in high...

5 Feb 50min

What a $42B Software Co. Really Spends on AI Tools

What a $42B Software Co. Really Spends on AI Tools

“I don't worry about being replaced by AI. I worry about being replaced by someone who's really good at using AI.”Atlassian has 10,000+ engineers currently split-testing the world’s top AI coding tool...

20 Jan 1h 7min

Populärt inom Business & ekonomi

framgangspodden
rss-jossan-nina
varvet
badfluence
rss-borsens-finest
uppgang-och-fall
avanzapodden
tabberaset
fill-or-kill
skaraborgspodden
svd-tech-brief
rss-inga-dumma-fragor-om-pengar
24fragor
rss-kort-lang-analyspodden-fran-di
lastbilspodden
bathina-en-podcast
rss-dagen-med-di
rikatillsammans-om-privatekonomi-rikedom-i-livet
borsmorgon
rss-hos-psykologen