A couple weeks ago, a popular account on twitter posted this: This sparked a bit of discussion, including this quote tweet: I think these tweets, particularly the second one, demonstrate some common misconceptions about evaluations and benchmarking that I’ve been seeing recently, so I figured this could be a useful case study to explore how …
Category Archives: Uncategorized
Eight Things to Know about Large Language Models
I’m sharing a draft of a slightly-opinionated survey paper I’ve been working on for the last couple of months: Eight Things to Know about Large Language Models. Here are the eight things: LLMs predictably get more capable with increasing investment, even without targeted innovation. Many important LLM behaviors emerge unpredictably as a byproduct of increasing …
Continue reading “Eight Things to Know about Large Language Models”
Why I Think More NLP Researchers Should Engage with AI Safety Concerns
Large language modeling research in NLP seems to be feeding into much more impactful technologies than we’re used to working with. While the positive potential for this technology could be tremendous, the downside risk is also potentially catastrophic, and it doesn’t look like we’re prepared to manage that risk. I’m starting a new research group …
Continue reading “Why I Think More NLP Researchers Should Engage with AI Safety Concerns”