New 'The Sing-Off' judge Sara Bareilles co-hosted the fourth hour of 'Today' (weekdays, 9AM ET on NBC) while Kathie Lee Gifford was off shooting her 'Hot in Cleveland' cameo. Bareilles, who's a sweetheart by nature, talked about the challenge of giving singers negative feedback, sang some a capella of her own and hoola-hooped with Hoda.

Bareilles struggled to figure out how to provide substantive criticism without coming off like a jerk. "It's hard to be critical sometimes. I see that now as I'm watching the show be aired, and I remember feeling self-conscious about wanting to point out places where they need to make improvements," she said.

NBC's A cappella talent competition 'The Sing-Off' returns for a new season tonight at 9PM ET. Host Nick Lachey made an appearance on 'Today' (9AM ET on NBC) to talk about the judging changes the show has made this season, and the inspiring musical atmosphere the show fosters. It's so contagious that Lachey has been working in the studio on a new project of his own.

'The Sing-Off' will have a new judges this year, as Sara Bareilles joins Ben Folds and rest of the crew. It will also have a longer season, as its moving to 16 episodes. Lachey said the new judges bring a wealth of energy, and experience, to a set that's already brimming with its own unique vibe. The show has struck such a nerve that not only is it creating A cappella groups as part of the competition, but singers are starting to form their own groups to compete for a spot in the competition.

Sara Bareilles, the woman responsible for 'Love Song' and most recently 'King of Anything,' has joined 'The Sing Off.'

"I'm extremely excited to be joining the panel of judges for 'The Sing-Off' this fall!" Bareilles said in a statement. "A cappella music is very near and dear to my heart, and I have been a huge fan of the show from the beginning. I love the opportunity to be a part of something that celebrates a genre of music that I just adore and believe in. I can't wait for it to start."

Sara Bareilles was more comfortable on the set of 'Chelsea Lately' (Weeknights, 11PM on E!) because of the more lax restrictions on language. A self-proclaimed potty mouth, Bareilles said, "I was so relieved when they told me, 'It's cool. You can totally swear on the show.'"

You pretty much have to swear if you want any chance of keeping up with Chelsea Handler, who went from a discussion of Cee Lo Green's hit song, 'F*** You,' into the diversity and virtues of the F-Word alone.

"You can always relate it to whatever situation you're in," she said of the song. "If you're happy. If you're p***ed. If you're breaking up. If you're over it. Whatever. You're like, 'F*** you!'"