Rehashing the Obvious: DMN’s Restaurant Star Rating System is Still Broken

The restaurant reviewers at the Dallas Morning News can’t win for losing. Each Wednesday when their reviews hit the web, readers who take the time to comment still seem confused by the number of stars assigned to a review. Sometimes there are more positive statements than negative, yet the restaurant receives only two stars.

It’s a difficult and complicated task to coordinate reviews and stars. But if you have to keep explaining your method, you should know whatever you are doing is not working and you are not serving your readers. Let’s go back.

On June 19, 2009, Leslie Brenner wrote:

“If a restaurant is serving brilliant main courses and charging $50 for them, that’s far less impressive to me than if it’s serving brilliant main courses and charging $22 for them, and I definitely consider than when assigning a rating. If you charge $50 per entree you can afford much more help in the kitchen. It’s a lot of money to charge, and my expectation is that the dish will be stellar. And if you’re asking a diner to pay that kind of money, the whole experience, including service and ambience, had better be stellar too. At the lower end of the financial scale, if I find a restaurant with good, honest cooking, where you pay, say $10 for an amazing chile relleno stuffed with brisket, that’s definitely appreciated and it’ll be rewarded.”

I disagreed and proposed a solution to the impossible task the DMN faces of aligning their stars. Divide restaurants by price points and use a different color star for each. That makes it easy to spot a four-star burger joint over a four-star $50 steak joint. (Dave Faries if you use that I will send you a bill.)

But, noooooooooo. Today, the DMN complicated the situation beyond repair. Now, dear reader, you have to evaluate “evolving stars” and “old stars” to balance when deciding where to spend your $50.

Oh that makes me sad and gives me a five-star headache. Love to hear your suggestions.

27 comments on “Rehashing the Obvious: DMN’s Restaurant Star Rating System is Still Broken

  1. To me, the stars should be the average of 3 sections.. Food, Service, and Value for your money. If you get the worlds greatest taco with the worlds greatest service, the fact that you spent $50 for a taco should bump the rating down, because unless it is made with a rare breed of lobster harvested from the frozen water deep within the moons surface, it isn’t balance.

  2. Keith, that seems inherently like what the DMN star system already is. I feel that DMN’s star rating is very clear and very simple. I’d be willing to bet that for every imbecile who complains that they don’t understand the star system, there are probably 20 readers that DO understand it.

    Nancy, your proposed “fix” is a joke. I suppose you think it’d be even clearer if we used a three-dimensional matrix of varying color gradients to rate restaurants, but the simple fact of the matter is the current DMN star system is simple solution to a multi-variable problem. It’s something that 95% of the population understands. You’re just trying to dig up dirt because… well, you’re D Magazine. Vapid articles are your specialty.

  3. Who reads the Dallas Morning News anymore?
    It should disappear and Dallas life will just continue

  4. I have no problem understanding the DMN rating system. However, I read the reviews. Perhaps if I just made a snap judgement by looking at stars only I would miss some decent restaurants.

  5. @chris: Wow, someone’s mommy did not love him enough. Chill out dude. Act like this is your daughter or best friend’s wife who wrote this, and you don’t agree with her methodology or point, and think about how you would politely disagree and make the same counterpoints. You don’t need vitriol to be a good writer or debate an issue. I respectfully disagree that she is just trying to dig up dirt — I think her point has some substance behind it and merits a discussion. Without calling anyone an imbecile.

  6. It seems the DMN keeps trying to re-educate its readers into changing a long held belief system. They expect us to adapt to their ‘new’ logic. Common sense dictates that 1 star is poor, 3 is mid-range, and 5 is great. It is a logical scale.
    Yet now we are told that 2 stars (or 40% of the scale) is Good and 3 stars or 50% is ‘Very Good.’ Not logical.

  7. I think TLS’s comment exemplifies the issue here. If you don’t read the review you might be misled by the star rating. Shouldn’t they sync up? Obviously, the start system is flawed. Part of the reason for having a star rating, is so the reader can quickly get a gist of the nature of the review.

    It cracks me up when someone on the DMN blog (like Brad or Leslie) gets all annoyed that they always have to re-explain the star system. Then they, or more likely some self-important reader, makes a condescending comment about how the other readers are imbeciles. How about this: if you have to explain it every week, maybe the problem is the star system, and not the readers.

    I stopped reading Brenner’s reviews a long time ago because I always felt they were off-base and I almost always disagreed with her. I would say that this is fine – her right to write bad reviews, my right to not read them – but it impacts local businesses and that sucks. But that’s another discussion. Anyway, when I saw she rated Kuby’s two-star, my first reaction (as usual) was WTF? Then I rememeber, oh that’s just Leslie and the stupid DMN review system.

    The problem is, someone from out of town or someone who isn’t familiar with the rating system, is likely to dismiss visiting the restaurant out of hand. There is an obvious diconnection here between writer and reader, and in my opnion the burden is on the writer to correct the issue effecitvely (obviously, just re-posting star explanations isn’t working).

    There must be a better way to spend my time…

  8. I agree with CJ. It’s hard to change people’s perceptions that a 2 out of 5 rating on anything is “good”. In school, a 2 out of 5 was a ‘D’, which meant you were barely passing.

    The intuitiveness of the ratings becomes even more mind-boggling when you compare the actual review to the rating. For instance, you read today’s review of Ferrari’s and see a 1-star rating and immediately expect the article to go on to describe a dining fiasco. Instead, you hear the reviewer describe an average and uninspiring visit while mentioning several dishes that she actually liked. That expectation of a 1-star rating being correlated to fiasco-type experiences only is being driven by the past precedent of the same reviewer.

    I don’t know why DMN just doesn’t adopt ratings that most people are more naturally familiar with. Most people expect that 1=poor, 2=below average, 3=average, 4=good and 5=superior.

  9. @Chris the D Bag

    You realize your point was completely invalidated by your very clever nickname right?

  10. There used to be a time when all Italian restaurants had to do was serve the classics and they were golden. Then time evolved and people wanted pappardelle cinghiale and big veal chops and risotto. Now according to LB you are not a real Italian restaurant if you don’t have sea urchin, chitarra pasta, bottarga, or other nouveau Italian dishes.

    Just from a blind eye, Arcodoro carries these items but got a less than favorable review. Nonna tries but didn’t get 4 stars.

    So if the consumer is the ultimate critic, who speaks with their dollars, we either must all not know what food is (because their is a lot of longetivity in the restaurant scene) or the DMN seriously has an agenda because no way someone like Ferrari’s or the Old Warsaw can be still be in business with one star in this economic climate if they were no good.

  11. As a Dallas native, I come to realize a major trait that characterizes the Dallas mindset: an inherent inferiority complex about our own culture. The cultural institutions / publications are obsessed with hiring “experts” from other major cities who come with an agenda wanting to “educate” our plebian minds with what “real” culture is — Bill Addison… Brenner… What you need is someone who is invested and engaged in what Dallas is, what it stands for and the context that Dallas operates in, its history and future. Changing the old rating system didn’t bring Dallas any more “credibility” as Addison. (We already have credibility) It was much more disruptive than it was productive.

    And Leslie’s comment about charging $50 for a dish: ignorant. Any passionate chef would love to serve premium quality food but there’s not enough demand. Charging $50 doesn’t mean you can “afford” anything: more kitchen staff, table cloths, fresh flowers, better training for servers. That can be just food costs alone. She’s the old guard. Take a look at Momofuku – extraordinary food, higher prices, average service, very casual, minimal atmosphere but I simply have to enjoy dinner there when I’m in NY. According to her, it would get 3 or 4 stars when David Chang is crossing the Rubicon of the culinary world.

    The more you charge, the lower your profit margin is. Generally, the less you charge for food, the better your food costs are because you can buy in bulk. There is more consumption at lower price points. It’s actually harder to be a successful expensive restaurant.

  12. Get over the rating system. It is simple to understand. Crappy restaurants don’t even get reviewed, so start with fair at the bottom. Stars are earned. A restaraunt doesn’t get a star just for existing.

    Now, Nancy, spend time worrying about your own business. Your pettiness is wearing thin.

  13. The DMN star system is working and has been for many years. Perhaps this is a subject D Magazine really needs to stay out of as it is really none of your business. Period.

  14. @Mitch, I agree that stars need to be earned. However, I do think that what Nancy is doing is important. I don’t always agree with Nancy, but this dialogue that she’s elected to instigate / start with Brenner is how culture moves forward. No can can operate in a vacuum. To be good at one’s profession means to stay informed with what our peers are doing. Petty fight is what you could call it, but it’s advancing and challenging and contributing to discourse of dining in Dallas. We’ve never had more engagement in the restaurant industry than now.

  15. JTT – Great & insightful comments. Ask Scott Ginsburg about expensive restaurants. Anyone remember Voltaire at Keller Springs and the Tollway? He proudly touted that it would be the most expensive restaurant in Dallas. How long did it last? I would like to know whatever became of all the Chihuly glass sculpture!

  16. JTT, thanks for getting it. managing reviews after the invent of the web has changed the way we all read. Used to get the paper at the door and open it up to READ a review. Now it’s a click away and comments from readers. I feel for the DMN, they are obviously struggling to clarify a crummy situation. I’m not picking a fight, this is a topic that I’ve written about for years.
    LJT, it is my understanding that a lot of the fixtures from Voltaire went up in flames when Ginsburg’s house burned.

  17. You are all still crazy…!
    Stars can work – but must be awarded with consistency over a substantial period of time.
    Ain’t nothin’ wrong with the hard a** approach of the New York Times, once everyone understands the game.

    BTW, let’s strive for – and reward – excellence, not coddle mediocrity.

  18. “The more you charge, the lower your profit margin is. Generally, the less you charge for food, the better your food costs are because you can buy in bulk. There is more consumption at lower price points. It’s actually harder to be a successful expensive restaurant.”
    I don’t think I understand this. Food cost is a balancing act in either type of restaurant but there are two ways to work at it. One is a percent of sales the other is a contribution to bottom line or flow thru.
    Food cost and labor go hand in hand. I had a chef friend tell me that his restaurant had a high food cost percent but the owner was delighted because they were making more money than ever. Would you make more over all money selling one lobster for $100 and a 50% food cost or 100 hamburgers for $1 at a 33% food cost? Take into account labor, taxes and utilities. The dollars saved in buying bulk are incidental. Food cost is controlling waste , proper purchasing and minimizing theft.
    Jtt’s comments ( that I agree with) regarding inherent inferiority complex about our own culture could also explain the massive amount of out of state restaurant concepts that have come and gone.

  19. @curmudgeon – Allow me to clarify.

    The issue is about the addressable (and capturable) market. How many people can afford burgers? Nearly everyone. How many people can afford lobsters? In this economy, nearly no one.

    When you operate an expensive restaurant, you are chasing after a TINY percentage of the addressable market with the disposable income who can afford to eat fine cuisine and appreciate it. The competition is fierce and the survival rate is low.

    Unlike burgers, where you can buy 200 patties in bulk, you probably will only sell 10 lobsters. The discounts for 200 patties are better than only 10 lobsters. It’s not incidental. It really adds up. ($1 burgers are not restaurant burgers, they are fast food — its more like $8 burgers)

    For high end cuisine, you need better trained kitchen staff who can do more than fry and stack a burger (the menu is more complex and changes for more expensive places) and better servers who are trainable and articulate. Training servers requires strong management = more costs. If you salary your kitchen staff, labor (and utilities) is fixed. Taxes are variable. Food costs are easier to control at a cheap place.

    There are a handful of expensive restaurants that are packed all the time, but it’s rare. After years of fine cuisine, Avner Samuel transformed Aurora into Nosh – lower price points.

  20. @JTT – I liked your comments re: Dallas’ self esteem issues.

    However, your analysis of restaurant profits is a bit puzzling. I’m no expert, but looking at some industry stats, I see that the average pre-tax net profit margin for full-service restaurants is about 2.5%.

    Cheaper price points make that 2.5% by taking a cut on the profit margin and selling volume (b/c, as you said, more people can afford it – and b/c more people can afford, there is fierce competition here too). The more expensive places sell fewer items and earn a higher margin (because their clientèle can afford it). It seemed like you were suggesting it is the other way around.

    The demand pool is less price sensitive in the higher-end niches. I would think the price inelasticity would actually lend more flexibility to the management of food costs in the higher-end. When you’re selling cheap food and squeaking out margins, the management of those margins becomes essential (cheap places go out of business all the time too).

    The fixed costs – the required labor, overhead, and so forth – evens out, on average. The discount on purchases is more or less irrelevant. What matters is the net materials costs after available discounts. Just because I don’t get a discount on a lobster doesn’t mean I can’t sell it for the same (or higher) markup as your burger.

    Ultimately, the business models are probably too different to make such generalizations.

  21. No, you can’t mark up the lobster the same way you can mark up a burger. Next time you order a bottle of wine, go for bust. The more expensive bottles have a lower mark-up because they need to move. Your $30 bottle of cab is marked up a lot more. Guests have a psychological cap on how much they are willing to spend.

    Controlling food costs is essential. Even decreasing food costs by 1% can make or break you because the margin is razor-thin. Mario Batali gets accolades from fellow chefs for mastering food costs.

    You can be a great chef (in your home) but you are not considered a professional chef unless you can master food costs – it’s the only way you can run/operate a restaurant. The discount on purchases is not irrelevant, but that’s not the only factor. Think about it: cheaper places use more frozen, non-perishable products = easier to control costs. Chicken breast is $3/pound. At a nice place, once you break down an entire fish, that portion you actually serve can be $40/pound.

    The demand pool is, in fact, price sensitive in higher-end niches = why wouldn’t it be? Have you seen how many houses are for sale on Preston? Rich people eat at cheap places, too, but do poor people eat at expensive places? 1 or 2x a year, at best.
    You are overestimating the market.

    What Brenner started doing, which I think is actually interesting, is accounting for value in the reviews (like all Yelpers do). She just did it ignorantly. She spent a year at Daniel, but she did not sign any of the checks to the food vendors and she (I strongly believe) wasn’t involved in any discussions about cost or menu pricing to make those judgments.

    Good food costs a lot of money. Your bill at Kroger vs your bill at Central Market?

  22. I’m not suggesting that cost control isn’t important. That pretty much goes for any company in any industry.

    Since we’re talking about food costs, the average gross profit margin in the industry is 60%. I realize that this figure probably includes other direct costs, but I’ll just go with it to keep it simple.

    In a burger joint, I can sell a burger for $5 with a food cost of $2 (60% gross profit margin). Similarly, I can sell a lobster in a high-end restaurant for $27 with a food cost of $11 (again 60%). Some items will have lower margins while others are higher. I’m guessing the asparagus side for $11 really helps pad the bottom line at some of the places around here. Maybe the margin there is 80%, while the margin on the white truffle diamond encrusted wagyu beef is 5%. It really doesn’t matter.

    Your point regarding the unpredictable nature of food costs in fine dining is well taken. The variability there creates a certain amount of risk. And good management is essential! However, I would think that if Kent Rathbun went down to Spicers garden for a new dish tonight and the cost was $1 extra per plate than he though because of some shift in the market, he would easily (relative to lower end places) be able to bump the entree price up a couple bucks to cover it.

    All consumers are price sensitive to some degree. I was just suggesting that the high-end clientèle are less sensitive, on average, than the low end. The lobster tail add-on doesn’t mean as much to Jerry Jones as it does to me. The guy who buys an SLS AMG 550 might not care as much about a $5k add-on as the guy buying the Honda Civic.

    The failure rate in the restaurant industry is extremely high. Low end and high end. And I would bet a ton of that has to do with sheer mismanagement.

  23. Ardy- you are dead on man. Management is the biggest factor in failure of a restaurant. I still think that JTT is incorrect with his food cost comparisons, when as a chef you buy say hamburger meat. It still costs the same at 10 pounds as it does at 100 pounds. There is no discount, the price is the price. Same with lobsters, the price is the price per pound.
    The only variance is from vendor to vendor.
    If both styles of restaurants employ managers and servers the cost is the same, the training is different because OF management.
    The salary of your staff, labor is not fixed it varies from day to day and shift to shift and again is either to high or to low because OF management (or lack of it).
    Either way 1 lobster or a hundred burgers at the end of the day it is a business and you have to turn a profit.


    Oh Leslie B., I simply must once again disagree
    With your standards of star ratings, Abacus a three?
    Worse yet, Kuby’s two stars, a judgment quite gutsy
    Do ya’ realize this rating’s below the (closed) Zinski?

    Your standard’s inconsistent to what Dallas expects
    For an informative discussion where to dine as directs.
    Perhaps skewed a bit from your west coast perspective
    In a writing style where stars don’t align with invective.

    While Hanna, freshman rater, is rarely critiqued
    Your comments, my dear, indicates readers quite piqued.
    At reviews seeming positive yet sparing few stars,
    Then flip-flopping those standards for tacos and burgers.

    My dear, just a tip, a review t’ain’t all about “me”,
    The Mercury paid a steep price for your edible vanity.
    Arguing about cheese in the dining room is sooo declasse’
    Sad backstory, I’m afraid, is people believe what you say.

    Big city, so small some say about Dallas,
    But like most of the South it’s hospitality magnanimous.
    But an old fashioned saying defines what who are:
    “You can catch more bees with honey than you can with vinegar.”