Hero image credit: Photo by Kelly

Another followup to my blog post:

A Weekend With Github Copilot

Up until now, my experiments with GitHub Copilot have used the Claude Sonnet 4 model in Agent mode. I figured I had learned about as much as I could from that one, so it was time to give another model a try: ChatGPT 5. Again, as with Claude Sonnet 4, this is a premium model and requires you to have the Github Copilot premium license.

Verbosity

The first thing I noticed was the responses from ChatGPT 5 were extremely terse in comparison to Claude Sonnet 4. Claude will provide a great deal of descriptive text as it goes, telling you what it’s doing with each step as it processes things, then providing a detailed overview of the results at the very end. It will often also include snippets of code that were added or removed in the chat window. You will often have to scroll up and down the chat window to get the full description of what Claude did to implement your prompt. Here’s a good example of that:

Claude Sonnet is quite the chatty AI

ChatGPT 5 is having none of that. Here, you will get a list of files that were examined and/or updated, along with line counts of added/removed lines. Here’s what I mean:

ChatGPT 5 showing list of files examined and modified

In order to see what code was changed, you have to click on each filename in the chat window. That will open a diff screen in the code window where you can see the details of what was added and removed, just like you were looking at a Git diff screen. Very simple, very basic, but also very easy to follow.

ChatGPT 5 code diff screen

After the list of files, ChatGPT 5 will provide a simple bulleted list of what it did.

The simple summary that ChatGPT 5 provides

And that’s it, most of the time. That’s all ChatGPT 5 will give you. It doesn’t create markdown summary files. It doesn’t wax philosophic about anything. Clean, simple, direct, and done.

I say about “most of the time”, because from time to time it will do something I never saw Claude Sonnet 4 do. It will make suggestions for what you might consider doing next. For example, I had it add a model, update the db context, add API endpoints for CRUD, and add web pages to handle the search and CRUD operations. It did all that just fine. And when it was done, I was presented with a list of things that it thought might be good next steps:

  • Implement remaining filters (keywords, geo radius).
  • Add validation attributes on domain/DTOs (e.g., required, date ranges).
  • Introduce paging/sorting for large result sets.
  • Add DTO separation to avoid exposing entity types directly.
  • Add tests around endpoint filtering logic.

And what’s more, they were actually good suggestions. Those were all things that should be on the TODO list after you implement your initial basic CRUD operations for a model.

Blazor

If you’ve read my previous posts, you know that Sonnet 4 didn’t do so great with Blazor. ChatGPT 5 did an incrementally better job, but still not perfect. It never tried to use the wrong version of Blazor, and as often as not the components worked correctly the first time you tried running the page. It also seemed to keep more consistency with the styling across multiple pages, so it didn’t use button groups on one page and dropdowns on the next, and checkboxes on yet another page like Sonnet often did. So that was better.

However, while Sonnet tried to be fairly fancy in its styling of Blazor pages, again ChatGPT went for the simple and basic route. Pages were very plain, with very little CSS added in. There were no attempts whatsoever to add icons or emoji characters, and there was very little use of color. Very simple, very basic from the start. You may like or dislike that, but the difference in styling was stark when you compare the two.

Here’s what Sonnet created:

The designs that Claude Sonnet 4 creates get a lot of style applied to them

And here’s what ChatGPT created:

ChatGPT 5, on the other hand, applies virtually no styling to a web UI by default

Entity Framework

Here’s where they are very much the same. ChatGPT 5 never once successfully created an EF migration. Well, it would create them, but they would never work. So you would have to tell it “hey, the ef migration isn’t getting applied”. And the problem was the same every single time. Instead of relying on the EF CLI to create the migrations like any normal developer would do, it would instead manually create the migration file and update the snapshot file manually. And every single time it would forget to create the designer file, which is required for EF to recognize that a migration has been added and needs to be applied to the database. EVERY. SINGLE. TIME.

So, the more things change, the more they stay the same?

Debugging Runtime Errors

ChatGPT 5 seemed to do a better job overall with diagnosing and debugging runtime issues. When you copy/paste log file content into the chat window, it would almost always figure out what was wrong on the first try and successfully resolve the code issue. The only time it was unable to figure it out on the first try was in relation to the above mentioned Entity Framework issues around migrations. That doesn’t trigger design time or even runtime errors, until you hit a page that tries to access a field in the database that doesn’t exist because the migration wasn’t applied. When that happens, getting an error like “c.StartDate doesn’t exist” and putting it into the chat window can take a few tries before it recognizes that the designer file didn’t get created. That example aside, it did quite well.

Code Quality

There appears to be very little difference in the quality of the code overall. They each do simple, repetitive tasks quite well. Model updates, API CRUD creation, basic coding things. And they seem to have the same struggles with things like Blazor and Entity Framework. Overall, the quality is similar to the kind of code an entry level developer might create or be tasked to work on. And you’ll get the same kind of output. And that’s fine. It gives you a workable starting point. Just don’t expect to “vibe code” an app you would deploy to production usage. They can’t do that.

Which To Use??

Well, there seem to be pros and cons to each. So it seems more a question of your preference at the moment. Do you want a lot of detail in your conversation? Do you want the AI to make a lot of design decisions for you? Then got with Claude Sonnet 4. If you want plain and simple, go with ChatGPT 5. It’s really up to you.