In the Summer vs In Summer

(Grammar Questions I Couldn't Answer)

In my class, I was giving students feedback on their writing mistakes.  I selected sentences, and the students had to identify the errors themselves in groups.

One sentence was:
In the summer, I looked at my friends swim at the pool and I wished I would be able to do it.

I was intending that the students would focus on the difference between look/watch as the error, but as so often happens in this activity, they picked out something that I wasn't even intending.

On student said it should be "in summer" instead of "in the summer".

My native speaker intuition told me that both "in summer" and "in the summer" were fine.  But why were they both fine?  Is there any difference between them?

I looked this one up in Practical English Usage by Michael Swan.

Swan says on page 63:

To talk about the seasons in general, we can say spring or the spring, summer or the summer, etc.  there is little difference.

But he doesn't explain why there is little difference.  Wouldn't a selection of different forms indicate a different meaning?