In my opinion, the comment definitely should not be deleted. Not because of a moderator's personal views on it, or through any sense of fairness, but simply because the job of a moderator is first and foremost to prevent abuse of the site for purposes for which it was not intended, and not to be an editor or curator (or at least not significantly more so than any active user assumes these roles because of the way this site operates).
I think it's important for both moderators and (probably more importantly) users who would flag items for moderator attention to understand this distinction. As a moderator, the temptation is (understandably) always there to be proactive and try to improve the site in any way possible. However, in my experience as a one-time forum moderator, this bureaucratic workload coupled with continually having to make value judgments in ambiguous circumstances can lead to burnout and an eventual disinclination to visit the site at all, so my advice to you would be: be careful how much of this you take on, because you may regret it later. This is especially likely if anyone might be inclined to question or disagree with your actions. (The one exception I would suggest to this is in strongly discouraging questions in which the asker displays an obvious disinterest in engaging with their problem, because the long-term results of allowing such questions--as rather graphically evidenced on Python.SO--seem to be quite severely corrosive.)
Getting back to the central point of your question, comments are very unobtrusive and it seems to me not worth the time and effort to delete them even if they contain redundant information. Maybe the commentator will come back and, seeing that their comment no longer adds value, delete it; maybe they feel that their contribution should be recorded and let it remain. Either way, it doesn't very much matter. In fact, the site concept seems very well thought through in general and with luck should require minimal moderator attention to maintain in a useful state, especially given that the Mathematica community is, on the whole, very reasonable, well-informed, and helpful.