Strange bug in code highlighting

[Matthew Woehlke]

I've been seeing this for quite some time (I think since KDE4 days,
even), but only just now realized what's happening. Sometimes, KDevelop
will (seemingly) randomly colorize blocks of code that don't seem to
match up to anything, often starting and ending color in the middle of a
word. This is most noticeable when color gets applied to a comment.

I just realized that the coloring that's happening is "correct"... for a
*different line*. See attached screen shot, which shows both the problem
and this observation; the coloring of the comment (top of screen shot)
is correct for the actual code two lines lower (bottom of screen shot).
This off-by-two problem seems to continue for the entire file.

Note that reloading the file (File → Reload) does not help. However,
forcing a reparse, via the "reload" action on the project folder
containing the file in question, seems to fix the issue, so I guess the
code model is getting out of sync with the actual file contents somehow.

Is this a known issue? Should I file a bug report (if so, against what
component?)?

Thanks,

[Sven Brauch]

Post by Matthew Woehlke
Is this a known issue? Should I file a bug report (if so, against what
component?)?

Yes, it's known. It is extremely difficult to trace though. I think the
reason is somehow related to the modification revision attached to the
parse job not fitting the contents, i.e. a race-condition between
reading the contents of the document and the modification revision. Then
if you insert a line in between these events, the syntax highlighting is
translated one line wrong and you get this effect.

It might also be related to saving the file at the "wrong" time, because
I'm not sure the transition from the "unsaved" to the "on-disk" state is
handled correctly when it happens at some random point during creating
the parse job.

Any insights on how to reproduce the issue are very welcome, I already
spent hours trying to find it but didn't achieve much, mostly because it
happens really really rarely.

Best,
Sven

[Aetf]

Post by Sven Brauch
Any insights on how to reproduce the issue are very welcome, I already
spent hours trying to find it but didn't achieve much, mostly because it
happens really really rarely.

This happens to me all the time when working on large projects. It seems
that the code model can't catch up when there are already parsing jobs in
the background and then I modify some file that's not yet reparsed, and
thus the previous version of the highlight data is used.

By large project I mean something like kdevelop itself, or the tensorflow
source code, which takes about half an hour to finish initial parsing.

Best,
Aetf

[Matthew Woehlke]

Yes, it's known. It is extremely difficult to trace though. [...]
Any insights on how to reproduce the issue are very welcome.

Hmm... here's an interesting data point. I had TTBOMK no active parsing
(KDevelop sitting idle for a while). Then I switched to a different git
branch (outside of KDevelop), followed immediately by doing a pull (so,
possibly, KDevelop started parsing the pre-pull files). I then opened a
file that I did not previously have open, and am seeing this issue.

Thus, at the least, I can say with some confidence that the problem
doesn't seem to depend on editing a file *in KDevelop*. There may be an
issue with a file being changed on disk, outside of KDevelop, while it
is being parsed.

A rough guess what may be happening:

- File changes on disk ("rev A").
- KDevelop starts to re-parse file.
- File changes on disk again ("rev B").
- KDevelop finishes parsing "rev A".
- User opens file in KDevelop.
- KDevelop highlights per "rev A", but file content is "rev B".

...although I was unsuccessful at trying to reproduce that intentionally.

[René J.V. Bertin]

FWIW, I'm not seeing any of this, yet I am on a slow(er) machine and I'd expect to encounter the issue there based on what I've read until now. One possible crucial difference in my set-up is that I never allow full project parsing. Instead I use the "on-demand" mode that only parses the documents you actually open (plus the headers they include). That's still plenty slow enough ...

What I do notice quite regularly is something you could call the "echo of past errors": lines that retain a (possibly slightly less saturated) pinkish background for no obvious reason. Related or completely unrelated?

R.

[Sven Brauch]

Post by René J.V. Bertin
What I do notice quite regularly is something you could call the
"echo of past errors": lines that retain a (possibly slightly less
saturated) pinkish background for no obvious reason. Related or
completely unrelated?

I think that's a different issue.

[Matthew Woehlke]

Post by René J.V. Bertin
FWIW, I'm not seeing any of this, yet I am on a slow(er) machine and I'd expect to encounter the issue there based on what I've read until now. One possible crucial difference in my set-up is that I never allow full project parsing. Instead I use the "on-demand" mode that only parses the documents you actually open (plus the headers they include). That's still plenty slow enough ...

I wouldn't not be at all surprised if full-project parsing needs to be
enabled for the glitch to manifest.

A full parse does take a while :-). I tend to go away and do something
else for a bit whenever that happens. (Fortunately, I don't have to
restart KDevelop often, so it is usually days or even weeks between
waiting on a full parse.)

[René J.V. Bertin]

Post by Matthew Woehlke

Post by Matthew Woehlke
(Fortunately, I don't have to

restart KDevelop often, so it is usually days or even weeks between
waiting on a full parse.)

YMMV as they say :)

Watch the memory usage for bigger projects, esp. if they also contain ObjC(++) files! I've seen it go well over 1Gb with only a few files open (= parsed), which is reason enough for me to quit the session when done on my poor 12Gb system. But we're straying :)

R.

[Matthew Woehlke]

Post by René J.V. Bertin

(Fortunately, I don't have to restart KDevelop often, so it is
usually days or even weeks between waiting on a full parse.)

YMMV as they say :)
Watch the memory usage for bigger projects, esp. if they also contain ObjC(++) files! I've seen it go well over 1Gb with only a few files open (= parsed), which is reason enough for me to quit the session when done on my poor 12Gb system. But we're straying :)

YMMV indeed. I consider 1 GiB to be *low*, or at least typical, memory
usage for KDevelop :-). Fortunately (and mostly due to not-KDevelop
memory hogs), I have 32 GiB.

(I have two instances open currently. Both are at about 1 GiB RES, and
my current swap is negligible, so that's probably a reasonable
reflection of actual memory requirements. One has about five
modest-sized projects loaded. The other has one modest-sized project,
but also Qt.)