CDT/summitfall2008/Indexing Session

Note: The contents of this page refer to a past CDT summit. For information on other and possible upcoming summits visit CDT/summits

Participants

Chris Recoskie, Vivian Kong, Mike Kucera, Andrew Ferguson, Markus Schorn, Sergey Prigonin

Parsing of inactive code

Support for inactive code as a second-class citizen in the editor. Needs to be safe - cannot degrade active regions.

Inside regions of inactive code:

Parse

• Two options
  • Save tokens and do it afterwards - there is quite a lot of context to save at each point.
  • Pre-processor co-operation with the parser so that for inactive code delimited by declns, or stmts, have the parser build the ast. For example, IToken.INACTIVE_START, IToken.INACTIVE_END. After IToken.INACTIVE_END proceed as though the inactive tokens never occurred.
    • DOM Parsers could use a new custom exception similar to BacktrackException
    • LPG Parsers could spawn new parsers as ambiguity resolution does
• Do not attempt to correctly track macro dictionary state. Macros expanded within inactive code, but the dictionary is not altered on entry or exit.

Build AST

• Compatibility: Clients external to CDT would get an AST without these sections (inactive tokens would not be requested on construction). Internal clients which share the AST would be updated to cope.

Build CModel

• Most likely have all elements be queryable for active/inactive state

Scopes, Binding resolution

• Probably not

Index

• Not for now

UI

• diagonal strike-through of icon, grey label
• allow to filter/hide inactive regions

Name resolution

Binding Resolution on non-name nodes

• Syntax highlight of overloaded operators
  • Binding resolution on nodes other than names for this purpose e.g. Binary operators, Unary operators
• Search for, or flag up occurrences of implicit conversions
• Some bindings have no name or operator to attach to (automatic variables going out of scope, implicit conversions)
• In the AST and Index, we don't have a name node (reference) to attach the binding to.
  • True for Macros also
• Open definition on impicit constructor call ("A a(5);") could jump to the constructor, as there is no value in staying in the same place. The node associated with the binding would be the declarator.
• We need a list of occurrences of places where implicit references occur. Automatic use of constructor and overloaded operators are seen as the priority.
• Mike may have time to work on this

Ambiguity resolution

• Maybe a different visit ordering of amiguity nodes can be defined to solve some issues
• Once a correct match has been found, maybe we can skip flushing some scopes
• No-one has a good understanding of the scope implementation and its associated CPPSemantic routines right now
• Duplicate name nodes (from alternate ast candidate branches) should not be added to the scope on population, and at the point of ambiguity resolution success then the successful node should be added to the scope.
• A tracing option to log the number of ambiguity nodes
• Content assist is seen as not interacting with this work
• Markus is willing to do this

Tests for absence of false positives

• That is tests for presence of true negatives
• We tend to write tests that check legal code resolves correctly. Testing that illegal code correctly generates problem bindings is also valid as
  • Ambiguity resolution works by counting errors
  • Increasing parser accuracy gets us nearer to a place where one day we could build a client that marks compilation problems on reconcile (?)

Template arguments

Markus will look at modelling template arguments. Subtleties around having an index representation of values as needed for enums and const variables.

Language defines 3 types of arguments: type, non-type and template. It may be possible to model template temaplate arguments as type arguments. Values could be modelled partitioned by type-family e.g. IIntegralValue. IBasicType currently also masquerades as a value - this should be cleaned up. Template arguments can be obtained by a notion of template position (nested depth and argument ordinal) - this would replace the untyped ObjectMap in ICppInstance.

Unknown bindings

We may not need to store unknown bindings for references. There are some problems with problem markings when names cannot be statically resolved within a template scope. For example we don't have a way of distinguishing unknown type from unknown value.

C++0x

The standard not due until around 2010. For practical purposes, nearer the end of this year. GCC plans to support this 2009. Some awareness of features in the team.

• Name resolution
• Index, AST modelling them

Which parser to extend? Performance/characteristics of the implementations needs to be understood.

Action: No commitment for any work for CDT 2009. Get an understanding of the standard. Start a wiki page and track the impacts of each language feature

Language Variants

Two ways of doing language variants - DOM Parser and extension objects, LPG Grammar hierarchies.

Language configurations vs Language variants - it makes more sense to have a language be configurable rather than a distinct language for each set of options. On the UI side we should consider language (C,C++,UPC) mappings, that indirect to specific languages (C89,C99,C90,C++0x,UPC). For example, the user marks files as C or C++, but can reconfigure which specific dialect is used for these. The language would instantiate a parser.

Some discussion of wanting to sync build and parser settings. Concern over the complexity.

Search-path-less Include resolution

A bunch of heuristics to guess include paths from project content. Two metrics of closeness for ambiguity resolution. Structure for hashing for include lookup for large search paths, lazily constructed. Have API so it can be determined if an include was explicitly configured or discovered automatically.

Add Include action

Adding an include for a name looked-up in the index. Choosing the right header to include needs some intelligence. This really needs meta-data/markup of headers depending on container (library, SDK) which gives a hint as to which headers are public or private. Maybe we should have a heuristic that C++ prefers headers without an extension.

The division of the search path into user and system includes could be used to make this automatically discoverable. Take the leftmost system (angle bracket) includes, combined with some shortest path strategy. No promises that this will be worked on.

see also: https://bugs.eclipse.org/bugs/show_bug.cgi?id=205045

EFS based indexing

• Markus willing to deal with the preprocessor
• Assume hierarchical URIs for relative include resolution
• Considered lower priority at this time
  • Being able to use a URI in the include path is a pre-requisite

Misc

• We should provide a way of determining whether a method is an override of an existing method.
  • Consider copying JDT's editor feature where a small upward triangle appears in the left gutter with an associated action of navigating to the overridden method
• We should provide a way of determining whether a method is overridden.
• we could double-buffer indexes - make current index read-only. Start a new index with new settings. Swap when done.
• Resume of indexing between eclipse stop and start, and cancelled.
• Sergey will continue to try to isolate remaining STL problems.
• Organize Includes action - seen as a little too difficult.
• We currently skip some implicit conversion checks completely - e.g. assignment lhs and rhs type checking
• Sergey rediscovered a variant of the incremental-changes-to-header problem: https://bugs.eclipse.org/bugs/show_bug.cgi?id=171834