EDT:Planning notes

This page contains a series of notes from meetings about EDT validation rules and related concerns.

You can return to EDT 0.8.2 Planning.

11 October 2012

Array assignment compatibility

Conclusion: add an array that is a corollary to dictionary, indexable by number.

Discussion:

We have something defined as an ANY array and had spoken of changing the definition from being an array of ANY --- an array that is an EList with a type parm as ANY -- to being an EList. If you declare a variable without giving a type, those operations get the type parm.

We were speaking of the AS operation being defined on EList. And that conversion operation would do this work if the type parms of the argument passed in to the type AS was compatible with the type parm of the variable. We need to define the conversion operation in the same way that we define all the array functions. When we say add element, we have an any in there. and the any is treated as a type parm. This would have a similar effect on the conversion operation. And the implementation of the conversion operation would do a shallow copy.

The AS would then be doing a copy, not only for value types, but for reference types as targets. If you want to treat an array as a different type, you can't do it unless you do a copy. If we assign a smallint array to an int array, or vice versa... okay, those are compatible. When you are copying values into the other array, if it is a value-type copy it is a value-type copy, but in the case of a reference, there is a reference-type copy. When we make the source target compatible, we need to be smart about what the operation does. Might be a value assignment, might be a reference assignment.

You must be able to go from one array to another array. The data field is an ANY array in a grid. An array of foo records AS ANY in the assignment. Makes sense, instead of doing an assignment, to do a copy. But it can't be a function invocation in a settings block.

How about leaving arrays as arrays and introduce lists? Even with an array, you can assign something other what is supposed to be assigned. An untyped array? In Java, is a runtime exception.

Dictionary is like this already. We don't even attempt to put type detail. Like the JavaScript array. A dictionary is accessible by string rather than by number.

Don't change the typed behavior of arrays. What about a corrollary to dictionary? It would make the RUI code work the same as in RBD. What does it mean to assign what we have already? The implication is that the utilities that return List types now would need to have a different capability to return this type of array. Or maybe have a factory method to create one from another. Or the assignment would always be a copy. Array to an array? Reference aassigment. An int array to this any array would be a copy.

The typed arrays ought to stay as typed arrays, and this dictionary-like thing is different, with assignments happening during a copy. We don't know all the things it might effect. It doesn't break the semantic of what we have.

If you are using an ANY array, you should go typeless. But an ANY array is such that you can't assign an int array to an ANY array; but you can assign an int array to this typeless array.

Will have to change the RUI grid from an ANY array to this new type. Similarly, the children field in RUI.

Bytes

Conclusion: follow the comparison rules in use for strings.

Discussion:

Make a distinction between object identity and equals. You might want to know if two objects are identical. We have not handled this distinction. Use triple equals for object identity?

Nulls

Conclusion: change nothing.

Discussion:

In Java, you can cast nulls. If you have a var of type some handler and assign it null and then assign a value is of type Any, what happens? Not tested with a reference type. Does it gen as a static invocation of a conversion, where null gets tapped in?

The parser doesn't understand when you place a question mark after the type. i as decimal and i as null. An exception will be thrown.

I have a nullable handler or some ref-type variable and is set to null. try AS any. does it blow up on the conversion or after the conversion? Assigned the handler AS ANY when the ANY is has a null value. If the ANY is not nullable, it would blow up after the conversion.

No null check for the conversion, but there is a conversion for the assignment. Nullable handler variable is assigned to a non-nullable ANY variable. If it's a decimal that you are talking about, it's a reference type and NULL should not be thown. but if it's i as Decimal(10), it's a value type and there should be an exception.

A nullable int as Dec is assigned to a nullable decimal, and the result is null. if you said i as decimal 10, what happens? still null. so.... it DOESN'T throw a pointer exception. but the NullValueException will occur if you remove the question mark... it IS a null value exception.

is important not to generate check for nulls if we can avoid it.

the NullValueException should happen. left is nullable, the right is not. can Decimal10 be nullable? i can have a variable of type Decimal10 and make it nullable.

Since we don't allow a question mark on the right-hand side of the AS expression. We get no help in the type system.

Nullability has to do with whether the variable or slot can hold a null. The Decimal10 value can never be null, but the slot can be nullable. It's not the type that's nullable, it's the place that is nullable. It means nothing to state that the type is nullable. if you assign to a nullable, the check should be present. The question mark should go on the name, not the type.

Delegates

Delegates should end with a semicolon or an END.

Concrete and abstract functions

At this moment, the empty statement (;) is removed so that the parser can process functions for IBM i access without requiring END and without allowing room for unused logic. Those concrete functions might otherwise be named "abstract" or some other keyword. In future, we might add the word "abstract" for abstract types and functions; for example, in classes.

20 September 2012

Last week we talked about delegate signature. Delegate must have the same signature to be supported in different contexts; no other check is necessary.

Content assist is ongoing work in relation to the model.

Rich UI is a separate compiler extension, including core, not generation.

The model getName and getPackageName are case sensitive. You want the field name to be the same as what the user requested.

Expression validation needs to be ported. Ternary, in, and array access. Planned for next week.

Most validation is done. Porting test cases is time-consuming.

Constants. The generators seem not to require changes.

No way to say that a field on an external type is const. We need to add that. A small parser change might be sufficient.

For content assist, annotation stuff is radically different. A lot of hard-coded references are in place. Content assist should be extensible.

But do we really need an extensible content assist? The code should handle much, with variability provided by templates.

Content assist for an annotation should be the same for all annotations. This work has not been slated to rewrite content assist the right way. We should get it going and open a defect for subsequent work.

Content assist must be able to work off the types that are in the system plugins. Must not be hard-coded. The most important aspect is having content assist work consistency and correctly against whatever is in the environment.

Will be working on the new model. And am removing the hard-coded references and will ensure that it looks in the environment.

We might soon be compile error free. Must consider carefully before doing a big merge. Getting close.

Must remove all the stuff from the parser that is not supported in EDT.

We'd like to do analysis against existing EGL code.

Old parser can parse old code, it had EGL to IR that went through old AST stuff. We can still get the old AST and the original code can be invoked to create the IRs.

We will need a different copy of the ASTs and so will need another copy of the EGL to MOF code and hook it up. We need this to be completely separate.

Don't throw away what was there, aside from what EDT does not support.

Bytes implementation. Bytes as byte arrays, perhaps. The current implementation fails. Another area of interest.

13 September 2012

Conclusion:

The EDT arrays continue to be called "lists," and the list type will have a new conversion operator... as ANY[], as INT[], and so forth.

the element type is explicit. the element is strongly typed.

Delegates with type compatibility. More strict in the branch than in 0.8.1. The situation is, when comparing two delegates that are the same signature but are different delegates, they are different. But if delegate 1 has delegate 2 as a parm and if delegate 2 has ...

two records with the same structure are not compatible. so shouldn't these be the same?

we have name compatibility. can you cast from one delegate type to another if the signatures match? no, casting means that a conversion is possible at all. structure compatibility is done in pascal and elsewhere.

stay strict until we have a reason to be open. the ruiwidgets has this exact case: two different delegates. do we generate anything for a delegate that would make a delegate type.

a delegate should not be a generated artifact. for that reason, we can choose to be looser or not. the compat rule between a function and a delegate is loose: is by signature, not by name. you'd think the delegate would be the same. so... let's make it work like funciton.

we want parms of type delegate to follow the same rule.

IDE extensibility. the utilities will use the old model? no. remove the old model. paul is working on content assist. he says, "we have these enumeration and system-library managers set up by the system environment. i didn't migrate those pieces..."

content assist should have nothing special in its processing than other lookups. the system parts were not indexed before. system parts will always be parts that are resolvable.

IDE extensibility. i have an extension point. you now have a command line parm to use an extension.

a syntactic issue about assigning arrays to arrays. the dataGrid expects an ANY array, and you assign an ANY array of customer; but one way to solve the problem is by having a specialization of mutable versus immutable arrays. if the system knows about immutability, it's a different type.

in dataGrids, do people add columns on a fly? we need a rationale. dictionaries lets you put anything there, and no one cares if you assign one dictionary to another -- no issues about element type. anything in that dictionary is really an ANY.

what we really need is an array that works like that. let List be a real type. what is the difference between ANY array and List? The difference is, if you assign a variable to type list and assign an arbitrary list, there are no restrictions on what you can put into it.

So the DataGrid is of type List. You have to know what you are putting in there and nothing will enforce the type. Dynamism is retained. An array of 1,2 assigned to an int array: still should work. A different literal syntax for a list versus array. "as List"? Sounds right. Alternatively, have a List constructor that suupports any number of arguments?

In the generated Java code, it's always a list of objects. Implementation of List and Array is the same. Is the same class.

x any [] = [ 1, 2 ];

y List (from egl.lang.EList) = [1, 2] as any[]

z int[]

y = z

y.addElement("abc")

appendElement, getElement, etc: you must use an interface for access, or ::, but not indexes.

how to syntactically declared an occurred thing: how to declare a fixed occurred element of type int.

x[3] int; it's an element named x that occurs 3 times. this is not an array: no array object. just a way of stating and referencing something that is a value. could be occurs of string.

x[3] string; a copy of all occurred elements.

occurred elements are fixed size: a static number of elements in the ordered set.

back to the arrays. we need a solution to assignment between lists. we have to allow grids to take arbitrary lists. just having an arbitrary list of things that have no type restriction, as we said earlier.

MOVE is our copy operation, by the way.

The problem with MOVE is that you can't use it inside of an initializer. In EGL, List is a different type than EList.

you can't assign an int array to an any array.

an array of specific records can't be cast as an any array. as operation between lists of different types just forces a copy.

need a conversion operation in EList that takes another typed list and does the conversion of every element.

x any[] = [1,2] is not compatible. x list = [1,2] as any ... will work. no! we do not use the name List after all... and arrays are called lists.

list type will have a new conversion operator... as ANY[]. as INT[].

the element type is explicit. strongly typed.

language elements in bugzilla. finalize the language for the .9 release.

object-oriented EGL by December.

validation: core. how about validation for extensions. "done" says Justin. that's how we proved that they are extensible. Rich UI hasn't been separated out yet. Are we separating out RUI from core? We may choose to package it, fine, but not now.

EGL parts need to be pulled out in a separate eglar. separate EUnit out of the core. EUnit tests aren't even out there. JavaObject and JavaObjectException... are not part of core. We are splitting Java out of core.

6 September 2012

java object can no longer change the instantiation rules.

array types were made more strict to match what was in master for 082. at least it matches what is in master now.

for a normal list, it must stay that way.

binary operations, and whether the right or left side matters: it tries no conversion, and if it must, it widens, and then narrows. more is now being allowed; for examploe, a string in a byte operator, ie string can be converted to int.

int to string is widening. operations are only defined on int, not string.

generic types on arrays. for a list type, we have the type parameter. if it finds an operation parm ANY it changes it to a generic type and converts it to the appropriate type based on the argument that is being passed in.

some parameters known as Elist. they were really arrays of a generic type. for appendAll, takes an EList. Validation requires another layer. the definitions, instead of EList, are now arrays of Any... the appropriate code in the serialization is arrays of generic types.

follows the pattern.... List of Any.

validates that the parameters are correct.

egl source from edt.compiler; from EList.egl.

no different between a list whose type parm is Any and an array of ANYs.

validation for set and move have not been done. various extensions registed with the compiler. if there is no validation for one statement -- the add statement ie you weren't using a valid data source.

call statement can support a. no return, b. returns, or c. returning to. javascript gen only supports returning to.

three kinds of validation. a platform validation is necessary, as is true for many validations. similarly, sql.

if not expression is validated out.

java gen still does not have class.

new stereotype syntax. you can compile stereotypes and annotations. but not the system parts. we have no way to convert mof classes. our system parts reference the mof things differently. mof to binding code no longer exists. ie, we say @operation; but operation has a flag on it to compile as a mof object. it serializes as an EClass.

if you're not trying to extend the language, things are working pretty well.

intern strings compared with ==; but we don't know which way to use. a few thousand edt compile errors. this change should have no effect on generation.

class, bytes. generator implementation was flawed.

a byte array have ops like a fixed-length char. bytes(10) has padding operations. what you want is to manipulate the byte array easily. can the byte substring be on the left? no.

we are going to redo bytes. we have no way to make a fixed-length array versus a list. should be able to get a range of values for any EList. can they be values as well as scalar values? yes.

some sort of static type in the model: for program, for example, to show that it is a static. can't make an instance. is an interface that will be implemented by these things.

compiler extension API. part validation, function, statement, and type validations. part, type and function validation are additive. statement validation is replacement.

particular extensions taking over an ASt node. how are annotations for members delegated? if a validation proxy is present, it is invoked. if you have anntations that are mutually exclusive. the annotation might know about the others (?), or might not.

when we go to create the mof element, we check with the compiler, which checks with the extensions. we don't know what criteria will be used. the core compiler will ask the edt compiler: i have this node, do you have this element. right there, the compiler can check. if multiple extensions want to extend the element, an error.

what is the plan for platform-specific validation? it must be extensible. you can't use RUI handler in Java... is handled in the generator. you have to use the validation framework in the generator to handle the validation. it is already handled. port rules from IR validators to the generators.

will start to write down what other outstanding language elements might be.

we need to talk about the removal of interning and the performance implications of that; but java already interns strings, because they are case insensitive.

is this new way supposed to solve a package case problem? when the variable and type names are the same, the result is weird in the generated code. that's the sort of reason that languages go to case-insensitive.

all the IDE utilities, while being deprecated. the old compiler will live next to the new ones; but we suggest that people use the new compiler. we may not be able to resurrect the old parser.

any new syntax we make needs to be added into the old compiler. any change must go in both places. don't sacrifice the language for deprecated tools. for example, F3, open on selection... why not just get the name and bind to it?

a whole set of tools like search are based on the old tools.

the extension points are being defined.

24 August 2012

Some 25000 test cases for validation; but the results would be different. They must be ported, with aspects of the language that are specific to the IBM Rational Business Developer removed.

They cover a lot of the validation issues.

The focus is on core validations. A small set of easily understood rules. Bitwise comparisons, for example. Per Tim: If the op is not supported by the type, you shouldn't convert it with an implicit "as"; but let's move on.

Algorithms for type and compatibility.

Arrays is a special case. If you have an int array and string array, they were incompatible. Per Tim, that's still true. Even if the array type (target) ... int to Any array ... you would get an Any. but you can't allow an assignment, because a string to the int array is wrong.

String bitwise operator String.... checks for an operator for those types.

String and String, was an error.

String and Int, was not. Because of the conversion.

Binary operations... if a widening conversion exists from one to another, that's valid: Int to string is a widening.

Plus operator is dependent on what type is on the left side. To be investigated.

Also, assignment compatibility. generic types must be resolved to a right type, to make sure. string array, with  :: . if you say a.appendElement.

part validation. certain part types have requirements. program must have a main function for example. parms are invalidated: will never support called programs.

other part types, you can use main() as a function name.

make program akin to handler, with an anno that tells what starts?

handler... anything implemented must be an interface type. and something must be defined for each method listed in the interface type.

class has extends. handler does not. handler validations are a subset of those in class.

should you be able to extend only a class? do not make that restriction now. (anything of type EGLClass is valid?) to be considered.

avoid cycles. cannot extend itself.

Interface... can extend other interfaces. checks for cycles in the hierarchy.

ET... same. make sure that parms are not const, unless primitive types. removed restrictions that all parms are IN.

o now, must specify a stereotype. o ensure that all subtypes in a user hierarchy of ETs are the same.

Library... the basic language rules. Cannot extend or implement anything.

functions are functions wherever they are.

Type validation. Different extensions can contribute different types, and the contributor must provide the validation. if you have a parameterized type (like a timestamp pattern), the validation comes from the extension.

On instantiation. If something is not nullable... should it be instantiated? It must have a public default constructor, if you can. A default constructor is present in any type for which you cannot define a constructor.

JavaScript has an implicit, default constructor. A private, default constructor? You cannot instantiate it; must throw an error. You don't have visibility to the function if you say "new." JavaObject has no default constructor at all.

If you make a constructor for JavaScript object, you must have an explicit default constructor. Stereotype should not change what instantiable means.

If there is an explicit constructor, then in order for folks to be able to invoke the default constructor, you must code the default constructor.

Static types are not full objects; that's why you can't declare a variable. Model needs that element in it. Provide a standard way to specify this in the MOF model.

All functions in a Program were marked private; but that caused a problem, because it genned the main as a private function. The generator should have special processing to handle main. In fact, it is private.

Types need to be able to exist as values. We have a type expression that returns that value, but we haven't implemented that well. int.type; and you can declare vars and parms of those. means more reusable code. can use it to construct an instance of that type. for future discussion. "Types as values."

Later, talk about statements. Design of the compiler extensibility. What is the core of the language. (Is XML in there?) What comes with the compiler, what is a compiler extension? Annotations.

Parts in core: JavaObject, JavaScriptObject. Core for Java, and a core for Javascript, for the generators. MofClass, and others that the compiler itself uses, Stereotype and Annotation. So... the JavaObject and JSObject are extensions to the core validation.

Four different extensible validators: part, type, statement, function validators. Function and statement can be extended. IBMiProgram... IBMi function. if you are changing the type, you must provide your own validation.

Annotation validation. Why is IBMiProgram not an annotation validator. We have a proxy function, and there is an anno on the proxy function. An anno in general is annotating some element; whatever it is annotating, is being given to the validator associated with that element.

can't put an anno on a statement.

the extension is replacing the IR. if you have two annos on a function, what do you do? If two extensions want to extend a part, perhaps throw an error.

at stereotype level, there was member annotations and mutual exclusions. the second said, out of a set of annotations, if you had one you can't have the other.

a function validator is for the case when you change the kind of function, not for any annotation. when do you decide to delegate validation?

validation of the annotaiton is not extensible. you specify in the definition.

there is an annotation validator.

can a stereotype validate the entire type.

XMLAttribute and XMLElement cannot be on the same field. They must be mutually exclusive.

how many validation messages, error codes?