%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%
%A  create.tex                GAP manual                    Thomas Breuer
%A                                                       Martin Schoenert
%%
%H  @(#)$Id: create.tex,v 4.42 2001/12/11 07:50:10 gap Exp $
%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Chapter{Creating New Objects}

This chapter is divided into three parts.

In the first part, it is explained how to create
filters (see~"Creating Categories", "Creating Representations",
"Creating Attributes and Properties", "Creating Other Filters"),
operations (see~"Creating Operations"),
families (see~"Creating Families"),
types (see~"Creating Types"),
and objects with given type (see~"Creating Objects").

In the second part, first a few small examples are given,
for dealing with the usual cases of
component objects (see~"Component Objects")
and positional objects (see~"Positional Objects"),
and for the implementation of new kinds of lists
(see~"Implementing New List Objects"
and~"Arithmetic Issues in the Implementation of New Kinds of Lists").
Finally, the external representation of objects is introduced
(see~"External Representation"),
as a tool for representation independent access to an object.

The third part deals with some rules concerning the organization
of the {\GAP} library;
namely, some commands for creating global variables are explained
(see~"Global Variables in the Library")
that correspond to the ones discussed in the first part of the chapter,
and the idea of distinguishing declaration and implementation part
of {\GAP} packages is outlined (see~"Declaration and Implementation Part").

See also Chapter~"An Example -- Residue Class Rings" for examples
how the functions from the first part are used,
and why it is useful to have a declaration part and an implementation part.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Categories}

\>NewCategory( <name>, <super> )

`NewCategory' returns a new category <cat> that has the name <name> and
is contained in the filter <super>,
see~"ref:Filters" in the Reference Manual.
This means that every object in <cat> lies automatically also in <super>.
We say also that <super> is an implied filter of <cat>.

For example, if one wants to create a category of group elements
then <super> should be `IsMultiplicativeElementWithInverse' or a
subcategory of it.
If no specific supercategory of <cat> is known,
<super> may be `IsObject'.

*@Eventually tools will be provided to display hierarchies of
categories etc.,
which will help to choose <super> appropriately.@*

The incremental rank (see~"ref:Filters" in the Reference Manual)
of <cat> is 1.

Two functions that return special kinds of categories are of importance.

\>CategoryCollections( <cat> )

For a category <cat>,
`CategoryCollections' returns the *collections category* of <cat>.
This is a category in that all collections of objects in <cat> lie.

For example, a permutation lies in the category `IsPerm',
and every dense list of permutations and every domain of permutations
lies in the collections category of `IsPerm'.

\>CategoryFamily( <cat> )

For a category <cat>,
`CategoryFamily' returns the *family category* of <cat>.
This is a category in that all families lie that know from their
creation that all their elements are in the category <cat>,
see~"Creating Families".

For example, a family of tuples is in the category
`CategoryFamily( IsTuple )',
and one can distinguish such a family from others by this category.
So it is possible to install methods for operations that require one
argument to be a family of tuples.

`CategoryFamily' is quite technical, and in fact of minor importance.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Representations}

\>NewRepresentation( <name>, <super>, <slots> )

`NewRepresentation' returns a new representation <rep> that has the name
<name> and is a subrepresentation of the representation <super>.
This means that every object in <rep> lies automatically also in <super>.
We say also that <super> is an implied filter of <rep>.

Each representation in {\GAP} is a subrepresentation of exactly one
of the four representations `IsInternalRep', `IsDataObjectRep',
`IsComponentObjectRep', `IsPositionalObjectRep'.
The data describing objects in the former two can be accessed only via
{\GAP} kernel functions, the data describing objects in the latter two
is accessible also in library functions, see~"Component Objects"
and~"Positional Objects" for the details.

The third argument <slots> is a list either of integers or of strings.
In the former case, <rep> must be `IsPositionalObjectRep' or a
subrepresentation of it, and <slots> tells what positions of the objects
in the representation <rep> may be bound.
In the latter case, <rep> must be `IsComponentObjectRep' or a
subrepresentation of, and <slots> lists the admissible names of
components that objects in the representation <rep> may have.
The admissible positions resp. component names of <super> need not be
be listed in <slots>.

The incremental rank (see~"ref:Filters" in the Reference Manual)
of <rep> is 1.

Note that for objects in the representation <rep>,
of course some of the component names and positions reserved via <slots>
may be unbound.

Examples for the use of `NewRepresentation' can be found
in~"Component Objects", "Positional Objects", and also
in~"A Second Attempt to Implement Elements of Residue Class Rings".


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Attributes and Properties}

\>NewAttribute( <name>, <filt> )
\>NewAttribute( <name>, <filt>, <rank> )

`NewAttribute' returns a new attribute <attr> with name <name>
(see also~"ref:Attributes" in the Reference Manual).
The filter <filt> describes the involved filters of <attr>
(see~"ref:Filters" in the Reference Manual).
That is, the argument for <attr> is expected to lie in <filt>.

Each method for <attr> that does *not* require its argument to lie
in <filt> must be installed using `InstallOtherMethod'.

Contrary to the situation with categories and representations,
the tester of <attr> does *not* imply <filt>.
This is exactly because of the possibility to install methods
that do not require <filt>.

For example, the attribute `Size' was created with second argument
a list or a collection,
but there is also a method for `Size' that is applicable to
a character table, which is neither a list nor a collection.

The optional third argument <rank> denotes the incremental rank
(see~"ref:Filters" in the Reference Manual) of the tester of <attr>,
the default value is 1.

\>NewAttribute( <name>, <filt>, "mutable" )!{mutable}
\>NewAttribute( <name>, <filt>, "mutable", <rank> )!{mutable}

If the third argument is the string `"mutable"', the stored values of the
new attribute are not forced to be immutable.
This is useful for an attribute whose value is some partial information
that may be completed later.
For example, there is an attribute `ComputedSylowSubgroups' for the list
holding those Sylow subgroups of a group that have been computed already
by the function `SylowSubgroup',
and this list is mutable because one may want to enter groups into it
as they are computed.

\>NewProperty( <name>, <filt> )
\)\fmark NewProperty( <name>, <filt>, <rank> )

`NewProperty' returns a new property <prop> with name <name>
(see also~"ref:Properties" in the Reference Manual).
The filter <filt> describes the involved filters of <prop>.
As in the case of attributes, <filt> is not implied by <prop>.

The optional third argument <rank> denotes the incremental rank
(see~"ref:Filters" in the Reference Manual) of the property <prop> itself,
i.e. *not* of its tester, the default value is 1.

Each method that is installed for an attribute or a property
via `InstallMethod' must require exactly one argument,
and this must lie in the filter <filt> that was entered as second
argument of `NewAttribute' resp. `NewProperty'.

As for any operation (see~"Creating Operations"),
for attributes and properties one can install a method taking an argument
that does not lie in <filt> via `InstallOtherMethod',
or a method for more than one argument;
in the latter case,
clearly the result value is *not* stored in any of the arguments.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Other Filters}

\>NewFilter( <name> )
\)\fmark NewFilter( <name>, <rank> )

`NewFilter' returns a simple filter with name <name>
(see~"ref:Other Filters" in the Reference Manual).
The optional second argument <rank> denotes the incremental rank
(see~"ref:Filters" in the Reference Manual) of the filter,
the default value is 1.

In order to change the value of <filt> for an object <obj>,
one can use logical implications (see~"Logical Implications") or
the functions

\>SetFilterObj( <obj>, <filt> )
\>ResetFilterObj( <obj>, <filt> )

`SetFilterObj' sets the value of <filt> (and of all filters implied by
<filt>) for <obj> to `true',

`ResetFilterObj' sets the value of <filt> for <obj> to `false' (but implied
filters of <filt> are not touched. This might create inconsistent situations
if applied carelessly).

The default value of <filt> for each object is `false'.

%T Categories and representations should not be operations,
%T the same for filters made by `NewFilter'!


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Operations}

\>NewOperation( <name>, <args-filts> )

`NewOperation' returns an operation <opr> with name <name>.
The list <args-filts> describes requirements about the arguments
of <opr>, namely the number of arguments must be equal to the length of
<args-filts>, and the $i$-th argument must lie in the filter
`<args-filts>[$i$]'.

Each method that is installed for <opr> via `InstallMethod' must require
that the $i$-th argument lies in the filter `<args-filts>[$i$]'.

One can install methods for other arguments tuples via
`InstallOtherMethod',
this way it is also possible to install methods for a different number
of arguments than the length of <args-filts>.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Families}

Families are probably the least obvious part of the {\GAP} type system,
so some remarks about the role of families are necessary.
When one uses {\GAP} as it is, one will (better: should) not meet
families at all.
The two situations where families come into play are the following.

First, since families are used to describe relations between arguments of
operations in the method selection mechanism
(see Chapter~"Method Selection" in this manual,
and also Chapter~"ref:Types of Objects" in the Reference Manual),
one has to prescribe such a relation in each method installation
(see~"Method Installation");
usual relations are `ReturnTrue' (which means that any relation of the
actual arguments is admissible), `IsIdenticalObj' (which means that
there are two arguments that lie in the same family),
and `IsCollsElms' (which means that there are two arguments,
the first being a collection of elements that lie in the same family
as the second argument).

Second ---and this is the more complicated situation---
whenever one creates a new kind of objects,
one has to decide what its family shall be.
If the new object shall be equal to existing objects,
for example if it is just represented in a different way,
there is no choice:
The new object must lie in the same family as all objects
that shall be equal to it.
So only if the new object is different (w.r.t.~the equality ```=''')
from all other {\GAP} objects, we are likely to create a new family
for it.
Note that enlarging an existing family by such new objects
may be problematic because of implications that have been
installed for all  objects of the family in question.
The choice of families depends on the applications one has in mind.
For example, if the new objects in question are not likely to be
arguments of operations for which family relations are relevant
(for example binary arithmetic operations),
one could create one family for all such objects,
and regard it as ``the family of all those {\GAP} objects that would
in fact not need a family''.
On the other extreme, if one wants to create domains of the new objects
then one has to choose the family in such a way that all intended
elements of a domain do in fact lie in the same family.
(Remember that a domain is a collection, see Chapter~"ref:Domains"
in the Reference Manual, and that a collection consists of elements
in the same family, see Chapter~"ref:Collections" and Section~"ref:Families"
in the Reference Manual.)

Let us look at an example.
Suppose that no permutations are available in {\GAP},
and that we want to implement permutations.
Clearly we want to support permutation groups,
but it is not a priori clear how to distribute the new permutations
into families.
We can put all permutations into one family;
this is how in fact permutations are implemented in {\GAP}.
But it would also be possible to put all permutations of a given degree
into a family of their own;
this would for example mean that for each degree,
there would be distinguished trivial permutations,
and that the stabilizer of the point `5' in the symmetric group on the
points $`1', `2', \ldots, `5'$ is not regarded as equal to the
symmetric group on $`1', `2', `3', `4'$.
Note that the latter approach would have the advantage that it is
no problem to construct permutations and permutation groups acting on
arbitrary (finite) sets,
for example by constructing first the symmetric group on the set
and then generating any desired permutation group as a subgroup of this
symmetric group.

So one aspect concerning a reasonable choice of families is
to make the families large enough for being able to form interesting
domains of elements in the family.
But on the other hand,
it is useful to choose the families small enough for admitting
meaningful relations between objects.
For example, the elements of different free groups in {\GAP}
lie in different families;
the multiplication of free group elements is installed only for the
case that the two operands lie in the same family,
with the effect that one cannot erroneously form the product of
elements from different free groups.
In this case, families appear as a tool for providing useful
restrictions.

As another example, note that an element and a collection containing
this element never lie in the same family,
by the general implementation of collections;
namely, the family of a collection of elements in the family <Fam>
is the collections family of <Fam> (see~"CollectionsFamily").
This means that for a collection, we need not (because we cannot)
decide about its family.


\>NewFamily( <name> )
\)\fmark NewFamily( <name>, <req> )
\)\fmark NewFamily( <name>, <req>, <imp> )
\)\fmark NewFamily( <name>, <req>, <imp>, <famfilter> )

`NewFamily' returns a new family <fam> with name <name>.
The argument <req>, if present, is a filter of which <fam> shall be a
subset.
If one tries to create an object in <fam> that does not lie in the filter
<req>, an error message is printed.
Also the argument <imp>, if present,
is a filter of which <fam> shall be a subset.
Any object that is created in the family <fam> will lie automatically in
the filter <imp>.

The filter <famfilter>, if given, specifies a filter that will hold for the
family <fam> (not for objects in <fam>).

Families are always represented as component objects
(see~"Component Objects").
This means that components can be used to store and access
useful information about the family.

There are a few functions in {\GAP} that construct families.
As an example, consider
(see also~"ref:Collection Families" in the Reference Manual)

\>CollectionsFamily( <fam> )

`CollectionsFamily' is an attribute that takes a family <fam> as
argument, and returns the family of all collections over <fam>,
that is, of all dense lists and domains that consist of objects in
<fam>.

The `NewFamily' call in the standard method of `CollectionsFamily'
is executed with second argument `IsCollection',
since every object in the collections family must be a collection,
and with third argument the collections categories of the involved
categories in the implied filter of <fam>.

If <fam> is a collections family then

\>ElementsFamily( <fam> )

returns the unique family with collections family <fam>;
note that by definition,
all elements in a collection lie in the same family,
so `ElementsFamily( <fam> )' is the family of each element
in any collection that has the family <fam>.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Types}

\>NewType( <fam>, <filt> )
\)\fmark NewType( <fam>, <filt>, <data> )

`NewType' returns the type given by the family <fam>
and the filter <filt>.
The optional third argument <data> is any object that denotes defining
data of the desired type.

For examples where `NewType' is used, see~"Component Objects",
"Positional Objects",
and the example in Chapter~"An Example -- Residue Class Rings".


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Creating Objects}

\>Objectify( <type>, <data> ) F

New objects are created by `Objectify'. <data> is a list or a record, and
<type> is the type that the desired object shall have. `Objectify'  turns
<data> into an object with type <type>. That is, <data> is  changed,  and
afterwards it will not be a list or a record unless  <type>  is  of  type
list resp. record.

If <data> is a list then `Objectify' turns it into a  positional  object,
if <data> is a record then `Objectify' turns it into a  component  object
(for examples, see~"Component Objects" and~"Positional Objects").

`Objectify' does also return the object that it made out of <data>.

For  examples  where  `Objectify'  is  used,   see~"Component   Objects",
"Positional Objects", and the example in Chapter~"An Example  --  Residue
Class Rings".

Attribute assignments will change the type of an object.  If  you  create
many objects, code of the form

\begintt
o:=Objectify(type,rec());
SetMyAttribute(o,value);
\endtt

will take a lot of time for type changes. You can avoid this  by  setting
the attributes immediately while the object is created, via:

\>ObjectifyWithAttributes(<obj>,<type>,<Attr1>,<val1>[,<Attr2>,<val2>...]) F

which changes the type of object <obj> to type <type> and sets  attribute
<Attr1> to <val1>, sets attribute <Attr2> to <val2> and so forth.

If the filter list of <type> includes that these attributes are set  (and
the properties also include values of the properties) and if  no  special
setter methods are installed for any of the involved attributes then they
are  set  simultaneously  without  type  changes  which  can  produce   a
substantial speedup.

If the conditions of the last sentence are  not  fulfilled,  an  ordinary
`Objectify'  with  subsequent  `Setter'  calls  for  the  attributes   is
performed, instead.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Component Objects}

A *component object* is an object in the representation
`IsComponentObjectRep' or a subrepresentation of it.
Such an object <cobj> is built from subobjects that can be accessed via
`<cobj>!.<name>', similar to components of a record.
Also analogously to records, values can be assigned to components of
<cobj> via `<cobj>!.<name>:= <val>'.
For the creation of component objects, see~"Creating Objects".

\>NamesOfComponents( <comobj> ) F

For a component object <comobj>, `NamesOfComponents' returns a list of
strings, which are the names of components currently bound in <comobj>.

One must be *very careful* when using the `!.' operator,
in order to interpret the component in the right way,
and even more careful when using the assignment to components using `!.',
in order to keep the information stored in <cobj> consistent.

First of all, in the access or assignment to a component as shown above,
<name> must be among the admissible component names
for the representation of <cobj>, see~"Creating Representations".
Second, preferably only few low level functions should use `!.',
whereas this operator should not occur in ``user interactions''.

Note that even if <cobj> claims that it is immutable, i.e., if <cobj>
is not in the category `IsMutable', access and assignment via `!.' work.
This is necessary for being able to store newly discovered information
in immutable objects.

The following example shows the implementation of an iterator
(see~"ref:Iterators" in the Reference Manual) for the domain of integers,
which is represented as component object.
See~"Positional Objects" for an implementation using positional objects.

The used succession of integers is $0, 1, -1, 2, -2, 3, -3, \ldots$,
that is, $a_n = n/2$ if $n$ is even,
and $a_n = (1-n)/2$ otherwise.

\begintt
IsIntegersIteratorCompRep := NewRepresentation( "IsIntegersIteratorRep",
    IsComponentObjectRep, [ "counter" ] );
\endtt

The above command creates a new representation (see~"NewRepresentation")
`IsIntegersIteratorCompRep',
as a subrepresentation of `IsComponentObjectRep',
and with one admissible component `counter'.
So no other components than `counter' will be needed.

\begintt
InstallMethod( Iterator,
    "method for `Integers'",
    [ IsIntegers ],
    function( Integers )
    return Objectify( NewType( IteratorsFamily,
                                   IsIterator
                               and IsIntegersIteratorCompRep ),
                      rec( counter := 0 ) );
    end );
\endtt

After the above method installation, one can already ask for
`Iterator( Integers )'.
Note that exactly the domain of integers is described by
the filter `IsIntegers'.

By the call to `NewType', the returned object lies in the family
containing all iterators, which is `IteratorsFamily',
it lies in the category `IsIterator' and in the representation
`IsIntegersIteratorCompRep';
furthermore, it has the component `counter' with value `0'.

What is missing now are methods for the two basic operations
of iterators, namely `IsDoneIterator' and `NextIterator'.
The former must always return `false', since there are infinitely
many integers.
The latter must return the next integer in the iteration,
and update the information stored in the iterator,
that is, increase the value of the component `counter'.

\begintt
InstallMethod( IsDoneIterator,
    "method for iterator of `Integers'",
    [ IsIterator and IsIntegersIteratorCompRep ],
    ReturnFalse );

InstallMethod( NextIterator,
    "method for iterator of `Integers'",
    [ IsIntegersIteratorCompRep ],
    function( iter )
    iter!.counter:= iter!.counter + 1;
    if iter!.counter mod 2 = 0 then
      return iter!.counter / 2;
    else
      return ( 1 - iter!.counter ) / 2;
    fi;
    end );
\endtt

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Positional Objects}

A *positional object* is an object in the representation
`IsPositionalObjectRep' or a subrepresentation of it.
Such an object <pobj> is built from subobjects that can be accessed via
`<pobj>![<pos>]', similar to positions in a list.
Also analogously to lists, values can be assigned to positions of
<pobj> via `<pobj>![<pos>]:= <val>'.
For the creation of positional objects, see~"Creating Objects".

One must be *very careful* when using the `![]' operator,
in order to interpret the position in the right way,
and even more careful when using the assignment to positions using `![]',
in order to keep the information stored in <pobj> consistent.

First of all, in the access or assignment to a position as shown above,
<pos> must be among the admissible positions
for the representation of <pobj>, see~"Creating Representations".
Second, preferably only few low level functions should use `![]',
whereas this operator should not occur in ``user interactions''.

Note that even if <pobj> claims that it is immutable, i.e., if <pobj>
is not in the category `IsMutable', access and assignment via `![]' work.
This is necessary for being able to store newly discovered information
in immutable objects.

The following example shows the implementation of an iterator
(see~"ref:Iterators" in the Reference Manual) for the domain of integers,
which is represented as positional object.
See~"Component Objects" for an implementation using component objects,
and more details.

\begintt
IsIntegersIteratorPosRep := NewRepresentation( "IsIntegersIteratorRep",
    IsPositionalObjectRep, [ 1 ] );
\endtt

The above command creates a new representation (see~"NewRepresentation")
`IsIntegersIteratorPosRep',
as a subrepresentation of `IsComponentObjectRep',
and with only the first position being admissible for storing data.

\begintt
InstallMethod( Iterator,
    "method for `Integers'",
    [ IsIntegers ],
    function( Integers )
    return Objectify( NewType( IteratorsFamily,
                                   IsIterator
                               and IsIntegersIteratorRep ),
                      [ 0 ] );
    end );
\endtt

After the above method installation, one can already ask for
`Iterator( Integers )'.
Note that exactly the domain of integers is described by
the filter `IsIntegers'.

By the call to `NewType', the returned object lies in the family
containing all iterators, which is `IteratorsFamily',
it lies in the category `IsIterator' and in the representation
`IsIntegersIteratorPosRep';
furthermore, the first position has value `0'.

What is missing now are methods for the two basic operations
of iterators, namely `IsDoneIterator' and `NextIterator'.
The former must always return `false', since there are infinitely
many integers.
The latter must return the next integer in the iteration,
and update the information stored in the iterator,
that is, increase the value stored in the first position.

\begintt
InstallMethod( IsDoneIterator,
    "method for iterator of `Integers'",
    [ IsIterator and IsIntegersIteratorPosRep ],
    ReturnFalse );

InstallMethod( NextIterator,
    "method for iterator of `Integers'",
    [ IsIntegersIteratorPosRep ],
    function( iter )
    iter![1]:= iter![1] + 1;
    if iter![1] mod 2 = 0 then
      return iter![1] / 2;
    else
      return ( 1 - iter![1] ) / 2;
    fi;
    end );
\endtt

It should be noted that one can of course install both the methods shown
in Section~"Component Objects" and "Positional Objects".
The call `Iterator( Integers )' will cause one of the methods to be
selected, and for the returned iterator, which will have one of the
representations we constructed, the right `NextIterator' method
will be chosen.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Implementing New List Objects}

This section gives some hints for the quite usual situation that one wants
to implement new objects that are lists.
More precisely, one either wants to deal with lists that have additional
features, or one wants that some objects also behave as lists.

A *list* in {\GAP} is an object in the category `IsList'.
Basic operations for lists are `Length', `\\[\\]', and `IsBound\\[\\]'
(see~"ref:Basic Operations for Lists" in the Reference Manual).

Note that the access to the position <pos> in the list <list>
via `<list>[<pos>]' is handled by the call `\\[\\]( <list>, <pos> )'
to the operation `\\[\\]'.
To explain the somewhat strange name `\\[\\]' of this operation,
note that non-alphanumeric characters like `[' and `]' may occur in
{\GAP} variable names only if they are escaped by a `\\' character.

Analogously, the check `IsBound( <list>[<pos>] )' whether the position
<pos> of the list <list> is bound is handled by the call
`IsBound\\[\\]( <list>, <pos> )' to the operation
`IsBound\\[\\]'.

For mutable lists, also assignment to positions and unbinding of
positions via the operations `\\[\\]\\:\\=' and `Unbind\\[\\]'
are basic operations.
The assignment `<list>[<pos>]:= <val>' is handled by the call
`\\[\\]\\:\\=( <list>, <pos>, <val> )',
and `Unbind( <list>[<pos>] )' is handled by the call
`Unbind\\[\\]( <list>, <pos> )'.

All other operations for lists, e.g., `Add', `Append', `Sum',
are based on these operations.
This means that it is sufficient to install methods for the new list
objects only for the basic operations.

So if one wants to implement new list objects then one creates them
as objects in the category `IsList', and installs methods for `Length',
`\\[\\]', and `IsBound\\[\\]'.
If the new lists shall be mutable, one needs to install also methods
for `\\[\\]\\:\\=' and `Unbind\\[\\]'.

One application for this is the implementation of *enumerators*
for domains.
An enumerator for the domain $D$ is a dense list whose entries are
in bijection with the elements of $D$.
If $D$ is large then it is not useful to write down all elements.
Instead one can implement such a bijection implicitly.
This works also for infinite domains.

In this situation, one implements a new representation of the
lists that are already available in {\GAP},
in particular the family of such a list is the same as the family of
the domain $D$.

But it is also possible to implement new kinds of lists that lie in
new families, and thus are not equal to lists that were available
in {\GAP} before.
An example for this is the implementation of matrices
whose multiplication via ```*''' is the Lie product of matrices.

In this situation, it makes no sense to put the new matrices into the
same family as the original matrices.
Note that the product of two Lie matrices shall be defined but not the
product of an ordinary matrix and a Lie matrix.
So it is possible to have two lists that have the same entries but that
are not equal w.r.t. ```=''' because they lie in different families.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Arithmetic Issues in the Implementation of New Kinds of Lists}

When designing a new kind of list objects in {\GAP},
defining the arithmetic behaviour of these objects is an issue.

There are situations where arithmetic operations of list objects
are unimportant in the sense that adding two such lists need not be
represented in a special way.
In such cases it might be useful either to support no arithmetics at all
for the new lists, or to enable the default arithmetic methods.
The former can be achieved by not setting the filters
`IsGeneralizedRowVector' and `IsMultiplicativeGeneralizedRowVector'
in the types of the lists,
the latter can be achieved by setting the filter `IsListDefault'.
(for details, see~"ref:Filters Controlling the Arithmetic Behaviour of Lists"
in the {\GAP} Reference Manual).
An example for ``wrapped lists'' with default behaviour are vector space
bases;
they are lists with additional properties concerning the computation of
coefficients, but arithmetic properties are not important.
So it is no loss to enable the default methods for these lists.

However, often the arithmetic behaviour of new list objects is important,
and one wants to keep these lists away from default methods for addition,
multiplication etc.
For example, the sum and the product of (compatible) block matrices shall
be represented as a block matrix, so the default methods for sum and
product of matrices shall not be applicable,
although the results will be equal to those of the default methods
in the sense that their entries at corresponding positions are equal.

So one does not set the filter `IsListDefault' in such cases,
and thus one can implement one's own methods for arithmetic operations.
% It should be stated explicitly what ``arithmetic operations'' means!
(Of course ``can'' means on the other hand that one *must* implement such
methods if one is interested in arithmetics of the new lists.)

The specific binary arithmetic methods for the new lists will usually cover
the case that both arguments are of the new kind,
and perhaps also the interaction between a list of the new kind and certain
other kinds of lists may be handled if this appears to be useful.

For the last situation, interaction between a new kind of lists and other
kinds of lists, {\GAP} provides already a setup.
Namely, there are the categories `IsGeneralizedRowVector' and
`IsMultiplicativeGeneralizedRowVector', which are concerned with the
additive and the multiplicative behaviour, respectively, of lists. 
For lists in these filters, the structure of the results of arithmetic
operations is prescribed (see~"ref:Additive Arithmetic for Lists" and
"ref:Multiplicative Arithmetic for Lists" in the {\GAP} Reference Manual).

For example,
if one implements block matrices in `IsMultiplicativeGeneralizedRowVector'
then automatically the product of such a block matrix and a (plain) list
of such block matrices will be defined as the obvious list of matrix
products, and a default method for plain lists will handle this
multiplication.
(Note that this method will rely on a method for computing the product of
the block matrices, and of course no default method is available for that.)
Conversely, if the block matrices are not in
`IsMultiplicativeGeneralizedRowVector' then the product of a block matrix
and a (plain) list of block matrices is not defined.
(There is no default method for it, and one can define the result and
provide a method for computing it.)

Thus if one decides to set the filters `IsGeneralizedRowVector' and
`IsMultiplicativeGeneralizedRowVector' for the new lists,
on the one hand one loses freedom in defining arithmetic behaviour,
but on the other hand one gains several default methods for a more
or less natural behaviour.

If a list in the filter `IsGeneralizedRowVector'
(`IsMultiplicativeGeneralizedRowVector') lies in `IsAttributeStoringRep',
the values of additive (multiplicative) nesting depth is stored in
the list and need not be calculated for each arithmetic operation.
One can then store the value(s) already upon creation of the lists,
with the effect that the default arithmetic operations will access
elements of these lists only if this is unavoidable.
For example, the sum of two plain lists of ``wrapped matrices'' with
stored nesting depths are computed via the method for adding two such
wrapped lists, and without accessing any of their rows
(which might be expensive).
In this sense, the wrapped lists are treated as black boxes.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{External Representation}

An operation is defined for elements rather than for objects in the sense
that if the arguments are replaced by objects that are equal to the old
arguments w.r.t. the equivalence relation ```=''' then the result must be
equal to the old result w.r.t.~```='''.

But the implementation of many methods is representation dependent in the
sense that certain representation dependent subobjects are accessed.

For example, a method that implements the addition of univariate
polynomials may access coefficients lists of its arguments
only if they are really stored,
while in the case of sparsely represented polynomials a different approach
is needed.

In spite of this, for many operations one does not want to write an own
method for each possible representations of each argument,
for example because none of the methods could in fact take advantage
of the actually given representations of the objects.
Another reason could be that one wants to install first a representation
independent method, and then add specific methods as they are needed to
gain more efficiency, by really exploiting the fact that the arguments
have certain representations.

For the purpose of admitting representation independent code,
one can define an *external representation* of objects in a given family,
install methods to compute this external representation for each
representation of the objects,
and then use this external representation of the objects whenever they
occur.

We cannot provide conversion functions that allow us to first convert
any object in question to one particular ``standard representation'',
and then access the data in the way defined for this representation,
simply because it may be impossible to choose such a ``standard
representation'' uniformly for all objects in the given family.

So the aim of an external representation of an object <obj> is a
different one, namely to describe the data from which <obj> is composed.
In particular, the external representation of <obj> is *not* one possible
(``standard'') representation of <obj>,
in fact the external representation of <obj> is in general different
from <obj> w.r.t.~```=''',
first  of all because the external representation of <obj> does in general
not lie in the same family as <obj>.

For example the external representation of a rational function is a list
of length two or three, the first entry being the zero coefficient,
the second being a list describing the coefficients and monomials of the
numerator, and the third, if bound, being a list describing the coefficients
and monomials of the denominator.
In particular, the external representation of a polynomial is a list
and not a polynomial.

The other way round, the external representation of <obj> encodes <obj>
in such a way that from this data and the family of <obj>,
one can create an object that is equal to <obj>.
Usually the external representation of an object is a list or a record.

Although the external representation of <obj> is by definition independent
of the actually available representations for <obj>,
it is usual that a representation of <obj> exists for which the
computation of the external representation is obtained by just
``unpacking'' <obj>,
in the sense that the desired data is stored in a component or a position
of <obj>, if <obj> is a component object (see~"Component Objects")
or a positional object (see~"Positional Objects").

To implement an external representation means to install methods for the
following two operations.

\>ExtRepOfObj( <obj> )
\>ObjByExtRep( <fam>, <data> )

`ExtRepOfObj' returns the external representation of its argument,
and `ObjByExtRep' returns an object in the family <fam> that has
external representation <data>.

Of course, `ObjByExtRep( FamilyObj( <obj> ), ExtRepOfObj( <obj> ) )'
must be equal to <obj>.
But it is *not* required that equal objects have equal external
representations.

Note that if one defines a new representation of objects for which an
external representation does already exist
then one *must* install a method to compute this external representation
for the objects in the new representation.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Mutability and Copying}

Any {\GAP} Object is either mutable or immutable. This can be tested
with the Operation `IsMutable'. The intended meaning of (im)mutability
is a mathematical one:  an immutable Object should never change in
such a way that it represents a different Element. Objects *may*
change in other ways, for instance to store more information, or
represent an element in a different way.

Immutability is enforced in different ways for built-in objects (like
records, or lists) and for external objects (made using `Objectify').

For built-in objects which are immutable, the kernel will prevent
you from changing them. Thus

\beginexample
gap> l := [1,2,4];
[ 1, 2, 4 ]
gap> MakeImmutable(l);
gap> l[3] := 5;
Lists Assignment: <list> must be a mutable list
not in any function
Entering break read-eval-print loop ...
you can 'quit;' to quit to outer loop, or
you can 'return;' and ignore the assignment to continue
brk> 
\endexample

For external Objects, the situation is different. An external Object which
claims to be immutable (i.e. its Type does not contain `IsMutable')
should not admit any Methods which change the Element it represents.
The kernel does *not* prevent the use of `!.' and `![' to change the
underlying data structure.
This is used for instance by the code that stores Attribute values for reuse.
In general, these `!' operations should only be used in Methods
which depend on the Representation of the Object. Furthermore, we would *not*
recommend users to install Methods which depend on the Representations of
Objects created by the library or by {\GAP} packages, as there is certainly no
guarantee of the representations being the same in future versions of {\GAP}.

Here we see an immutable Object (the group $S_4$), in which we improperly
install a new component.

\beginexample
gap> g := SymmetricGroup(IsPermGroup,4);
Sym( [ 1 .. 4 ] )
gap> IsMutable(g);
false
gap> NamesOfComponents(g);
[ "GeneratorsOfMagmaWithInverses", "Size", "MovedPoints", "NrMovedPoints" ]
gap> g!.silly := "rubbish";
"rubbish"
gap> NamesOfComponents(g);
[ "GeneratorsOfMagmaWithInverses", "Size", "MovedPoints", "NrMovedPoints", 
  "silly" ]
gap> g!.silly;
"rubbish"
\endexample

On the other hand, if we form an immutable externally represented list, we
find that there is no Method for changing it using list assignment

\beginexample
gap> e := Enumerator(g);
<enumerator of perm group>
gap> IsMutable(e);
false
gap> IsList(e);
true
gap> e[3];
(1,4)(2,3)
gap> e[3] := false;
Error, no method found! For debugging hints type ?Recovery from NoMethodFound
Error, no 1st choice method found for `ASS_LIST' on 3 arguments called from
... lines omitted here ...
\endexample

When we consider copying Objects, another filter `IsCopyable', enters
the game and we find that `ShallowCopy' and `StructuralCopy' behave quite
differently. Objects can be divided for this purpose into three:
mutable Objects, immutable but copyable Objects, and non-copyable
objects (called constants).

A mutable or copyable  Object should have a Method for the Operation
`ShallowCopy', which should make a new mutable Object, sharing its top-level
subobjects with the original. The exact definition of top-level subobject may
be defined by the implementor for new kinds of Object.

`ShallowCopy' applied to a constant simply returns the constant.

`StructuralCopy' is expected to be much less used than
`ShallowCopy'. Applied to a mutable object, it returns a new mutable
object which shares no mutable sub-objects with the input. Applied to
an immutable Object (even a copyable one), it just returns the
object. It is not an Operation (indeed, it's a rather special kernel
function).

\beginexample
gap> e1 := StructuralCopy(e);
<enumerator of perm group>
gap> IsMutable(e1);
false
gap> e2 := ShallowCopy(e);
[ (), (1,2)(3,4), (1,4)(2,3), (1,3)(2,4), (2,3,4), (1,2,4), (1,4,3), (1,3,2), 
  (2,4,3), (1,2,3), (1,4,2), (1,3,4), (3,4), (1,2), (1,4,2,3), (1,3,2,4), 
  (2,3), (1,2,4,3), (1,4), (1,3,4,2), (2,4), (1,2,3,4), (1,4,3,2), (1,3) ]
gap> 
\endexample

There are two other related functions: `Immutable', which makes a new
immutable object which shares no mutable subobjects with its input and
`MakeImmutable' which changes an object and its mutable subobjects *in
place* to be immutable. It should only be used on ``new'' Objects that
you have just created, and which cannot share mutable subobjects with
anything else.

Both `Immutable' and `MakeImmutable' work on external objects by just
resetting the `IsMutable' filter in the Object's type. This should make
ineligible any methods that might change the Object. As a consequence,
you must allow for the possibility of immutable versions of any
objects you create.

So, if you are implementing your own external Objects. The rules amount to the
following:

\beginlist%ordered

\item{1.}
You decide if your Objects should be mutable or copyable or constants, by
fixing whether their Type includes `IsMutable' or `IsCopyable'.

\item{2.}
You install Methods for your objects respecting that decision:

\itemitem{$\bullet$}%unordered
for constants -- no methods change the underlying elements;

\itemitem{$\bullet$}
for copyables -- you provide a method for `ShallowCopy';

\itemitem{$\bullet$}
for mutables  -- you may have methods that change the underlying elements
and these should explicitly require `IsMutable'.

\endlist

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Global Variables in the Library}

Global variables in the {\GAP} library are usually read-only in order to
avoid their being overwritten accidentally.

\>BindGlobal( <name>, <val> ) F

sets the global variable named by the string <name> to the value <val>,
and makes it read-only.
An error is given if the global variable corresponding to <name>
already had a value bound.

\>DeclareAttribute( <name>, <filt>[, "mutable"][, <rank>] ) F
\>DeclareCategory( <name>, <super> ) F
\>DeclareFilter( <name>, <rank> ) F
\>DeclareProperty( <name>, <filt>[, <rank>] ) F
\>DeclareRepresentation( <name>, <super>, <slots> ) F

The different types of filters (see Sections~"Creating Categories",
"Creating Representations", "Creating Attributes and Properties",
"Creating Other Filters") that are used in the {\GAP} library are
assigned by the above `Declare<Something>' functions which make the
variable with name <name> (a string) automatically read-only.
The only other difference between `New<Something>' and
`Declare<Something>' is that `DeclareAttribute' and `DeclareProperty'
also bind read-only global variables with names `Has<name>' and `Set<name>'
for the tester and setter of the attribute
(see Section~"ref:Setter and Tester for Attributes" in the Reference Manual).
For the meaning of the other arguments of `Declare<Something>',
see~"NewAttribute", "NewCategory", "NewFilter", "NewProperty", and
"NewRepresentation".

\>DeclareOperation( <name>, <args-filts> ) F
\>DeclareGlobalFunction( <name> ) F

declare operations and other global functions used in the {\GAP} library,
respectively,
are assigned to the read-only variable with name <name> (a string).
For the meaning of the other arguments of `DeclareOperation',
see~"NewOperation".

{\GAP} functions that are not operations and that are intended to be
called by users should be notified to {\GAP} in the declaration part
of the respective package
(see Section~"Declaration and Implementation Part")
via `DeclareGlobalFunction', which returns a function that serves as a
place holder for the function that will be installed later,
and that will print an error message if it is called.
See also~"DeclareSynonym".

\>InstallGlobalFunction( <gvar>, <func> ) F

A global function declared with `DeclareGlobalFunction' can be given its
value <func> via `InstallGlobalFunction'; 
<gvar> is the global variable (*not* a string) named with the 
<name> argument of the call to `DeclareGlobalFunction'. For example,
a declaration like

\begintt
DeclareGlobalFunction( "SumOfTwoCubes" );
\endtt

in the ``declaration part'' (see Section~"Declaration and Implementation
Part") might have a corresponding ``implementation part'' of:

\begintt
InstallGlobalFunction( SumOfTwoCubes, function(x, y) return x^3 + y^3; end);
\endtt

*Note:* <func> must be a function which has *not* been declared as a
`GlobalFunction' itself. Otherwise completion files
(see~"ref:Completion Files" in the reference manual) get confused!

\>DeclareGlobalVariable( <name>[, <description>] ) F

For global variables that are *not* functions,
instead of using `BindGlobal' one can also declare the variable with
`DeclareGlobalVariable' 
which creates a new global variable named by the string <name>.
If the second argument <description> is entered then this must be
a string that describes the meaning of the global variable.
`DeclareGlobalVariable' shall be used in the declaration part of the
respective package (see~"Declaration and Implementation Part"),
values can then be assigned to the new variable with `InstallValue' or
`InstallFlushableValue', in the implementation part
(again, see~"Declaration and Implementation Part").

\>InstallValue( <gvar>, <value> ) F
\>InstallFlushableValue( <gvar>, <value> ) F

`InstallValue' assigns the value <value> to the global variable <gvar>.
`InstallFlushableValue' does the same but additionally provides that
each call of `FlushCaches' (see~"FlushCaches")
will assign a structural copy of <value> to <gvar>.

`InstallValue' does *not* work if <value> is an ``immediate object''
(i.e., an internally represented small integer or finite field element).
Furthermore, `InstallFlushableValue' works only if <value> is a list.
(Note that `InstallFlushableValue' makes sense only for *mutable*
global variables.)

\>DeclareSynonym( <name>, <value> ) F

assigns the string <name> to a global variable as a synonym for <value>.
Two typical intended usages are to declare an ``and-filter'', e.g.

\begintt
DeclareSynonym( "IsGroup", IsMagmaWithInverses and IsAssociative );
\endtt

and (mainly for compatibility reasons) to provide a previously declared
global function with an alternative name, e.g.

\begintt
DeclareGlobalFunction( "SizeOfSomething" );
DeclareSynonym( "OrderOfSomething", SizeOfSomething );
\endtt

*Note:* Before using `DeclareSynonym' in the way of this second example,
one should determine whether the synonym is really needed. Perhaps an
extra index entry in the documentation would be sufficient. 

When declaring a synonym that is to be an attribute `DeclareSynonymAttr'
should be used.

\>DeclareSynonymAttr( <name>, <value> ) F

assigns the string <name> to an attribute global variable as a synonym for
<value>.  Two typical intended usages are to provide a previously declared
attribute or property with an alternative name, e.g.

\begintt
DeclareAttribute( "GeneratorsOfDivisionRing", IsDivisionRing );
DeclareSynonymAttr( "GeneratorsOfField", GeneratorsOfDivisionRing );
\endtt

and to declare an attribute that is an ``and-filter'', e.g.

\begintt
DeclareSynonymAttr( "IsField", IsDivisionRing and IsCommutative );
\endtt

Also see~"DeclareSynonym". (The comments made there also pertain to
`DeclareSynonymAttr'.)

\>FlushCaches() O

`FlushCaches' resets the value of each global variable that has
been declared with `DeclareGlobalVariable' and for which the initial
value has been set with `InstallFlushableValue' to this initial value.

`FlushCaches' should be used only for debugging purposes,
since the involved global variables include for example lists that store
finite fields and cyclotomic fields used in the current {\GAP} session,
in order to avoid that these fields are constructed anew in each call
to `GF' and `CF' (see~"ref:GaloisField" and~"ref:CyclotomicField"
in the Reference Manual).


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Section{Declaration and Implementation Part}

Each package of {\GAP} code consists of two parts,
the *declaration part* that defines the new categories and operations for
the objects the package deals with,
and the *implementation part* where the corresponding methods are
installed.
The declaration part should be representation independent,
representation dependent information should be dealt with in the
implementation part.

{\GAP} functions that are not operations and that are intended to be
called by users should be notified to {\GAP} in the declaration part via
`DeclareGlobalFunction'.
Values for these functions can be installed in the implementation part
via `InstallGlobalFunction'.

Calls to the following functions belong to the declaration part.

`DeclareAttribute',
`DeclareCategory',
`DeclareFilter',
`DeclareOperation',
`DeclareGlobalFunction',
`DeclareSynonym',
`DeclareSynonymAttr',
`DeclareProperty',
`InstallTrueMethod'.

See~"DeclareAttribute",
"DeclareCategory",
"DeclareFilter",
"DeclareOperation",
"DeclareGlobalFunction",
"DeclareSynonym",
"DeclareSynonymAttr",
"DeclareProperty",
"InstallTrueMethod".

Calls to the following functions belong to the implementation part.

`DeclareRepresentation',
`InstallGlobalFunction',
`InstallMethod',
`InstallImmediateMethod',
`InstallOtherMethod',
`NewFamily',
`NewType',
`Objectify'.

See~"DeclareRepresentation",
"InstallGlobalFunction",
"InstallMethod",
"InstallImmediateMethod",
"InstallOtherMethod",
"NewFamily",
"NewType",
"Objectify".

\indextt{DeclareRepresentation!belongs to implementation part}
Whenever both a `New<Something>' and a `Declare<Something>' variant
of a function exist (see~"Global Variables in the Library"),
the use of `Declare<Something>' is recommended
because this protects the variables in question from being overwritten.
Note that there are *no* functions `DeclareFamily' and `DeclareType'
since families and types are created dynamically,
hence usually no global variables are associated to them.
Further note that `DeclareRepresentation' is regarded as belonging to the
implementation part,
because usually representations of objects are accessed only in very
few places, and all code that involves a particular representation
is contained in one file;
additionally, representations of objects are often not interesting
for the user, so there is no need to provide a user interface
or documentation about representations.

It should be emphasized that ``declaration'' means only an explicit
notification of mathematical or technical terms or of concepts to {\GAP}.
For example, declaring a category or property with name `IsInteresting'
does of course not tell {\GAP} what this shall mean,
and it is necessary to implement possibilities to create objects that
know already that they lie in `IsInteresting' in the case that it is a
category, or to install implications or methods in order to
compute for a given object whether `IsInteresting' is `true' or `false'
for it in the case that `IsInteresting' is a property.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%
%E

