Editor’s note: We're pleased to host Leo on the StepZen blog for an exploration of generic error codes and the GraphQL Spec. Building on the fact that proposals #698 and #708 have already received broad approval from the GraphQL community, Leo outlines some of the tasks in front of us, and contributes a piece of the work needed, should the community decide to introduce generic error codes in the GraphQL spec. #GraphQL-community

The GraphQL spec defines a series of validations to perform when executing a GraphQL query, and return an error message when some validation fails. However, it does not assign an error code to the validation, or impose GraphQL servers to return the error code alongside the error message.

If generic error codes were supported by the spec, then GraphQL servers could better check if all required validations are being satisfied, since tracking down specific error codes is easier than searching for equivocal error messages (i.e. every GraphQL server is currently returning its own error message).

In addition, validation do not match 1 to 1 with an error. In some cases, a single validation could return 2 or more different error messages. For instance, graphql-js returns these different error messages concerning validation 5.6.1. Values of Correct Type:

  • Field "${type.name}.${fieldDef.name}" of required type "${typeStr}" was not provided.
  • Field "${node.name.value}" is not defined by type "${parentType.name}".
  • Expected value of type "${inspect(type)}", found ${print(node)}.

When GraphQL server vendors browse the GraphQL spec as to learn of all validations to satisfy, they may not be aware of all the nuances within each validation, and they may miss some potential circumstance that makes the validation fail. By introducing generic error codes, each of the nuances should be defined in the schema and given a code all for themselves. Then, in the case above, we could have:

Generic error codeError message
5.6.1.aField "${type.name}.${fieldDef.name}" of required type "${typeStr}" was not provided.
5.6.1.bField "${node.name.value}" is not defined by type "${parentType.name}".
5.6.1.cExpected value of type "${inspect(type)}", found ${print(node)}.

Generic error codes have been requested for the GraphQL spec, in these two issues:

As proposed in the first of these issues, in addition to adding certainty when handling errors, returning generic error codes could allow a generic test suite to assert that validations are implemented properly across a variety of GraphQL servers.

As a pre-requirement to introduce generic error codes to the GraphQL spec, the exhaustive list of errors that can be produced should be first ascertained. With this objective in mind, I extracted the list of all error messages from graphql-js. Being the reference implementation of a GraphQL server, graphql-js is the best (possibly most complete) source of intended behavior, as it cannot (must not) deviate from what's defined in the spec.

The logic throwing errors in graphql-js sometimes indicates which section from the spec does the error message belong to, sometimes not. So I made it a task to also complete this information, pairing the source code with spec section. This, in turn, can help fix the discrepancies between graphql-js and the GraphQL spec concerning which validations are executed and which ones are missing.

To perform this task, I have searched for all occurrences of "new GraphQLError" in graphql-js, extracted their location in the source code and error message and then, for each, I found out which section from the spec they deal with. The table in the section below contains the results.

In the future, if the community decides to introduce generic error codes to the GraphQL spec, the following tasks should be performed:

  • Check if the list of error messages is comprehensive (i.e. do other GraphQL servers perform extra validations, not present in graphql-js, which should also be defined in the schema?)
  • Add a new column code to the table, and fill it up with a unique code per error message
  • Transfer the comprehensive list of errors, including their error codes, to the spec
  • Define in the GraphQL spec that, whenever a generic error code has been defined (i.e. certainly for all GraphQL spec errors, and possibly for custom validations under the domain of the API), an entry code with the corresponding value should be added under the error entry returned in the GraphQL response (alongside the message entry)

Error mapping: from graphql-js to GraphQL spec

The table below contains 3 colums:

  • Message: The error message returned via new GraphQLError in the graphql-js source code
  • Spec section: Which validation from the GraphQL spec it deals with
  • Source code: File and line number where the new GraphQLError code appears

In the Spec section column, it may contain these extra strings:

  • "Internal": when it relates to some internal functionality from graphql-js to build the schema, which is not documented in the schema, and does not need be documented in the schema
  • "Missing in spec": validations which aren't found in the GraphQL spec, but should be there. They are all sensible validations, such as 'Cannot extend type "${typeName}" because it is not defined.', which we'd normally expect to be the case and, as such, we may take for granted (but should be defined in the spec nevertheless)
  • "Optional": custom validations from graphql-js
MessageSpec sectionSource code
Syntax Error: ${description}2. Languagesrc/error/syntaxError.ts#L14
Must provide operation name if query contains multiple operations.6.1. Executing Requestssrc/execution/execute.ts#L299
Unknown operation named "${operationName}".6.1. Executing Requestssrc/execution/execute.ts#L318
Must provide an operation.6.1. Executing Requestssrc/execution/execute.ts#L320
Schema is not configured to execute ${operation.operation} operation.6.2. Executing Operationssrc/execution/execute.ts#L362
Expected Iterable, but did not find one for field "${info.parentType.name}.${info.fieldName}".Internal - 3.11. Listsrc/execution/execute.ts#L719
Abstract type "${returnType.name}" must resolve to an Object type at runtime for field "${info.parentType.name}.${info.fieldName}". Either the "${returnType.name}" type should provide a "resolveType" function or each possible type should provide an "isTypeOf" function.Internal - 3.6. Objectssrc/execution/execute.ts#L859
Support for returning GraphQLObjectType from resolveType was removed in graphql-js@16.0.0 please return type name instead.Internal - 3.6. Objectssrc/execution/execute.ts#L868
Abstract type "${returnType.name}" must resolve to an Object type at runtime for field "${info.parentType.name}.${info.fieldName}" withInternal - 3.6. Objectssrc/execution/execute.ts#L874
Abstract type "${returnType.name}" was resolved to a type "${runtimeTypeName}" that does not exist inside the schema.Internal - 3.6. Objectssrc/execution/execute.ts#L882
Abstract type "${returnType.name}" was resolved to a non-object type "${runtimeTypeName}".Internal - 3.6. Objectssrc/execution/execute.ts#L889
Runtime Object type "${runtimeType.name}" is not a possible type for "${returnType.name}".Internal - 3.6. Objectssrc/execution/execute.ts#L896
Expected value of type "${returnType.name}" but got: ${inspect(result)}.Internalsrc/execution/execute.ts#L953
Schema is not configured to execute subscription operation.3.3.1. Root Operation Typessrc/execution/subscribe.ts#L195
The subscription field "${fieldName}" is not defined.5.3.1. Field Selectionssrc/execution/subscribe.ts#L212
Too many errors processing variables, error limit reached. Execution aborted.Internalsrc/execution/values.ts#L54
Variable "$${varName}" expected value of type "${varTypeStr}" which cannot be used as an input type.5.8.2. Variables Are Input Typessrc/execution/values.ts#L87
Variable "$${varName}" of required type "${varTypeStr}" was not provided.6.1.2. Coercing Variable Valuessrc/execution/values.ts#L101
Variable "$${varName}" of non-null type "${varTypeStr}" must not be null.6.1.2. Coercing Variable Valuessrc/execution/values.ts#L114
Variable "$${varName}" got invalid value ${inspect(invalidValue)}6.1.2. Coercing Variable Valuessrc/execution/values.ts#L132
Argument "${name}" of required type "${inspect(argType)} was not provided."6.4.1. Coercing Field Argumentssrc/execution/values.ts#L177
Argument "${name}" of required type "${inspect(argType)} was provided the variable "$${variableName}" which was not provided a runtime value."6.4.1. Coercing Field Argumentssrc/execution/values.ts#L198
Argument "${name}" of non-null type "${inspect(argType)} must not be null."6.4.1. Coercing Field Argumentssrc/execution/values.ts#L210
Argument "${name}" has invalid value ${print(valueNode)}.6.4.1. Coercing Field Argumentssrc/execution/values.ts#L223
Expected name to be a non-empty string.B.3. Lexical Tokenssrc/type/assertName.ts#L15
Names must only contain _a-zA-Z0-9 but "${name}" does not.B.3. Lexical Tokenssrc/type/assertName.ts#L20
Names must start with _a-zA-Z but "${name}" does not.B.3. Lexical Tokenssrc/type/assertName.ts#L26
Enum values cannot be named: ${name}B.4. Document Syntaxsrc/type/assertName.ts#L42
Enum "${this.name}" cannot represent value: ${inspect(outputValue)}3.9. Enumssrc/type/definition.ts#L1408
Enum "${this.name}" cannot represent non-string value: ${valueStr}.3.9. Enumssrc/type/definition.ts#L1418
Value "${inputValue}" does not exist in "${this.name}" enum.3.9. Enumssrc/type/definition.ts#L1426
Enum "${this.name}" cannot represent non-enum value: ${valueStr}.3.9. Enumssrc/type/definition.ts#L1441
Value "${valueStr}" does not exist in "${this.name}" enum.3.9. Enumssrc/type/definition.ts#L1451
Int cannot represent non-integer value: ${inspect(coercedValue)}3.5.1. Intsrc/type/scalars.ts#L42
Int cannot represent non 32-bit signed integer value:3.5.1. Intsrc/type/scalars.ts#L47
Int cannot represent non-integer value: ${inspect(inputValue)}3.5.1. Intsrc/type/scalars.ts#L57
Int cannot represent non 32-bit signed integer value: ${inputValue}3.5.1. Intsrc/type/scalars.ts#L62
Int cannot represent non-integer value: ${print(valueNode)}3.5.1. Intsrc/type/scalars.ts#L71
Int cannot represent non 32-bit signed integer value: ${valueNode.value}3.5.1. Intsrc/type/scalars.ts#L78
Float cannot represent non numeric value: ${inspect(coercedValue)}3.5.2. Floatsrc/type/scalars.ts#L105
Float cannot represent non numeric value: ${inspect(inputValue)}3.5.2. Floatsrc/type/scalars.ts#L114
Float cannot represent non numeric value: ${print(valueNode)}3.5.2. Floatsrc/type/scalars.ts#L123
String cannot represent value: ${inspect(outputValue)}3.5.3. Stringsrc/type/scalars.ts#L151
String cannot represent a non string value: ${inspect(inputValue)}3.5.3. Stringsrc/type/scalars.ts#L158
String cannot represent a non string value: ${print(valueNode)}3.5.3. Stringsrc/type/scalars.ts#L167
Boolean cannot represent a non boolean value: ${inspect(coercedValue)}3.5.4. Booleansrc/type/scalars.ts#L189
Boolean cannot represent a non boolean value: ${inspect(inputValue)}3.5.4. Booleansrc/type/scalars.ts#L196
Boolean cannot represent a non boolean value: ${print(valueNode)}3.5.4. Booleansrc/type/scalars.ts#L205
ID cannot represent value: ${inspect(outputValue)}3.5.5. IDsrc/type/scalars.ts#L228
ID cannot represent value: ${inspect(inputValue)}3.5.5. IDsrc/type/scalars.ts#L240
ID cannot represent a non-string and non-integer value: print(valueNode)3.5.5. IDsrc/type/scalars.ts#L245
Name "${name}" must not begin with "__", which is reserved by GraphQL introspection.4. Introspection => Reserved Namessrc/utilities/assertValidName.ts#L28
Expected non-nullable type "${inspect(type)}" not to be null.3.12. Non-Null => Input Coercionsrc/utilities/coerceInputValue.ts#L64
Expected type "${type.name}" to be an object.`) | 3.10. Input Object => Input Coercion | src/utilities/coerceInputValue.ts#L93
Field "${field.name}" of required type "${typeStr}" was not provided.3.10. Input Object => Input Coercionsrc/utilities/coerceInputValue.ts#L112
Field "${fieldName}" is not defined by type "${type.name}".3.10. Input Object => Input Coercionsrc/utilities/coerceInputValue.ts#L138
Expected type "${type.name}". ${error.message}3.5. Scalars => Input Coercion | 3.9. Enums => Input Coercionsrc/utilities/coerceInputValue.ts#L163
Expected type "${type.name}".3.5. Scalars => Input Coercion | 3.9. Enums => Input Coercionsrc/utilities/coerceInputValue.ts#L179
Schema does not define the required query root type.3.3.1. Root Operation Typessrc/utilities/getOperationRootType.ts#L23
Schema is not configured for mutations.3.3.1. Root Operation Typessrc/utilities/getOperationRootType.ts#L34
Schema is not configured for subscriptions.3.3.1. Root Operation Typessrc/utilities/getOperationRootType.ts#L45
Can only have query, mutation and subscription operations.3.3.1. Root Operation Typessrc/utilities/getOperationRootType.ts#L53
Too many validation errors, error limit reached. Validation aborted.Internalsrc/validation/validate.ts#L62
The ${defName} definition is not executable.5.1.1. Executable Definitionssrc/validation/rules/ExecutableDefinitionsRule.ts#L30
Cannot query field "${fieldName}" on type "${type.name}".5.3.1. Field Selectionssrc/validation/rules/FieldsOnCorrectTypeRule.ts#L58
Fragment cannot condition on non composite type "${typeStr}".5.5.1.3. Fragments On Composite Typessrc/validation/rules/FragmentsOnCompositeTypesRule.ts#L32
Fragment "${node.name.value}" cannot condition on non composite type "${typeStr}".5.5.1.3. Fragments On Composite Typessrc/validation/rules/FragmentsOnCompositeTypesRule.ts#L45
Unknown argument "${argName}" on field "${parentType.name}.${fieldDef.name}".5.4.1. Argument Namessrc/validation/rules/KnownArgumentNamesRule.ts#L39
Unknown argument "${argName}" on directive "@${directiveName}".5.4.1. Argument Namessrc/validation/rules/KnownArgumentNamesRule.ts#L87
Unknown directive "@${name}".5.7.1. Directives Are Definedsrc/validation/rules/KnownDirectivesRule.ts#L54
Directive "@${name}" may not be used on ${candidateLocation}.5.7.2. Directives Are In Valid Locationssrc/validation/rules/KnownDirectivesRule.ts#L62
Unknown fragment "${fragmentName}".5.5.1.2. Fragment Spread Type Existencesrc/validation/rules/KnownFragmentNamesRule.ts#L22
Unknown type "${typeName}".5.5.1.2. Fragment Spread Type Existencesrc/validation/rules/KnownTypeNamesRule.ts#L63
This anonymous operation must be the only defined operation.5.2.2.1. Lone Anonymous Operationsrc/validation/rules/LoneAnonymousOperationRule.ts#L29
Cannot define a new schema within a schema extension.Missing in spec - 3.3. Schemasrc/validation/rules/LoneSchemaDefinitionRule.ts#L27
Must provide only one schema definition.Missing in spec - 3.3. Schemasrc/validation/rules/LoneSchemaDefinitionRule.ts#L37
Cannot spread fragment "${spreadName}" within itself5.5.2.2. Fragment spreads must not form cyclessrc/validation/rules/NoFragmentCyclesRule.ts#L78
Variable "$${varName}" is not defined by operation "${operation.name.value}".5.8.3. All Variable Uses Definedsrc/validation/rules/NoUndefinedVariablesRule.ts#L32
Fragment "${fragName}" is never used.5.5.1.4. Fragments Must Be Usedsrc/validation/rules/NoUnusedFragmentsRule.ts#L49
Variable "$${variableName}" is never used in operation "${operation.name.value}".5.8.4. All Variables Usedsrc/validation/rules/NoUnusedVariablesRule.ts#L36
Fields "${responseName}" conflict because ${reasonMsg}. Use different aliases on the fields to fetch both if this was intentional.5.3.2. Field Selection Mergingsrc/validation/rules/OverlappingFieldsCanBeMergedRule.ts#L83
Fragment cannot be spread here as objects of type "${parentTypeStr}" can never be of type "${fragTypeStr}".5.5.2.3. Fragment spread is possiblesrc/validation/rules/PossibleFragmentSpreadsRule.ts#L38
Fragment "${fragName}" cannot be spread here as objects of type "${parentTypeStr}" can never be of type "${fragTypeStr}".5.5.2.3. Fragment spread is possiblesrc/validation/rules/PossibleFragmentSpreadsRule.ts#L57
Cannot extend non-${kindStr} type "${typeName}".Missing in spec - 3.4.3. Type Extensionssrc/validation/rules/PossibleTypeExtensionsRule.ts#L68
Cannot extend type "${typeName}" because it is not defined.Missing in spec - 3.4.3. Type Extensionssrc/validation/rules/PossibleTypeExtensionsRule.ts#L82
Field "${fieldDef.name}" argument "${argDef.name}" of type "${argTypeStr}" is required, but it was not provided.5.4.2.1. Required Argumentssrc/validation/rules/ProvidedRequiredArgumentsRule.ts#L50
Directive "@${directiveName}" argument "${argName}" of type "${argType}" is required, but it was not provided.5.4.2.1. Required Argumentssrc/validation/rules/ProvidedRequiredArgumentsRule.ts#L112
Field "${fieldName}" must not have a selection since type "${typeStr}" has no subfields.5.3.3. Leaf Field Selectionssrc/validation/rules/ScalarLeafsRule.ts#L29
Field "${fieldName}" of type "${typeStr}" must have a selection of subfields. Did you mean "${fieldName} { ... }"?5.3.3. Leaf Field Selectionssrc/validation/rules/ScalarLeafsRule.ts#L39
Subscription "${operationName}" must not select an introspection top level field.5.2.3.1. Single root fieldsrc/validation/rules/SingleFieldSubscriptionsRule.ts#L56
Subscription "${operationName}" must not select an introspection top level field.5.2.3.1. Single root fieldsrc/validation/rules/SingleFieldSubscriptionsRule.ts#L69
Argument "${parentName}(${argName}:)" can only be defined once.5.4.2. Argument Uniquenesssrc/validation/rules/UniqueArgumentDefinitionNamesRule.ts#L69
There can be only one argument named "${argName}".5.4.2. Argument Uniquenesssrc/validation/rules/UniqueArgumentNamesRule.ts#L38
Directive "@${directiveName}" already exists in the schema. It cannot be redefined.Missing in spec - 3.13. Directives => Validationsrc/validation/rules/UniqueDirectiveNamesRule.ts#L24
There can be only one directive named "@${directiveName}".Missing in spec - 3.13. Directives => Validationsrc/validation/rules/UniqueDirectiveNamesRule.ts#L34
The directive "@${directiveName}" can only be used once at this location.5.7.3. Directives Are Unique Per Locationsrc/validation/rules/UniqueDirectivesPerLocationRule.ts#L79
Enum value "${typeName}.${valueName}" already exists in the schema. It cannot also be defined in this type extension.3.9. Enums => Type Validationsrc/validation/rules/UniqueEnumValueNamesRule.ts#L50
Enum value "${typeName}.${valueName}" can only be defined once.3.9. Enums => Type Validationsrc/validation/rules/UniqueEnumValueNamesRule.ts#L56
Field "${typeName}.${fieldName}" already exists in the schema. It cannot also be defined in this type extension.3.6. Objects => Type Validationsrc/validation/rules/UniqueFieldDefinitionNamesRule.ts#L62
Field "${typeName}.${fieldName}" can only be defined once.3.6. Objects => Type Validationsrc/validation/rules/UniqueFieldDefinitionNamesRule.ts#L69
There can be only one fragment named "${fragmentName}".5.5.1.1. Fragment Name Uniquenesssrc/validation/rules/UniqueFragmentNamesRule.ts#L24
There can be only one input field named "${fieldName}".5.6.3. Input Object Field Uniquenesssrc/validation/rules/UniqueInputFieldNamesRule.ts#L41
There can be only one operation named "${operationName.value}".5.2.1.1. Operation Name Uniquenesssrc/validation/rules/UniqueOperationNamesRule.ts#L24
Type for ${operation} already defined in the schema. It cannot be redefined.Missing in spec - 3.3.1. Root Operation Typessrc/validation/rules/UniqueOperationTypesRule.ts#L47
There can be only one ${operation} type in schema.Missing in spec - 3.3.1. Root Operation Typessrc/validation/rules/UniqueOperationTypesRule.ts#L54
Type "${typeName}" already exists in the schema. It cannot also be defined in this type definition.Missing in spec - 3.4. Typessrc/validation/rules/UniqueTypeNamesRule.ts#L31
There can be only one type named "${typeName}".Missing in spec - 3.4. Typessrc/validation/rules/UniqueTypeNamesRule.ts#L41
There can be only one variable named "$${variableName}".5.8.1. Variable Uniquenesssrc/validation/rules/UniqueVariableNamesRule.ts#L31
Field "${type.name}.${fieldDef.name}" of required type "${typeStr}" was not provided.5.6.1. Values of Correct Typesrc/validation/rules/ValuesOfCorrectTypeRule.ts#L58
Field "${node.name.value}" is not defined by type "${parentType.name}".5.6.1. Values of Correct Typesrc/validation/rules/ValuesOfCorrectTypeRule.ts#L75
Expected value of type "${inspect(type)}", found ${print(node)}.5.6.1. Values of Correct Typesrc/validation/rules/ValuesOfCorrectTypeRule.ts#L87
Expected value of type "${typeStr}", found ${print(node)}.5.6.1. Values of Correct Typesrc/validation/rules/ValuesOfCorrectTypeRule.ts#L118
Expected value of type "${typeStr}", found ${print(node)}.5.6.1. Values of Correct Typesrc/validation/rules/ValuesOfCorrectTypeRule.ts#L133
Expected value of type "${typeStr}", found ${print(node)};5.6.1. Values of Correct Typesrc/validation/rules/ValuesOfCorrectTypeRule.ts#L145
Variable "$${variableName}" cannot be non-input type "${typeName}".5.8.2. Variables Are Input Typessrc/validation/rules/VariablesAreInputTypesRule.ts#L33
Variable "$${varName}" of type "${varTypeStr}" used in position expecting type "${typeStr}".5.8.5. All Variable Usages are Allowedsrc/validation/rules/VariablesInAllowedPositionRule.ts#L63
The field ${parentType.name}.${fieldDef.name} is deprecated. ${deprecationReason}Optional - 4.2. Introspection => Deprecationsrc/validation/rules/custom/NoDeprecatedCustomRule.ts#L30
Directive "@${directiveDef.name}" argument "${argDef.name}" is deprecated. ${deprecationReason}Optional - 4.2. Introspection => Deprecationsrc/validation/rules/custom/NoDeprecatedCustomRule.ts#L44
Field "${parentType.name}.${fieldDef.name}" argument "${argDef.name}" is deprecated. ${deprecationReason}Optional - 4.2. Introspection => Deprecationsrc/validation/rules/custom/NoDeprecatedCustomRule.ts#L54
The input field ${inputObjectDef.name}.${inputFieldDef.name} is deprecated. ${deprecationReason}Optional - 4.2. Introspection => Deprecationsrc/validation/rules/custom/NoDeprecatedCustomRule.ts#L69
The enum value "${enumTypeDef.name}.${enumValueDef.name}" is deprecated. ${deprecationReason}Optional - 4.2. Introspection => Deprecationsrc/validation/rules/custom/NoDeprecatedCustomRule.ts#L84
GraphQL introspection has been disabled, but the requested query contained the field "${node.name.value}".Optional - 4. Introspectionsrc/validation/rules/custom/NoSchemaIntrospectionCustomRule.ts#L29

Wrapping up

Supporting generic error codes in the GraphQL spec would help enforce the correctness of all required validations by GraphQL servers. The proposals (#698 and #708) have already received broad approval from the community. This article contributes a bit of work to help make them hopefully happen.