GraphQL: Mapping Error Messages to Generic Error Codes
Leonardo LosovizEditor’s note: We're pleased to host Leo on the StepZen blog for an exploration of generic error codes and the GraphQL Spec. Building on the fact that proposals #698 and #708 have already received broad approval from the GraphQL community, Leo outlines some of the tasks in front of us, and contributes a piece of the work needed, should the community decide to introduce generic error codes in the GraphQL spec. #GraphQL-community
The GraphQL spec defines a series of validations to perform when executing a GraphQL query, and return an error message when some validation fails. However, it does not assign an error code to the validation, or impose GraphQL servers to return the error code alongside the error message.
If generic error codes were supported by the spec, then GraphQL servers could better check if all required validations are being satisfied, since tracking down specific error codes is easier than searching for equivocal error messages (i.e. every GraphQL server is currently returning its own error message).
In addition, validation do not match 1 to 1 with an error. In some cases, a single validation could return 2 or more different error messages. For instance, graphql-js returns these different error messages concerning validation 5.6.1. Values of Correct Type:
- Field "${type.name}.${fieldDef.name}" of required type "${typeStr}" was not provided.
- Field "${node.name.value}" is not defined by type "${parentType.name}".
- Expected value of type "${inspect(type)}", found ${print(node)}.
When GraphQL server vendors browse the GraphQL spec as to learn of all validations to satisfy, they may not be aware of all the nuances within each validation, and they may miss some potential circumstance that makes the validation fail. By introducing generic error codes, each of the nuances should be defined in the schema and given a code all for themselves. Then, in the case above, we could have:
| Generic error code | Error message |
| --- | --- |
| 5.6.1.a | Field "${type.name}.${fieldDef.name}" of required type "${typeStr}" was not provided. |
| 5.6.1.b | Field "${node.name.value}" is not defined by type "${parentType.name}". |
| 5.6.1.c | Expected value of type "${inspect(type)}", found ${print(node)}. |
Generic error codes have been requested for the GraphQL spec, in these two issues:
As proposed in the first of these issues, in addition to adding certainty when handling errors, returning generic error codes could allow a generic test suite to assert that validations are implemented properly across a variety of GraphQL servers.
As a pre-requirement to introduce generic error codes to the GraphQL spec, the exhaustive list of errors that can be produced should be first ascertained. With this objective in mind, I extracted the list of all error messages from graphql-js. Being the reference implementation of a GraphQL server, graphql-js is the best (possibly most complete) source of intended behavior, as it cannot (must not) deviate from what's defined in the spec.
The logic throwing errors in graphql-js sometimes indicates which section from the spec does the error message belong to, sometimes not. So I made it a task to also complete this information, pairing the source code with spec section. This, in turn, can help fix the discrepancies between graphql-js and the GraphQL spec concerning which validations are executed and which ones are missing.
To perform this task, I have searched for all occurrences of "new GraphQLError" in graphql-js, extracted their location in the source code and error message and then, for each, I found out which section from the spec they deal with. The table in the section below contains the results.
In the future, if the community decides to introduce generic error codes to the GraphQL spec, the following tasks should be performed:
- Check if the list of error messages is comprehensive (i.e. do other GraphQL servers perform extra validations, not present in
graphql-js, which should also be defined in the schema?) - Add a new column
codeto the table, and fill it up with a unique code per error message - Transfer the comprehensive list of errors, including their error codes, to the spec
- Define in the GraphQL spec that, whenever a generic error code has been defined (i.e. certainly for all GraphQL spec errors, and possibly for custom validations under the domain of the API), an entry
codewith the corresponding value should be added under the error entry returned in the GraphQL response (alongside themessageentry)
Error mapping: from graphql-js to GraphQL spec
The table below contains 3 colums:
- Message: The error message returned via
new GraphQLErrorin thegraphql-jssource code - Spec section: Which validation from the GraphQL spec it deals with
- Source code: File and line number where the
new GraphQLErrorcode appears
In the Spec section column, it may contain these extra strings:
- "Internal": when it relates to some internal functionality from
graphql-jsto build the schema, which is not documented in the schema, and does not need be documented in the schema - "Missing in spec": validations which aren't found in the GraphQL spec, but should be there. They are all sensible validations, such as 'Cannot extend type "${typeName}" because it is not defined.', which we'd normally expect to be the case and, as such, we may take for granted (but should be defined in the spec nevertheless)
- "Optional": custom validations from
graphql-js
| Message | Spec section | Source code | | --- | --- | --- | | Syntax Error: ${description} | 2. Language | src/error/syntaxError.ts#L14 | | Must provide operation name if query contains multiple operations. | 6.1. Executing Requests | src/execution/execute.ts#L299 | | Unknown operation named "${operationName}". | 6.1. Executing Requests | src/execution/execute.ts#L318 | | Must provide an operation. | 6.1. Executing Requests | src/execution/execute.ts#L320 | | Schema is not configured to execute ${operation.operation} operation. | 6.2. Executing Operations | src/execution/execute.ts#L362 | | Expected Iterable, but did not find one for field "${info.parentType.name}.${info.fieldName}". | Internal - 3.11. List | src/execution/execute.ts#L719 | | Abstract type "${returnType.name}" must resolve to an Object type at runtime for field "${info.parentType.name}.${info.fieldName}". Either the "${returnType.name}" type should provide a "resolveType" function or each possible type should provide an "isTypeOf" function. | Internal - 3.6. Objects | src/execution/execute.ts#L859 | | Support for returning GraphQLObjectType from resolveType was removed in graphql-js@16.0.0 please return type name instead. | Internal - 3.6. Objects | src/execution/execute.ts#L868 | | Abstract type "${returnType.name}" must resolve to an Object type at runtime for field "${info.parentType.name}.${info.fieldName}" with | Internal - 3.6. Objects | src/execution/execute.ts#L874 | | Abstract type "${returnType.name}" was resolved to a type "${runtimeTypeName}" that does not exist inside the schema. | Internal - 3.6. Objects | src/execution/execute.ts#L882 | | Abstract type "${returnType.name}" was resolved to a non-object type "${runtimeTypeName}". | Internal - 3.6. Objects | src/execution/execute.ts#L889 | | Runtime Object type "${runtimeType.name}" is not a possible type for "${returnType.name}". | Internal - 3.6. Objects | src/execution/execute.ts#L896 | | Expected value of type "${returnType.name}" but got: ${inspect(result)}. | Internal | src/execution/execute.ts#L953 | | Schema is not configured to execute subscription operation. | 3.3.1. Root Operation Types | src/execution/subscribe.ts#L195 | | The subscription field "${fieldName}" is not defined. | 5.3.1. Field Selections | src/execution/subscribe.ts#L212 | | Too many errors processing variables, error limit reached. Execution aborted. | Internal | src/execution/values.ts#L54 | | Variable "$${varName}" expected value of type "${varTypeStr}" which cannot be used as an input type. | 5.8.2. Variables Are Input Types | src/execution/values.ts#L87 | | Variable "$${varName}" of required type "${varTypeStr}" was not provided. | 6.1.2. Coercing Variable Values | src/execution/values.ts#L101 | | Variable "$${varName}" of non-null type "${varTypeStr}" must not be null. | 6.1.2. Coercing Variable Values | src/execution/values.ts#L114 | | Variable "$${varName}" got invalid value ${inspect(invalidValue)} | 6.1.2. Coercing Variable Values | src/execution/values.ts#L132 | | Argument "${name}" of required type "${inspect(argType)} was not provided." | 6.4.1. Coercing Field Arguments | src/execution/values.ts#L177 | | Argument "${name}" of required type "${inspect(argType)} was provided the variable "$${variableName}" which was not provided a runtime value." | 6.4.1. Coercing Field Arguments | src/execution/values.ts#L198 | | Argument "${name}" of non-null type "${inspect(argType)} must not be null." | 6.4.1. Coercing Field Arguments | src/execution/values.ts#L210 | | Argument "${name}" has invalid value ${print(valueNode)}. | 6.4.1. Coercing Field Arguments | src/execution/values.ts#L223 | | Expected name to be a non-empty string. | B.3. Lexical Tokens | src/type/assertName.ts#L15 | | Names must only contain [_a-zA-Z0-9] but "${name}" does not. | B.3. Lexical Tokens | src/type/assertName.ts#L20 | | Names must start with [_a-zA-Z] but "${name}" does not. | B.3. Lexical Tokens | src/type/assertName.ts#L26 | | Enum values cannot be named: ${name} | B.4. Document Syntax | src/type/assertName.ts#L42 | | Enum "${this.name}" cannot represent value: ${inspect(outputValue)} | 3.9. Enums | src/type/definition.ts#L1408 | | Enum "${this.name}" cannot represent non-string value: ${valueStr}. | 3.9. Enums | src/type/definition.ts#L1418 | | Value "${inputValue}" does not exist in "${this.name}" enum. | 3.9. Enums | src/type/definition.ts#L1426 | | Enum "${this.name}" cannot represent non-enum value: ${valueStr}. | 3.9. Enums | src/type/definition.ts#L1441 | | Value "${valueStr}" does not exist in "${this.name}" enum. | 3.9. Enums | src/type/definition.ts#L1451 | | Int cannot represent non-integer value: ${inspect(coercedValue)} | 3.5.1. Int | src/type/scalars.ts#L42 | | Int cannot represent non 32-bit signed integer value: | 3.5.1. Int | src/type/scalars.ts#L47 | | Int cannot represent non-integer value: ${inspect(inputValue)} | 3.5.1. Int | src/type/scalars.ts#L57 | | Int cannot represent non 32-bit signed integer value: ${inputValue} | 3.5.1. Int | src/type/scalars.ts#L62 | | Int cannot represent non-integer value: ${print(valueNode)} | 3.5.1. Int | src/type/scalars.ts#L71 | | Int cannot represent non 32-bit signed integer value: ${valueNode.value} | 3.5.1. Int | src/type/scalars.ts#L78 | | Float cannot represent non numeric value: ${inspect(coercedValue)} | 3.5.2. Float | src/type/scalars.ts#L105 | | Float cannot represent non numeric value: ${inspect(inputValue)} | 3.5.2. Float | src/type/scalars.ts#L114 | | Float cannot represent non numeric value: ${print(valueNode)} | 3.5.2. Float | src/type/scalars.ts#L123 | | String cannot represent value: ${inspect(outputValue)} | 3.5.3. String | src/type/scalars.ts#L151 | | String cannot represent a non string value: ${inspect(inputValue)} | 3.5.3. String | src/type/scalars.ts#L158 | | String cannot represent a non string value: ${print(valueNode)} | 3.5.3. String | src/type/scalars.ts#L167 | | Boolean cannot represent a non boolean value: ${inspect(coercedValue)} | 3.5.4. Boolean | src/type/scalars.ts#L189 | | Boolean cannot represent a non boolean value: ${inspect(inputValue)} | 3.5.4. Boolean | src/type/scalars.ts#L196 | | Boolean cannot represent a non boolean value: ${print(valueNode)} | 3.5.4. Boolean | src/type/scalars.ts#L205 | | ID cannot represent value: ${inspect(outputValue)} | 3.5.5. ID | src/type/scalars.ts#L228 | | ID cannot represent value: ${inspect(inputValue)} | 3.5.5. ID | src/type/scalars.ts#L240 | | ID cannot represent a non-string and non-integer value: print(valueNode) | 3.5.5. ID | src/type/scalars.ts#L245 | | Name "${name}" must not begin with "__", which is reserved by GraphQL introspection. | 4. Introspection => Reserved Names | src/utilities/assertValidName.ts#L28 | | Expected non-nullable type "${inspect(type)}" not to be null. | 3.12. Non-Null => Input Coercion | src/utilities/coerceInputValue.ts#L64 | | Expected type "${type.name}" to be an object.`) | 3.10. Input Object => Input Coercion | src/utilities/coerceInputValue.ts#L93 | | Field "${field.name}" of required type "${typeStr}" was not provided. | 3.10. Input Object => Input Coercion | src/utilities/coerceInputValue.ts#L112 | | Field "${fieldName}" is not defined by type "${type.name}". | 3.10. Input Object => Input Coercion | src/utilities/coerceInputValue.ts#L138 | | Expected type "${type.name}". ${error.message} | 3.5. Scalars => Input Coercion | 3.9. Enums => Input Coercion | src/utilities/coerceInputValue.ts#L163 | | Expected type "${type.name}". | 3.5. Scalars => Input Coercion | 3.9. Enums => Input Coercion | src/utilities/coerceInputValue.ts#L179 | | Schema does not define the required query root type. | 3.3.1. Root Operation Types | src/utilities/getOperationRootType.ts#L23 | | Schema is not configured for mutations. | 3.3.1. Root Operation Types | src/utilities/getOperationRootType.ts#L34 | | Schema is not configured for subscriptions. | 3.3.1. Root Operation Types | src/utilities/getOperationRootType.ts#L45 | | Can only have query, mutation and subscription operations. | 3.3.1. Root Operation Types | src/utilities/getOperationRootType.ts#L53 | | Too many validation errors, error limit reached. Validation aborted. | Internal | src/validation/validate.ts#L62 | | The ${defName} definition is not executable. | 5.1.1. Executable Definitions | src/validation/rules/ExecutableDefinitionsRule.ts#L30 | | Cannot query field "${fieldName}" on type "${type.name}". | 5.3.1. Field Selections | src/validation/rules/FieldsOnCorrectTypeRule.ts#L58 | | Fragment cannot condition on non composite type "${typeStr}". | 5.5.1.3. Fragments On Composite Types | src/validation/rules/FragmentsOnCompositeTypesRule.ts#L32 | | Fragment "${node.name.value}" cannot condition on non composite type "${typeStr}". | 5.5.1.3. Fragments On Composite Types | src/validation/rules/FragmentsOnCompositeTypesRule.ts#L45 | | Unknown argument "${argName}" on field "${parentType.name}.${fieldDef.name}". | 5.4.1. Argument Names | src/validation/rules/KnownArgumentNamesRule.ts#L39 | | Unknown argument "${argName}" on directive "@${directiveName}". | 5.4.1. Argument Names | src/validation/rules/KnownArgumentNamesRule.ts#L87 | | Unknown directive "@${name}". | 5.7.1. Directives Are Defined | src/validation/rules/KnownDirectivesRule.ts#L54 | | Directive "@${name}" may not be used on ${candidateLocation}. | 5.7.2. Directives Are In Valid Locations | src/validation/rules/KnownDirectivesRule.ts#L62 | | Unknown fragment "${fragmentName}". | 5.5.1.2. Fragment Spread Type Existence | src/validation/rules/KnownFragmentNamesRule.ts#L22 | | Unknown type "${typeName}". | 5.5.1.2. Fragment Spread Type Existence | src/validation/rules/KnownTypeNamesRule.ts#L63 | | This anonymous operation must be the only defined operation. | 5.2.2.1. Lone Anonymous Operation | src/validation/rules/LoneAnonymousOperationRule.ts#L29 | | Cannot define a new schema within a schema extension. | Missing in spec - 3.3. Schema | src/validation/rules/LoneSchemaDefinitionRule.ts#L27 | | Must provide only one schema definition. | Missing in spec - 3.3. Schema | src/validation/rules/LoneSchemaDefinitionRule.ts#L37 | | Cannot spread fragment "${spreadName}" within itself | 5.5.2.2. Fragment spreads must not form cycles | src/validation/rules/NoFragmentCyclesRule.ts#L78 | | Variable "$${varName}" is not defined by operation "${operation.name.value}". | 5.8.3. All Variable Uses Defined | src/validation/rules/NoUndefinedVariablesRule.ts#L32 | | Fragment "${fragName}" is never used. | 5.5.1.4. Fragments Must Be Used | src/validation/rules/NoUnusedFragmentsRule.ts#L49 | | Variable "$${variableName}" is never used in operation "${operation.name.value}". | 5.8.4. All Variables Used | src/validation/rules/NoUnusedVariablesRule.ts#L36 | | Fields "${responseName}" conflict because ${reasonMsg}. Use different aliases on the fields to fetch both if this was intentional. | 5.3.2. Field Selection Merging | src/validation/rules/OverlappingFieldsCanBeMergedRule.ts#L83 | | Fragment cannot be spread here as objects of type "${parentTypeStr}" can never be of type "${fragTypeStr}". | 5.5.2.3. Fragment spread is possible | src/validation/rules/PossibleFragmentSpreadsRule.ts#L38 | | Fragment "${fragName}" cannot be spread here as objects of type "${parentTypeStr}" can never be of type "${fragTypeStr}". | 5.5.2.3. Fragment spread is possible | src/validation/rules/PossibleFragmentSpreadsRule.ts#L57 | | Cannot extend non-${kindStr} type "${typeName}". | Missing in spec - 3.4.3. Type Extensions | src/validation/rules/PossibleTypeExtensionsRule.ts#L68 | | Cannot extend type "${typeName}" because it is not defined. | Missing in spec - 3.4.3. Type Extensions | src/validation/rules/PossibleTypeExtensionsRule.ts#L82 | | Field "${fieldDef.name}" argument "${argDef.name}" of type "${argTypeStr}" is required, but it was not provided. | 5.4.2.1. Required Arguments | src/validation/rules/ProvidedRequiredArgumentsRule.ts#L50 | | Directive "@${directiveName}" argument "${argName}" of type "${argType}" is required, but it was not provided. | 5.4.2.1. Required Arguments | src/validation/rules/ProvidedRequiredArgumentsRule.ts#L112 | | Field "${fieldName}" must not have a selection since type "${typeStr}" has no subfields. | 5.3.3. Leaf Field Selections | src/validation/rules/ScalarLeafsRule.ts#L29 | | Field "${fieldName}" of type "${typeStr}" must have a selection of subfields. Did you mean "${fieldName} { ... }"? | 5.3.3. Leaf Field Selections | src/validation/rules/ScalarLeafsRule.ts#L39 | | Subscription "${operationName}" must not select an introspection top level field. | 5.2.3.1. Single root field | src/validation/rules/SingleFieldSubscriptionsRule.ts#L56 | | Subscription "${operationName}" must not select an introspection top level field. | 5.2.3.1. Single root field | src/validation/rules/SingleFieldSubscriptionsRule.ts#L69 | | Argument "${parentName}(${argName}:)" can only be defined once. | 5.4.2. Argument Uniqueness | src/validation/rules/UniqueArgumentDefinitionNamesRule.ts#L69 | | There can be only one argument named "${argName}". | 5.4.2. Argument Uniqueness | src/validation/rules/UniqueArgumentNamesRule.ts#L38 | | Directive "@${directiveName}" already exists in the schema. It cannot be redefined. | Missing in spec - 3.13. Directives => Validation | src/validation/rules/UniqueDirectiveNamesRule.ts#L24 | | There can be only one directive named "@${directiveName}". | Missing in spec - 3.13. Directives => Validation | src/validation/rules/UniqueDirectiveNamesRule.ts#L34 | | The directive "@${directiveName}" can only be used once at this location. | 5.7.3. Directives Are Unique Per Location | src/validation/rules/UniqueDirectivesPerLocationRule.ts#L79 | | Enum value "${typeName}.${valueName}" already exists in the schema. It cannot also be defined in this type extension. | 3.9. Enums => Type Validation | src/validation/rules/UniqueEnumValueNamesRule.ts#L50 | | Enum value "${typeName}.${valueName}" can only be defined once. | 3.9. Enums => Type Validation | src/validation/rules/UniqueEnumValueNamesRule.ts#L56 | | Field "${typeName}.${fieldName}" already exists in the schema. It cannot also be defined in this type extension. | 3.6. Objects => Type Validation | src/validation/rules/UniqueFieldDefinitionNamesRule.ts#L62 | | Field "${typeName}.${fieldName}" can only be defined once. | 3.6. Objects => Type Validation | src/validation/rules/UniqueFieldDefinitionNamesRule.ts#L69 | | There can be only one fragment named "${fragmentName}". | 5.5.1.1. Fragment Name Uniqueness | src/validation/rules/UniqueFragmentNamesRule.ts#L24 | | There can be only one input field named "${fieldName}". | 5.6.3. Input Object Field Uniqueness | src/validation/rules/UniqueInputFieldNamesRule.ts#L41 | | There can be only one operation named "${operationName.value}". | 5.2.1.1. Operation Name Uniqueness | src/validation/rules/UniqueOperationNamesRule.ts#L24 | | Type for ${operation} already defined in the schema. It cannot be redefined. | Missing in spec - 3.3.1. Root Operation Types | src/validation/rules/UniqueOperationTypesRule.ts#L47 | | There can be only one ${operation} type in schema. | Missing in spec - 3.3.1. Root Operation Types | src/validation/rules/UniqueOperationTypesRule.ts#L54 | | Type "${typeName}" already exists in the schema. It cannot also be defined in this type definition. | Missing in spec - 3.4. Types | src/validation/rules/UniqueTypeNamesRule.ts#L31 | | There can be only one type named "${typeName}". | Missing in spec - 3.4. Types | src/validation/rules/UniqueTypeNamesRule.ts#L41 | | There can be only one variable named "$${variableName}". | 5.8.1. Variable Uniqueness | src/validation/rules/UniqueVariableNamesRule.ts#L31 | | Field "${type.name}.${fieldDef.name}" of required type "${typeStr}" was not provided. | 5.6.1. Values of Correct Type | src/validation/rules/ValuesOfCorrectTypeRule.ts#L58 | | Field "${node.name.value}" is not defined by type "${parentType.name}". | 5.6.1. Values of Correct Type | src/validation/rules/ValuesOfCorrectTypeRule.ts#L75 | | Expected value of type "${inspect(type)}", found ${print(node)}. | 5.6.1. Values of Correct Type | src/validation/rules/ValuesOfCorrectTypeRule.ts#L87 | | Expected value of type "${typeStr}", found ${print(node)}. | 5.6.1. Values of Correct Type | src/validation/rules/ValuesOfCorrectTypeRule.ts#L118 | | Expected value of type "${typeStr}", found ${print(node)}. | 5.6.1. Values of Correct Type | src/validation/rules/ValuesOfCorrectTypeRule.ts#L133 | | Expected value of type "${typeStr}", found ${print(node)}; | 5.6.1. Values of Correct Type | src/validation/rules/ValuesOfCorrectTypeRule.ts#L145 | | Variable "$${variableName}" cannot be non-input type "${typeName}". | 5.8.2. Variables Are Input Types | src/validation/rules/VariablesAreInputTypesRule.ts#L33 | | Variable "$${varName}" of type "${varTypeStr}" used in position expecting type "${typeStr}". | 5.8.5. All Variable Usages are Allowed | src/validation/rules/VariablesInAllowedPositionRule.ts#L63 | | The field ${parentType.name}.${fieldDef.name} is deprecated. ${deprecationReason} | Optional - 4.2. Introspection => Deprecation | src/validation/rules/custom/NoDeprecatedCustomRule.ts#L30 | | Directive "@${directiveDef.name}" argument "${argDef.name}" is deprecated. ${deprecationReason} | Optional - 4.2. Introspection => Deprecation | src/validation/rules/custom/NoDeprecatedCustomRule.ts#L44 | | Field "${parentType.name}.${fieldDef.name}" argument "${argDef.name}" is deprecated. ${deprecationReason} | Optional - 4.2. Introspection => Deprecation | src/validation/rules/custom/NoDeprecatedCustomRule.ts#L54 | | The input field ${inputObjectDef.name}.${inputFieldDef.name} is deprecated. ${deprecationReason} | Optional - 4.2. Introspection => Deprecation | src/validation/rules/custom/NoDeprecatedCustomRule.ts#L69 | | The enum value "${enumTypeDef.name}.${enumValueDef.name}" is deprecated. ${deprecationReason} | Optional - 4.2. Introspection => Deprecation | src/validation/rules/custom/NoDeprecatedCustomRule.ts#L84 | | GraphQL introspection has been disabled, but the requested query contained the field "${node.name.value}". | Optional - 4. Introspection | src/validation/rules/custom/NoSchemaIntrospectionCustomRule.ts#L29 |
Wrapping up
Supporting generic error codes in the GraphQL spec would help enforce the correctness of all required validations by GraphQL servers. The proposals (#698 and #708) have already received broad approval from the community. This article contributes a bit of work to help make them hopefully happen.
