Typeless systems, such as NoSQL Databases, JSON, JavaScript and Python, have gained popularity and are very useful in practice because the world’s data does not conform nicely to a consistent, rigid structure.

Even if it did, it is almost impossible to capture all facets of the data and predict future use of this data.

However, the data is also not completely schemaless, and any application using the data will enforce a well-defined schema.

NoSQL opponents complain that this popularity with NoSQL reverts application development back to the hierarchical database days of the ’60s, ignoring the reality that the data in these typeless systems do, in fact, have consistent structure and that their schema is well-designed and evolves orderly with application development processes.

With the advent of GraphQL, we can introduce a type system over these systems without taking away from the flexibility they introduced, bringing order back into what appears to be disorder.

Let’s walk through this using one of our favorite backend databases, MongoDB.

Suppose we have a GraphQL schema for data that represents a customer:

type Customer {
   id: ID!
   name: String!
   email: String
   age: Int
   joined: Date!
}

The above says that a customer has five attributes and each must be of type as shown. The presence of “!” in GraphQL indicates the field is non-nullable, akin to NOT NULL in SQL.

Now assume that your MongoDB collection contains the following three JSON-style documents:

   [
      {
         “id”: “12345”,
         “name”: “John Doe”,
         “email”: “john.doe@example.com”,
         “age”: 20,
         “joined”: “2021-01-10”,
         “children”: [“John”, “Mary”]
      },
      {
         “id”: “23456”,
         “name”: “Jane Doe”,
         “email”: “jane.doe@example.com”,
         “age”: 30,
         “joined”: “2020-01-10”,
         “hobbies”: [“surfing”, “reading”]
      },
      {
         “id”: “34567”,
         “name”: “John Smith”,
         “email”: “john.smith@example.com”,
         “age”: 40.1
         “offices”: [“StepZen”]
      }
      ]

As you can see, the data has a fairly consistent structure with a few exceptions. The obvious ones are that each document has differentiating characteristics. John Doe has children, Jane Doe has hobbies, and John Smith has offices. The “joined” property appears to be common, but on closer inspection, we see that one of the documents is missing this field. And finally, we notice that if we just considered Mr. and Mrs. Doe, we might think age could be represented as an integer, but Mr. Smith’s age is clearly not an integer.

How would this data fit our GraphQL model? There are obvious issues:

1. The field joined cannot be set to be ! since in the last document, it is missing. Easy fix, right? Convert it into joined “Date” as opposed to joined “Date!” But what if, out of the million documents, this is the only record that does not seem to have any value for joined? With this increased perspective, removing the ! now seems like a bad idea as this anomaly could represent bad data.

2. The documents seem to have different extra fields. This is a pretty common pattern even when data is in strictly typed systems like relational database management systems, which is why BLOBS (Binary Large Objects), XML and JSON were introduced. With GraphQL, we could do something like:

      type Customer {
      ...
         children: [String!]
         hobbies: [String!]
         office: [String!]
      }

ensuring that all three records can be mapped. Note that the fields children, hobbies and office do not have ! after them, indicating they are optional. (The String! descriptor in the list means that an item in the list cannot be null). However, this seems suboptimal too; after all, we may get hundreds of such fields over millions of records and that leads to excessive bloat and application logic. We’ll show you a better way below.

3. Our GraphQL schema assumes type INT for age, which works for Mr. and Mrs. Doe but is not compatible with Mr. Smith’s age, which would be better handled with type FLOAT, unless it is typecast to an INT.

These are just a few simple examples that illustrate these marriage issues between strongly typed (GraphQL) and weakly typed (in this case JSON).

However, our team at StepZen has found some excellent solutions using GraphQL for these and other issues.

1. A GraphQL type can be JSON. That means that while the data within that JSON is opaque to GraphQL, any data that is shaped like JSON can be assigned to that field, preserving the type flexibility of the underlying typeless system. For example, if there was a field:

      type Customer {
      ...
         extras: JSON
      }

then all records could be passed through the GraphQL layer.

2. A GraphQL type can be a UNION type. So we could declare:

      union Extra =Hobbies | Children | Offices
      type Customer {
      ...
         extras: [Extra!]
      }

and this keeps the representation compact. Furthermore, the records are still strongly typed, except the field extras, which can be one or more of the three types. Additionally, we can use a fragment with a type condition to capture design specific to the Extra type:

   {
      customers (id: “12345”) {
         name
         extras: {
            __typeName
            … on hobbies {
               level
            }
            … on children {
               name
            }
            … on offices {
               location
            }
         }
      }
   }

In the above, the GraphQL query is explicitly saying, “If an element of extras is of type hobbies, then return the level, etc.” This allows the GraphQL API to do the right thing to return the data in the right shape, removing the burden from the application.

In all of the above, the developer of the GraphQL API is making some explicit choices about what types she wants that approximately match the types of the backends. What if we had the reverse problem? What if we were to inspect the backend and automatically derive the GraphQL types from the underlying data? For such a system:

1. Sample enough records to see the diversity. MongoDB has a find where you can retrieve multiple records. REST APIs might have pagination models. SQL systems have limits and offsets. Either way, a dozen or so records is a good sample.

2. Make some sensible choices. As mentioned above, the data in these systems are not completely schemaless; they contain some light schema. Find the largest common sub-structures and declare the uncommon ones as UNION or JSON.

3. Pick the right types and when in doubt, drop a !

If you check out JSON2SDL, you will find one such implementation built by the company we work for, StepZen. In fact, we have built a set of introspectors that make these kinds of sensible choices, including going against a wide variety of REST, SQL and NoSQL backends. And we are currently releasing to the public our REST2GraphQL introspector.

The problem of type mismatches is an important issue to address for GraphQL to ultimately realize its potential. However, as a language, it has many features that make the mismatch problem less severe. And tools such as StepZen’s JSON2SDL make the job more and more automatic. The paradigm mismatch between a NoSQL database such as MongoDB and GraphQL can be easily bridged. And that is true for other backends like REST and SQL too. All signs point to GraphQL becoming the default, strongly typed API layer for accessing all sorts of backends.

---

This article was originally published in The New Stack: