In the last post, we discussed a rough architecture for using event logs to track state in fractional source of truth systems, but we didn’t touch on a critical aspect of this architecture: conflict resolution. What happens (going back to the example in the previous post) if you try to group two people together, but one of them is no longer there? Before figuring that out, we need to cover three classes of conflicts, in the context of the scenario from the previous post.

Data

import {LegacyInput} from '../edge_types';

export let payload: LegacyInput = {
  people: [
    {
      id: 1,
      age: 35,
      years_licensed: 18,
      name: "Jimmy Jams",
      ssn: "111-11-1111"
    },
    {
      id: 2,
      age: 34,
      years_licensed: 18,
      name: "Joanie Jams",
      ssn: "222-22-2222"
    },
    {
      id: 3,
      age: 17,
      years_licensed: 1,
      name: "Jimmy Jams II",
      ssn: "333-33-3333"
    }
  ],
  vehicles: [
    {
      id: 1,
      make: "GMC",
      model: "Sonoma",
      year: 2005,
      vin: "12345678901234"
    },
    {
      id: 2,
      make: "Chevy",
      model: "Sonoma",
      year: 2005,
      vin: "43210987654321"
    }
  ]
};

Operations

let events: event[] = [
  ["set", "exclusion", 1, 2],
  ["set", "group", 1, [1, 2]],
  ["set", "group", 2, [3]],
  ["set", "groupLimit", 1, 30000],
  ["set", "groupLimit", 2, 50000],
  ["set", "vehicleGroupAssignment", 1, 1],
  ["set", "vehicleGroupAssignment", 1, 2],
  ["remove", "exclusion", 1, 2]
];

1. Automatically Resolvable Conflicts

Any conflict that your code can resolve without user input is going to be the easiest case to handle. This would be the case in the above example for the "set", "exclusion" event above if person 1 was removed (even though, in this case, you’d likely compact away that set operation - we’ll cover compaction in post 3). Given you know you can throw away that exclusion, you can pop it off the list, and perhaps choose to notify the user that it happened, but there’s no expectation that user interaction would be required.

2. Warnable Conflicts

Some conflicts present a scenario whereby your code can take a stab at what the user wants, but you might want the user to confirm data. A case here could be the same removal of person 1 combined with the "set", "group", "1" event. We may be able to assume that group 1 can continue on with 1 person in it, but we might want to warn the user to confirm that fact, while at the same time rewriting the event in the log.

3. Error conflicts

Finally, there will be some conflicts where you can’t move forward and you need to let the user know that they will need to re-enter information. Policy will tend to define these and they should not be common. One scenario in our example could be the removal of person 1 and person 2 - this will cause the removal of group 1, and both the vehicles are assigned to that group. It might be less confusing to the user in cases like this to clear the log on their behalf (or present them the option to do so), otherwise your system leans further and further out on the limb of assumptions about what the user wants.

What could these look like in practice? The first step is to enable strictNullChecks in our TypeScript config - this will give us an idea of areas we will need to cover:

tsc output screenshot

This output actually surfaces a bug with our prior implementation: we should expect to see at least one error for each case handled, but we don’t have anything for exclusions - we never actually tie those exclusions to the input data! We’ll fix that bug in our implementation, which you can find here.

Some notable changes from our previous version of the implementation:

  • We’ve consolidated the events, combining the command with the event name to cut down on unique combinations (most things can’t be removed!)
  • We return an EventResponse from application of a single event and a HandlerResult from the application of the set instead of mutating with a void returning function; these look like: ```typescript interface EventResponse { output: Output; message?: Message; event: event; }

type HandlerResult = { output: Output; messages: Message[]; events: event[]; };

with `HandlerResult` containing 0..n messages for display to the user, and a new set of events that can be persisted in place of the original set.

## Output

We end up with output like this after running our new version with the following scenarios:

### The original legacy payload
- same output as before
- no messages for the user
- same event log given back

```shell
{"groups":[{"id":1,"vehicles":[{"id":1,"make":"GMC","model":"Sonoma","year":2005,"vin":"12345678901234"},{"id":2,"make":"Chevy","model":"Sonoma","year":2005,"vin":"43210987654321"}],"drivers":[{"id":1,"age":35,"years_licensed":18,"name":"Jimmy Jams","ssn":"111-11-1111"},{"id":2,"age":34,"years_licensed":18,"name":"Joanie Jams","ssn":"222-22-2222"}],"limit":30000},{"id":2,"vehicles":[],"drivers":[{"id":3,"age":17,"years_licensed":1,"name":"Jimmy Jams II","ssn":"333-33-3333"}],"limit":50000}],"exclusions":[]}
[]
[["set","exclusion",1,2],["set","group",1,[1,2]],["set","group",2,[3]],["set","groupLimit",1,30000],["set","groupLimit",2,50000],["set","vehicleGroupAssignment",1,1],["set","vehicleGroupAssignment",1,2],["remove","exclusion",1,2]]

The legacy payload with person 1 removed

  • output adjusted to compensate for one less person
  • three messages for the user: 2 regarding exclusion setting for the removed person, and a warning regarding a group being missing a person
  • a new event log returned with exclusion setting / removing replaced with noop events
{"groups":[{"id":1,"vehicles":[{"id":1,"make":"GMC","model":"Sonoma","year":2005,"vin":"12345678901234"},{"id":2,"make":"Chevy","model":"Sonoma","year":2005,"vin":"43210987654321"}],"drivers":[{"id":2,"age":34,"years_licensed":18,"name":"Joanie Jams","ssn":"222-22-2222"}],"limit":30000},{"id":2,"vehicles":[],"drivers":[{"id":3,"age":17,"years_licensed":1,"name":"Jimmy Jams II","ssn":"333-33-3333"}],"limit":50000}],"exclusions":[]}
[{"level":"NOTIFY","content":"Exclusion Removed"},{"level":"WARN","content":"Some drivers were removed from this group"},{"level":"NOTIFY","content":"Exclusion Not Removed"}]
[["noop","none",0,["set","exclusion",1,2]],["set","group",1,[2]],["set","group",2,[3]],["set","groupLimit",1,30000],["set","groupLimit",2,50000],["set","vehicleGroupAssignment",1,1],["set","vehicleGroupAssignment",1,2],["noop","none",0,["remove","exclusion",1,2]]]

The legacy payload with person 1 & 2 removed

  • Messages truncated down to the error message
  • Very limited output
  • The last event in the new event set will be the error-causing event (as a noop)
{"groups":[{"id":2,"vehicles":[],"drivers":[{"id":3,"age":17,"years_licensed":1,"name":"Jimmy Jams II","ssn":"333-33-3333"}],"limit":50000}],"exclusions":[]}
[{"level":"ERROR","content":"Vehicle no longer assigned, group removed"}]
[["noop",0,["setExclusion",1,2]],["noop",0,["setGroup",1,[1,2]]],["setGroup",2,[3]],["noop",0,["setGroupLimit",1,30000]],["setGroupLimit",2,50000],["noop",0,["setVehicleGroupAssignment",1,1]]]

We now have a set of hooks for applying an event log to (potentially) changing data, but it has come at a pretty obvious complexity cost, but this is ok! Something to keep in mind is that you typically can’t avoid paying this cost, but with this approach the complexity is centralized in the portion of your codebase where you apply the event log, instead of strewn throughout the application. Admittedly, this event log can get a bit unwieldy over time, especially if a long time passes in between flushing it - to combat that, we’ll discuss strategies for compacting the log in the next post.