Cannot successfully implement color picker on visual using table mapping

I am using the table type dataViewMapping.

I'm trying to store and retrieve colors, however colors don't save successfully.

I'm creating my selection Ids using the code from #77:

private static getSelectionIds(dataView: DataView, host: IVisualHost): powerbi.visuals.ISelectionId[] {
            return dataView.table.identity.map((identity: DataViewScopeIdentity) => {
                const categoryColumn: DataViewCategoryColumn = {
                    source: dataView.table.columns[0],
                    values: null,
                    identity: [identity]
                };
                return host.createSelectionIdBuilder()
                    .withCategory(categoryColumn, 0)
                    .createSelectionId();
            });

Then I populate the default colors:

                this.GANTTEntries = options.dataViews[0].table.rows.map((v, i, a) => 	
                                    [
                                        v[columnNames['stage']],												//Stage name
                                        a.map(i => i[columnNames['stage']]).indexOf(v[columnNames['stage']]),	//Stage surrogate index
                                    ])
                        .filter((v, i) => v[1] === i)														     //Only keep stages groups (based on index)
                        .map((v, i) =>  {
                                            let defaultColor: Fill = {
                                                solid: {
                                                    color: this.host.colorPalette.getColor(options.dataViews[0].table.rows[i][2].toString()).value
                                                }
                                            }
                                            return {
                                                        id: i,
                                                        content: v[0],
                                                        color: getCategoricalObjectValue<Fill>(options.dataViews[0].categorical.categories[0], i, 'colorSelector', 'fill', defaultColor).solid.color,
                                                        selectionId:    selectionIds[i]
                                                    }
                                        });

Finally I implement my enumerate method:

       public enumerateObjectInstances(options: EnumerateVisualObjectInstancesOptions): VisualObjectInstance[] | VisualObjectInstanceEnumerationObject {

            let objectName = options.objectName;
            let objectEnumeration: VisualObjectInstance[] = [];

            switch (objectName) {

                case 'colorSelector':

                    for (let GANTTElement of this.GANTTEntries) {

                        //debugger;

                        objectEnumeration.push({
                            objectName: objectName,
                            displayName: GANTTElement.content.toString(),
                            properties: {
                                fill: {
                                    solid: {
                                        color: GANTTElement.color
                                    }
                                }
                            },
                            selector: GANTTElement.selectionId.getSelector()
                        });
                    }
                    
                break;
            }

            return(objectEnumeration);
        }

Color selections immediately revert to the default. The immediate reason is that there is no stored value (objects), but I am not sure how this all plumbs up and there are no examples that I can find other than the post for #77, which does not seem to work. I can't find the object in the dataview anywhere, and getCategoricalObjectValue returns the default value in all cases (as it can't either). This has taken longer than to implement the rest of the visual. Why is this so difficult?