Read Models

Dave Redfern

Published: 21 Feb 01:22 in Standalone

Introduction

Read-Models

GitHub Actions Build Status

Read-Models are a companion resource to a Doctrine ORM entity based project. They provide an active-record style data access layer, designed for presentational purposes only. This allows your domain objects to remain completely focused on managing your data and not getting sidelined with presentational concerns.

To further highlight this tight integration, read-models uses DBAL and the DBAL type system under-the-hood. Any registered types will be used during model hydration and even embeddables can be reused.

Note that unlike standard active-record packages, there is no write support at all nor will this be added. This package is purely focused on reading and querying data with objects / query builders for use in the presentation layer.

A lot of the internal arrangement is heavily inspired by Laravels Eloquent and other active-record projects including GranadaORM (IdiORM), PHP ActiveRecord and others.

Current Features

  • active-record query model
  • read-only - no ability to change your db through the built-in methods
  • read-only models - no mutation methods, models are immutable once loaded
  • support for attribute casting
  • support for embedded objects via attribute casting
  • support for exporting as JSON / Array data (configurable)
  • relationships (1:1, 1:m, m:m, 1:m reversed)
  • identity map
  • pluggable attribute casting / cast to value-objects

Requirements

  • PHP 8.0+
  • mb_string
  • doctrine/dbal
  • somnambulist/collection

Installation

Install using composer, or checkout / pull the files from github.com.

  • composer require somnambulist/read-models
  • instantiate Manager with your connection mappings and any attribute casters
  • add models extending the Model class, being sure to set the table name property
  • load some data: <model>::find()

For example:

use Doctrine\DBAL\DriverManager;
use Somnambulist\Components\ReadModels\Manager;
use Somnambulist\Components\ReadModels\TypeCasters\DoctrineTypeCaster;

new Manager(
[
    User::class => $conn = DriverManager::getConnection(['url' => 'sqlite://db.sqlite']),
    'default'   => $conn,
],
[
    new DoctrineTypeCaster($conn),
]
);

Usage

Extend Somnambulist\Components\ReadModels\Model and add casts, define relationships, exports etc.

class User extends Model
{

    protected string $table = 'users';
}

You can add a default table alias by setting the property: $tableAlias. Other defaults can be overridden by defining the property:

class User extends Model
{

    protected string $table = 'tbl_users';
    protected ?string $tableAlias = 'u';
    protected string $primaryKey = 'uuid';

}

Note: properties are defined with types and must follow those defined in the base class.

To load a record:

$model = User::find(1);

$results = User::query()->whereColumn('name', 'like', '%bob%')->orderBy('created_at', 'desc')->limit(5)->fetch();

Access properties directly or via method calls:

$model = User::find(1);

$model->id;
$model->id();
$model->created_at;
$model->createdAt();

You cannot set, unset, or change the returned models.

You can define attribute mutators in the same way as Laravels Eloquent:

class User extends Model
{

    protected function getUsernameAttribute($username)
    {
        return Str::capitalize($username);
    }
}

// user:{username: bob was here} -> returns Bob Was Here

User::find(1)->username();

Note: these methods should be protected as they expect the current value to be passed from the loaded model attributes.

Or create virtual properties, that exist at run time:

class User extends Model
{

    protected function getAnniversayDayAttribute()
    {
        return $this->created_at->format('l');
    }
}

// user:{created_at: '2019-07-15 14:23:21'} -> "Monday"

User::find(1)->anniversay_day;
User::find(1)->anniversayDay();

Or for micro-optimizations, add the method directly:

class User extends Model
{

    public function anniversayDay()
    {
        return $this->created_at->format('l');
    }
}

// user:{created_at: '2019-07-15 14:23:21'} -> "Monday"

User::find(1)->anniversayDay();

Note: to access properties via the magic __get/call the property name must be a valid PHP property/method name. Keys that start with numbers (for example), will not work. Any virtual methods / properties should be documented using @property-read tags on the class level docblock comment. Additionally: virtual methods can be tagged using @method.

Note: to get a raw attribute value, use ->getRawAttribute(). This will return null if the attribute is not found, but could also return null for the specified key.

When returning sets of Model objects, the returned set can be customised per model to allow for specific filters on the collection or other behaviour. Override the collectionClass property with the class name to use. This class must implement the Collection contract from the somnambulist/collection project and must have extract() and add() methods.

Querying

Querying Data

Like with many other ActiveRecord implementation, underlying the Model is a ModelBuilder that wraps the standard Doctrine DBAL QueryBuilder with some convenience methods. The underlying query can be accessed to allow for even more complex queries, however you should consider using straight SQL at that point to fetch the primary ID's and then loading models from those IDs after the fact.

All queries will start with either: with() or query(). The following methods are available:

  • andHaving
  • count returns the count of the query at this point
  • expression returns the DBAL ExpressionBuilder
  • fetch runs the current query, returning results
  • fetchFirstOrFail runs the current query, returning the first result or throwing an exception
  • fetchFirstOrNull runs the current query, returning the first result or null
  • findBy returns a Collection matching the criteria
  • findOneBy returns the first result matching the criteria or null
  • groupBy
  • having
  • innerJoin
  • join
  • leftJoin
  • limit set the maximum results per page
  • offset set the start of the results
  • orderBy
  • orHaving
  • orWhere add an arbitrarily complex OR <expression> clause including multiple values
  • orWhereBetween add an OR <column> BETWEEN <start> AND <end> clause
  • orWhereColumn
  • orWhereIn
  • orWhereNotBetween
  • orWhereNotIn
  • orWhereNotNull
  • orWhereNull
  • rightJoin
  • select select specific columns or add additional properties (see [Select Notes](#Select Notes))
  • where add an arbitrarily complex AND <expression> clause including multiple values
  • whereBetween add an AND <column> BETWEEN <start> AND <end> clause
  • whereColumn
  • whereIn
  • whereNotBetween
  • whereNotIn
  • whereNotNull
  • whereNull
  • wherePrimaryKey specifically search for the primary key with the specified value

These methods can be chained together, the underlying DBAL QueryBuilder called to add more complex expressions.

Note: the query builder does not produce "optimised" SQL. It is dependent on the developer to profile and test the generated SQL queries. Certain types of query (nested sub-selects or dependent WHERE clauses) may not perform very well.

Note: where and orWhere are not for basic column queries. These methods are for custom SQL that must contain named placeholders with an array of values. Use whereColumn for a basic column query. These methods allow you to create complex nested WHERE criteria or use sub-selects etc and add that SQL to your query.

where will accept a callback as the expression. This will be passed the current ModelBuilder as the only argument.

Correlated Sub-Queries

It is possible to pass ModelBuilder instances to the select() method. This will add the builder as a SELECT (query) AS .... When using this form, the second argument is the alias. If not set then sub_select_(n+1) will be used as a placeholder.

Note that when using this, you still need to actively select the columns e.g. select('*') either before or after select($builder), otherwise no fields will be selected when the query is finally run - apart from the sub-select.

select() can accept a callback. It will be provided the current ModelBuilder as the only argument. You can manipulate the query however you like in the callback. Similar to using a builder instance, you still need to select columns as the callback will not trigger any defaults to be added.

findBy and findOneBy

From 1.1.0 findBy and findOneBy have been added to the ModelBuilder. These allow for basic AND WHERE x = y type queries that will return multiple or one result. The methods have the same signature as the Doctrine EntityRepository (except orderBy defaults to an empty array). Use them when you wish to quickly find record(s) with simple criteria:

// return the first 10 matches where user is_active=1 AND email=bob@example.org
// ordered by created_at desc

$results = User::query()->findBy(['is_active' => 1, 'email' => 'bob@example.org'], ['created_at' => 'DESC'], 10);

Select Notes

When changing the selected columns, bear in mind that the identity map will return the same instance and that instance is the first loaded instance. If you only load a couple of attributes then you may have issues later on. Additionally: some logic may require or be dependent on the full model being loaded e.g.: virtual properties.

For relationships, the required keys to setup and query that relationship will be automatically added to any query to ensure it can still function. This may not work in all cases, so ensure that you have sufficient tests for any data fetches.

Model Scopes

From 2.0.0 local scopes can be defined on the Model. A scope provides a convenient way to add re-usable query parts. To add a scope create a method that starts with scope. It will be passed a ModelBuilder instance and then arguments that are needed. For example; on a User model it may be useful to have quick methods for fetching only active users:

<?php

class User extends Model
{

    public function scopeOnlyActive(ModelBuilder $builder)
    {
        $builder->whereColumn('is_active', '=', 1);
    }
}

To use the scope when querying for objects, add it without the scope prefix on the query call:

$users = User::query()->onlyActive()->fetch();

Arguments can be passed in as well and type hinted:

<?php

class User extends Model
{

    public function scopeActiveIs(ModelBuilder $builder, bool $state)
    {
        $builder->whereColumn('is_active', '=', (int)$state);
    }
}

$users = User::query()->activeIs(false)->fetch();

Type Casting

Casting Data

Read models utilises an extended AttributeCaster system to convert scalar types to objects. Several casters are provided including a DoctrineTypeCaster. Using the DoctrineTypeCaster allows directly casting to any known (registered) DBAL type. Add the doctrine type to the casts array for the attribute:

class User extends Model
{

    protected $casts = [
        'created_at' => 'datetime',
        'updated_at' => 'datetime',
        'uuid' => 'uuid',
        'website_url' => 'url',
        'location' => 'resource:geometry',
    ];
}

Certain DBAL types expect a resource to work with (e.g. Creof GeoSpatial Postgres types). Prefix the type with resource: and the string will be converted to a resource and passed through.

Note: types requiring converting to a resource may require explicitly registering by calling either:

  • Manager::instance()->caster()->add($doctrineCaster, ['resource:...']) or,
  • Manager::instance()->caster()->extend('geometry', ['resource:...'])

Embeddables / Casting to Value Objects

Just like Doctrine ORM, you can embed and hydrate value-objects into the read models. These can be the exact same VOs used in the main domain (this is safe, VOs are immutable). Like with Doctrine, these are mapped as types against the attribute name you want the resulting VO to be loaded to:

class UserContact extends Model
{

    protected $casts = [
        'contact' => Contact::class, // or 'contact' if that alias was registered
    ];
}

The Contact class has the following signature:

class Contact
{

    public function __construct($name, ?PhoneNumber $phone, ?EmailAddress $email)
    {
    }
}

To cast this a custom AttributeCaster is needed. All casters must implement the interface to be registered in the Managers master caster system. For the above Contact VO, the caster could be:

use Somnambulist\Components\ReadModels\Contracts\AttributeCaster;

class ContactCaster implements AttributeCaster
{
    public function types(): array
    {
        return ['contact', Contact::class];
    }

    public function supports(string $type): bool
    {
        return in_array($type, $this->types());
    }

    public function cast(array &$attributes, string $attribute, string $type): void
    {
        $attributes['contact'] = new Contact(
            $attributes['name'],
            $attributes['contact_phone'] ? new PhoneNumber($attributes['contact_phone']) : null,
            $attributes['contact_email'] ? new EmailAddress($attributes['contact_email']) : null,
        );

        unset($attributes['name'], $attributes['contact_phone'], $attributes['contact_email']);
    }
}

When the Contact is created, the name, contact_phone and contact_email attributes are used to build the value-object. This is then set to the key contact. As the attributes array is passed by reference, the original attributes can be removed (unset()), though this will mean other casters will not be able to access them.

It is recommended to guard against empty attribute arrays, or where the attributes only contain partial results.

The types() method is used to pre-register the names that the caster will respond to. Any number of casters can be registered, however the type name must be unique. If you require variations use either class names, or prefixed names to distinguish.

The result is something like:

...
  #attributes: array:3 [
    "id" => 1
    "user_id" => 1
    "contact" => Contact^ {#54
      -name: "A Contact"
      -phone: Somnambulist\Domain\Entities\Types\PhoneNumber^ {#138
        -value: "+12345678901"
      }
      -email: Somnambulist\Domain\Entities\Types\Identity\EmailAddress^ {#139
        -value: "a.contact@example.com"
      }
    }
  }
...

Built-in Casters

The following built-in casters are available:

  • AreaCaster - cast to an Area value-object (requires: somnambulist/domain)
  • CoordinateCaster - cast to a Coordinate value-object (requires: somnambulist/domain)
  • DistanceCaster - cast to an Area value-object (requires: somnambulist/domain)
  • DoctrineTypeCaster - cast to any register Doctrine type
  • ExternalIdentityCaster - cast to an ExternalIdentity value-object (requires: somnambulist/domain)
  • MoneyCaster - cast to a Money value-object (requires: somnambulist/domain)

The built-in casters can be registered in a DI container for re-use.

Relationships

Relationships

Define a relationship between models by adding a method named for that relationship. For example: A User has many Roles:

class User extends Models
{

    public function roles()
    {
        return $this->hasMany(Role:class);
    }
}

If you leave the method public, the relationship can be directly accessed for method chaining, however you can make it protected and still access the relationship using: getRelationship(). The benefit of a protected relationship, is fewer exposed methods and not confusing other devs with both a property and method for the same thing.

Using the Relationship

Relationships can operate in two ways:

  • Returning the relationship object that represents the linked data allowing it to be queried,
  • Returning the actual, loaded, objects by executing the default relationship query.

To access the relationship object, call the method that defines it - provided it is public.

For example: to access the roles relationship and modify the query:

User::find(1)->roles()->select(...)...->fetch();

By accessing the relationship object you have full control over the query that will be executed and can limit what is selected, or add conditionals etc. This method will require explicitly calling ->fetch() to actually run the query.

To fetch and load the related models immediately call the relationship name as a property. This will execute the underlying query object and return either a Collection instance or the object. The return type depends on the relationship that was defined. One to one relationships or 1:m inversed, will always return the mapped object. One to many or many to many will always produce a Collection of objects.

For 1:1 and 1:m reversed, they can be configured to optionally return an empty object if there is no result e.g.: A User was disconnected from an Account but this is valid, so to prevent calls to null, the account() relationship could be set to not return null allowing chaining to an empty Account object.

For example: to fetch the first Role of the mapped Roles to a User:

User::find(1)->roles->first();

The call to ->first() returns the first object from the loaded Collection and is being run from the Collection object. Under the hood, the attribute accessor accessed the relationship method, fetching the rules and executing the default query, returning the collection and mapping it into the parent model.

Note: calls into relationships are always eager i.e.: all records will be returned without any limits. Further: if default includes are defined, cascade queries may be issued. It is not recommended to perform this step in loops as it can lead to n+1 queries. To better optimise use eager loading to reduce the queries needed.

Eager Load Relationships

As read-models is based on Eloquent and how that operates, you can eager load models in the same way! Either define the property to always load data, or start with a with call.

$users = User::with('roles.permissions')->whereColumn('name', 'LIKE', '%bob')->limit(3)->fetch();

Will fetch the users, the roles and the permissions for the roles. And just like Eloquent you can also specify specific fields:

$users = User::with('roles:name.permissions:name')->whereColumn('name', 'LIKE', '%bob')->limit(3)->fetch();

Will only load the name of the role and the permission... except! Read-Models will ensure that the keys are loaded so that the models can be attached to the user.

Currently read-models supports:

  • one-to-one (hasOne)
  • one-to-many (hasMany)
  • many-to-many (belongsToMany)
  • one-to-many inverse (belongsTo)

The table and foreign field names can be customised or leave the library to attempt to guess at them.

As Doctrine does not support through type relationships, these are not implemented. An additional note: there is no notion of a pivot table. An un-typed intermediary table like this is a sign your data domain is missing an object participant and this should be implemented. Again, Doctrine does not allow this type of table structure.

Note: the join-table is required for many-to-many relationships. It is not guessed.

Identity Map

Identity Map

Read-Models uses an identity map to ensure that you only ever have one instance, and the same instance of the object. Right now this is a very simple implementation that does not deal with updating the shared instance, so once loaded it is whatever the original data is. A future update may change this.

The identity map is used to resolve and track relationships between models and provides an aliasing system to match foreign key names to a class name.

Because of the identity map, it means you can perform object comparisons without needing to use an equality method; User object with id 1 will be the same as a reloaded User object with id 1.

There is a slight penalty during hydration for the lookups, however this does result in far fewer objects in memory at any one time (for example a User with permissions).

The identity map does need clearing at the end of a request and if running in a long running process be sure to periodically call Manager::instance()->map()->clear().

The Identity Map can be accessed from the Manager.

For Symfony based projects there are 2 kernel event listeners that can be subscribed:

  • IdentityMapClearerSubscriber clears the map on request / error
  • IdentityMapClearerMessengerSubscriber clears the map after handling messages in Messenger

IdentityMap Test Listener

When working with PHPUnit the identity map will preserve loading entities across tests unless cleared. A listener is included that will clear the identity map before and after every test case for you.

Add the following to your phpunit.xml config file to enable the automatic clearing:

<phpunit>
    <!-- other unit config excluded -->
    
    <extensions>
        <extension class="Somnambulist\Components\ReadModels\PHPUnit\PHPUnitListener"/>
    </extensions>
</phpunit>

In addition: if you do not use a dependency injection container in your tests, you may need to setup a bootstrap that will configure the Manager with suitable defaults. Remember: the Manager must be instantiated before it can be accessed statically.

Exporting

Exporting Models

The models can be exported as an array or JSON by calling into the exporter: ->export() or using ->jsonSerialize(). Using export() allows fine-grained control over what is being exported.

Additionally: a default export scheme can be set by overriding the $exports key and setting the attributes and relationships that should be exported by default. The attribute keys can be overridden for example: to not export surrogate keys / to export a UUID as the primary id.

For example:

class User extends Model
{
    protected $exports = [
        'attributes'    => ['uuid' => 'id', 'name', 'date_of_birth',],
        'relationships' => ['addresses', 'contacts'],
    ];
}

This will export the UUID field as id along with the name and date_of_birth. Any addresses and contacts will be exported, using whatever rules are defined on those.

Note: the export rules can be overridden, but it is on a per object basis. If you wish to apply the same rules to a collection of objects, use the collection methods to foreach over the models and apply the rules to all items.

The exporter will only touch attributes and relationships; no other properties. For attributes it will convert objects to arrays / strings if possible, however it currently cannot access private inherited properties.

Export Relationship Attributes

Just like with eager loading; specific attributes can be exported via the relationships, and just like eager loading the syntax is the same. Add a colon and then comma separate the attributes you wish to export from that relationship. For collections, these will be passed to all Models in that collection.

For example:

User::find(1)->export()->with('roles:name.permissions:name')->toArray();

This will export the User with all roles but only the name and all permissions per role but only the permission name.

These rules can be defined on the default exports too.

Profiling / Testing

Profiling

If you use Symfony; using the standard Doctrine DBAL connection from your entity manager will automatically ensure that ALL SQL queries are added to the profiler without having to do anything else! You get full insight into the query that was executed, the data bound etc. For further insights consider using an application profiler such as:

For other frameworks; as DBAL is used, hook into the Configuration object and add an SQL logger instance that can report to your frameworks profiler.

Test Suite

The test suite uses an SQlite database file named "users.db" that simulates a possible User setup with Roles, Permissions, Contacts and Addresses. Before running the test suite, be sure to generate some test data using: tests/resources/seed.php. This console app has a couple of commands:

  • db:create - builds the table structure
  • db:seed - generate base records and --records=XX random records
  • db:destroy - deletes all test data and tables

For the test suite to run and be able to test various relationships / eager loading etc a reasonable number of test records are needed. The suite was built against a random sample of 150 records.

The DataGenerator attempts some amount of random allocation of addresses, contacts and roles to each user; however data integrity was not the goal, merely usable data.

To run the tests: vendor/bin/phpunit.

Return to article