clojure-demos/resources/public/documents/datalog-demo.org

Datalog DSL Guide

• Intro to datalog
  • Entities Attributes & values
  • Creating a DATABASE
    • Creating a Schema
  • Transacting data
    • Transacting related data
  • Querying the databases
    • Looking up an entity
    • Pull a tree of data
    • Querying your dataset

Intro to datalog

This is a short guide on writing common datalog queries to allow you to query databases like datascript, datalevin, datahike datomic & xtdb.

These examples are all aimed at datascript which is a browser client side database allowing interactivity int the examples.

You can find some further guides and information on the subject at these locations. https://max-datom.com/ http://www.learndatalogtoday.org/ https://www.youtube.com/watch?v=oo-7mN9WXTw

Blog of the dev who makes datascript https://tonsky.me/blog/the-web-after-tomorrow/

Entities Attributes & values

Before getting started with data log it vital to understand data is stored as a list of values, the entity id which is a way to look up and group related attributes, attributes are the name of the thing you want to store so :user/name for example and the value to store under this name, the entity id will be generated when inserting data.

Sometimes the database may store other details like transaction time and revoked data dependant on the characteristics of the underlying database.

This is an example of how a EAV triplet would be represented. #+BEGIN EXAMPLE [1 | :user/name | "Daisy"]

#+END_EXAMPLE

This is an example of how a EAVT quadruplet would be represented, T in this instance is the Transaction ID if multiple values are inserted at once they would have the same transaction ID. #+BEGIN EXAMPLE [1 | :user/name | "Daisy" | 100]

#+END_EXAMPLE

Creating a DATABASE

Datalog databases can be schema less but a lot of the power comes from creating a schema specifying uniqueness and relations on the stored fields.

You can create a new database using create-conn as below then empty hash map is simply a blank schema,

(def demo-conn (d/create-conn {}))

@demo-conn

Using the connection we can just start inserting data, using standard hash maps and lists structures, we always specify the attribute and the value when transacting.

(def demo-conn (d/create-conn {}))

(d/transact!
 demo-conn
 [{:user/name "Brooke" :user/img "me.jpg"}
  {:user/name "Kalvin" :user/img "you.jpg"}])

@demo-conn

Creating a Schema

Not all datalog databases let you create a schema, but datascript does this allows us to add constraints and relations between the stored data.

In this example we are saying :user/name is unique and :user/rooms has a many to one relationship, when we later transact data it will use the constraints to stop things like duplicates from being created.

  (def schema   {:user/name {:db/unique :db.unique/identity}
                 :user/rooms {:db/cardinality :db.cardinality/many
                              :db/valueType :db.type/ref}

                 ;; needs to be set so we can aggregate into the find query
                 :diagram/objects {:db/cardinality :db.cardinality/many
                                   :db/valueType :db.type/ref}
                 })
  (def demo-conn (d/create-conn schema))

Transacting data

Transacting this data would mean Brooke would be inserted once but the image will be updated to you.jpg

(str (d/transact! demo-conn [{:user/name "Brooke" :user/img "you.jpg"}]))

Transacting related data

We can insert bulk data and include relationship information by specifying negative id's in the transaction maps. The example below adds circle and square to :diagram/objects the negatives being replaced by the real entity ids.

(d/transact! demo-conn [{:db/id -1 :object/name "circle"}
                             {:db/id -2 :object/name "square"}
                             {:db/id -3 :object/name "rectangle"}
                             {:diagram/objects [-1 -2]}])
(str @demo-conn)

Querying the databases

There are three types of queries in datalog entity lookup's pulling a tree of data or querying with d/q.

Looking up an entity

d/entity is used to find the entity id, using any unique piece of data for example the user Brooke exists once so the entity db/id will be returned which can be used for further queries.

(d/entity @demo-conn [:user/name "Brooke"])

Pull a tree of data

Pull is used with entity id's once you know the entity you can specify what data you want to view '[*] being the most common looking up all keys, you can also specify the attributes your interested in looking up including there relations to make a more specific view.

(d/pull @demo-conn '[*] 1)
(d/pull @demo-conn '[:user/name :user/rooms] 1)

Querying your dataset

Querying in datalog is all about binding variables to your entities attributes and values which you can use in you conditions or to return in the result set.

In this example we return the user-id and user/name in the find clause which we looked up in the where clause by finding all attributes :user/name the binding the entity id and username to variables on each match to display in the find clause.

  (d/q '[:find ?user-entity ?user-name  :where
         [?user-entity :user/name ?user-name]] @demo-conn)

(d/q '[:find [(pull ?e [*]) ]
       :where
       [?e :object/name ?objects]]
     @demo-conn)

Fetch a diagram and look up the associated objects

(d/q '[:find [(pull ?diagram-entity [* {:diagram/objects [:object/name]}])]
       :where
       [?diagram-entity :diagram/objects ?objects]]
     @demo-conn)

Write some more as needed better examples in the src code.