Query Block: Method to get last run datetime of map

[Manoj Chaurasia]

Isn't there a method to get the last run datetime I do not want to use the "Net Change" function, but would like to use it as parameter in the "Filter".

[Todd Bailey]

Unfortunately there is not a way do this today. The request is not new, and it is already on the backlog to get into a future release. Adding @Dawn Ronaynewho is the PM on Scribe Online so she is also aware.

[Brendan Peterson]

You could take a different approach - if you use "something" as a query to step into your map, you could then use the Scribe API Connector to pull back the LastRunTime of the map you are currently in, and pass that into a Fetch Block as a filter parameter.

 

That should give you what you need with a slightly different pattern to get there.

 

Brendan

[Charles Stitman]

Any move on this

It's kinda critical to our replacement of an local hosted scribe.

[Ash Arendsz]

Hi Charles,

 

Could you elaborate more on your use case

 

- Netchange feature in scribe maps is one of many ways to retrieve deltas for an ETL batch run, other ways does include:

 

 

 

Depending on what system you are reading from and which connector your are using and the fields available to you:

 

- Implementing your own netchange or high water mark by persisting it in a lookup table somewhere in a database or file.

 

- By using Filter on a lastmodified field and using a scribe dateadd -15 m formula to only process the last 15 minutes of transactions for example and run this every 15 mins for example or on top of each hour for the last hour of transactions.

 

If the filter formula is not possible for any reason, you can add an If block in your map and filterout using different criterias and filtering rules in the If block (using AND or OR logic).

 

Other ways include using a webhook style integration to trigger changes(deltas to a scribe REST endpoint in a TCI Scribe or TCI Flogo endpoint) and to be processed one record at a time as the changes/events are triggered.