1. Questions & Answers
  2. Something Else
  3. Questions
+1

Query Management

Joanna Russell 4 years ago in Something Else • updated by mark 4 years ago • 5

Hi,

I was wondering how you handle query management and if anybody has built/used a layer over XTRF? I'm specifically interested in cases where query volumes are unmanageable for solutions such as simple gsheets. Have you found any off-the-shelf solutions or built something from scratch? 

Thanks in advance

Joanna

Viktor Mizsei 4 years ago

Somehow, you have to save the data to file. There are several approach, but if you have a lot of data it's not an an easy job. 

Try these:

  1. Xtrf API: create views with the necessary info, and get it through the api
  2. Xtrf API - CSV: same as the first option, but get the result in csv. It's faster a little bit
  3. Create macro, which creates a csv or XML with the necessary columns and run the macro through API. 

If you have the data, you can create dashboards in Excel, Power BI or Google Data Studio 

If you access the database you have more options.

mark 4 years ago

We use option one above. We use the API to connect to google Big Query and access that data through various avenues...

Viktor Mizsei 4 years ago

Mark, did you managed to get big data with API? The API's lack, that you can only get 1000 rows per request, so you have to create a small application to get the whole data, and if a new items appear meantime, you have duplicate rows, or you will miss some. 

That's why the CSV version would be better, but it runs about 10 minutes.  

Dennis Rosenbaum 4 years ago

We like and use the CSV version as well. Though we don't like to do things manually, so we have created a periodic job that sends a single view as CSV every x minutes/hours/days to our inbox. After this a mailreader retrieves the CSV, parses it and puts them where we need it.

Fully automated, so very usefull. Though I don't like to send a CSV every x time as that creates a load on our servers as well as that from XTRF.

mark 4 years ago

@Viktor... I stand corrected. We actually use the second option. We use the API to pull a CSV but we manipulate it automatically and send it to our database rather than a manual step. We process the CSV once a day, so we use it for reporting purposes not for production.

Customer support service by UserEcho