automatic updating of dataset content?

Asked by Adam L

Hi there,

I'm trying to use datasets to display upcoming session times. I have uploaded the session and show times to the dataset from a CSV file and they are displaying just fine - no problems there.

What i'm after is automation of the whole process. The session times change daily, and i'd like the dataset to update with no user intervention. I've got the CSV file automatically updating with SQL server integration services, now i just need Xibo to auto-update the contents of the dataset from this file on some sort of schedule/interval.

Is this possible? Running release 1.6.3.

Many thanks
-Adam

Question information

Language:
English Edit question
Status:
Answered
For:
Xibo Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:

This question was reopened

Revision history for this message
Dan Garner (dangarner) said :
#1

Adam,

You could do this through the XiboAPI in 1.6.3 - we have added the necessary methods to import a csv file. You'd need a small application running on a timer (Xibo can't do that for you, you would have to make your own provisions).

There is a good (basic) example API client written in PHP in lp:xibo/1.6 which you could use as a starting point.

If you aren't a programmer you could ask the community on here or one of the companies in the directory http://xibo.org.uk/directory

Thanks,
Dan

Revision history for this message
Adam L (adlamann) said :
#2

I'm a programmer in training - I've got pretty close using the API via python - but having a bit of trouble with the DataSetImportCsv API call (have managed to list the datasets, get the ID, upload a file and get it's ID).

Can anyone give me an example I could work off?

Revision history for this message
Dan Garner (dangarner) said :
#3

there is an example in example_oauth\index.php

After you have the file ID you just need to call DataSetImportCSV as per line 513 of that file.

Sorry I dont have more time for a complete answer at the moment.

Revision history for this message
Adam L (adlamann) said :
#5

Hi again,

I've managed to get it all working via the API using Python. One little thing though - every time I upload a new copy of the CSV file, it generates a new FileID and creates a corresponding file in the content\temp directory. As i plan to refresh every hour or so, this directory is going to fill up. How do I clean out this temp directory, or stop a new copy from being uploaded each time? Is there some sort of overwrite function i'm missing?

Thanks for your assistance.

Cheers
Adam

Revision history for this message
Dan Garner (dangarner) said :
#6

The file should be automatically deleting once the import has completed - it is strange that it isn't... if you pop debugging and auditing on, do you get anything in the logs?

Revision history for this message
Adam L (adlamann) said :
#7

I'm getting a permission denied:

2 2014-09-16 13:02:14 <errormsg>unlink(C:\Applications\Xibo\content/temp/61): Permission denied</errormsg>
<errornum>2</errornum>
<errortype>Warning</errortype>
<scriptname>C:\Applications\Xibo\xampp\htdocs\xibo\lib\data\datasetdata.data.class.php</scriptname>
<scriptlinenum>361</scriptlinenum>

I've tried giving 'everyone' full control permissions over that directory, but still the same message. Something to do with the strange backslash/forward slash combination it's attempting to use for the 'unlink' path perhaps?

Revision history for this message
Dan Garner (dangarner) said :
#8

I don't think its the unsual path name (although that could indeed be improved to be more human readable).

Seems to be a more general problem with file ownership when running in windows. Can you try changing the ownership of the file after the file upload call, but before the importcsv call?

Revision history for this message
Adam L (adlamann) said :
#9

Hi Dan,

Okay just tried that (changed ownership of temp file to 'everyone' and also granted 'full control' NTFS permissions to 'everyone' just to be sure), but still the same 'Permission Denied' error in the Xibo logs as above.

Can I manually clear files out of this directory without messing anything up as an interim measure?

Cheers
Adam

Revision history for this message
Dan Garner (dangarner) said :
#10

Hi, yes please do manually clean those up. I can't understand why it wont delete those files, but I do also have the same problem here on my windows test PC.

I'll pop a bug in for it so I don't forget to investigate.

Revision history for this message
Adam L (adlamann) said :
#11

Cheers Dan.
I've found another issue with the CSV import via the API - I can't seem to import into anything beyond dataset ID #1.

The fields in the datasets (2 and over) will clear the data with the overwrite flag set, but nothing will import. It does some strange overwriting of dataset 1 instead. I've tried adding a few more datasets (3 and 4) but the same result. I can only get an import happening with #1.

Should I open another question for this one?

Revision history for this message
Dan Garner (dangarner) said :
#12

Hi Adam,

I think that might be related to your spreadSheetMapping ? The column id's in there should refer to each dataset - I suspect yours are all for datasetid 1?

We should probably test that the datasetcolumnids in the mapping exist in the dataset before we run anything (https://bugs.launchpad.net/xibo/+bug/1371883).

Keep it in this question for now :)

Revision history for this message
Adam L (adlamann) said :
#13

ah - sorry, my bad. I assumed all the column ID's would reset for each dataset. Probably should have carefully read the API some more before posting that. I've written a new API call that lists all my column ID's, and sorted out my mapping. That bit is working now.

Leaving this open for the linked bug.

Thanks for all your help Dan.

Revision history for this message
Dan Garner (dangarner) said :
#14

I think this is because the code forgot to close the file pointer.

Revision history for this message
Matt (v-matt-l) said :
#15

Adam, would you care to share your solution. I have been trying to figure this out without much success.

Can you help with this problem?

Provide an answer of your own, or ask Adam L for more information if necessary.

To post a message you must log in.