Data Sync - Product Catalogue

[SadGeek]

Hi, I'm interested in your thoughts on this ..

I'm developing a product catalogue app on the iPhone and the catalogue will have minor changes every few weeks or so. The product information will contain images (which may change or new images added).

The question is, what would be the best plan of action to perform the updates.

First thoughts are that we could just do a new release of the App every month, so at least all the data is contained on the device and there is no delay while it's updating.

The other option I guess would be for the App to download a simple sync style XML (perhaps Xml is a bit heavy for a mobile device?) / Json document from the server and from that it can work out what products need to be downloaded, updated or deleted. It can then update the document when the product has been updated on the device.

Have a separate product SQLite database. When the App starts up it checks for updates and if a new database exists then it will download and install the database. Hammer / Nut solution for small updates!

I'd appreciate anyone's thoughts / experience on the topic.

Cheers

Steve

[dageshi]

Originally Posted by SadGeek

Hi, I'm interested in your thoughts on this ..

I'm developing a product catalogue app on the iPhone and the catalogue will have minor changes every few weeks or so. The product information will contain images (which may change or new images added).

The question is, what would be the best plan of action to perform the updates.

First thoughts are that we could just do a new release of the App every month, so at least all the data is contained on the device and there is no delay while it's updating.

The other option I guess would be for the App to download a simple sync style XML (perhaps Xml is a bit heavy for a mobile device?) / Json document from the server and from that it can work out what products need to be downloaded, updated or deleted. It can then update the document when the product has been updated on the device.

Have a separate product SQLite database. When the App starts up it checks for updates and if a new database exists then it will download and install the database. Hammer / Nut solution for small updates!

I'd appreciate anyone's thoughts / experience on the topic.

Cheers

Steve

Well, I have done something similar (on a back burner at the moment) this is the way I do it.

I add a TIMESTAMP field to each table I want to sync to tell when that product was last updated
and also a "status" field which marks the product as "deleted" or "active"

The client keeps a record of the last time it updated itself, sends it's last sync date to the server which then checks to see what's been updated since the date the client sent.

It then send back a list of the primary keys for the products that have been updated (or need to be deleted). The client saves those primary keys to it's local database and starts downloading them. Once it's downloaded them it updates the database in one go.

The advantage of doing it like this for me is it's highly fault tolerant (especially when using SQLITE transactions), the updates get downloaded quietly in the background and saved ready to be applied in one go. When they are applied everything is done in one go so the database goes cleanly from the old version to the new. Also it doesn't matter how long the client has gone without updating it just takes longer to download the data in the background is all.

However I am dealing with fairly large amounts of data and complicated db structures which need updating in one go, for what your doing it might be overkill.

[SadGeek]

Originally Posted by dageshi

Well, I have done something similar (on a back burner at the moment) this is the way I do it.

I add a TIMESTAMP field to each table I want to sync to tell when that product was last updated
and also a "status" field which marks the product as "deleted" or "active"

The client keeps a record of the last time it updated itself, sends it's last sync date to the server which then checks to see what's been updated since the date the client sent.

It then send back a list of the primary keys for the products that have been updated (or need to be deleted). The client saves those primary keys to it's local database and starts downloading them. Once it's downloaded them it updates the database in one go.

The advantage of doing it like this for me is it's highly fault tolerant (especially when using SQLITE transactions), the updates get downloaded quietly in the background and saved ready to be applied in one go. When they are applied everything is done in one go so the database goes cleanly from the old version to the new. Also it doesn't matter how long the client has gone without updating it just takes longer to download the data in the background is all.

However I am dealing with fairly large amounts of data and complicated db structures which need updating in one go, for what your doing it might be overkill.

Thanks dageshi, what are your plans with the update. Download everything in the background and then update the lot when the user next starts up the App. I don't think you're planning to do this but if you updated via seperate thread then you would have to store temp versions of the data. e.g. A performs a search, 5 products come back (but in the background the product A has been deleted). So if the user taps Product A there is now no such record. So as I you say, updating in one go is the best approach.

Many thanks for you post.

[dageshi]

Originally Posted by SadGeek

Thanks dageshi, what are your plans with the update. Download everything in the background and then update the lot when the user next starts up the App. I don't think you're planning to do this but if you updated via seperate thread then you would have to store temp versions of the data. e.g. A performs a search, 5 products come back (but in the background the product A has been deleted). So if the user taps Product A there is now no such record. So as I you say, updating in one go is the best approach.

Many thanks for you post.

Due the amount of data involved and more importantly the relative slow access speeds to the iphone flash memory (or so it seems to me) when the updates are ready I ask them if they want to update now and then throw up a update progress screen in a modal view. Updgrade proceeds and then window closes.

My app is a travel guide based on wikitravel.org data, the data is all the wikitravel info for an entire country. For China that runs to about 55MB including 50-60 images at 1024x786 each. In that there is about 5MB+ of just text based content split up over several thousand entries. Updating that lot + tags and other data takes a fair amount of time unfortunately especially since sqlite indexes don't help they actually hinder.

The only other approach I can suggest for you is that you could do a hybrid version. Download and insert the data into the database as you get it in the background, but have a special field that marks each row as "active" or "inactive" when you've downloaded it all you can mark everything from inactive to active in one go and delete your old entries. I'm guessing this might be alot quicker, less intrustive in your circumstances. So basically

1) send client last sync date to server
2) server sends back list of primary keys that need updating
3) client starts to download these in the background and saves them to the db marking them as "inactive"
4) either at startup or maybe in iOS 4 when the app goes into the background, mark all the inactive rows as active and delete those that need deleting.

I think doing it at startup is the best option though.