Best way to load remote data (XML, XML to Local SQLite) into UITableView?
Hello Everyone,
I'm currently developing an app and my major bottleneck is getting data into the program. The data is highly dynamic and changes several time a day so local storage is not an option. Currently, I have all data loaded on application load via a remote XML file (the XML file is created programatically via a PHP script that queries a MySQL db that houses the data). The problem with this is that it is very slow to parse even though the XML file is only about 67k (using NSXMLParser). It takes about 25 seconds to parse the file, during which time the UI is completely locked up (I put an animated overlay for now but ultimately it would be awesome to be able to load the data into the view asynchronously or just reduce the load time to something under 4 seconds). I've tried using a static locally stored XML file rather than a programatically generated one but it still suffers the same parsing times. I've seen SQLite in use but it's hindered by the fact that you can't load data remotely. I've seen a few people using PLIST files but I haven't been able to create one programatically and I'm also not sure how to deal with creating an NSSArray with the data. I know there are apps out there that have large amounts of data updated on a continuous stream but they don't seem to suffer from the problems I'm having. Am I missing something obvious? Thank you so much in advance!
you can easily download the file asynchronously... Check out this post. It shows how to download an HTML file, but the same thing applies, just pass an XML file to it.
How are you parsing the XML? Are you using NSXMLParser? That is the only way to go, if not.
SQLite is not option for remote data pull, you already pointed out why. If you have problems with the NSXMLParser class try out TouchXML. It is a replacement for the missing NSXML class tree which is not ported to Cocoa touch.
To not lock up the system you can go deep with POSIX commands in plain C, but I suggest you look up NSThread for threading or NSOperationQueue if the task is more complex (as the name states, NSO.Q. queses threads) but you won't give up comfortable threading.
I will try both of these solutions. Thanks! I tried out KissXML awhile ago but wasn't successful.
One hint: if you try TouchXML and wonder about the lack of documentation for it, keep in mind that it is a replacement for NSXML, so just use commands analog to it, (e.g. NSXMLElement would be CXMLElement).
you can easily download the file asynchronously... Check out this post. It shows how to download an HTML file, but the same thing applies, just pass an XML file to it.
How are you parsing the XML? Are you using NSXMLParser? That is the only way to go, if not.
I am using NSXMLParser but it locks the main thread. I can't really see where to invoke the XML parser in that script you linked to. I may try out TouchXML and see if it simply parses out the XML quicker.
you can easily download the file asynchronously... Check out this post. It shows how to download an HTML file, but the same thing applies, just pass an XML file to it.
How are you parsing the XML? Are you using NSXMLParser? That is the only way to go, if not.
Hi Dutch, dun mind I chap in. Any idea how to read a .plist from a website and replace the one in my app's folder? Does this work? Thanks.
I am using NSXMLParser but it locks the main thread. I can't really see where to invoke the XML parser in that script you linked to. I may try out TouchXML and see if it simply parses out the XML quicker.
Perhaps there is an issue in your parsing code. I have not experienced such delays.
I really don't know what could be wrong with my parsing code. Would you mind taking a look for anything obvious?
Code:
@implementation XMLParser
- (XMLParser *) initXMLParser {
[super init];
appDelegate = (XMLAppDelegate *)[[UIApplication sharedApplication] delegate];
return self;
}
- (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName
namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qualifiedName
attributes:(NSDictionary *)attributeDict {
if([elementName isEqualToString:@"Books"]) {
//Initialize the array.
appDelegate.books = [[NSMutableArray alloc] init];
}
else if([elementName isEqualToString:@"Book"]) {
//Initialize the book.
aBook = [[Book alloc] init];
//Extract the attribute here.
aBook.bookID = [[attributeDict objectForKey:@"id"] integerValue];
NSLog(@"Reading id value :%i", aBook.bookID);
}
NSLog(@"Processing Element: %@", elementName);
}
- (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string {
if(!currentElementValue)
currentElementValue = [[NSMutableString alloc] initWithString:string];
else
[currentElementValue appendString:string];
NSLog(@"Processing Value: %@", currentElementValue);
}
- (void)parser:(NSXMLParser *)parser didEndElement:(NSString *)elementName
namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName {
if([elementName isEqualToString:@"Books"])
return;
//There is nothing to do if we encounter the Books element here.
//If we encounter the Book element howevere, we want to add the book object to the array
// and release the object.
if([elementName isEqualToString:@"Book"]) {
[appDelegate.books addObject:aBook];
[aBook release];
aBook = nil;
}
else
[aBook setValue:currentElementValue forKey:elementName];
[currentElementValue release];
currentElementValue = nil;
}
- (void) dealloc {
[aBook release];
[currentElementValue release];
[super dealloc];
}
@end
wow, after looking at my code, I just removed the NSLog calls and it reduced the parsing time down to less than 1 second! I can't beleive I missed that. This is one of those moments where you want to punch yourself in the face lol
Best way to load remote data (XML, XML to Local SQLite) into UITableView?
Thank you so much in advance!
Linear Mode
