Bulk storing in SQLite while parsing in iOS

[ashwinr87]

In my iOS application, I am parsing an XML from a SOAP based webservice and I am storing it into the SQLITE database. The problem that I am facing it is that, it is very slow. It takes about 18 seconds for about 310 rows of data.

Here is what is a sample of my XML -

<return>
<batteryID>1234</batteryID>
<batteryVersionNum>1</batteryVersionNum>
<conceptCode>abc</conceptCode>
<conceptDescription>abc</conceptDescription>
<effectiveEndTime>2010-11-23</effectiveEndTime>
<effectiveStartTime>2010-11-23</effectiveStartTime>
</return>
<return>
<batteryID>2345</batteryID>
<batteryVersionNum>1</batteryVersionNum>
<conceptCode>bac</conceptCode>
<conceptDescription>bac</conceptDescription>
<effectiveEndTime>2010-11-23</effectiveEndTime>
<effectiveStartTime>2010-11-23</effectiveStartTime>
</return>

I use NSXML parser for parsing the XML. For every return tag() I encounter, I create a new instance of my entity -

- (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName
  namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qualifiedName
    attributes:(NSDictionary *)attributeDict
{
    if ([elementName isEqualToString:@"return"])
    {
        // Blank lab panel object
        objLabPanel = [NSEntityDescription insertNewObjectForEntityForName:@"LabPanels" inManagedObjectContext:managedObjectContext];
        mainElement = elementName;
    }
}

Once I encounter a return end element (), I save the object to the database -

- (void)parser:(NSXMLParser *)parser didEndElement:(NSString *)elementName
  namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName {

    if ([elementName isEqualToString:@"return"])
    {
        [objPatient addLabPanelsObject:objLabPanel];

        // Save
        NSError *error = nil;
        BOOL saveObj = FALSE;
        saveObj = [managedObjectContext save:&error];

        if (saveObj == FALSE)
        {
            NSLog (@"Error: %@", error);
        }
    }
    else if ([elementName isEqualToString:@"batteryID"] && [mainElement    isEqualToString:@"return"])
    {
    objLabPanel.labPanelBatteryId = elementValue;
    }
    // getting other values from the xml
    . 
    .
    .
    }

Hence for every return tag I encounter, I do a save to the database and so for my 310 or so records, I would be saving it 310 times. So by doing a [managedObjectContext save:&error]; , I guess that it opens the database, saves the data and closes the database and hence it takes a lot of time. Am I right in thinking this way?

Is there a way I can save all the objLabPanel's to an array and then do a bulk insert to the database i.e. bulk insert the entire array of objLabPanel's into the SQLITE database at once?

It would be great if someone could help me out with this.

[nobre84]

You should save the managedObjectContext only once , after all your inserts are done. That will speed up the process a lot. If you expect the number of entries to grow in the future, you can save your context every 500 records added or so, to be on the safe side.

[ashwinr87]

Thank you very much.. I just tried that and found out it is working..
one question.. how would I save it every 500 records or so? I would need to keep a kind of counter or something right?

Originally Posted by nobre84

You should save the managedObjectContext only once , after all your inserts are done. That will speed up the process a lot. If you expect the number of entries to grow in the future, you can save your context every 500 records added or so, to be on the safe side.

[nobre84]

If the data itself doesn't have any numbering info you can use, you can declare a static int inside the function to count it... Every time you reach 500 , you save the context and set it to zero. Be sure to still save the context at the end of the parsing, because you could be in the middle of a 500-batch and still didn't save before finishing

[ashwinr87]

yeah.. that was what I was thinking about doing too..
thank you for the reply...

Originally Posted by nobre84

If the data itself doesn't have any numbering info you can use, you can declare a static int inside the function to count it... Every time you reach 500 , you save the context and set it to zero. Be sure to still save the context at the end of the parsing, because you could be in the middle of a 500-batch and still didn't save before finishing