Plier fails to import data

[Nicklas Nordborg]

Summary from the BASE mailing list. The entire thread is available at: ​http://www.mail-archive.com/basedb-users@lists.sourceforge.net/msg00778.html

---

i am running a test base 2.3.2 on mysql 5.1.14-beta (i must apologies again, i was thinking to be running on the reccomended 5.0) the stack (as in base report) is

net.sf.basedb.core.BaseException: Table 'base2dynamic.D2Spot' doesn't 
exist
  at net.sf.basedb.core.AbstractSqlQuery.iterate(AbstractSqlQuery.java:193)
  at 
net.sf.basedb.core.BioAssaySet.countSpotsAndReporters(BioAssaySet.java:1096)
  at net.sf.basedb.core.BioAssaySet.onBeforeCommit(BioAssaySet.java:315)
  at net.sf.basedb.core.DbControl.commit(DbControl.java:365)
  at 
se.lu.thep.affymetrix.AbstractExternalBinaryPlugin.run(AbstractExternalBinaryPlugin.java:264)
  at 
net.sf.basedb.core.PluginExecutionRequest.invoke(PluginExecutionRequest.java:88)
  at 
net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java:420)

I think I partly know what is happening. There are two problems, the first causing the second.

1. The Plier plugin is creating a root bioassayset but is never

inserting any data to it. If no data is inserted, then the missing table isn't created.

2. Before everything is committed to the database the root bioassayset

tries to count the number spots that has been inserted. This fails because the table isn't there and there is no check if it exists at this place.

Number 2 is a bug in the BASE core and should be easy to fix. This will be included in the 2.4 release in about two weeks.

Why number 1 is happening is hard to tell. The Plier plug-in seems to execute without errors. Maybe there is something wrong with the output from the the 'apt-probeset-summarize' program. In that case, the plug-in will just ignore it and continue without inserting any data. I guess the developer of the Plier plug-in knows more about this...

[base]

further info:

• the version of apt used is the latest 1.8.
• the failure happens during the creation of the new root bioassay, and no new node appears under the plier analysis tree.
• the affymetrix chip is a snp one.
• when i have loaded the new reporters for the chip i have not created any new reporter list.
• i have tried with only one cel file (instead of 4), same result.
• the same procedure worked fine for another array design (mg_u74av2)

[Peter Johansson]

I suspect the reason behind the problem is that the output from apt has been changed from 1.6 to 1.8. Though I need to test this to be certain. If this is the case I suggest we modify the parsing of the output to suit 1.8 and simply require apt 1.8. I'll be back when I've done the testing.

[Peter Johansson]

I have tested with both apt 1.6 and apt 1.8. They succeed equally well on my test set.

Just like you indicated in your comment, the problem could be due to the array design. From the data I have available it is difficult to tell whether this is an issue caused by apt binary or whether it is caused by the plugin implementation.

To test this, I would try running apt-summarize on the problematic dataset outside of BASE. Does the binary work on SNP data? If so what does the output look like? How does it differ from a output from a non-problematic dataset?

[base]

To test this, I would try running apt-summarize on the problematic dataset outside of BASE. Does the binary work on SNP data? If so what does the output look like? How does it differ from a output from a non-problematic dataset?

dear peter, thank you for the suggestions and apologies for the delay (i am just back to work). i tried the apt: the problem is with the size of the array, that gives memory problems to apt which is rather parsimonious with error messages. I am trying some of the options suggested by affymetrix (rma-sketch, setting a block-size), but still have problems. contacting affymetrix now.

[base]

... my previous posting was clearly premature...

for SNP arrays one must use apt-probeset-genotype (briefly tested and worked ok).

it needs .cdf and also .chrx files.

there must be a pool of cel files.

i was trying to check out the plugin from svn but i'm getting 'Floating point exception': can you help? thanks!!

[Jari Häkkinen]

The problem is that snp chips cannot be used with either RMAExpress or Plier. To resolve this issue the plug-in must be increased with further features, or if it is not possible to add features to this plug-ins another snp specific plug-in must be created.

I'll add a note to the plug-in documentation explainig that the plug-in currently only supports expression chips.

[Jari Häkkinen]

(In [416]) Refs #73. Added note about non snp data support.

[Jari Häkkinen]

The ​APT plug-in supports snp-chips (i.e., implements apt-probeset-genotype).