Opened 10 years ago

Closed 10 years ago

#73 closed enhancement (wontfix)

Plier fails to import data

Reported by: nicklas Owned by: jari
Priority: major Milestone:
Component: se.lu.thep.affymetrix Keywords:
Cc:

Description

Summary from the BASE mailing list. The entire thread is available at: http://www.mail-archive.com/basedb-users@lists.sourceforge.net/msg00778.html


i am running a test base 2.3.2 on mysql 5.1.14-beta (i must apologies again, i was thinking to be running on the reccomended 5.0) the stack (as in base report) is

net.sf.basedb.core.BaseException: Table 'base2dynamic.D2Spot' doesn't 
exist
  at net.sf.basedb.core.AbstractSqlQuery.iterate(AbstractSqlQuery.java:193)
  at 
net.sf.basedb.core.BioAssaySet.countSpotsAndReporters(BioAssaySet.java:1096)
  at net.sf.basedb.core.BioAssaySet.onBeforeCommit(BioAssaySet.java:315)
  at net.sf.basedb.core.DbControl.commit(DbControl.java:365)
  at 
se.lu.thep.affymetrix.AbstractExternalBinaryPlugin.run(AbstractExternalBinaryPlugin.java:264)
  at 
net.sf.basedb.core.PluginExecutionRequest.invoke(PluginExecutionRequest.java:88)
  at 
net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java:420)

I think I partly know what is happening. There are two problems, the first causing the second.

  1. The Plier plugin is creating a root bioassayset but is never

inserting any data to it. If no data is inserted, then the missing table isn't created.

  1. Before everything is committed to the database the root bioassayset

tries to count the number spots that has been inserted. This fails because the table isn't there and there is no check if it exists at this place.

Number 2 is a bug in the BASE core and should be easy to fix. This will be included in the 2.4 release in about two weeks.

Why number 1 is happening is hard to tell. The Plier plug-in seems to execute without errors. Maybe there is something wrong with the output from the the 'apt-probeset-summarize' program. In that case, the plug-in will just ignore it and continue without inserting any data. I guess the developer of the Plier plug-in knows more about this...

Change History (8)

comment:1 Changed 10 years ago by base

further info:

  • the version of apt used is the latest 1.8.
  • the failure happens during the creation of the new root bioassay, and no new node appears under the plier analysis tree.
  • the affymetrix chip is a snp one.
  • when i have loaded the new reporters for the chip i have not created any new reporter list.
  • i have tried with only one cel file (instead of 4), same result.
  • the same procedure worked fine for another array design (mg_u74av2)

comment:2 Changed 10 years ago by peter

I suspect the reason behind the problem is that the output from apt has been changed from 1.6 to 1.8. Though I need to test this to be certain. If this is the case I suggest we modify the parsing of the output to suit 1.8 and simply require apt 1.8. I'll be back when I've done the testing.

comment:3 follow-up: Changed 10 years ago by peter

I have tested with both apt 1.6 and apt 1.8. They succeed equally well on my test set.

Just like you indicated in your comment, the problem could be due to the array design. From the data I have available it is difficult to tell whether this is an issue caused by apt binary or whether it is caused by the plugin implementation.

To test this, I would try running apt-summarize on the problematic dataset outside of BASE. Does the binary work on SNP data? If so what does the output look like? How does it differ from a output from a non-problematic dataset?

comment:4 in reply to: ↑ 3 Changed 10 years ago by base

To test this, I would try running apt-summarize on the problematic dataset outside of BASE. Does the binary work on SNP data? If so what does the output look like? How does it differ from a output from a non-problematic dataset?

dear peter, thank you for the suggestions and apologies for the delay (i am just back to work). i tried the apt: the problem is with the size of the array, that gives memory problems to apt which is rather parsimonious with error messages. I am trying some of the options suggested by affymetrix (rma-sketch, setting a block-size), but still have problems. contacting affymetrix now.

comment:5 Changed 10 years ago by base

... my previous posting was clearly premature...

for SNP arrays one must use apt-probeset-genotype (briefly tested and worked ok).

it needs .cdf and also .chrx files.

there must be a pool of cel files.

i was trying to check out the plugin from svn but i'm getting 'Floating point exception': can you help? thanks!!

comment:6 Changed 10 years ago by jari

  • Type changed from defect to enhancement

The problem is that snp chips cannot be used with either RMAExpress or Plier. To resolve this issue the plug-in must be increased with further features, or if it is not possible to add features to this plug-ins another snp specific plug-in must be created.

I'll add a note to the plug-in documentation explainig that the plug-in currently only supports expression chips.

comment:7 Changed 10 years ago by jari

(In [416]) Refs #73. Added note about non snp data support.

comment:8 Changed 10 years ago by jari

  • Resolution set to wontfix
  • Status changed from new to closed

The APT plug-in supports snp-chips (i.e., implements apt-probeset-genotype).

Note: See TracTickets for help on using tickets.