Drupal Feeds: Prevent import of multiple items with the same Unique field value

Date: 
Thursday, May 17, 2018 - 12:30

https://www.drupal.org/project/feeds/issues/2973628

Is there any way to import only one instance of an item with multiple variations, all sharing the same GUID? The importer should accept only the first instance and reject any future instance matching that GUID in the same import

I'm importing some products from Ebay and using the productid as GUID, but some products have variations of size, color, sellers, etc - but use the same productid:

Title:Mens Short Sleeve Dress Shirt Slim Fit Button Down Casual - Var1
GUID:173117424201
Title:Mens Short Sleeve Dress Shirt Slim Fit Button Down Casual - Var2
GUID:173117424201
Title:Mens Short Sleeve Dress Shirt Slim Fit Button Down Casual - Var3
GUID:173117424201

I end up with only one variation, which is what I want. The problem is that if there are 99 variations, the feed imports the item 99 times, each time overwriting the previous. This creates a serious performance and resource problem.

Feed Tamper has a "Unique" option, but that only makes the elements in a multivalued field unique.

I need the item to update with future imports so unchecking "Update existing nodes" is not an option

Drupal 7.58
Feeds 7.2..0-beta4

Does anyone know of any workarounds?
Thanks in advance

Project: 
Status: 
completed
Priority: 
high
Applied Status: 
Website: 
shopperita.com
Stage: 
Stage
Production

Comments

Posted by developer7 on 05/22/2018 - 11:27

I was attempting to use field validation module to cause import of variations to fail, but it does not seem that field variation checks feeds default field value. Here is what I did:
1. Created a rule to fill a field after node creation of the specific node type
2. Set field validation to check that the field is empty

My hope was that once the first variation is imported, and the field is filled, all subsequent variations would fail. I then use rules scheduler to change the state of the field to empty after three minutes so that update to the node would still be possible.

Is there any way to get the validation to work? I read several issues dealing with field validation with feeds, but most of them deal with validating incoming values, not checking the default value of a field.

Even if that worked I still have questions:
1. Would such error cause the complete feed import to abort/stall?
2. Would throwing a possible 100 errors cause an equally severe performance issue as importing all variations?

https://www.drupal.org/project/feeds/issues/2973628#comment-12624648

Posted by muhammad on 05/23/2018 - 00:06

This task is complete and ready for review.

Screenshot:
https://www.cloudhq.net/c/b7c25100cb

URL:
http://dev.shopperita.com/admin/config/shops/ksearch

Project Management