Skip to content

Comments

Add feature for importing CSV file#877

Open
alserious wants to merge 9 commits intoaminalaee:mainfrom
alserious:csv-import
Open

Add feature for importing CSV file#877
alserious wants to merge 9 commits intoaminalaee:mainfrom
alserious:csv-import

Conversation

@alserious
Copy link
Contributor

  1. Add an import button and a modal window for selecting a CSV file in the list page.
  2. Add flags: column_import_list, column_import_exclude_list, can_import by default disabled.
  3. If the model has relations, the association will be according to the returned data __str__ method of the SQLAlchemy model. Relation models in a CSV file are separated by a comma.

Screenshot 2025-02-03 144529

Screenshot 2025-02-03 144541

import_csv

image

@alserious
Copy link
Contributor Author

@aminalaee Сan you review, plz?

@twoodwark twoodwark mentioned this pull request Oct 29, 2025
1 task
for column, value in row.items():
if column not in columns:
continue
if value and "," in value:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alserious dont you think it would be easier and more robust for importing relations by ID (or PK) only instead of relying on str? Addresses can be just string field containing string Address 1, Address 2.

Also, I think we should disable O2M and M2M import: as it could easily be done by adding/importing two entries in Address table (which is correct way of doing it).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mmzeynalli Thanks for the review. I agree that importing data without specifying relationships is a more correct and simpler method, but in practice, we need to import a lot of data like:
|username|address|role|...|
|sam|relation to address(new york)|relation to role(author)| ...|

And then manually filling in the relationships is problematic. Maybe make this optional? With or without importing relationships?

for row in data:
n_row = []
for value in row.getlist(relation_name):
if value == str(relation_obj):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is really ineffective. If we at least had list of PKs, we could have used: .where(id.in_(list))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it seems like this is the only way, as long as we don't use a primary key list. If we remove the relationship import, it won't be a problem.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets not do this, it can be problematic for for duplicate data if there are two rows data with 'Address 1'. Let user to do the following:

  1. Import first data, no relations
  2. Filter out in admin newly imported
  3. Export that data and get IDs
  4. Update 2nd csv
  5. Upload 2nd csv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants