[Feature Request]: Add option to use CRC checksum if MD5 is not there due to multipart upload #3181
Description
Related Template(s)
GCS_Avro_to_Cloud_Spanner
What feature(s) are you requesting?
If exported data is re-uploaded to a GCS bucket using multipart upload for performance, export fails with MD5 checksum not matching.
'null' not matching with 'xyz'
Root Cause:
Since we uploaded using multipart API we did not get MD5 on object we only have CRC in such cases.
import checks MD5 in manifest file to the content that we uploaded.
Request
Import should not have mandatory integrity check.
Have an option to skip integrity check using some input param.
Keep CRC as well in manifest file generated in export (Cloud_Spanner_to_GCS_Avro) and provide an option to use CRC instead of MD5 in such cases when object data has missing MD5.