Update and reminder on use of Google Drive; constraints checking

1) Here’s an update and reminder on the use of the Google Drive. You may now use subfolders (level/depth 1 only) or zip archives when uploading files into the benchmarks folders, or just continue with uploading files as is.

The guides for using are also copied again here:

• • Please follow the provided folder structure for uploading submission files:
    • there is a dedicated subfolder for the alpha round (and later on also one for the final round); and
    • within there are subfolders for each benchmark — upload your submissions into those subfolders.
  • Each team may upload submission files anytime, upon which these files are automatically downloaded to our servers for evaluation.
    • Submission files are: the DEF file and the post-layout netlist — no other files from the benchmark ZIP archives have to be re-uploaded again.
    • You can upload your submission files as follows:
      • directly as is (what you did until now);
      • within a subfolder (use level/depth of 1 only, i.e., no further subfolders inside that subfolder; do not use folder names containing ‘results’);
      • or as zip archive.
    • DEF and netlist files that go together must use the same basename, e.g., trial1.def and trial1.v. You may still upload multiple trials at once; they will be handled in separate runs.
      • This also applies to subfolder submission.
      • For zip submission, you cannot put multiple trials into one archive; each trial must go into a separate zip.
    • Re-uploaded submission files with the same name (option offered by Drive: “Replace existing file”) are not re-evaluated. Thus, you want to either select the Drive option “Keep both files” when re-uploading, or use files of different names when re-uploading.
    • Once uploaded and processed, renaming of files will also not trigger re-evaluation.
  • Results will be returned/uploaded into the same benchmark subfolder. Results will include scores and report files as generated by our evaluation scripts.
  • Participants will receive an email notification once their results are available.
  • You may also keep any other files in the Drive, and you’re free to just put them in the home directory or organize in folders separate from the provided folder structure. These other files won’t be downloaded for evaluation.

2) The constraints checking is not fully working yet. Thus, even when you receive a score without any processing errors, it does not mean that your solution is valid. Please follow the constraints proactively, and we will let you know as soon as the constraints checking works.