user/spack: (pmmx) FTBFS: Error: the "target" field in compilers.yaml accepts only target families

See also:

Note that any environment variables used to specify configuration file locations do NOT seem to work with the spack compiler find command, and a new compilers.yaml file will be written to ./etc/spack/linux/. This file may have precedence over user-specified configuration locations. (This is a Spack bug, see above).

Anyway, here is what is automatically detected:

builder on x86_64 ~/packages/user/spack/src/spack $ ./bin/spack compiler find
==> Added 1 new compiler to /root/packages/user/spack/src/spack/etc/spack/linux/compilers.yaml
    gcc@8.5.0
==> Compilers are defined in the following files:
    /root/packages/user/spack/src/spack/etc/spack/linux/compilers.yaml
builder on x86_64 ~/packages/user/spack/src/spack $ cat /root/packages/user/spack/src/spack/etc/spack/linux/compilers.yaml
compilers:
- compiler:
    spec: gcc@8.5.0
    paths:
      cc: /usr/local/bin/gcc
      cxx: /usr/local/bin/g++
      f77: null
      fc: null
    flags: {}
    operating_system: adelie1
    target: i686
    modules: []
    environment: {}
    extra_rpaths: []

Here is how Spack relates architectures:

builder on x86_64 ~/packages/user/spack/src/spack $ ./bin/spack arch --known-targets
Generic architectures (families)
    aarch64  arm  ppc  ppc64  ppc64le  ppcle  riscv64  sparc  sparc64  x86  x86_64  x86_64_v2  x86_64_v3  x86_64_v4

GenuineIntel - x86
    i686  pentium2  pentium3  pentium4  prescott

GenuineIntel - x86_64
    nocona  nehalem   sandybridge  haswell    skylake  cannonlake      cascadelake
    core2   westmere  ivybridge    broadwell  mic_knl  skylake_avx512  icelake

AuthenticAMD - x86_64
    k10  bulldozer  piledriver  zen  steamroller  zen2  zen3  excavator

IBM - ppc64
    power7  power8  power9

IBM - ppc64le
    power8le  power9le

Cavium - aarch64
    thunderx2

Fujitsu - aarch64
    a64fx

ARM - aarch64
    graviton  graviton2  graviton3

Apple - aarch64
    m1

SiFive - riscv64
    u74mc

The error comes from here, because i686 is not considered a microarch "family", but belongs to the family we want:

# from lib/spack/spack/compilers/__init__.py
def get_compilers(config, cspec=None, arch_spec=None):
    compilers = []

    for items in config:
        items = items["compiler"]
        if cspec and items["spec"] != str(cspec):
            continue

        # If an arch spec is given, confirm that this compiler
        # is for the given operating system
        os = items.get("operating_system", None)
        if arch_spec and os != arch_spec.os:
            continue

        # If an arch spec is given, confirm that this compiler
        # is for the given target. If the target is 'any', match
        # any given arch spec. If the compiler has no assigned
        # target this is an old compiler config file, skip this logic.
        target = items.get("target", None)

        try:
            current_target = archspec.cpu.TARGETS[str(arch_spec.target)]
            family = str(current_target.family)
        except KeyError:
            # TODO: Check if this exception handling makes sense, or if we
            # TODO: need to change / refactor tests
            family = arch_spec.target
        except AttributeError:
            assert arch_spec is None

        if arch_spec and target and (target != family and target != "any"):
            # If the family of the target is the family we are seeking,
            # there's an error in the underlying configuration
            if archspec.cpu.TARGETS[target].family == family:
                msg = (
                    'the "target" field in compilers.yaml accepts only '
                    'target families [replace "{0}" with "{1}"'
                    ' in "{2}" specification]'
                )
                msg = msg.format(str(target), family, items.get("spec", "??"))
                raise ValueError(msg)
            continue

        compilers.append(_compiler_from_config_entry(items))

    return compilers

Changing i686 (derived) to x86 (family) does work. Or, as suggested in the thread above, clearing the from field in lib/spack/llnl/util/cpu/microarchitectures.json for i686 also works because it tricks Spack into thinking it is a family.

It is confusing because documentation suggests this is a correct way to specify target, which of course is not the literal string x86_64 as would be the family name:

$ spack install zlib target=sandybridge

So does this mean target= is different from the target in compilers.yaml? I'm not sure.

We definitely do not want to edit microarchitectures.json because Spack is a unique package in that our users will be compiling packages on their own systems for their target CPUs.

This file may need to be patched (it is how Spack detects CPU type): https://github.com/archspec/archspec/blob/master/archspec/cpu/detect.py#L95

We also need to consider, if a user installs their own compiler(s), then they are fully permitted to use them with Spack, so distributing a compilers.yaml with our package is not an option.

compilers.yaml is generated by:

https://github.com/spack/spack/blob/1dcb5d1fa75476fb4f8744e7f1a5a2512ba1ea93/lib/spack/spack/compilers/init.py#L713


    def _default_make_compilers(cmp_id, paths):
        operating_system, compiler_name, version = cmp_id
        compiler_cls = spack.compilers.class_for_compiler_name(compiler_name)
        spec = spack.spec.CompilerSpec(compiler_cls.name, version)
        paths = [paths.get(x, None) for x in ("cc", "cxx", "f77", "fc")]
        target = archspec.cpu.host()
        compiler = compiler_cls(spec, operating_system, str(target.family), paths)
        return [compiler]

archspec.cpu.host() is implemented here:

https://github.com/archspec/archspec/blob/77640e572725ad97f18e63a04857155752ace045/archspec/cpu/detect.py#L234

def host():
    """Detects the host micro-architecture and returns it."""
    # Retrieve a dictionary with raw information on the host's cpu
    info = raw_info_dictionary()

    # Get a list of possible candidates for this micro-architecture
    candidates = compatible_microarchitectures(info)
    ...

compatible_microarchitectures(info) is implemented here, and seems to rely on a "compatibility check" to decorate based on CPU feature flags what is the correct family:

https://github.com/archspec/archspec/blob/77640e572725ad97f18e63a04857155752ace045/archspec/cpu/detect.py#L226


def compatible_microarchitectures(info):
    """Returns an unordered list of known micro-architectures that are
    compatible with the info dictionary passed as argument.
    Args:
        info (dict): dictionary containing information on the host cpu
    """
    architecture_family = _machine()
    # If a tester is not registered, be conservative and assume no known
    # target is compatible with the host
    tester = COMPATIBILITY_CHECKS.get(architecture_family, lambda x, y: False)
    return [x for x in TARGETS.values() if tester(info, x)] or [
        generic_microarchitecture(architecture_family)
    ]

Note 1: the line architecture_family = _machine() is ... not really precise. More accurately:

diff --git a/lib/spack/external/archspec/cpu/detect.py b/lib/spack/external/archspec/cpu/detect.py
index a7cc4481f6..a527a3c17d 100644
--- a/lib/spack/external/archspec/cpu/detect.py
+++ b/lib/spack/external/archspec/cpu/detect.py
@@ -222,12 +222,12 @@ def compatible_microarchitectures(info):
     Args:
         info (dict): dictionary containing information on the host cpu
     """
-    architecture_family = _machine()
+    uname = _machine()
     # If a tester is not registered, be conservative and assume no known
     # target is compatible with the host
-    tester = COMPATIBILITY_CHECKS.get(architecture_family, lambda x, y: False)
+    tester = COMPATIBILITY_CHECKS.get(uname, lambda x, y: False)
     return [x for x in TARGETS.values() if tester(info, x)] or [
-        generic_microarchitecture(architecture_family)
+        generic_microarchitecture(uname)
     ]

Note 2: info contains e.g. /proc/cpuinfo or equivalent, which we will ignore for now:

$ ./quicktest 2>&1 | grep ^{ | head -n 1 | sed -e "s/'/\"/g" | jq
{
  "processor": "0",
  "vendor_id": "AuthenticAMD",
  "cpu family": "23",
  "model": "49",
  "model name": "AMD Ryzen Threadripper 3970X 32-Core Processor",
  "stepping": "0",
  "microcode": "0x8301025",
  "cpu MHz": "3076.174",
  "cache size": "512 KB",
  "physical id": "0",
  "siblings": "64",
  "core id": "0",
  "cpu cores": "32",
  "apicid": "0",
  "initial apicid": "0",
  "fpu": "yes",
  "fpu_exception": "yes",
  "cpuid level": "16",
  "wp": "yes",
  "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sme sev sev_es xsavec xsaveopt",
  "bugs": "sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass",
  "bogomips": "7400.17",
  "TLB size": "3072 4K pages",
  "clflush size": "64",
  "cache_alignment": "64",
  "address sizes": "43 bits physical, 48 bits virtual",
  "power management": "ts ttp tm hwpstate cpb eff_freq_ro [13] [14]"
}

Let's pretend the generic_microarchitecture() branch is taken.

_machine() is implemented here:

def _machine():
    """ "Return the machine architecture we are on"""
    operating_system = platform.system()

    # If we are not on Darwin, trust what Python tells us
    if operating_system != "Darwin":
        return platform.machine()
    ...

and this returns either x86_64 or i686. The former is considered a "family", while the latter is not. This is the root cause.

generic_microarchitecture() begins with:

class Microarchitecture(object):
    """Represents a specific CPU micro-architecture.

    Args:
        name (str): name of the micro-architecture (e.g. skylake).
    ...

There is confusion around the terminology involved. As far as I can tell, this is what is intended:

"micro-architecture" refers to skylake or i686
"generic architecture" is synonymous with "family" which means x86_64 or x86

The proposed fix is therefore:

diff --git a/lib/spack/spack/compilers/__init__.py b/lib/spack/spack/compilers/__init__.py
index 05ffaf9f6c..e988bad307 100644
--- a/lib/spack/spack/compilers/__init__.py
+++ b/lib/spack/spack/compilers/__init__.py
@@ -710,7 +710,7 @@ def _default_make_compilers(cmp_id, paths):
         compiler_cls = spack.compilers.class_for_compiler_name(compiler_name)
         spec = spack.spec.CompilerSpec(compiler_cls.name, version)
         paths = [paths.get(x, None) for x in ("cc", "cxx", "f77", "fc")]
-        target = archspec.cpu.host()
+        target = archspec.cpu.TARGETS[str(archspec.cpu.host())]
         compiler = compiler_cls(spec, operating_system, str(target.family), paths)
         return [compiler]

I'm not fully following the intent behind the family decorator logic:

    def decorator(func):
        COMPATIBILITY_CHECKS.update({family: func for family in architecture_family})
        return func

Discussion with @becker33 on Slack has led to this proposed patch:

diff --git a/lib/spack/external/archspec/cpu/detect.py b/lib/spack/external/archspec/cpu/detect.py
index a7cc4481f6..cadd593ca2 100644
--- a/lib/spack/external/archspec/cpu/detect.py
+++ b/lib/spack/external/archspec/cpu/detect.py
@@ -92,7 +92,8 @@ def _machine():
 
     # If we are not on Darwin, trust what Python tells us
     if operating_system != "Darwin":
-        return platform.machine()
+        machine = platform.machine()
+        return TARGETS[machine].family.name if machine in TARGETS else machine
 
     # On Darwin it might happen that we are on M1, but using an interpreter
     # built for x86_64. In that case "platform.machine() == 'x86_64'", so we

changed the description

added BETA5 label

Upstream issue: https://github.com/archspec/archspec/issues/102

mentioned in commit 630236d0

added UPSTREAM label

closed with commit 843d1bfd

mentioned in commit 843d1bfd

user/spack: (pmmx) FTBFS: Error: the "target" field in compilers.yaml accepts only target families

Designs

Child items 0

Activity