- Generated by
1.9.1
|
Hardware Locality (hwloc)
3.0.0a1-git
|
The behavior of the hwloc library and tools may be tuned thanks to the following environment variables.
Enforce the discovery from the given XML file as if hwloc_topology_set_xml() had been called. This file may have been generated earlier with lstopo file.xml. For convenience, the XML backend provides empty binding hooks which just return success. To have hwloc still actually call OS-specific hooks, HWLOC_THISSYSTEM should be set 1 in the environment too, to assert that the loaded file is really the underlying system. See also Importing and exporting topologies from/to XML files.
Enforce the discovery through a synthetic description string as if hwloc_topology_set_synthetic() had been called. For convenience, this backend provides empty binding hooks which just return success. See also Synthetic topologies.
Switch to reading the topology from the specified Linux filesystem root instead of the main file-system root. This directory may have been saved previously from another machine with hwloc-gather-topology.
One should likely also set HWLOC_COMPONENTS=linux,stop so that non-Linux backends are disabled (the -i option of command-line tools takes care of both).
Not using the main file-system root causes hwloc_topology_is_thissystem() to return 0. For convenience, this backend provides empty binding hooks which just return success. To have hwloc still actually call OS-specific hooks, HWLOC_THISSYSTEM should be set 1 in the environment too, to assert that the loaded file is really the underlying system.
Force the x86 backend to read dumped CPUIDs from the given directory instead of executing actual x86 CPUID instructions. This directory may have been saved previously from another machine with hwloc-gather-cpuid.
One should likely also set HWLOC_COMPONENTS=x86,stop so that non-x86 backends are disabled (the -i option of command-line tools takes care of both).
It causes hwloc_topology_is_thissystem() to return 0. For convenience, this backend provides empty binding hooks which just return success. To have hwloc still actually call OS-specific hooks, HWLOC_THISSYSTEM should be set 1 in the environment too, to assert that the loaded CPUID dump is really the underlying system.
Allow components to annotate the topology even if they are usually excluded by global components by default. For instance, setting this variable to 1 enables the addition of PCI vendor and model string info attributes to a XML topology that was generated without those names (if pciaccess was missing).
Assume that the selected backend provides the topology for the system on which we are running, even if it is not the OS-specific backend but the XML backend for instance. This means making the binding functions actually call the OS-specific system calls and really do binding, while the XML backend would otherwise provide empty hooks just returning success. This can be used for efficiency reasons to first detect the topology once, save it to a XML file, and quickly reload it later through the XML backend, but still having binding functions actually do bind.
It enforces the return value of hwloc_topology_is_thissystem(), as if HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM was set with hwloc_topology_set_flags(). It also enables support for the variable HWLOC_THISSYSTEM_ALLOWED_RESOURCES.
Change the locality of I/O devices behind the specified PCI buses. If no I/O locality information is available or if the BIOS reports incorrect information, this variable allows to move a I/O device tree (OS and/or PCI devices with optional bridges) near a custom set of processors.
Localities are given either inside the environment variable itself, or in the pointed file. They may be separated either by semi-colons or by line-breaks. Invalid localities are ignored, and it is possible to insert comments between actual localities by starting the line with # or //.
Each locality contains a domain/bus specification (in hexadecimal numbers as usual) followed by a whitespace and a cpuset:
0001 <cpuset> specifies the locality of all buses in PCI domain 0000. 0000:0f <cpuset> specifies only PCI bus 0f in domain 0000. 0002:04-0a <cpuset> specifies a range of buses (from 04 to 0a) within domain 0002. Domain/bus specifications should usually match entire hierarchies of buses behind a bridge (including primary, secondary and subordinate buses). For instance, if hostbridge 0000:00 is above other bridges/switches with buses 0000:01 to 0000:09, the variable should be HWLOC_PCI_LOCALITY="0000:00-09 <cpuset>", otherwise hwloc will try to extend the given locality to match the entire hierarchy. A single hierarchy of buses cannot be split between two localities.
If the variable is defined to empty or invalid, no forced PCI locality is applied but hwloc's internal automatic locality quirks are disabled, which means the exact PCI locality reported by the platform is used.
Show or hide NUMA nodes that correspond to NVIDIA GPU memory. By default they are ignored on POWER platforms to avoid interleaved memory being allocated on all CPUs and GPUs by mistake.
Setting this environment variable to 0 hides the NUMA nodes (default on POWER). Setting to 1 exposes these NUMA nodes (default on non-POWER platforms such as NVIDIA Grace Hopper).
These NUMA nodes may be recognized by the GPUMemory subtype. They also have a PCIBusID info attribute to identify the corresponding GPU.
Expose Windows processor groups as hwloc Group objects. By default (0), these groups are disabled because they may be incompatible with the hierarchy of resources that hwloc builds (leading to warnings). Setting this variable to 1 reenables the addition of these groups to the topology.
This variable does not impact the querying of Windows processor groups using the dedicated API in hwloc/windows.h, this feature is always supported.
Enable or disable some use of NUMA distances and memory target/initiator information to improve the locality of NUMA nodes, especially CPU-less nodes.
Bits in the value of this environment variable enable different features: Bit 0 enables the gathering of NUMA distances from the operating system. Bit 1 further enables the use of NUMA distances to improve the locality of CPU-less nodes. Bit 2 enables the use of target/initiator information. By default, all bits are set (7).
Disable or enable all heuristics to guess memory subtypes and tiers. By default, hwloc only uses heuristics that are likely correct and disables those that are unlikely.
Enforce the memory tiers from the given semi-colon separated list. Each entry specifies a bitmask (nodeset) of NUMA nodes and their subtype. Nodes not listed in any entry are not placed in any tier.
If an empty value or none is given, tiers are entirely disabled.
If set, this variable forces the rebuilding of memory tiers. This is mostly useful when importing a XML topology from an old hwloc version which was not able to guess memory subtypes and tiers.
Enable or disable object grouping based on distances. By default (1), hwloc uses distance matrices between objects (either read from the OS or given by the user) to find groups of close objects. These groups are described by adding intermediate Group objects in the topology. Setting this environment variable to 0 will disable this grouping.
See also HWLOC_GROUPING_VERBOSE for verbose messages about grouping.
Relax distance comparison during grouping. By default, objects may be grouped if their distances form a minimal distance graph. When setting this variable to 0.02, and when HWLOC_DISTANCES_ADD_FLAG_GROUP_INACCURATE is given, these distances do not have to be strictly equal anymore, they may just be equal with a 2% error.
If set to try instead of a numerical value, hwloc will try to group with perfect accuracy (0, the default), then with 0.01, 0.02, 0.05 and finally 0.1.
Numbers given in this environment variable should always use a dot as a decimal mark (for instance 0.01 instead of 0,01).
Change the ranking policy when guessing CPU kinds from frequencies and core types. hwloc tries to rank CPU kinds that are energy efficiency first, and then CPUs that are rather high-performance and power hungry.
By default, if available, the OS-provided efficiency is used for ranking. Otherwise, the frequency and/or core types are used when available.
This environment variable may be set to coretype+frequency, coretype+frequency_strict, coretype, frequency, frequency_base, frequency_max, forced_efficiency, no_forced_efficiency, default, or none.
Change the use of the max frequency in the Linux backend. hwloc tries to read the base and max frequencies of each core on Linux. Some hardware features such as Intel Turbo Boost Max 3.0 make some cores report slightly higher max frequencies than others in the same CPU package. Despite having slightly different frequencies, these cores are considered identical instead of exposing an hybrid CPU. Hence, by default (adjust=10), hwloc uniformizes the max frequencies of cores that have the same base frequency (higher values are downgraded by up to 10%). When available, the CPU capacity value is also adjusted accordingly.
If this environment variable is set to adjust=X, the 10% threshold is replaced with X. If set to 1, max frequencies are not adjusted anymore, some homogeneous processors may appear hybrid because of this. If set to 0, max frequencies are entirely ignored.
Uniformize max frequency, base frequency and Linux capacity to force a single homogeneous kind of CPUs. Setting to 1 enables this uniformization.
Try to enable the discovery of CPU kinds on Linux. This is enabled by default except on old AMD CPUs known to be homogeneous. Setting to 1 always enables it, while 0 always disables it. Setting to cppc=0 enables it without using ACPI CPPC nominal frequency. Setting to midr=0 enables it without using the ARM MIDR register.
Get the set of allowed resources from the native operating system even if the topology was loaded from XML or synthetic description, as if HWLOC_TOPOLOGY_FLAG_THISSYSTEM_ALLOWED_RESOURCES was set with hwloc_topology_set_flags(). This variable requires the topology to match the current system (see the variable HWLOC_THISSYSTEM).
This is useful when the topology is not loaded directly from the local machine (e.g. for performance reason) and it comes with all resources, but the running process is restricted to only a part of the machine (for instance because of Linux Cgroup/Cpuset).
Totally ignore administrative restrictions such as Linux Cgroups and consider all resources (PUs and NUMA nodes) as allowed. This is different from setting HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED which gathers all resources but marks the unavailable ones as disallowed.
Enable or disable the given comma-separated list of components (if they do not conflict with each other). Component names prefixed with - are disabled (a single phase may also be disabled).
Once the end of the list is reached, hwloc falls back to enabling the remaining components (sorted by priority) that do not conflict with the already enabled ones, and unless explicitly disabled in the list. If stop is met, the enabling loop immediately stops, no more component is enabled.
This is useful for understanding whether a topology issue comes from the native operating system backend or from the x86 backend: Setting to x86,stop or linux,stop will test one backend without the other.
If xml or synthetic components are selected, the corresponding XML filename or synthetic description string should be passed in HWLOC_XMLFILE or HWLOC_SYNTHETIC respectively.
Since this variable is the low-level and more generic way to select components, it takes precedence over environment variables for selecting components.
If the variable is set to an empty string (or set to a single comma), no specific component is loaded first, all components are loaded in priority order.
See Selecting which components to use for details.
The gl component is disabled by default since it may cause hwloc to hang when trying to connect to the X server. Setting this variable to 1 will let the component be enabled automatically as any other component.
Users are advised to verify that lstopo runs fine with the HWLOC_GL=1 on a platform before enabling the variable more globally on that specific platform.
Change the default search directory for plugins. By default, $libdir/hwloc is used. The variable may contain several colon-separated directories.
Prevent plugins from being loaded if their filename (without path) is listed. Plugin filenames may be found in verbose messages outputted when HWLOC_PLUGINS_VERBOSE=1.
Use the new AMD SMI library for querying RSMI OS devices. If set to rocm instead, the old ROCm SMI library will be used. The default is AMD SMI is available, otherwise ROCm SMI.
Enable or disable verbose reporting of errors. The hwloc library may issue warnings and errors to the standard error stream when it detects a problem during topology discovery, for instance if the operating system (or user) gives contradictory topology information.
Each error message is tagged with one or several keywords that control whether it should be displayed or not:
critical for critical errors such as invalid hardware topology information or invalid configuration. components and plugins for internal component and backends, and when they are dynamically loaded as plugins. core for issues in the library core, usually caused by discovery backends but not caught there. os for operating system specific errors. pci for PCI (and CXL) discovery. rsmi, cuda, nvml, l0, opencl and gl for IO-specific discovery. xml and synthetic for XML and synthetic topologies. bind for CPU or memory binding. misc for other error messages. user for errors caused by user-given configuration (environment variables, XML input path, etc.). By default (critical), hwloc only shows critical errors. If the variable is set to all (default in lstopo), all errors are displayed, for instance a failure to initialize CUDA or NVML. If set to empty or none, no hwloc error messages are shown.
The variable may contain a comma-separated list of keywords, possibly prefixed with a dash to negate them. For instance all,-cuda enables all error messages except the non-critical ones emitted by the CUDA discovery. Long keywords such as components may be shortened down to 4 characters as long as there is no possible confusion (e.g. opencl cannot be shortened).
This supersedes the former HWLOC_HIDE_ERRORS, HWLOC_XML_VERBOSE and HWLOC_SYNTHETIC_VERBOSE variables. It also includes some error messages about components and plugins previously enabled through HWLOC_COMPONENTS_VERBOSE and HWLOC_PLUGINS_VERBOSE.
Disable all verbose messages that are enabled by default when –enable-debug is passed to configure. When set to more than 1, even more verbose messages are displayed. The default is 1.
Enable or disable some verbose messages during grouping. If this variable is set to 1, some debug messages will be displayed during distance-based grouping of objects even if debug was not specific at configure time. This is useful when trying to find an interesting distance grouping accuracy.
Display messages when components are registered or enabled. This is the recommended way to list the available components with their priority (all of them are registered at startup).
Error messages about components are controled by the components keyword in the HWLOC_SHOW_ERRORS variable.
Display verbose information about plugins: list which directories are scanned, which files are loaded, and which components are successfully loaded.
Error messages about plugins are controled by the plugins keyword in the HWLOC_SHOW_ERRORS variable.
1.9.1