Published on

ACI - Fault Messages

Authors
  • avatar
    Name
    Jimmy Lai
    Twitter

Introduction

Cisco ACI System Messages Reference Guide

Cisco APIC System Faults/Events Message Reference

if __name__ == '__main__':
    parent_parser = argparse.ArgumentParser(description='APIC Fault summary')
    parent_parser.add_argument('apicUrl', help='APIC URL (http or https should be included)', type=str,
                               default="https://sandboxapicdc.cisco.com/")
    parent_parser.add_argument('--username', type=str, help='username', default="admin")
    parent_parser.add_argument('--pwd', type=str, help='password', default="!v3G@!4@Y")
    parent_parser.add_argument('--json', type=str,
                               help='load faults from json file (expects full json response by APIC)')

    args = parent_parser.parse_args()

    if re.match('http', args.apicUrl) is None:
        args.apicUrl = 'https://' + args.apicUrl
    if args.pwd is None:
        args.pwd = getpass.getpass()
    apic_faults = None
    if args.json is not None:
        with open(args.json) as fp:
            apic_faults = json.load(fp)['imdata']

    main(apicUrl=args.apicUrl, name=args.username, pwd=args.pwd, documentation=False, apic_faults=apic_faults)

def main(apicUrl=None, name=None, pwd=None, documentation=False, apic_faults=None):
    faults = {}
    headers = {
         'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

    session = requests.Session()
    if apic_faults is None:
        apic_faults = getApicFaults(session, apicUrl, name, pwd)
    for x in apic_faults:
        code = x['faultInst']['attributes']['code']
        if code not in faults:
            faults[code] = {'occurrences': 0, 'instances': [], 'documentation': {}}
        faults[code]['occurrences'] += 1
        faults[code]['instances'].append(x['faultInst']['attributes'])
    if documentation == True:
        getFaultDocumentation(session, apicUrl, faults)
    printFaultSummary(faults)
    printFaultDocumentation(faults, documentation)
FAULT SUMMARY (grouped / sorted by # of occurrences)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

FAULT: F0546, access , occurred: 568 , warning , topology/pod-1/node-102/sys/phys-[eth1/49]/phys/fault-F0546 , Port is down, reason:link-up-connected(disabled), used by:Discovery , 2023-11-26T01:07:00.660+00:00 , interface-physical-down
FAULT: F0467, tenant , occurred: 43 , minor , topology/pod-1/node-102/local/svc-policyelem-id-0/uni/epp/fv-[uni/tn-SDCJK/ap-SALES-APPLICATION/epg-SALES-FE]/node-102/stpathatt-[eth1/20]/nwissues/fault-F0467 , Configuration failed for uni/tn-SDCJK/ap-SALES-APPLICATION/epg-SALES-FE node 102 eth1/20 due to Invalid Path Configuration, debug message: invalid-path: Either the EpG/L3Out is not associated with a domain or the domain does not have this interface assigned to it; , 2023-11-27T06:04:51.634+00:00 , configuration-failed
FAULT: F96976, infra , occurred: 20 , warning , topology/pod-1/node-102/sys/phys-[eth1/49]/fault-F96976 , TCA: Egress Error Drop Packets rate(eqptEgrDropPkts5min:errorRate) value 13 raised above threshold 10 , 2023-11-26T03:45:00.631+00:00 , threshold-crossed
FAULT: F96760, infra , occurred: 20 , warning , topology/pod-1/node-102/sys/phys-[eth1/49]/fault-F96760 , TCA: Egress Buffer Drop Packets rate(eqptEgrDropPkts5min:bufferRate) value 10 raised above threshold 10 , 2023-11-26T03:45:00.639+00:00 , threshold-crossed
FAULT: F100480, infra , occurred: 20 , warning , topology/pod-1/node-102/sys/phys-[eth1/49]/fault-F100480 , TCA: Ingress Error Drop Packets rate(eqptIngrDropPkts5min:errorRate) value 22 raised above threshold 10 , 2023-11-26T03:45:00.641+00:00 , threshold-crossed
FAULT: F100264, infra , occurred: 20 , warning , topology/pod-1/node-102/sys/phys-[eth1/49]/fault-F100264 , TCA: Ingress Buffer Drop Packets rate(eqptIngrDropPkts5min:bufferRate) value 10 raised above threshold 10 , 2023-11-26T03:45:00.642+00:00 , threshold-crossed
FAULT: F100696, infra , occurred: 20 , warning , topology/pod-1/node-102/sys/phys-[eth1/49]/fault-F100696 , TCA: Ingress Forwarding Drop Packets rate(eqptIngrDropPkts5min:forwardingRate) value 11 raised above threshold 10 , 2023-11-26T03:45:00.644+00:00 , threshold-crossed
FAULT: F381328, infra , occurred: 20 , warning , topology/pod-1/node-102/sys/phys-[eth1/49]/fault-F381328 , TCA: CRC Align Errors current value(eqptIngrErrPkts5min:crcLast) value 1% raised above threshold 0% and value is recovering  , 2023-11-26T03:46:02.671+00:00 , threshold-crossed
FAULT: F110176, infra , occurred: 19 , warning , topology/pod-1/node-102/sys/ctx-[vxlan-2195457]/bd-[vxlan-16154557]/svi-[vlan6]/fault-F110176 , TCA: ingress drop bytes rate(l2IngrBytes5min:dropRate) value 24202 raised above threshold 10000 , 2023-11-27T06:05:33.221+00:00 , threshold-crossed
FAULT: F112128, infra , occurred: 19 , warning , topology/pod-1/node-102/sys/ctx-[vxlan-2195457]/bd-[vxlan-16154557]/svi-[vlan6]/fault-F112128 , TCA: ingress drop packets rate(l2IngrPkts5min:dropRate) value 10 raised above threshold 10 , 2023-11-27T06:05:33.223+00:00 , threshold-crossed
FAULT: F3083, tenant , occurred: 12 , major , uni/tn-MyTest/ap-default_APP/epg-EPG_1024/cep-4A:CD:BB:C0:00:00/ip-[2222::65:9]/fault-F3083 , In Tenant MyTest, 2222::65:9 is detected on multiple MACs (Context: 2555905. First 3 fvCEps: uni/tn-MyTest/ap-default_APP/epg-EPG_1025/cep-4A:CD:BB:C0:00:00; uni/tn-MyTest/ap-default_APP/epg-EPG_1026/cep-4A:CD:BB:C0:00:00; uni/tn-MyTest/ap-default_APP/epg-EPG_1027/cep-4A:CD:BB:C0:00:00;). This can happen when different MACs                     are sourcing traffic with the same IP due to NIC teaming, duplicate external IP configuration on external devices, ect. , 2023-11-27T14:58:57.941+00:00 , config-error
FAULT: F110344, infra , occurred: 9 , warning , topology/pod-1/node-102/local/svc-policyelem-id-0/uni/epp/fv-[uni/tn-MyTest/ap-default_APP/epg-EPG_1025]/fault-F110344 , TCA: ingress drop bytes rate(l2IngrBytesPart5min:dropRate) value 12479 raised above threshold 10000 , 2023-11-27T14:59:03.167+00:00 , threshold-crossed
FAULT: F112296, infra , occurred: 9 , warning , topology/pod-1/node-102/local/svc-policyelem-id-0/uni/epp/fv-[uni/tn-MyTest/ap-default_APP/epg-EPG_1025]/fault-F112296 , TCA: ingress drop packets rate(l2IngrPktsPart5min:dropRate) value 11 raised above threshold 10 , 2023-11-27T14:59:03.169+00:00 , threshold-crossed
FAULT: F0756, infra , occurred: 5 , minor , uni/tn-PRIVATE/out-L3OUT_SIMILE_WIND/lnodep-L3OUT_SIMILE_WIND_nodeProfile/lifp-L3OUT_SIMILE_WIND_vpcIpv4/bfdIfP/rsIfPol/fault-F0756 , Could not resolve the target bfdIfPol-BFD_l3out_WIND to form a named relationship. Using a default target uni/tn-common/bfdIfPol-default instead , 2023-11-27T14:51:19.733+00:00 , resolution-failed
FAULT: F0523, tenant , occurred: 4 , minor , uni/tn-ALI-TEST/ap-Production_AP/epg-BACKEND_BD_EPG/fault-F0523 , Configuration failed for EPG BACKEND_BD_EPG due to BD IDs Not Allocated,Context Not present. Configure Context and/or attach Context to Bridge-group domain for application EPG or attach Context to ESG for security group. , 2023-11-26T21:52:37.811+00:00 , configuration-failed
FAULT: F0974, infra , occurred: 3 , warning , uni/tn-PRIVATE/out-L3OUT_SIMILE_WIND/instP-Ext_EPG_SIMILE_WIND/rsprov-permit-all/fault-F0974 , Failed to form relation to MO brc-permit-all of class vzBrCP in context  , 2023-11-27T14:51:19.746+00:00 , resolution-failed
FAULT: F1295, infra , occurred: 3 , minor , topology/pod-1/node-102/local/svc-policyelem-id-0/uni/fabric/time-default/issues/fault-F1295 , Datetime Policy Configuration for default  failed due to : provider-is-not-present-under-default-pol , 2023-11-27T10:46:15.171+00:00 , configuration-failed
FAULT: F1298, tenant , occurred: 2 , minor , uni/epp/oob-[uni/tn-mgmt/mgmtp-default/oob-default]/node-103/polDelSt/fault-F1298 , For tenant mgmt, management profile default, deployment of out-band EPG default failed on node 103. Reason Node Cannot Deploy EPG , 2023-11-28T02:42:56.761+00:00 , configuration-failed
FAULT: F0952, infra , occurred: 2 , warning , uni/tn-ALI-TEST/BD-Production-Segment-BACKEND_BD/rsctx/fault-F0952 , Failed to form relation to MO ctx-Management of class fvCtx in context  , 2023-11-26T21:52:22.904+00:00 , resolution-failed
FAULT: F1021, infra , occurred: 2 , warning , uni/tn-PRIVATE/out-L3OUT_SIMILE_WIND/rsl3DomAtt/fault-F1021 , Failed to form relation to MO uni/l3dom-DOMAIN_L3_TEST of class extnwDomP , 2023-11-27T14:51:19.632+00:00 , resolution-failed
FAULT: F1003, infra , occurred: 2 , warning , uni/infra/attentp-test1_aaep/rsdomP-[uni/phys-test1_phy]/fault-F1003 , Failed to form relation to MO uni/phys-test1_phy of class physDomP , 2023-11-27T03:09:40.208+00:00 , resolution-failed
FAULT: F1299, infra , occurred: 2 , minor , uni/infra/nodecfgcont/node-1102/rsinterfacePolProfile-[uni/infra/accportprof-1-1102-1-31]/source-[uni/infra/nprof-1-1102-1-31]/fault-F1299 , Interface profile configuration 1-1102-1-31 has not been deployed on node 1102 because: Node Not Leaf For Infra Policies , 2023-11-26T10:33:05.178+00:00 , configuration-failed
FAULT: F1014, infra , occurred: 2 , warning , uni/infra/nprof-LEAF-102-PROFILE/rsaccPortP-[uni/infra/accportprof-L102]/fault-F1014 , Failed to form relation to MO uni/infra/accportprof-L102 of class infraAccPortP , 2023-11-28T04:04:13.234+00:00 , resolution-failed
FAULT: F0981, infra , occurred: 2 , warning , uni/phys-test9PhyDom/rsvlanNs/fault-F0981 , Failed to form relation to MO uni/infra/vlanns-[test9VlanPool]-dynamic of class fvnsVlanInstP , 2023-11-27T05:31:48.741+00:00 , resolution-failed
FAULT: F0989, infra , occurred: 2 , warning , uni/infra/funcprof/accportgrp-1-1102-1-31/rsattEntP/fault-F0989 , Failed to form relation to MO uni/infra/attentp-allvlans of class infraAttEntityP , 2023-11-27T05:43:30.903+00:00 , resolution-failed
FAULT: F1318, infra , occurred: 2 , major , topology/pod-1/node-102/sys/ops/psgp/fault-F1318 , No psu detected, look at the fault description! , 2023-11-26T01:06:52.596+00:00 , equipment-psu-missing
FAULT: F1360, access , occurred: 2 , warning , topology/pod-1/node-102/sys/coop/inst/dom-overlay-1/oracle-[10.0.248.1/32]/fault-F1360 , Operational state is down, reason:Unspecified , 2023-11-26T03:44:51.629+00:00 , protocol-coop-adjacency-down
FAULT: F1207, access , occurred: 2 , warning , topology/pod-1/node-101/sys/arp/inst/dom-overlay-1/if-[vlan14]/sadj-[10.0.0.1/32]/fault-F1207 , Static ARP adjacency is Down, reason Up , 2023-11-26T01:08:07.830+00:00 , protocol-arp-adjacency-down
FAULT: F3057, external , occurred: 1 , warning , licensecont/manager/fault-F3057 , APIC smart licensing network settings are not configured. Navigate to System -> Smart Licensing window to configure network settings. , 2023-11-26T01:12:04.848+00:00 , network-settings-not-configured
FAULT: F0104, infra , occurred: 1 , critical , topology/pod-1/node-1/sys/caggr-[po1.1]/fault-F0104 , Bond Interface po1.1 on node 1 of fabric ACI Fabric1 with hostname apic1 is now down , 2023-11-27T17:11:13.423+00:00 , port-down
FAULT: F1527, infra , occurred: 1 , warning , topology/pod-1/node-1/sys/ch/p-[/tmp/iFabric]-f-[tmpfs]/fault-F1527 , Storage unit /tmp/iFabric on Node 1 mounted at /tmp/iFabric is 75% full , 2023-11-28T04:07:26.362+00:00 , equipment-full
FAULT: F2069, tenant , occurred: 1 , minor , uni/tn-ACCTEST/out-test/lnodep-test_nodeProfile/fault-F2069 , Configuration failed for Tenant ACCTEST, Name test, Logical Node Profile test_nodeProfile due to RsNodeL3OutAtt RsPathL3OutAtt Misconfiguration , 2023-11-27T13:01:47.631+00:00 , configuration-failed
FAULT: F3571, infra , occurred: 1 , warning , uni/tn-PRIVATE/out-L3OUT_SIMILE_WIND/lnodep-L3OUT_SIMILE_WIND_nodeProfile/lifp-L3OUT_SIMILE_WIND_vpcIpv4/rspathL3OutAtt-[topology/pod-1/protpaths-101-102/pathep-[VPC_POLICY_GROUP_path_2]]/peerP-[10.10.10.4]/rspeerToProfile-[uni/tn-PRIVATE/prof-export_vs_firewall]-export/fault-F3571 , Failed to form relation to MO uni/tn-PRIVATE/prof-export_vs_firewall of class rtctrlProfile , 2023-11-27T14:51:19.719+00:00 , resolution-failed
FAULT: F1020, infra , occurred: 1 , warning , uni/tn-PRIVATE/out-L3OUT_SIMILE_WIND/rsectx/fault-F1020 , Failed to form relation to MO ctx-VRF-01 of class fvCtx in context  , 2023-11-27T14:51:19.764+00:00 , resolution-failed
FAULT: F1127, infra , occurred: 1 , warning , uni/tn-DEV/brc-vzAny/subj-icmp/rssubjFiltAtt-icmp/fault-F1127 , Failed to form relation to MO flt-icmp of class vzFilter in context  , 2023-11-28T04:43:19.868+00:00 , resolution-failed
FAULT: F1012, infra , occurred: 1 , warning , uni/infra/nprof-LEAF_101/leaves-LEAF-101-typ-range/rsaccNodePGrp/fault-F1012 , Failed to form relation to MO uni/infra/funcprof/accnodepgrp-STP of class infraAccNodePGrp , 2023-11-27T21:25:15.063+00:00 , resolution-failed
FAULT: F0967, infra , occurred: 1 , warning , uni/tn-PROD/ap-PROD_ANP/epg-TEST-1_EPG/rscons-TEST_CRT/fault-F0967 , Failed to form relation to MO brc-TEST_CRT of class vzBrCP in context  , 2023-11-26T16:02:43.845+00:00 , resolution-failed
FAULT: F4308, tenant , occurred: 1 , warning , uni/tn-DSP/ctx-DSP_vrf/ctxrtsummpol-DSP_RSP/fault-F4308 , Rtsum subnet or Node is missing in Route Summarization policy , 2023-11-27T06:07:55.264+00:00 , config-error
FAULT: F110473, infra , occurred: 1 , warning , uni/tn-MyTest/ctx-KA_VN/fault-F110473 , TCA: ingress drop bytes rate(l2IngrBytesAg15min:dropRate) value 154475 raised above threshold 100000 , 2023-11-27T15:04:04.558+00:00 , threshold-crossed
FAULT: F112425, infra , occurred: 1 , warning , uni/tn-MyTest/ctx-KA_VN/fault-F112425 , TCA: ingress drop packets rate(l2IngrPktsAg15min:dropRate) value 164 raised above threshold 100 , 2023-11-27T15:04:04.566+00:00 , threshold-crossed
FAULT: F0956, infra , occurred: 1 , warning , uni/tn-APP1_Tn/ap-APP1_Prod_AppProfile/epg-APP1_Prod_WebServers_EPG/rsdomAtt-[uni/phys-APP1_PhysDom]/fault-F0956 , Failed to form relation to MO uni/phys-APP1_PhysDom of class infraDomP , 2023-11-28T03:11:09.232+00:00 , resolution-failed
FAULT: F0849, infra , occurred: 1 , minor , topology/pod-1/node-101/local/svc-policyelem-id-0/uni/infra/accportprof-SDCJK-INTERFACE-PROFILE-1/hports-E40-41-PC-BE-SERVERS-typ-range/selectorissues/fault-F0849 , Configuration failed due to port-configured-for-apic for uni/infra/accportprof-SDCJK-INTERFACE-PROFILE-1/hports-E40-41-PC-BE-SERVERS-typ-range for node Leaf-1 , 2023-11-27T06:26:36.242+00:00 , configuration-failed
FAULT: F0416, infra , occurred: 1 , major , topology/pod-1/node-101/sys/extch-101/extchslot-1/extchc/fault-F0416 , Unknown software card id for node Leaf-1 with ID 101 , 2023-11-26T08:19:26.525+00:00 , equipment-card-failed
FAULT: F103824, infra , occurred: 1 , critical , topology/pod-1/node-201/sys/ch/scslot-6/sc/sensor-1/fault-F103824 , TCA: normalized temperature current value(eqptTemp5min:normalizedLast) value 68% fell below threshold 80% , 2023-11-26T03:45:56.518+00:00 , threshold-crossed

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++