22 Feb 2021

#### Service IP advertisement - improved!

Calico has been capable of advertising Service ClusterIPs and ExternalIPs for some time now, extending Kubernetes service reachability outside of your cluster. Building upon that, Calico v3.18.0 supports [advertising Service LoadBalancer IP addresses](https://docs.projectcalico.org/archive/v3.18/networking/advertise-service-ips#advertise-service-load-balancer-ip-addresses){: data-proofer-ignore=""} as well.

With this, you can now use an external IP allocator for LoadBalancer type Services (for example, MetalLB) and Calico will advertise those addresses into your BGP infrastructure so they can be reached by the outside world.

In addition to LoadBalancer IP support, Calico now respects the “node.kubernetes.io/exclude-from-external-load-balancers” label, which allows control over which nodes in the cluster should advertise service IPs.

**References**:
 - Add service loadbalancer IP advertisement option to BGPConfiguration API. [libcalico-go #1349](https://github.com/projectcalico/libcalico-go/pull/1349) (@caseydavenport)
 - Ability to advertise LoadBalancer IPs over BGP [confd #422](https://github.com/projectcalico/confd/pull/422) (@caseydavenport)
 - Respect label for excluding nodes from service advertisement. [confd #420](https://github.com/projectcalico/confd/pull/420) (@caseydavenport)

#### eBPF data plane support for HostEndpoints

In Calico v3.18.0, the eBPF data plane now has support for Calico’s host protection capabilities via the use of HostEndpoints and GlobalNetworkPolicy. Normal, pre-DNAT and apply-on-forward policy types are implemented. Untracked policy is not yet implemented.  As part of this work we were able to make the upgrade process from iptables to eBPF mode much smoother; existing connections should no longer be disrupted (but they'll need to be restarted to take advantage of eBPF mode).

**References**:
- eBPF mode now supports host endpoints.  Wildcard host endpoints, pre-DNAT policy and apply on forward policy is supported but doNotTrack policy is not yet supported.  Wildcard host endpoints do not apply to the loopback device in eBPF mode.  This change also adds IPIP and VXLAN bypass and policing and failsafe ports, bringing the eBPF dataplane into line with iptables. [felix #2664](https://github.com/projectcalico/felix/pull/2664) (@fasaxc)

#### IPAM Prometheus Metrics

Calico’s IP address management system powers a number of important capabilities. In Calico v3.18, we’ve introduced some simple Prometheus metrics to enable closer monitoring of Calico’s IPAM behavior. For instance, active IP allocations, IP blocks, and borrowed IP addresses on a per-node basis.

For more information, see the [kube-controllers prometheus metrics documentation](https://docs.projectcalico.org/archive/v3.18/reference/kube-controllers/prometheus){: data-proofer-ignore=""}.

**References**:
 - Add add field PrometheusMetricsPort to KubeControllersConfiguration [libcalico-go #1362](https://github.com/projectcalico/libcalico-go/pull/1362) (@caseydavenport)
 - Expose Prometheus metrics for IP allocation state [kube-controllers #607](https://github.com/projectcalico/kube-controllers/pull/607) (@caseydavenport)

#### Operator improvements

The version of the operator that ships with Calico v3.18.0 now includes support for the ImageSet resource, which allows you to specify precise image digests for each Calico component image.

#### Bug fixes

**General:**
 - On non-AMD64 platforms, fix that confd was unable to restart bird. [node #652](https://github.com/projectcalico/node/pull/652) (@halfcrazy)
 - Fix that Typha was unable to deduplicate Calico v3 Node resources.  This caused excessive CPU usage in Felix. [typha #449](https://github.com/projectcalico/typha/pull/449) (@fasaxc)
 - Mark aws-src-dst-check report ready when successful. This fix addresses the issue with calico-node not being ready in EKS setup using Calico CNI. [felix #2592](https://github.com/projectcalico/felix/pull/2592) (@realgaurav)
 - Bugfix: HostEndpoint policy was incorrectly enforced on connections from a pod to a service IP, even when that service IP was resolved to another workload on the same host. [felix #2582](https://github.com/projectcalico/felix/pull/2582) (@neiljerram)
 - Update Kubernetes client libraries to fix hung connections to the Kubernetes API server. [libcalico-go #1356](https://github.com/projectcalico/libcalico-go/pull/1356) (@caseydavenport)
 - Automatically delete HostEndpoints for deleted nodes. [libcalico-go #1355](https://github.com/projectcalico/libcalico-go/pull/1355) (@caseydavenport)
 - Remove bgppeer KeepOriginalNextHop default to allow the CRD to apply cleanly in upgrades. [libcalico-go #1353](https://github.com/projectcalico/libcalico-go/pull/1353) (@lwr20)
 - Fix releasing of improperly formatted handles as a result of host-local IPAM migration bug. [libcalico-go #1345](https://github.com/projectcalico/libcalico-go/pull/1345) (@caseydavenport)
 - Fix that IP pools with blockSize of /32 did not function properly [confd #330](https://github.com/projectcalico/confd/pull/330) (@halfcrazy)
 - Fix a bug that prevented datastore readiness from being reset during IPAM migration. [cni-plugin #1013](https://github.com/projectcalico/cni-plugin/pull/1013) (@mgleung)
 - Fix an IPAM auto assignment bug that caused IP addresses to be leaked in dual stack environments when assignment partially succeeded. [cni-plugin #1010](https://github.com/projectcalico/cni-plugin/pull/1010) (@mgleung)
 - Fix bug that prevented pods from being torn down in some situations [cni-plugin #1008](https://github.com/projectcalico/cni-plugin/pull/1008) (@caseydavenport)
 - Fix a bug that made the UPDATE_CNI_BINARIES environment variable not behave as expected. [cni-plugin #979](https://github.com/projectcalico/cni-plugin/pull/979) (@mgleung)
 - Fix potential memory-leak in kube-controllers [kube-controllers #594](https://github.com/projectcalico/kube-controllers/pull/594) (@duanjiong)

**eBPF data plane:**

 - Fix that, in eBPF mode, a Log rule would result in an error instead of being ignored.  Log rules are not supported but they should be ignored, not cause a failure. [felix #2684](https://github.com/projectcalico/felix/pull/2684) (@fasaxc)
 - In eBPF mode, enable FIB lookup acceleration in VXLAN mode.  This should improve performance in VXLAN mode.  Previously, this was disabled whenever an overlay was enabled but that was due to a problem with IPIP so we can safely enable the FIB with VXLAN mode. [felix #2669](https://github.com/projectcalico/felix/pull/2669) (@fasaxc)
 - In BPF tunnel mode, fix that connections to node ports that are backed by host-networked pods would be returned as DSR packets instead of being returned over the tunnel.  This caused AWS NLBs to fail unless the chosen host also hosted the backing pod. [felix #2620](https://github.com/projectcalico/felix/pull/2620) (@tomastigera)
 - In eBPF mode, fix that node port conntrack entries could be improperly cleaned up if there was no traffic for some time.  In particular, long-lived connections from AWS load balancers were affected. [felix #2588](https://github.com/projectcalico/felix/pull/2588) (@tomastigera)
 - In eBPF mode, return nodeport traffic from the backing pod to the same interface that it arrived at.  This prevents incorrect routing in environments like EKS, which use source-based routing. [felix #2549](https://github.com/projectcalico/felix/pull/2549) (@tomastigera)

**Windows:**
 - Fix network id comparison for Windows OS 2004 [cni-plugin #1020](https://github.com/projectcalico/cni-plugin/pull/1020) (@song-jiang)
 - Remove host route on endpoint with vxlan [cni-plugin #1011](https://github.com/projectcalico/cni-plugin/pull/1011) (@song-jiang)

#### Other changes

**General:**
 - For improved security, the Typha/Felix API now requires TLS v1.2+. [typha #457](https://github.com/projectcalico/typha/pull/457) (@fasaxc)
 - Address monitor now respects the CALICO_STARTUP_LOGLEVEL environment variable [node #770](https://github.com/projectcalico/node/pull/770) (@ottosulin)
 - calico/node for ARM64 now supports NFT mode [node #755](https://github.com/projectcalico/node/pull/755) (@caseydavenport)
 - calico/node will attempt to auto-discover the correct subnet information if the given IP does not include a subnet. [node #593](https://github.com/projectcalico/node/pull/593) (@njuptlzf)
 - Typha now exports a discovery API for other components to use. [typha #462](https://github.com/projectcalico/typha/pull/462) (@fasaxc)
 - Felix now uses the common discovery logic from the Typha repo. [felix #2613](https://github.com/projectcalico/felix/pull/2613) (@fasaxc)
 - Add an info log summarisation system to felix to reduce the volume of "background" dataplane resync logs. Log one summary log every 60s rater than logging every resync from every component. [felix #2605](https://github.com/projectcalico/felix/pull/2605) (@fasaxc)
 - `calicoctl` now supports multiple kubeconfig files via KUBECONFIG env var, just as `kubectl` would:
 - Support for K8S_CURRENT_CONTEXT [libcalico-go #1335](https://github.com/projectcalico/libcalico-go/pull/1335) (@frozenprocess)
 - The CNI plugin now logs at info level by default, this makes it much easier to diagnose IPAM related issues, for example. [cni-plugin #1001](https://github.com/projectcalico/cni-plugin/pull/1001) (@fasaxc)
 - Ensure the CNI plugin logs out an error to its own log rather than relying on the CNI client to log out final errors. [cni-plugin #1000](https://github.com/projectcalico/cni-plugin/pull/1000) (@fasaxc)
 - Serialise IPAM updates using a file lock on disk in order to reduce load on the API server. Concurrent updates are almost always to the same block so they only contend rather than improving scale. [cni-plugin #985](https://github.com/projectcalico/cni-plugin/pull/985) (@fasaxc)
 - kube-controllers runs a non-root by default [kube-controllers #565](https://github.com/projectcalico/kube-controllers/pull/565) (@caseydavenport)
 - Allow configuring the location of the kube-controllers status file [kube-controllers #548](https://github.com/projectcalico/kube-controllers/pull/548) (@johanfleury)
 - Add helm v3 chart to GitHub release artifacts [calico #4365](https://github.com/projectcalico/calico/pull/4365) (@lwr20)
 - A `BGPPeer` resource can now be configured not to specify an explicit source address, which may be helpful for multi-homed nodes.  See `sourceAddress` at https://docs.projectcalico.org/reference/resources/bgppeer. [calico #4253](https://github.com/projectcalico/calico/pull/4253) (@neiljerram)

**eBPF data plane:**

 - In eBPF mode, generic IP protocols (i.e. everything apart from TCP, UDP and ICMP) are now supported in a limited fashion (similarly to iptables). Such protocols are connection-tracked based on IP and protocol number only. [felix #2666](https://github.com/projectcalico/felix/pull/2666) (@fasaxc)
 - The calico-bpf diagnostic command is now available in calico/node as "calico-node -bpf ..." [node #786](https://github.com/projectcalico/node/pull/786) (@fasaxc)
 - Fixes https://github.com/projectcalico/calico/issues/4242 [felix #2630](https://github.com/projectcalico/felix/pull/2630) (@tomastigera)
 - The eBPF programs are now built with clang flag -target bpf and they use libbpf for helper definitions.  This enables the correct features in clang and improves build stability. Thanks to @netoptimizer for his PR and guidance on how to do this correctly. [felix #2608](https://github.com/projectcalico/felix/pull/2608) (@fasaxc)
 - In eBPF mode, squash spammy log from conntrack cleanup. [felix #2585](https://github.com/projectcalico/felix/pull/2585) (@fasaxc)
 - In BPF mode, skip second conntrack lookup when flow is established.  Tighten up the RPF check to ensure this is safe. [felix #2520](https://github.com/projectcalico/felix/pull/2520) (@fasaxc)
