Merge "Merge a594533df0f6 ("Merge tag 'drm-next-2022-12-13' of git://anongit.freedesktop.org/drm/drm") into android-mainline" into android-mainlinediff --git a/Documentation/ABI/testing/sysfs-driver-intel-i915-hwmon b/Documentation/ABI/testing/sysfs-driver-intel-i915-hwmon
new file mode 100644
index 0000000..2d6a472
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-driver-intel-i915-hwmon
@@ -0,0 +1,75 @@
+What: /sys/devices/.../hwmon/hwmon<i>/in0_input
+Date: February 2023
+KernelVersion: 6.2
+Contact: intel-gfx@lists.freedesktop.org
+Description: RO. Current Voltage in millivolt.
+
+ Only supported for particular Intel i915 graphics platforms.
+
+What: /sys/devices/.../hwmon/hwmon<i>/power1_max
+Date: February 2023
+KernelVersion: 6.2
+Contact: intel-gfx@lists.freedesktop.org
+Description: RW. Card reactive sustained (PL1/Tau) power limit in microwatts.
+
+ The power controller will throttle the operating frequency
+ if the power averaged over a window (typically seconds)
+ exceeds this limit.
+
+ Only supported for particular Intel i915 graphics platforms.
+
+What: /sys/devices/.../hwmon/hwmon<i>/power1_rated_max
+Date: February 2023
+KernelVersion: 6.2
+Contact: intel-gfx@lists.freedesktop.org
+Description: RO. Card default power limit (default TDP setting).
+
+ Only supported for particular Intel i915 graphics platforms.
+
+What: /sys/devices/.../hwmon/hwmon<i>/power1_max_interval
+Date: February 2023
+KernelVersion: 6.2
+Contact: intel-gfx@lists.freedesktop.org
+Description: RW. Sustained power limit interval (Tau in PL1/Tau) in
+ milliseconds over which sustained power is averaged.
+
+ Only supported for particular Intel i915 graphics platforms.
+
+What: /sys/devices/.../hwmon/hwmon<i>/power1_crit
+Date: February 2023
+KernelVersion: 6.2
+Contact: intel-gfx@lists.freedesktop.org
+Description: RW. Card reactive critical (I1) power limit in microwatts.
+
+ Card reactive critical (I1) power limit in microwatts is exposed
+ for client products. The power controller will throttle the
+ operating frequency if the power averaged over a window exceeds
+ this limit.
+
+ Only supported for particular Intel i915 graphics platforms.
+
+What: /sys/devices/.../hwmon/hwmon<i>/curr1_crit
+Date: February 2023
+KernelVersion: 6.2
+Contact: intel-gfx@lists.freedesktop.org
+Description: RW. Card reactive critical (I1) power limit in milliamperes.
+
+ Card reactive critical (I1) power limit in milliamperes is
+ exposed for server products. The power controller will throttle
+ the operating frequency if the power averaged over a window
+ exceeds this limit.
+
+ Only supported for particular Intel i915 graphics platforms.
+
+What: /sys/devices/.../hwmon/hwmon<i>/energy1_input
+Date: February 2023
+KernelVersion: 6.2
+Contact: intel-gfx@lists.freedesktop.org
+Description: RO. Energy input of device or gt in microjoules.
+
+ For i915 device level hwmon devices (name "i915") this
+ reflects energy input for the entire device. For gt level
+ hwmon devices (name "i915_gtN") this reflects energy input
+ for the gt.
+
+ Only supported for particular Intel i915 graphics platforms.
diff --git a/Documentation/accel/index.rst b/Documentation/accel/index.rst
new file mode 100644
index 0000000..2b43c9a
--- /dev/null
+++ b/Documentation/accel/index.rst
@@ -0,0 +1,17 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+====================
+Compute Accelerators
+====================
+
+.. toctree::
+ :maxdepth: 1
+
+ introduction
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
diff --git a/Documentation/accel/introduction.rst b/Documentation/accel/introduction.rst
new file mode 100644
index 0000000..6f31af1
--- /dev/null
+++ b/Documentation/accel/introduction.rst
@@ -0,0 +1,110 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============
+Introduction
+============
+
+The Linux compute accelerators subsystem is designed to expose compute
+accelerators in a common way to user-space and provide a common set of
+functionality.
+
+These devices can be either stand-alone ASICs or IP blocks inside an SoC/GPU.
+Although these devices are typically designed to accelerate
+Machine-Learning (ML) and/or Deep-Learning (DL) computations, the accel layer
+is not limited to handling these types of accelerators.
+
+Typically, a compute accelerator will belong to one of the following
+categories:
+
+- Edge AI - doing inference at an edge device. It can be an embedded ASIC/FPGA,
+ or an IP inside a SoC (e.g. laptop web camera). These devices
+ are typically configured using registers and can work with or without DMA.
+
+- Inference data-center - single/multi user devices in a large server. This
+ type of device can be stand-alone or an IP inside a SoC or a GPU. It will
+ have on-board DRAM (to hold the DL topology), DMA engines and
+ command submission queues (either kernel or user-space queues).
+ It might also have an MMU to manage multiple users and might also enable
+ virtualization (SR-IOV) to support multiple VMs on the same device. In
+ addition, these devices will usually have some tools, such as profiler and
+ debugger.
+
+- Training data-center - Similar to Inference data-center cards, but typically
+ have more computational power and memory b/w (e.g. HBM) and will likely have
+ a method of scaling-up/out, i.e. connecting to other training cards inside
+ the server or in other servers, respectively.
+
+All these devices typically have different runtime user-space software stacks,
+that are tailored-made to their h/w. In addition, they will also probably
+include a compiler to generate programs to their custom-made computational
+engines. Typically, the common layer in user-space will be the DL frameworks,
+such as PyTorch and TensorFlow.
+
+Sharing code with DRM
+=====================
+
+Because this type of devices can be an IP inside GPUs or have similar
+characteristics as those of GPUs, the accel subsystem will use the
+DRM subsystem's code and functionality. i.e. the accel core code will
+be part of the DRM subsystem and an accel device will be a new type of DRM
+device.
+
+This will allow us to leverage the extensive DRM code-base and
+collaborate with DRM developers that have experience with this type of
+devices. In addition, new features that will be added for the accelerator
+drivers can be of use to GPU drivers as well.
+
+Differentiation from GPUs
+=========================
+
+Because we want to prevent the extensive user-space graphic software stack
+from trying to use an accelerator as a GPU, the compute accelerators will be
+differentiated from GPUs by using a new major number and new device char files.
+
+Furthermore, the drivers will be located in a separate place in the kernel
+tree - drivers/accel/.
+
+The accelerator devices will be exposed to the user space with the dedicated
+261 major number and will have the following convention:
+
+- device char files - /dev/accel/accel*
+- sysfs - /sys/class/accel/accel*/
+- debugfs - /sys/kernel/debug/accel/accel*/
+
+Getting Started
+===============
+
+First, read the DRM documentation at Documentation/gpu/index.rst.
+Not only it will explain how to write a new DRM driver but it will also
+contain all the information on how to contribute, the Code Of Conduct and
+what is the coding style/documentation. All of that is the same for the
+accel subsystem.
+
+Second, make sure the kernel is configured with CONFIG_DRM_ACCEL.
+
+To expose your device as an accelerator, two changes are needed to
+be done in your driver (as opposed to a standard DRM driver):
+
+- Add the DRIVER_COMPUTE_ACCEL feature flag in your drm_driver's
+ driver_features field. It is important to note that this driver feature is
+ mutually exclusive with DRIVER_RENDER and DRIVER_MODESET. Devices that want
+ to expose both graphics and compute device char files should be handled by
+ two drivers that are connected using the auxiliary bus framework.
+
+- Change the open callback in your driver fops structure to accel_open().
+ Alternatively, your driver can use DEFINE_DRM_ACCEL_FOPS macro to easily
+ set the correct function operations pointers structure.
+
+External References
+===================
+
+email threads
+-------------
+
+* `Initial discussion on the New subsystem for acceleration devices <https://lkml.org/lkml/2022/7/31/83>`_ - Oded Gabbay (2022)
+* `patch-set to add the new subsystem <https://lkml.org/lkml/2022/10/22/544>`_ - Oded Gabbay (2022)
+
+Conference talks
+----------------
+
+* `LPC 2022 Accelerators BOF outcomes summary <https://airlied.blogspot.com/2022/09/accelerators-bof-outcomes-summary.html>`_ - Dave Airlie (2022)
diff --git a/Documentation/admin-guide/devices.txt b/Documentation/admin-guide/devices.txt
index 9764d6ed..06c525e 100644
--- a/Documentation/admin-guide/devices.txt
+++ b/Documentation/admin-guide/devices.txt
@@ -3080,6 +3080,11 @@
...
255 = /dev/osd255 256th OSD Device
+ 261 char Compute Acceleration Devices
+ 0 = /dev/accel/accel0 First acceleration device
+ 1 = /dev/accel/accel1 Second acceleration device
+ ...
+
384-511 char RESERVED FOR DYNAMIC ASSIGNMENT
Character devices that request a dynamic allocation of major
number will take numbers starting from 511 and downward,
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 31c6747..7680d38 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3793,12 +3793,15 @@
shutdown the other cpus. Instead use the REBOOT_VECTOR
irq.
- nomodeset Disable kernel modesetting. DRM drivers will not perform
- display-mode changes or accelerated rendering. Only the
- system framebuffer will be available for use if this was
- set-up by the firmware or boot loader.
+ nomodeset Disable kernel modesetting. Most systems' firmware
+ sets up a display mode and provides framebuffer memory
+ for output. With nomodeset, DRM and fbdev drivers will
+ not load if they could possibly displace the pre-
+ initialized output. Only the system framebuffer will
+ be available for use. The respective drivers will not
+ perform display-mode changes or accelerated rendering.
- Useful as fallback, or for testing and debugging.
+ Useful as error fallback, or for testing and debugging.
nomodule Disable module load
diff --git a/Documentation/devicetree/bindings/display/allwinner,sun6i-a31-mipi-dsi.yaml b/Documentation/devicetree/bindings/display/allwinner,sun6i-a31-mipi-dsi.yaml
index 7910831..c731fbd 100644
--- a/Documentation/devicetree/bindings/display/allwinner,sun6i-a31-mipi-dsi.yaml
+++ b/Documentation/devicetree/bindings/display/allwinner,sun6i-a31-mipi-dsi.yaml
@@ -12,9 +12,14 @@
properties:
compatible:
- enum:
- - allwinner,sun6i-a31-mipi-dsi
- - allwinner,sun50i-a64-mipi-dsi
+ oneOf:
+ - enum:
+ - allwinner,sun6i-a31-mipi-dsi
+ - allwinner,sun50i-a64-mipi-dsi
+ - allwinner,sun50i-a100-mipi-dsi
+ - items:
+ - const: allwinner,sun20i-d1-mipi-dsi
+ - const: allwinner,sun50i-a100-mipi-dsi
reg:
maxItems: 1
@@ -59,7 +64,6 @@
- phys
- phy-names
- resets
- - vcc-dsi-supply
- port
allOf:
@@ -68,7 +72,9 @@
properties:
compatible:
contains:
- const: allwinner,sun6i-a31-mipi-dsi
+ enum:
+ - allwinner,sun6i-a31-mipi-dsi
+ - allwinner,sun50i-a100-mipi-dsi
then:
properties:
@@ -78,16 +84,22 @@
required:
- clock-names
+ else:
+ properties:
+ clocks:
+ maxItems: 1
+
- if:
properties:
compatible:
contains:
- const: allwinner,sun50i-a64-mipi-dsi
+ enum:
+ - allwinner,sun6i-a31-mipi-dsi
+ - allwinner,sun50i-a64-mipi-dsi
then:
- properties:
- clocks:
- minItems: 1
+ required:
+ - vcc-dsi-supply
unevaluatedProperties: false
diff --git a/Documentation/devicetree/bindings/display/bridge/renesas,dsi.yaml b/Documentation/devicetree/bindings/display/bridge/renesas,dsi.yaml
new file mode 100644
index 0000000..131d5b6
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/bridge/renesas,dsi.yaml
@@ -0,0 +1,182 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/bridge/renesas,dsi.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Renesas RZ/G2L MIPI DSI Encoder
+
+maintainers:
+ - Biju Das <biju.das.jz@bp.renesas.com>
+
+description: |
+ This binding describes the MIPI DSI encoder embedded in the Renesas
+ RZ/G2L alike family of SoC's. The encoder can operate in DSI mode, with
+ up to four data lanes.
+
+allOf:
+ - $ref: /schemas/display/dsi-controller.yaml#
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - renesas,r9a07g044-mipi-dsi # RZ/G2{L,LC}
+ - const: renesas,rzg2l-mipi-dsi
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ items:
+ - description: Sequence operation channel 0 interrupt
+ - description: Sequence operation channel 1 interrupt
+ - description: Video-Input operation channel 1 interrupt
+ - description: DSI Packet Receive interrupt
+ - description: DSI Fatal Error interrupt
+ - description: DSI D-PHY PPI interrupt
+ - description: Debug interrupt
+
+ interrupt-names:
+ items:
+ - const: seq0
+ - const: seq1
+ - const: vin1
+ - const: rcv
+ - const: ferr
+ - const: ppi
+ - const: debug
+
+ clocks:
+ items:
+ - description: DSI D-PHY PLL multiplied clock
+ - description: DSI D-PHY system clock
+ - description: DSI AXI bus clock
+ - description: DSI Register access clock
+ - description: DSI Video clock
+ - description: DSI D-PHY Escape mode transmit clock
+
+ clock-names:
+ items:
+ - const: pllclk
+ - const: sysclk
+ - const: aclk
+ - const: pclk
+ - const: vclk
+ - const: lpclk
+
+ resets:
+ items:
+ - description: MIPI_DSI_CMN_RSTB
+ - description: MIPI_DSI_ARESET_N
+ - description: MIPI_DSI_PRESET_N
+
+ reset-names:
+ items:
+ - const: rst
+ - const: arst
+ - const: prst
+
+ power-domains:
+ maxItems: 1
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Parallel input port
+
+ port@1:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description: DSI output port
+
+ properties:
+ endpoint:
+ $ref: /schemas/media/video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ properties:
+ data-lanes:
+ description: array of physical DSI data lane indexes.
+ minItems: 1
+ items:
+ - const: 1
+ - const: 2
+ - const: 3
+ - const: 4
+
+ required:
+ - data-lanes
+
+ required:
+ - port@0
+ - port@1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - interrupt-names
+ - clocks
+ - clock-names
+ - resets
+ - reset-names
+ - power-domains
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/r9a07g044-cpg.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ dsi0: dsi@10850000 {
+ compatible = "renesas,r9a07g044-mipi-dsi", "renesas,rzg2l-mipi-dsi";
+ reg = <0x10850000 0x20000>;
+ interrupts = <GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 144 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 145 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 146 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 147 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "seq0", "seq1", "vin1", "rcv",
+ "ferr", "ppi", "debug";
+ clocks = <&cpg CPG_MOD R9A07G044_MIPI_DSI_PLLCLK>,
+ <&cpg CPG_MOD R9A07G044_MIPI_DSI_SYSCLK>,
+ <&cpg CPG_MOD R9A07G044_MIPI_DSI_ACLK>,
+ <&cpg CPG_MOD R9A07G044_MIPI_DSI_PCLK>,
+ <&cpg CPG_MOD R9A07G044_MIPI_DSI_VCLK>,
+ <&cpg CPG_MOD R9A07G044_MIPI_DSI_LPCLK>;
+ clock-names = "pllclk", "sysclk", "aclk", "pclk", "vclk", "lpclk";
+ resets = <&cpg R9A07G044_MIPI_DSI_CMN_RSTB>,
+ <&cpg R9A07G044_MIPI_DSI_ARESET_N>,
+ <&cpg R9A07G044_MIPI_DSI_PRESET_N>;
+ reset-names = "rst", "arst", "prst";
+ power-domains = <&cpg>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&du_out_dsi0>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ data-lanes = <1 2 3 4>;
+ remote-endpoint = <&adv7535_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.yaml b/Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.yaml
index 5bb23e9..d976380 100644
--- a/Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.yaml
+++ b/Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.yaml
@@ -23,6 +23,7 @@
- mediatek,mt8173-dpi
- mediatek,mt8183-dpi
- mediatek,mt8186-dpi
+ - mediatek,mt8188-dp-intf
- mediatek,mt8192-dpi
- mediatek,mt8195-dp-intf
diff --git a/Documentation/devicetree/bindings/display/msm/dpu-common.yaml b/Documentation/devicetree/bindings/display/msm/dpu-common.yaml
new file mode 100644
index 0000000..8ffbc30
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/dpu-common.yaml
@@ -0,0 +1,52 @@
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/dpu-common.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display DPU common properties
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+ - Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
+ - Rob Clark <robdclark@gmail.com>
+
+description: |
+ Common properties for QCom DPU display controller.
+
+properties:
+ interrupts:
+ maxItems: 1
+
+ power-domains:
+ maxItems: 1
+
+ operating-points-v2: true
+ opp-table:
+ type: object
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+ description: |
+ Contains the list of output ports from DPU device. These ports
+ connect to interfaces that are external to the DPU hardware,
+ such as DSI, DP etc.
+
+ patternProperties:
+ "^port@[0-9a-f]+$":
+ $ref: /schemas/graph.yaml#/properties/port
+
+ # at least one port is required
+ required:
+ - port@0
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - clocks
+ - interrupts
+ - power-domains
+ - operating-points-v2
+ - ports
+
+additionalProperties: true
diff --git a/Documentation/devicetree/bindings/display/msm/dpu-msm8998.yaml b/Documentation/devicetree/bindings/display/msm/dpu-msm8998.yaml
deleted file mode 100644
index 253665c..0000000
--- a/Documentation/devicetree/bindings/display/msm/dpu-msm8998.yaml
+++ /dev/null
@@ -1,223 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
-%YAML 1.2
----
-$id: http://devicetree.org/schemas/display/msm/dpu-msm8998.yaml#
-$schema: http://devicetree.org/meta-schemas/core.yaml#
-
-title: Qualcomm Display DPU dt properties for MSM8998 target
-
-maintainers:
- - AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
-
-description: |
- Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
- sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
- bindings of MDSS and DPU are mentioned for MSM8998 target.
-
-properties:
- compatible:
- items:
- - const: qcom,msm8998-mdss
-
- reg:
- maxItems: 1
-
- reg-names:
- const: mdss
-
- power-domains:
- maxItems: 1
-
- clocks:
- items:
- - description: Display AHB clock
- - description: Display AXI clock
- - description: Display core clock
-
- clock-names:
- items:
- - const: iface
- - const: bus
- - const: core
-
- interrupts:
- maxItems: 1
-
- interrupt-controller: true
-
- "#address-cells": true
-
- "#size-cells": true
-
- "#interrupt-cells":
- const: 1
-
- iommus:
- items:
- - description: Phandle to apps_smmu node with SID mask for Hard-Fail port0
-
- ranges: true
-
-patternProperties:
- "^display-controller@[0-9a-f]+$":
- type: object
- description: Node containing the properties of DPU.
- additionalProperties: false
-
- properties:
- compatible:
- items:
- - const: qcom,msm8998-dpu
-
- reg:
- items:
- - description: Address offset and size for mdp register set
- - description: Address offset and size for regdma register set
- - description: Address offset and size for vbif register set
- - description: Address offset and size for non-realtime vbif register set
-
- reg-names:
- items:
- - const: mdp
- - const: regdma
- - const: vbif
- - const: vbif_nrt
-
- clocks:
- items:
- - description: Display ahb clock
- - description: Display axi clock
- - description: Display mem-noc clock
- - description: Display core clock
- - description: Display vsync clock
-
- clock-names:
- items:
- - const: iface
- - const: bus
- - const: mnoc
- - const: core
- - const: vsync
-
- interrupts:
- maxItems: 1
-
- power-domains:
- maxItems: 1
-
- operating-points-v2: true
- opp-table:
- type: object
-
- ports:
- $ref: /schemas/graph.yaml#/properties/ports
- description: |
- Contains the list of output ports from DPU device. These ports
- connect to interfaces that are external to the DPU hardware,
- such as DSI, DP etc. Each output port contains an endpoint that
- describes how it is connected to an external interface.
-
- properties:
- port@0:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF1 (DSI1)
-
- port@1:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF2 (DSI2)
-
- required:
- - port@0
- - port@1
-
- required:
- - compatible
- - reg
- - reg-names
- - clocks
- - interrupts
- - power-domains
- - operating-points-v2
- - ports
-
-required:
- - compatible
- - reg
- - reg-names
- - power-domains
- - clocks
- - interrupts
- - interrupt-controller
- - iommus
- - ranges
-
-additionalProperties: false
-
-examples:
- - |
- #include <dt-bindings/clock/qcom,mmcc-msm8998.h>
- #include <dt-bindings/interrupt-controller/arm-gic.h>
- #include <dt-bindings/power/qcom-rpmpd.h>
-
- mdss: display-subsystem@c900000 {
- compatible = "qcom,msm8998-mdss";
- reg = <0x0c900000 0x1000>;
- reg-names = "mdss";
-
- clocks = <&mmcc MDSS_AHB_CLK>,
- <&mmcc MDSS_AXI_CLK>,
- <&mmcc MDSS_MDP_CLK>;
- clock-names = "iface", "bus", "core";
-
- #address-cells = <1>;
- #interrupt-cells = <1>;
- #size-cells = <1>;
-
- interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-controller;
- iommus = <&mmss_smmu 0>;
-
- power-domains = <&mmcc MDSS_GDSC>;
- ranges;
-
- display-controller@c901000 {
- compatible = "qcom,msm8998-dpu";
- reg = <0x0c901000 0x8f000>,
- <0x0c9a8e00 0xf0>,
- <0x0c9b0000 0x2008>,
- <0x0c9b8000 0x1040>;
- reg-names = "mdp", "regdma", "vbif", "vbif_nrt";
-
- clocks = <&mmcc MDSS_AHB_CLK>,
- <&mmcc MDSS_AXI_CLK>,
- <&mmcc MNOC_AHB_CLK>,
- <&mmcc MDSS_MDP_CLK>,
- <&mmcc MDSS_VSYNC_CLK>;
- clock-names = "iface", "bus", "mnoc", "core", "vsync";
-
- interrupt-parent = <&mdss>;
- interrupts = <0>;
- operating-points-v2 = <&mdp_opp_table>;
- power-domains = <&rpmpd MSM8998_VDDMX>;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- dpu_intf1_out: endpoint {
- remote-endpoint = <&dsi0_in>;
- };
- };
-
- port@1 {
- reg = <1>;
- dpu_intf2_out: endpoint {
- remote-endpoint = <&dsi1_in>;
- };
- };
- };
- };
- };
-...
diff --git a/Documentation/devicetree/bindings/display/msm/dpu-qcm2290.yaml b/Documentation/devicetree/bindings/display/msm/dpu-qcm2290.yaml
deleted file mode 100644
index c5824e1..0000000
--- a/Documentation/devicetree/bindings/display/msm/dpu-qcm2290.yaml
+++ /dev/null
@@ -1,222 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
-%YAML 1.2
----
-$id: http://devicetree.org/schemas/display/msm/dpu-qcm2290.yaml#
-$schema: http://devicetree.org/meta-schemas/core.yaml#
-
-title: Qualcomm Display DPU dt properties for QCM2290 target
-
-maintainers:
- - Loic Poulain <loic.poulain@linaro.org>
-
-description: |
- Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
- sub-blocks like DPU display controller and DSI. Device tree bindings of MDSS
- and DPU are mentioned for QCM2290 target.
-
-properties:
- compatible:
- items:
- - const: qcom,qcm2290-mdss
-
- reg:
- maxItems: 1
-
- reg-names:
- const: mdss
-
- power-domains:
- maxItems: 1
-
- clocks:
- items:
- - description: Display AHB clock from gcc
- - description: Display AXI clock
- - description: Display core clock
-
- clock-names:
- items:
- - const: iface
- - const: bus
- - const: core
-
- interrupts:
- maxItems: 1
-
- interrupt-controller: true
-
- "#address-cells": true
-
- "#size-cells": true
-
- "#interrupt-cells":
- const: 1
-
- iommus:
- items:
- - description: Phandle to apps_smmu node with SID mask for Hard-Fail port0
- - description: Phandle to apps_smmu node with SID mask for Hard-Fail port1
-
- ranges: true
-
- interconnects:
- items:
- - description: Interconnect path specifying the port ids for data bus
-
- interconnect-names:
- const: mdp0-mem
-
- resets:
- items:
- - description: MDSS_CORE reset
-
-patternProperties:
- "^display-controller@[0-9a-f]+$":
- type: object
- description: Node containing the properties of DPU.
- additionalProperties: false
-
- properties:
- compatible:
- items:
- - const: qcom,qcm2290-dpu
-
- reg:
- items:
- - description: Address offset and size for mdp register set
- - description: Address offset and size for vbif register set
-
- reg-names:
- items:
- - const: mdp
- - const: vbif
-
- clocks:
- items:
- - description: Display AXI clock from gcc
- - description: Display AHB clock from dispcc
- - description: Display core clock from dispcc
- - description: Display lut clock from dispcc
- - description: Display vsync clock from dispcc
-
- clock-names:
- items:
- - const: bus
- - const: iface
- - const: core
- - const: lut
- - const: vsync
-
- interrupts:
- maxItems: 1
-
- power-domains:
- maxItems: 1
-
- operating-points-v2: true
- opp-table:
- type: object
-
- ports:
- $ref: /schemas/graph.yaml#/properties/ports
- description: |
- Contains the list of output ports from DPU device. These ports
- connect to interfaces that are external to the DPU hardware,
- such as DSI. Each output port contains an endpoint that
- describes how it is connected to an external interface.
-
- properties:
- port@0:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF1 (DSI1)
-
- required:
- - port@0
-
- required:
- - compatible
- - reg
- - reg-names
- - clocks
- - interrupts
- - power-domains
- - operating-points-v2
- - ports
-
-required:
- - compatible
- - reg
- - reg-names
- - power-domains
- - clocks
- - interrupts
- - interrupt-controller
- - iommus
- - ranges
-
-additionalProperties: false
-
-examples:
- - |
- #include <dt-bindings/clock/qcom,dispcc-qcm2290.h>
- #include <dt-bindings/clock/qcom,gcc-qcm2290.h>
- #include <dt-bindings/interrupt-controller/arm-gic.h>
- #include <dt-bindings/interconnect/qcom,qcm2290.h>
- #include <dt-bindings/power/qcom-rpmpd.h>
-
- mdss: mdss@5e00000 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "qcom,qcm2290-mdss";
- reg = <0x05e00000 0x1000>;
- reg-names = "mdss";
- power-domains = <&dispcc MDSS_GDSC>;
- clocks = <&gcc GCC_DISP_AHB_CLK>,
- <&gcc GCC_DISP_HF_AXI_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>;
- clock-names = "iface", "bus", "core";
-
- interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-controller;
- #interrupt-cells = <1>;
-
- interconnects = <&mmrt_virt MASTER_MDP0 &bimc SLAVE_EBI1>;
- interconnect-names = "mdp0-mem";
-
- iommus = <&apps_smmu 0x420 0x2>,
- <&apps_smmu 0x421 0x0>;
- ranges;
-
- mdss_mdp: display-controller@5e01000 {
- compatible = "qcom,qcm2290-dpu";
- reg = <0x05e01000 0x8f000>,
- <0x05eb0000 0x2008>;
- reg-names = "mdp", "vbif";
-
- clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
- <&dispcc DISP_CC_MDSS_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
- <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
- clock-names = "bus", "iface", "core", "lut", "vsync";
-
- operating-points-v2 = <&mdp_opp_table>;
- power-domains = <&rpmpd QCM2290_VDDCX>;
-
- interrupt-parent = <&mdss>;
- interrupts = <0>;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- dpu_intf1_out: endpoint {
- remote-endpoint = <&dsi0_in>;
- };
- };
- };
- };
- };
-...
diff --git a/Documentation/devicetree/bindings/display/msm/dpu-sc7180.yaml b/Documentation/devicetree/bindings/display/msm/dpu-sc7180.yaml
deleted file mode 100644
index 4890bc2..0000000
--- a/Documentation/devicetree/bindings/display/msm/dpu-sc7180.yaml
+++ /dev/null
@@ -1,235 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
-%YAML 1.2
----
-$id: http://devicetree.org/schemas/display/msm/dpu-sc7180.yaml#
-$schema: http://devicetree.org/meta-schemas/core.yaml#
-
-title: Qualcomm Display DPU dt properties for SC7180 target
-
-maintainers:
- - Krishna Manikandan <quic_mkrishn@quicinc.com>
-
-description: |
- Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
- sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
- bindings of MDSS and DPU are mentioned for SC7180 target.
-
-properties:
- compatible:
- items:
- - const: qcom,sc7180-mdss
-
- reg:
- maxItems: 1
-
- reg-names:
- const: mdss
-
- power-domains:
- maxItems: 1
-
- clocks:
- items:
- - description: Display AHB clock from gcc
- - description: Display AHB clock from dispcc
- - description: Display core clock
-
- clock-names:
- items:
- - const: iface
- - const: ahb
- - const: core
-
- interrupts:
- maxItems: 1
-
- interrupt-controller: true
-
- "#address-cells": true
-
- "#size-cells": true
-
- "#interrupt-cells":
- const: 1
-
- iommus:
- items:
- - description: Phandle to apps_smmu node with SID mask for Hard-Fail port0
-
- ranges: true
-
- interconnects:
- items:
- - description: Interconnect path specifying the port ids for data bus
-
- interconnect-names:
- const: mdp0-mem
-
- resets:
- items:
- - description: MDSS_CORE reset
-
-patternProperties:
- "^display-controller@[0-9a-f]+$":
- type: object
- description: Node containing the properties of DPU.
- additionalProperties: false
-
- properties:
- compatible:
- items:
- - const: qcom,sc7180-dpu
-
- reg:
- items:
- - description: Address offset and size for mdp register set
- - description: Address offset and size for vbif register set
-
- reg-names:
- items:
- - const: mdp
- - const: vbif
-
- clocks:
- items:
- - description: Display hf axi clock
- - description: Display ahb clock
- - description: Display rotator clock
- - description: Display lut clock
- - description: Display core clock
- - description: Display vsync clock
-
- clock-names:
- items:
- - const: bus
- - const: iface
- - const: rot
- - const: lut
- - const: core
- - const: vsync
-
- interrupts:
- maxItems: 1
-
- power-domains:
- maxItems: 1
-
- operating-points-v2: true
- opp-table:
- type: object
-
- ports:
- $ref: /schemas/graph.yaml#/properties/ports
- description: |
- Contains the list of output ports from DPU device. These ports
- connect to interfaces that are external to the DPU hardware,
- such as DSI, DP etc. Each output port contains an endpoint that
- describes how it is connected to an external interface.
-
- properties:
- port@0:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF1 (DSI1)
-
- port@2:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF0 (DP)
-
- required:
- - port@0
-
- required:
- - compatible
- - reg
- - reg-names
- - clocks
- - interrupts
- - power-domains
- - operating-points-v2
- - ports
-
-required:
- - compatible
- - reg
- - reg-names
- - power-domains
- - clocks
- - interrupts
- - interrupt-controller
- - iommus
- - ranges
-
-additionalProperties: false
-
-examples:
- - |
- #include <dt-bindings/clock/qcom,dispcc-sc7180.h>
- #include <dt-bindings/clock/qcom,gcc-sc7180.h>
- #include <dt-bindings/interrupt-controller/arm-gic.h>
- #include <dt-bindings/interconnect/qcom,sdm845.h>
- #include <dt-bindings/power/qcom-rpmpd.h>
-
- display-subsystem@ae00000 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "qcom,sc7180-mdss";
- reg = <0xae00000 0x1000>;
- reg-names = "mdss";
- power-domains = <&dispcc MDSS_GDSC>;
- clocks = <&gcc GCC_DISP_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>;
- clock-names = "iface", "ahb", "core";
-
- interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-controller;
- #interrupt-cells = <1>;
-
- interconnects = <&mmss_noc MASTER_MDP0 &mc_virt SLAVE_EBI1>;
- interconnect-names = "mdp0-mem";
-
- iommus = <&apps_smmu 0x800 0x2>;
- ranges;
-
- display-controller@ae01000 {
- compatible = "qcom,sc7180-dpu";
- reg = <0x0ae01000 0x8f000>,
- <0x0aeb0000 0x2008>;
-
- reg-names = "mdp", "vbif";
-
- clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
- <&dispcc DISP_CC_MDSS_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_ROT_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>,
- <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
- clock-names = "bus", "iface", "rot", "lut", "core",
- "vsync";
-
- interrupt-parent = <&mdss>;
- interrupts = <0>;
- power-domains = <&rpmhpd SC7180_CX>;
- operating-points-v2 = <&mdp_opp_table>;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- dpu_intf1_out: endpoint {
- remote-endpoint = <&dsi0_in>;
- };
- };
-
- port@2 {
- reg = <2>;
- dpu_intf0_out: endpoint {
- remote-endpoint = <&dp_in>;
- };
- };
- };
- };
- };
-...
diff --git a/Documentation/devicetree/bindings/display/msm/dpu-sc7280.yaml b/Documentation/devicetree/bindings/display/msm/dpu-sc7280.yaml
deleted file mode 100644
index 584d646..0000000
--- a/Documentation/devicetree/bindings/display/msm/dpu-sc7280.yaml
+++ /dev/null
@@ -1,239 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
-%YAML 1.2
----
-$id: http://devicetree.org/schemas/display/msm/dpu-sc7280.yaml#
-$schema: http://devicetree.org/meta-schemas/core.yaml#
-
-title: Qualcomm Display DPU dt properties for SC7280
-
-maintainers:
- - Krishna Manikandan <quic_mkrishn@quicinc.com>
-
-description: |
- Device tree bindings for MSM Mobile Display Subsystem (MDSS) that encapsulates
- sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
- bindings of MDSS and DPU are mentioned for SC7280.
-
-properties:
- compatible:
- const: qcom,sc7280-mdss
-
- reg:
- maxItems: 1
-
- reg-names:
- const: mdss
-
- power-domains:
- maxItems: 1
-
- clocks:
- items:
- - description: Display AHB clock from gcc
- - description: Display AHB clock from dispcc
- - description: Display core clock
-
- clock-names:
- items:
- - const: iface
- - const: ahb
- - const: core
-
- interrupts:
- maxItems: 1
-
- interrupt-controller: true
-
- "#address-cells": true
-
- "#size-cells": true
-
- "#interrupt-cells":
- const: 1
-
- iommus:
- items:
- - description: Phandle to apps_smmu node with SID mask for Hard-Fail port0
-
- ranges: true
-
- interconnects:
- items:
- - description: Interconnect path specifying the port ids for data bus
-
- interconnect-names:
- const: mdp0-mem
-
- resets:
- items:
- - description: MDSS_CORE reset
-
-patternProperties:
- "^display-controller@[0-9a-f]+$":
- type: object
- description: Node containing the properties of DPU.
- additionalProperties: false
-
- properties:
- compatible:
- const: qcom,sc7280-dpu
-
- reg:
- items:
- - description: Address offset and size for mdp register set
- - description: Address offset and size for vbif register set
-
- reg-names:
- items:
- - const: mdp
- - const: vbif
-
- clocks:
- items:
- - description: Display hf axi clock
- - description: Display sf axi clock
- - description: Display ahb clock
- - description: Display lut clock
- - description: Display core clock
- - description: Display vsync clock
-
- clock-names:
- items:
- - const: bus
- - const: nrt_bus
- - const: iface
- - const: lut
- - const: core
- - const: vsync
-
- interrupts:
- maxItems: 1
-
- power-domains:
- maxItems: 1
-
- operating-points-v2: true
- opp-table:
- type: object
-
- ports:
- $ref: /schemas/graph.yaml#/properties/ports
- description: |
- Contains the list of output ports from DPU device. These ports
- connect to interfaces that are external to the DPU hardware,
- such as DSI, DP etc. Each output port contains an endpoint that
- describes how it is connected to an external interface.
-
- properties:
- port@0:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF1 (DSI)
-
- port@1:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF5 (EDP)
-
- required:
- - port@0
-
- required:
- - compatible
- - reg
- - reg-names
- - clocks
- - interrupts
- - power-domains
- - operating-points-v2
- - ports
-
-required:
- - compatible
- - reg
- - reg-names
- - power-domains
- - clocks
- - interrupts
- - interrupt-controller
- - iommus
- - ranges
-
-additionalProperties: false
-
-examples:
- - |
- #include <dt-bindings/clock/qcom,dispcc-sc7280.h>
- #include <dt-bindings/clock/qcom,gcc-sc7280.h>
- #include <dt-bindings/interrupt-controller/arm-gic.h>
- #include <dt-bindings/interconnect/qcom,sc7280.h>
- #include <dt-bindings/power/qcom-rpmpd.h>
-
- display-subsystem@ae00000 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "qcom,sc7280-mdss";
- reg = <0xae00000 0x1000>;
- reg-names = "mdss";
- power-domains = <&dispcc DISP_CC_MDSS_CORE_GDSC>;
- clocks = <&gcc GCC_DISP_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>;
- clock-names = "iface",
- "ahb",
- "core";
-
- interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-controller;
- #interrupt-cells = <1>;
-
- interconnects = <&mmss_noc MASTER_MDP0 &mc_virt SLAVE_EBI1>;
- interconnect-names = "mdp0-mem";
-
- iommus = <&apps_smmu 0x900 0x402>;
- ranges;
-
- display-controller@ae01000 {
- compatible = "qcom,sc7280-dpu";
- reg = <0x0ae01000 0x8f000>,
- <0x0aeb0000 0x2008>;
-
- reg-names = "mdp", "vbif";
-
- clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
- <&gcc GCC_DISP_SF_AXI_CLK>,
- <&dispcc DISP_CC_MDSS_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>,
- <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
- clock-names = "bus",
- "nrt_bus",
- "iface",
- "lut",
- "core",
- "vsync";
-
- interrupt-parent = <&mdss>;
- interrupts = <0>;
- power-domains = <&rpmhpd SC7280_CX>;
- operating-points-v2 = <&mdp_opp_table>;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- dpu_intf1_out: endpoint {
- remote-endpoint = <&dsi0_in>;
- };
- };
-
- port@1 {
- reg = <1>;
- dpu_intf5_out: endpoint {
- remote-endpoint = <&edp_in>;
- };
- };
- };
- };
- };
-...
diff --git a/Documentation/devicetree/bindings/display/msm/dpu-sdm845.yaml b/Documentation/devicetree/bindings/display/msm/dpu-sdm845.yaml
deleted file mode 100644
index 7d10373..0000000
--- a/Documentation/devicetree/bindings/display/msm/dpu-sdm845.yaml
+++ /dev/null
@@ -1,217 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
-%YAML 1.2
----
-$id: http://devicetree.org/schemas/display/msm/dpu-sdm845.yaml#
-$schema: http://devicetree.org/meta-schemas/core.yaml#
-
-title: Qualcomm Display DPU dt properties for SDM845 target
-
-maintainers:
- - Krishna Manikandan <quic_mkrishn@quicinc.com>
-
-description: |
- Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
- sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
- bindings of MDSS and DPU are mentioned for SDM845 target.
-
-properties:
- compatible:
- items:
- - const: qcom,sdm845-mdss
-
- reg:
- maxItems: 1
-
- reg-names:
- const: mdss
-
- power-domains:
- maxItems: 1
-
- clocks:
- items:
- - description: Display AHB clock from gcc
- - description: Display core clock
-
- clock-names:
- items:
- - const: iface
- - const: core
-
- interrupts:
- maxItems: 1
-
- interrupt-controller: true
-
- "#address-cells": true
-
- "#size-cells": true
-
- "#interrupt-cells":
- const: 1
-
- iommus:
- items:
- - description: Phandle to apps_smmu node with SID mask for Hard-Fail port0
- - description: Phandle to apps_smmu node with SID mask for Hard-Fail port1
-
- ranges: true
-
- resets:
- items:
- - description: MDSS_CORE reset
-
-patternProperties:
- "^display-controller@[0-9a-f]+$":
- type: object
- description: Node containing the properties of DPU.
- additionalProperties: false
-
- properties:
- compatible:
- items:
- - const: qcom,sdm845-dpu
-
- reg:
- items:
- - description: Address offset and size for mdp register set
- - description: Address offset and size for vbif register set
-
- reg-names:
- items:
- - const: mdp
- - const: vbif
-
- clocks:
- items:
- - description: Display ahb clock
- - description: Display axi clock
- - description: Display core clock
- - description: Display vsync clock
-
- clock-names:
- items:
- - const: iface
- - const: bus
- - const: core
- - const: vsync
-
- interrupts:
- maxItems: 1
-
- power-domains:
- maxItems: 1
-
- operating-points-v2: true
- opp-table:
- type: object
-
- ports:
- $ref: /schemas/graph.yaml#/properties/ports
- description: |
- Contains the list of output ports from DPU device. These ports
- connect to interfaces that are external to the DPU hardware,
- such as DSI, DP etc. Each output port contains an endpoint that
- describes how it is connected to an external interface.
-
- properties:
- port@0:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF1 (DSI1)
-
- port@1:
- $ref: /schemas/graph.yaml#/properties/port
- description: DPU_INTF2 (DSI2)
-
- required:
- - port@0
- - port@1
-
- required:
- - compatible
- - reg
- - reg-names
- - clocks
- - interrupts
- - power-domains
- - operating-points-v2
- - ports
-
-required:
- - compatible
- - reg
- - reg-names
- - power-domains
- - clocks
- - interrupts
- - interrupt-controller
- - iommus
- - ranges
-
-additionalProperties: false
-
-examples:
- - |
- #include <dt-bindings/clock/qcom,dispcc-sdm845.h>
- #include <dt-bindings/clock/qcom,gcc-sdm845.h>
- #include <dt-bindings/interrupt-controller/arm-gic.h>
- #include <dt-bindings/power/qcom-rpmpd.h>
-
- display-subsystem@ae00000 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "qcom,sdm845-mdss";
- reg = <0x0ae00000 0x1000>;
- reg-names = "mdss";
- power-domains = <&dispcc MDSS_GDSC>;
-
- clocks = <&gcc GCC_DISP_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>;
- clock-names = "iface", "core";
-
- interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-controller;
- #interrupt-cells = <1>;
-
- iommus = <&apps_smmu 0x880 0x8>,
- <&apps_smmu 0xc80 0x8>;
- ranges;
-
- display-controller@ae01000 {
- compatible = "qcom,sdm845-dpu";
- reg = <0x0ae01000 0x8f000>,
- <0x0aeb0000 0x2008>;
- reg-names = "mdp", "vbif";
-
- clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
- <&dispcc DISP_CC_MDSS_AXI_CLK>,
- <&dispcc DISP_CC_MDSS_MDP_CLK>,
- <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
- clock-names = "iface", "bus", "core", "vsync";
-
- interrupt-parent = <&mdss>;
- interrupts = <0>;
- power-domains = <&rpmhpd SDM845_CX>;
- operating-points-v2 = <&mdp_opp_table>;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- dpu_intf1_out: endpoint {
- remote-endpoint = <&dsi0_in>;
- };
- };
-
- port@1 {
- reg = <1>;
- dpu_intf2_out: endpoint {
- remote-endpoint = <&dsi1_in>;
- };
- };
- };
- };
- };
-...
diff --git a/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml b/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml
index 3b609c1..f2c1437 100644
--- a/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml
+++ b/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml
@@ -49,6 +49,7 @@
maxItems: 1
phy-names:
+ deprecated: true
const: dsi
"#address-cells": true
@@ -80,6 +81,9 @@
operating-points-v2: true
+ opp-table:
+ type: object
+
ports:
$ref: "/schemas/graph.yaml#/properties/ports"
description: |
@@ -131,7 +135,6 @@
- clocks
- clock-names
- phys
- - phy-names
- assigned-clocks
- assigned-clock-parents
- power-domains
diff --git a/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml b/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml
index 1342d74..819de5c 100644
--- a/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml
+++ b/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml
@@ -16,6 +16,7 @@
compatible:
enum:
- qcom,dsi-phy-14nm
+ - qcom,dsi-phy-14nm-2290
- qcom,dsi-phy-14nm-660
- qcom,dsi-phy-14nm-8953
diff --git a/Documentation/devicetree/bindings/display/msm/mdp5.txt b/Documentation/devicetree/bindings/display/msm/mdp5.txt
index 43d1127..65d03c5 100644
--- a/Documentation/devicetree/bindings/display/msm/mdp5.txt
+++ b/Documentation/devicetree/bindings/display/msm/mdp5.txt
@@ -2,37 +2,9 @@
Description:
-This is the bindings documentation for the Mobile Display Subsytem(MDSS) that
-encapsulates sub-blocks like MDP5, DSI, HDMI, eDP etc, and the MDP5 display
+This is the bindings documentation for the MDP5 display
controller found in SoCs like MSM8974, APQ8084, MSM8916, MSM8994 and MSM8996.
-MDSS:
-Required properties:
-- compatible:
- * "qcom,mdss" - MDSS
-- reg: Physical base address and length of the controller's registers.
-- reg-names: The names of register regions. The following regions are required:
- * "mdss_phys"
- * "vbif_phys"
-- interrupts: The interrupt signal from MDSS.
-- interrupt-controller: identifies the node as an interrupt controller.
-- #interrupt-cells: specifies the number of cells needed to encode an interrupt
- source, should be 1.
-- power-domains: a power domain consumer specifier according to
- Documentation/devicetree/bindings/power/power_domain.txt
-- clocks: device clocks. See ../clocks/clock-bindings.txt for details.
-- clock-names: the following clocks are required.
- * "iface"
- * "bus"
- * "vsync"
-- #address-cells: number of address cells for the MDSS children. Should be 1.
-- #size-cells: Should be 1.
-- ranges: parent bus address space is the same as the child bus address space.
-
-Optional properties:
-- clock-names: the following clocks are optional:
- * "lut"
-
MDP5:
Required properties:
- compatible:
diff --git a/Documentation/devicetree/bindings/display/msm/mdss-common.yaml b/Documentation/devicetree/bindings/display/msm/mdss-common.yaml
new file mode 100644
index 0000000..27d7242
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/mdss-common.yaml
@@ -0,0 +1,83 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/mdss-common.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display MDSS common properties
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+ - Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
+ - Rob Clark <robdclark@gmail.com>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
+ sub-blocks like DPU display controller, DSI and DP interfaces etc.
+
+properties:
+ reg:
+ maxItems: 1
+
+ reg-names:
+ const: mdss
+
+ power-domains:
+ maxItems: 1
+
+ clocks:
+ minItems: 2
+ maxItems: 4
+
+ clock-names:
+ minItems: 2
+ maxItems: 4
+
+ interrupts:
+ maxItems: 1
+
+ interrupt-controller: true
+
+ "#address-cells": true
+
+ "#size-cells": true
+
+ "#interrupt-cells":
+ const: 1
+
+ iommus:
+ minItems: 1
+ items:
+ - description: Phandle to apps_smmu node with SID mask for Hard-Fail port0
+ - description: Phandle to apps_smmu node with SID mask for Hard-Fail port1
+
+ ranges: true
+
+ interconnects:
+ minItems: 1
+ items:
+ - description: Interconnect path from mdp0 (or a single mdp) port to the data bus
+ - description: Interconnect path from mdp1 port to the data bus
+
+ interconnect-names:
+ minItems: 1
+ items:
+ - const: mdp0-mem
+ - const: mdp1-mem
+
+ resets:
+ items:
+ - description: MDSS_CORE reset
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - power-domains
+ - clocks
+ - interrupts
+ - interrupt-controller
+ - iommus
+ - ranges
+
+additionalProperties: true
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,mdss.yaml
new file mode 100644
index 0000000..ba04602
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,mdss.yaml
@@ -0,0 +1,196 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Mobile Display SubSystem (MDSS)
+
+maintainers:
+ - Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
+ - Rob Clark <robdclark@gmail.com>
+
+description:
+ This is the bindings documentation for the Mobile Display Subsytem(MDSS) that
+ encapsulates sub-blocks like MDP5, DSI, HDMI, eDP, etc.
+
+properties:
+ compatible:
+ enum:
+ - qcom,mdss
+
+ reg:
+ minItems: 2
+ maxItems: 3
+
+ reg-names:
+ minItems: 2
+ items:
+ - const: mdss_phys
+ - const: vbif_phys
+ - const: vbif_nrt_phys
+
+ interrupts:
+ maxItems: 1
+
+ interrupt-controller: true
+
+ "#interrupt-cells":
+ const: 1
+
+ power-domains:
+ maxItems: 1
+ description: |
+ The MDSS power domain provided by GCC
+
+ clocks:
+ minItems: 1
+ items:
+ - description: Display abh clock
+ - description: Display axi clock
+ - description: Display vsync clock
+
+ clock-names:
+ minItems: 1
+ items:
+ - const: iface
+ - const: bus
+ - const: vsync
+
+ "#address-cells":
+ const: 1
+
+ "#size-cells":
+ const: 1
+
+ ranges: true
+
+ resets:
+ items:
+ - description: MDSS_CORE reset
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - interrupts
+ - interrupt-controller
+ - "#interrupt-cells"
+ - power-domains
+ - clocks
+ - clock-names
+ - "#address-cells"
+ - "#size-cells"
+ - ranges
+
+patternProperties:
+ "^mdp@[1-9a-f][0-9a-f]*$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,mdp5
+
+ "^dsi@[1-9a-f][0-9a-f]*$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,mdss-dsi-ctrl
+
+ "^phy@[1-9a-f][0-9a-f]*$":
+ type: object
+ properties:
+ compatible:
+ enum:
+ - qcom,dsi-phy-14nm
+ - qcom,dsi-phy-14nm-660
+ - qcom,dsi-phy-14nm-8953
+ - qcom,dsi-phy-20nm
+ - qcom,dsi-phy-28nm-hpm
+ - qcom,dsi-phy-28nm-lp
+
+ "^hdmi-phy@[1-9a-f][0-9a-f]*$":
+ type: object
+ properties:
+ compatible:
+ enum:
+ - qcom,hdmi-phy-8084
+ - qcom,hdmi-phy-8660
+ - qcom,hdmi-phy-8960
+ - qcom,hdmi-phy-8974
+ - qcom,hdmi-phy-8996
+
+ "^hdmi-tx@[1-9a-f][0-9a-f]*$":
+ type: object
+ properties:
+ compatible:
+ enum:
+ - qcom,hdmi-tx-8084
+ - qcom,hdmi-tx-8660
+ - qcom,hdmi-tx-8960
+ - qcom,hdmi-tx-8974
+ - qcom,hdmi-tx-8994
+ - qcom,hdmi-tx-8996
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,gcc-msm8916.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ mdss@1a00000 {
+ compatible = "qcom,mdss";
+ reg = <0x1a00000 0x1000>,
+ <0x1ac8000 0x3000>;
+ reg-names = "mdss_phys", "vbif_phys";
+
+ power-domains = <&gcc MDSS_GDSC>;
+
+ clocks = <&gcc GCC_MDSS_AHB_CLK>,
+ <&gcc GCC_MDSS_AXI_CLK>,
+ <&gcc GCC_MDSS_VSYNC_CLK>;
+ clock-names = "iface",
+ "bus",
+ "vsync";
+
+ interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ mdp@1a01000 {
+ compatible = "qcom,mdp5";
+ reg = <0x01a01000 0x89000>;
+ reg-names = "mdp_phys";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+
+ clocks = <&gcc GCC_MDSS_AHB_CLK>,
+ <&gcc GCC_MDSS_AXI_CLK>,
+ <&gcc GCC_MDSS_MDP_CLK>,
+ <&gcc GCC_MDSS_VSYNC_CLK>;
+ clock-names = "iface",
+ "bus",
+ "core",
+ "vsync";
+
+ iommus = <&apps_iommu 4>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ mdp5_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,msm8998-dpu.yaml b/Documentation/devicetree/bindings/display/msm/qcom,msm8998-dpu.yaml
new file mode 100644
index 0000000..b02adba
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,msm8998-dpu.yaml
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,msm8998-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display DPU dt properties for MSM8998 target
+
+maintainers:
+ - AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
+
+$ref: /schemas/display/msm/dpu-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,msm8998-dpu
+
+ reg:
+ items:
+ - description: Address offset and size for mdp register set
+ - description: Address offset and size for regdma register set
+ - description: Address offset and size for vbif register set
+ - description: Address offset and size for non-realtime vbif register set
+
+ reg-names:
+ items:
+ - const: mdp
+ - const: regdma
+ - const: vbif
+ - const: vbif_nrt
+
+ clocks:
+ items:
+ - description: Display ahb clock
+ - description: Display axi clock
+ - description: Display mem-noc clock
+ - description: Display core clock
+ - description: Display vsync clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: bus
+ - const: mnoc
+ - const: core
+ - const: vsync
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,mmcc-msm8998.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-controller@c901000 {
+ compatible = "qcom,msm8998-dpu";
+ reg = <0x0c901000 0x8f000>,
+ <0x0c9a8e00 0xf0>,
+ <0x0c9b0000 0x2008>,
+ <0x0c9b8000 0x1040>;
+ reg-names = "mdp", "regdma", "vbif", "vbif_nrt";
+
+ clocks = <&mmcc MDSS_AHB_CLK>,
+ <&mmcc MDSS_AXI_CLK>,
+ <&mmcc MNOC_AHB_CLK>,
+ <&mmcc MDSS_MDP_CLK>,
+ <&mmcc MDSS_VSYNC_CLK>;
+ clock-names = "iface", "bus", "mnoc", "core", "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmpd MSM8998_VDDMX>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ endpoint {
+ remote-endpoint = <&dsi1_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,msm8998-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,msm8998-mdss.yaml
new file mode 100644
index 0000000..cf52ff7
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,msm8998-mdss.yaml
@@ -0,0 +1,268 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,msm8998-mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm MSM8998 Display MDSS
+
+maintainers:
+ - AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
+ sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
+ bindings of MDSS are mentioned for MSM8998 target.
+
+$ref: /schemas/display/msm/mdss-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,msm8998-mdss
+
+ clocks:
+ items:
+ - description: Display AHB clock
+ - description: Display AXI clock
+ - description: Display core clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: bus
+ - const: core
+
+ iommus:
+ maxItems: 1
+
+patternProperties:
+ "^display-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,msm8998-dpu
+
+ "^dsi@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,mdss-dsi-ctrl
+
+ "^phy@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-phy-10nm-8998
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,mmcc-msm8998.h>
+ #include <dt-bindings/clock/qcom,rpmcc.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-subsystem@c900000 {
+ compatible = "qcom,msm8998-mdss";
+ reg = <0x0c900000 0x1000>;
+ reg-names = "mdss";
+
+ clocks = <&mmcc MDSS_AHB_CLK>,
+ <&mmcc MDSS_AXI_CLK>,
+ <&mmcc MDSS_MDP_CLK>;
+ clock-names = "iface", "bus", "core";
+
+ #address-cells = <1>;
+ #interrupt-cells = <1>;
+ #size-cells = <1>;
+
+ interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ iommus = <&mmss_smmu 0>;
+
+ power-domains = <&mmcc MDSS_GDSC>;
+ ranges;
+
+ display-controller@c901000 {
+ compatible = "qcom,msm8998-dpu";
+ reg = <0x0c901000 0x8f000>,
+ <0x0c9a8e00 0xf0>,
+ <0x0c9b0000 0x2008>,
+ <0x0c9b8000 0x1040>;
+ reg-names = "mdp", "regdma", "vbif", "vbif_nrt";
+
+ clocks = <&mmcc MDSS_AHB_CLK>,
+ <&mmcc MDSS_AXI_CLK>,
+ <&mmcc MNOC_AHB_CLK>,
+ <&mmcc MDSS_MDP_CLK>,
+ <&mmcc MDSS_VSYNC_CLK>;
+ clock-names = "iface", "bus", "mnoc", "core", "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmpd MSM8998_VDDMX>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dpu_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dpu_intf2_out: endpoint {
+ remote-endpoint = <&dsi1_in>;
+ };
+ };
+ };
+ };
+
+ dsi@c994000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0c994000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <4>;
+
+ clocks = <&mmcc MDSS_BYTE0_CLK>,
+ <&mmcc MDSS_BYTE0_INTF_CLK>,
+ <&mmcc MDSS_PCLK0_CLK>,
+ <&mmcc MDSS_ESC0_CLK>,
+ <&mmcc MDSS_AHB_CLK>,
+ <&mmcc MDSS_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+ assigned-clocks = <&mmcc BYTE0_CLK_SRC>, <&mmcc PCLK0_CLK_SRC>;
+ assigned-clock-parents = <&dsi0_phy 0>, <&dsi0_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmpd MSM8998_VDDCX>;
+
+ phys = <&dsi0_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&dpu_intf1_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ };
+ };
+ };
+ };
+
+ dsi0_phy: phy@c994400 {
+ compatible = "qcom,dsi-phy-10nm-8998";
+ reg = <0x0c994400 0x200>,
+ <0x0c994600 0x280>,
+ <0x0c994a00 0x1e0>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&mmcc MDSS_AHB_CLK>,
+ <&rpmcc RPM_SMD_XO_CLK_SRC>;
+ clock-names = "iface", "ref";
+
+ vdds-supply = <&pm8998_l1>;
+ };
+
+ dsi@c996000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0c996000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <5>;
+
+ clocks = <&mmcc MDSS_BYTE1_CLK>,
+ <&mmcc MDSS_BYTE1_INTF_CLK>,
+ <&mmcc MDSS_PCLK1_CLK>,
+ <&mmcc MDSS_ESC1_CLK>,
+ <&mmcc MDSS_AHB_CLK>,
+ <&mmcc MDSS_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+ assigned-clocks = <&mmcc BYTE1_CLK_SRC>, <&mmcc PCLK1_CLK_SRC>;
+ assigned-clock-parents = <&dsi1_phy 0>, <&dsi1_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmpd MSM8998_VDDCX>;
+
+ phys = <&dsi1_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi1_in: endpoint {
+ remote-endpoint = <&dpu_intf2_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi1_out: endpoint {
+ };
+ };
+ };
+ };
+
+ dsi1_phy: phy@c996400 {
+ compatible = "qcom,dsi-phy-10nm-8998";
+ reg = <0x0c996400 0x200>,
+ <0x0c996600 0x280>,
+ <0x0c996a00 0x10e>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&mmcc MDSS_AHB_CLK>,
+ <&rpmcc RPM_SMD_XO_CLK_SRC>;
+ clock-names = "iface", "ref";
+
+ vdds-supply = <&pm8998_l1>;
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-dpu.yaml b/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-dpu.yaml
new file mode 100644
index 0000000..a7b382f
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-dpu.yaml
@@ -0,0 +1,84 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,qcm2290-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display DPU dt properties for QCM2290 target
+
+maintainers:
+ - Loic Poulain <loic.poulain@linaro.org>
+
+$ref: /schemas/display/msm/dpu-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,qcm2290-dpu
+
+ reg:
+ items:
+ - description: Address offset and size for mdp register set
+ - description: Address offset and size for vbif register set
+
+ reg-names:
+ items:
+ - const: mdp
+ - const: vbif
+
+ clocks:
+ items:
+ - description: Display AXI clock from gcc
+ - description: Display AHB clock from dispcc
+ - description: Display core clock from dispcc
+ - description: Display lut clock from dispcc
+ - description: Display vsync clock from dispcc
+
+ clock-names:
+ items:
+ - const: bus
+ - const: iface
+ - const: core
+ - const: lut
+ - const: vsync
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-qcm2290.h>
+ #include <dt-bindings/clock/qcom,gcc-qcm2290.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-controller@5e01000 {
+ compatible = "qcom,qcm2290-dpu";
+ reg = <0x05e01000 0x8f000>,
+ <0x05eb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus", "iface", "core", "lut", "vsync";
+
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmpd QCM2290_VDDCX>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-mdss.yaml
new file mode 100644
index 0000000..d6f043a
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-mdss.yaml
@@ -0,0 +1,198 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,qcm2290-mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm QCM220 Display MDSS
+
+maintainers:
+ - Loic Poulain <loic.poulain@linaro.org>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
+ sub-blocks like DPU display controller and DSI. Device tree bindings of MDSS
+ are mentioned for QCM2290 target.
+
+$ref: /schemas/display/msm/mdss-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,qcm2290-mdss
+
+ clocks:
+ items:
+ - description: Display AHB clock from gcc
+ - description: Display AXI clock
+ - description: Display core clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: bus
+ - const: core
+
+ iommus:
+ maxItems: 2
+
+ interconnects:
+ maxItems: 1
+
+ interconnect-names:
+ maxItems: 1
+
+patternProperties:
+ "^display-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,qcm2290-dpu
+
+ "^dsi@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-ctrl-6g-qcm2290
+
+ "^phy@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-phy-14nm-2290
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-qcm2290.h>
+ #include <dt-bindings/clock/qcom,gcc-qcm2290.h>
+ #include <dt-bindings/clock/qcom,rpmcc.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interconnect/qcom,qcm2290.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ mdss@5e00000 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "qcom,qcm2290-mdss";
+ reg = <0x05e00000 0x1000>;
+ reg-names = "mdss";
+ power-domains = <&dispcc MDSS_GDSC>;
+ clocks = <&gcc GCC_DISP_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>;
+ clock-names = "iface", "bus", "core";
+
+ interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ interconnects = <&mmrt_virt MASTER_MDP0 &bimc SLAVE_EBI1>;
+ interconnect-names = "mdp0-mem";
+
+ iommus = <&apps_smmu 0x420 0x2>,
+ <&apps_smmu 0x421 0x0>;
+ ranges;
+
+ display-controller@5e01000 {
+ compatible = "qcom,qcm2290-dpu";
+ reg = <0x05e01000 0x8f000>,
+ <0x05eb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus", "iface", "core", "lut", "vsync";
+
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmpd QCM2290_VDDCX>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dpu_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+ };
+ };
+
+ dsi@5e94000 {
+ compatible = "qcom,dsi-ctrl-6g-qcm2290";
+ reg = <0x05e94000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <4>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE0_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC0_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK_SRC>, <&dispcc DISP_CC_MDSS_PCLK0_CLK_SRC>;
+ assigned-clock-parents = <&dsi0_phy 0>, <&dsi0_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmpd QCM2290_VDDCX>;
+
+ phys = <&dsi0_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&dpu_intf1_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ };
+ };
+ };
+ };
+
+ dsi0_phy: phy@5e94400 {
+ compatible = "qcom,dsi-phy-14nm-2290";
+ reg = <0x05e94400 0x100>,
+ <0x05e94500 0x300>,
+ <0x05e94800 0x188>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
+ clock-names = "iface", "ref";
+ vcca-supply = <&vreg_dsi_phy>;
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sc7180-dpu.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sc7180-dpu.yaml
new file mode 100644
index 0000000..bd590a6
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sc7180-dpu.yaml
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sc7180-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display DPU dt properties for SC7180 target
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+
+$ref: /schemas/display/msm/dpu-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,sc7180-dpu
+
+ reg:
+ items:
+ - description: Address offset and size for mdp register set
+ - description: Address offset and size for vbif register set
+
+ reg-names:
+ items:
+ - const: mdp
+ - const: vbif
+
+ clocks:
+ items:
+ - description: Display hf axi clock
+ - description: Display ahb clock
+ - description: Display rotator clock
+ - description: Display lut clock
+ - description: Display core clock
+ - description: Display vsync clock
+
+ clock-names:
+ items:
+ - const: bus
+ - const: iface
+ - const: rot
+ - const: lut
+ - const: core
+ - const: vsync
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sc7180.h>
+ #include <dt-bindings/clock/qcom,gcc-sc7180.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-controller@ae01000 {
+ compatible = "qcom,sc7180-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_ROT_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus", "iface", "rot", "lut", "core",
+ "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ power-domains = <&rpmhpd SC7180_CX>;
+ operating-points-v2 = <&mdp_opp_table>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+ endpoint {
+ remote-endpoint = <&dp_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sc7180-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sc7180-mdss.yaml
new file mode 100644
index 0000000..13e396d
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sc7180-mdss.yaml
@@ -0,0 +1,304 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sc7180-mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SC7180 Display MDSS
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
+ sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
+ bindings of MDSS are mentioned for SC7180 target.
+
+$ref: /schemas/display/msm/mdss-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,sc7180-mdss
+
+ clocks:
+ items:
+ - description: Display AHB clock from gcc
+ - description: Display AHB clock from dispcc
+ - description: Display core clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: ahb
+ - const: core
+
+ iommus:
+ maxItems: 1
+
+ interconnects:
+ maxItems: 1
+
+ interconnect-names:
+ maxItems: 1
+
+patternProperties:
+ "^display-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sc7180-dpu
+
+ "^displayport-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sc7180-dp
+
+ "^dsi@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,mdss-dsi-ctrl
+
+ "^phy@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-phy-10nm
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sc7180.h>
+ #include <dt-bindings/clock/qcom,gcc-sc7180.h>
+ #include <dt-bindings/clock/qcom,rpmh.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interconnect/qcom,sdm845.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-subsystem@ae00000 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "qcom,sc7180-mdss";
+ reg = <0xae00000 0x1000>;
+ reg-names = "mdss";
+ power-domains = <&dispcc MDSS_GDSC>;
+ clocks = <&gcc GCC_DISP_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>;
+ clock-names = "iface", "ahb", "core";
+
+ interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ interconnects = <&mmss_noc MASTER_MDP0 &mc_virt SLAVE_EBI1>;
+ interconnect-names = "mdp0-mem";
+
+ iommus = <&apps_smmu 0x800 0x2>;
+ ranges;
+
+ display-controller@ae01000 {
+ compatible = "qcom,sc7180-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_ROT_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus", "iface", "rot", "lut", "core",
+ "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ power-domains = <&rpmhpd SC7180_CX>;
+ operating-points-v2 = <&mdp_opp_table>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dpu_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+ dpu_intf0_out: endpoint {
+ remote-endpoint = <&dp_in>;
+ };
+ };
+ };
+ };
+
+ dsi@ae94000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0ae94000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <4>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE0_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC0_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK_SRC>, <&dispcc DISP_CC_MDSS_PCLK0_CLK_SRC>;
+ assigned-clock-parents = <&dsi_phy 0>, <&dsi_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmhpd SC7180_CX>;
+
+ phys = <&dsi_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&dpu_intf1_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ };
+ };
+ };
+
+ dsi_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-187500000 {
+ opp-hz = /bits/ 64 <187500000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-358000000 {
+ opp-hz = /bits/ 64 <358000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>;
+ };
+ };
+ };
+
+ dsi_phy: phy@ae94400 {
+ compatible = "qcom,dsi-phy-10nm";
+ reg = <0x0ae94400 0x200>,
+ <0x0ae94600 0x280>,
+ <0x0ae94a00 0x1e0>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "iface", "ref";
+ vdds-supply = <&vreg_dsi_phy>;
+ };
+
+ displayport-controller@ae90000 {
+ compatible = "qcom,sc7180-dp";
+
+ reg = <0xae90000 0x200>,
+ <0xae90200 0x200>,
+ <0xae90400 0xc00>,
+ <0xae91000 0x400>,
+ <0xae91400 0x400>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <12>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_AUX_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_LINK_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_LINK_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_PIXEL_CLK>;
+ clock-names = "core_iface", "core_aux", "ctrl_link",
+ "ctrl_link_iface", "stream_pixel";
+ assigned-clocks = <&dispcc DISP_CC_MDSS_DP_LINK_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_DP_PIXEL_CLK_SRC>;
+ assigned-clock-parents = <&dp_phy 0>, <&dp_phy 1>;
+ phys = <&dp_phy>;
+ phy-names = "dp";
+
+ operating-points-v2 = <&dp_opp_table>;
+ power-domains = <&rpmhpd SC7180_CX>;
+
+ #sound-dai-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ dp_in: endpoint {
+ remote-endpoint = <&dpu_intf0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dp_out: endpoint { };
+ };
+ };
+
+ dp_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-270000000 {
+ opp-hz = /bits/ 64 <270000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-540000000 {
+ opp-hz = /bits/ 64 <540000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>;
+ };
+
+ opp-810000000 {
+ opp-hz = /bits/ 64 <810000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sc7280-dpu.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sc7280-dpu.yaml
new file mode 100644
index 0000000..924059b
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sc7280-dpu.yaml
@@ -0,0 +1,98 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sc7280-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display DPU dt properties for SC7280
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+
+$ref: /schemas/display/msm/dpu-common.yaml#
+
+properties:
+ compatible:
+ const: qcom,sc7280-dpu
+
+ reg:
+ items:
+ - description: Address offset and size for mdp register set
+ - description: Address offset and size for vbif register set
+
+ reg-names:
+ items:
+ - const: mdp
+ - const: vbif
+
+ clocks:
+ items:
+ - description: Display hf axi clock
+ - description: Display sf axi clock
+ - description: Display ahb clock
+ - description: Display lut clock
+ - description: Display core clock
+ - description: Display vsync clock
+
+ clock-names:
+ items:
+ - const: bus
+ - const: nrt_bus
+ - const: iface
+ - const: lut
+ - const: core
+ - const: vsync
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sc7280.h>
+ #include <dt-bindings/clock/qcom,gcc-sc7280.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-controller@ae01000 {
+ compatible = "qcom,sc7280-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&gcc GCC_DISP_SF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus",
+ "nrt_bus",
+ "iface",
+ "lut",
+ "core",
+ "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ power-domains = <&rpmhpd SC7280_CX>;
+ operating-points-v2 = <&mdp_opp_table>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ endpoint {
+ remote-endpoint = <&edp_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sc7280-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sc7280-mdss.yaml
new file mode 100644
index 0000000..a3de174
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sc7280-mdss.yaml
@@ -0,0 +1,422 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sc7280-mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SC7280 Display MDSS
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem (MDSS) that encapsulates
+ sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
+ bindings of MDSS are mentioned for SC7280.
+
+$ref: /schemas/display/msm/mdss-common.yaml#
+
+properties:
+ compatible:
+ const: qcom,sc7280-mdss
+
+ clocks:
+ items:
+ - description: Display AHB clock from gcc
+ - description: Display AHB clock from dispcc
+ - description: Display core clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: ahb
+ - const: core
+
+ iommus:
+ maxItems: 1
+
+ interconnects:
+ maxItems: 1
+
+ interconnect-names:
+ maxItems: 1
+
+patternProperties:
+ "^display-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sc7280-dpu
+
+ "^displayport-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sc7280-dp
+
+ "^dsi@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,mdss-dsi-ctrl
+
+ "^edp@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sc7280-edp
+
+ "^phy@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ enum:
+ - qcom,sc7280-dsi-phy-7nm
+ - qcom,sc7280-edp-phy
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sc7280.h>
+ #include <dt-bindings/clock/qcom,gcc-sc7280.h>
+ #include <dt-bindings/clock/qcom,rpmh.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interconnect/qcom,sc7280.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-subsystem@ae00000 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "qcom,sc7280-mdss";
+ reg = <0xae00000 0x1000>;
+ reg-names = "mdss";
+ power-domains = <&dispcc DISP_CC_MDSS_CORE_GDSC>;
+ clocks = <&gcc GCC_DISP_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>;
+ clock-names = "iface",
+ "ahb",
+ "core";
+
+ interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ interconnects = <&mmss_noc MASTER_MDP0 &mc_virt SLAVE_EBI1>;
+ interconnect-names = "mdp0-mem";
+
+ iommus = <&apps_smmu 0x900 0x402>;
+ ranges;
+
+ display-controller@ae01000 {
+ compatible = "qcom,sc7280-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&gcc GCC_DISP_SF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus",
+ "nrt_bus",
+ "iface",
+ "lut",
+ "core",
+ "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ power-domains = <&rpmhpd SC7280_CX>;
+ operating-points-v2 = <&mdp_opp_table>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dpu_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dpu_intf5_out: endpoint {
+ remote-endpoint = <&edp_in>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+ dpu_intf0_out: endpoint {
+ remote-endpoint = <&dp_in>;
+ };
+ };
+ };
+ };
+
+ dsi@ae94000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0ae94000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <4>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE0_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC0_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK_SRC>;
+ assigned-clock-parents = <&mdss_dsi_phy 0>, <&mdss_dsi_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmhpd SC7280_CX>;
+
+ phys = <&mdss_dsi_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&dpu_intf1_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ };
+ };
+ };
+
+ dsi_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-187500000 {
+ opp-hz = /bits/ 64 <187500000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-358000000 {
+ opp-hz = /bits/ 64 <358000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>;
+ };
+ };
+ };
+
+ mdss_dsi_phy: phy@ae94400 {
+ compatible = "qcom,sc7280-dsi-phy-7nm";
+ reg = <0x0ae94400 0x200>,
+ <0x0ae94600 0x280>,
+ <0x0ae94900 0x280>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "iface", "ref";
+
+ vdds-supply = <&vreg_dsi_supply>;
+ };
+
+ edp@aea0000 {
+ compatible = "qcom,sc7280-edp";
+ pinctrl-names = "default";
+ pinctrl-0 = <&edp_hot_plug_det>;
+
+ reg = <0xaea0000 0x200>,
+ <0xaea0200 0x200>,
+ <0xaea0400 0xc00>,
+ <0xaea1000 0x400>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <14>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_EDP_AUX_CLK>,
+ <&dispcc DISP_CC_MDSS_EDP_LINK_CLK>,
+ <&dispcc DISP_CC_MDSS_EDP_LINK_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_EDP_PIXEL_CLK>;
+ clock-names = "core_iface",
+ "core_aux",
+ "ctrl_link",
+ "ctrl_link_iface",
+ "stream_pixel";
+ assigned-clocks = <&dispcc DISP_CC_MDSS_EDP_LINK_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_EDP_PIXEL_CLK_SRC>;
+ assigned-clock-parents = <&mdss_edp_phy 0>, <&mdss_edp_phy 1>;
+
+ phys = <&mdss_edp_phy>;
+ phy-names = "dp";
+
+ operating-points-v2 = <&edp_opp_table>;
+ power-domains = <&rpmhpd SC7280_CX>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ edp_in: endpoint {
+ remote-endpoint = <&dpu_intf5_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ mdss_edp_out: endpoint { };
+ };
+ };
+
+ edp_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-270000000 {
+ opp-hz = /bits/ 64 <270000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-540000000 {
+ opp-hz = /bits/ 64 <540000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ };
+
+ opp-810000000 {
+ opp-hz = /bits/ 64 <810000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ };
+ };
+ };
+
+ mdss_edp_phy: phy@aec2a00 {
+ compatible = "qcom,sc7280-edp-phy";
+
+ reg = <0xaec2a00 0x19c>,
+ <0xaec2200 0xa0>,
+ <0xaec2600 0xa0>,
+ <0xaec2000 0x1c0>;
+
+ clocks = <&rpmhcc RPMH_CXO_CLK>,
+ <&gcc GCC_EDP_CLKREF_EN>;
+ clock-names = "aux",
+ "cfg_ahb";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+ };
+
+ displayport-controller@ae90000 {
+ compatible = "qcom,sc7280-dp";
+
+ reg = <0xae90000 0x200>,
+ <0xae90200 0x200>,
+ <0xae90400 0xc00>,
+ <0xae91000 0x400>,
+ <0xae91400 0x400>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <12>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_AUX_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_LINK_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_LINK_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_DP_PIXEL_CLK>;
+ clock-names = "core_iface",
+ "core_aux",
+ "ctrl_link",
+ "ctrl_link_iface",
+ "stream_pixel";
+ assigned-clocks = <&dispcc DISP_CC_MDSS_DP_LINK_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_DP_PIXEL_CLK_SRC>;
+ assigned-clock-parents = <&dp_phy 0>, <&dp_phy 1>;
+ phys = <&dp_phy>;
+ phy-names = "dp";
+
+ operating-points-v2 = <&dp_opp_table>;
+ power-domains = <&rpmhpd SC7280_CX>;
+
+ #sound-dai-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dp_in: endpoint {
+ remote-endpoint = <&dpu_intf0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dp_out: endpoint { };
+ };
+ };
+
+ dp_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-270000000 {
+ opp-hz = /bits/ 64 <270000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-540000000 {
+ opp-hz = /bits/ 64 <540000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>;
+ };
+
+ opp-810000000 {
+ opp-hz = /bits/ 64 <810000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sdm845-dpu.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sdm845-dpu.yaml
new file mode 100644
index 0000000..5719b45
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sdm845-dpu.yaml
@@ -0,0 +1,90 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sdm845-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display DPU dt properties for SDM845 target
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+
+$ref: /schemas/display/msm/dpu-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,sdm845-dpu
+
+ reg:
+ items:
+ - description: Address offset and size for mdp register set
+ - description: Address offset and size for vbif register set
+
+ reg-names:
+ items:
+ - const: mdp
+ - const: vbif
+
+ clocks:
+ items:
+ - description: Display GCC bus clock
+ - description: Display ahb clock
+ - description: Display axi clock
+ - description: Display core clock
+ - description: Display vsync clock
+
+ clock-names:
+ items:
+ - const: gcc-bus
+ - const: iface
+ - const: bus
+ - const: core
+ - const: vsync
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sdm845.h>
+ #include <dt-bindings/clock/qcom,gcc-sdm845.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-controller@ae01000 {
+ compatible = "qcom,sdm845-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "gcc-bus", "iface", "bus", "core", "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ power-domains = <&rpmhpd SDM845_CX>;
+ operating-points-v2 = <&mdp_opp_table>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ endpoint {
+ remote-endpoint = <&dsi1_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sdm845-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sdm845-mdss.yaml
new file mode 100644
index 0000000..31ca6f9
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sdm845-mdss.yaml
@@ -0,0 +1,270 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sdm845-mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SDM845 Display MDSS
+
+maintainers:
+ - Krishna Manikandan <quic_mkrishn@quicinc.com>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
+ sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
+ bindings of MDSS are mentioned for SDM845 target.
+
+$ref: /schemas/display/msm/mdss-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,sdm845-mdss
+
+ clocks:
+ items:
+ - description: Display AHB clock from gcc
+ - description: Display core clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: core
+
+ iommus:
+ maxItems: 2
+
+ interconnects:
+ maxItems: 2
+
+ interconnect-names:
+ maxItems: 2
+
+patternProperties:
+ "^display-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sdm845-dpu
+
+ "^dsi@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,mdss-dsi-ctrl
+
+ "^phy@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-phy-10nm
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sdm845.h>
+ #include <dt-bindings/clock/qcom,gcc-sdm845.h>
+ #include <dt-bindings/clock/qcom,rpmh.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-subsystem@ae00000 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "qcom,sdm845-mdss";
+ reg = <0x0ae00000 0x1000>;
+ reg-names = "mdss";
+ power-domains = <&dispcc MDSS_GDSC>;
+
+ clocks = <&gcc GCC_DISP_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>;
+ clock-names = "iface", "core";
+
+ interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ iommus = <&apps_smmu 0x880 0x8>,
+ <&apps_smmu 0xc80 0x8>;
+ ranges;
+
+ display-controller@ae01000 {
+ compatible = "qcom,sdm845-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "gcc-bus", "iface", "bus", "core", "vsync";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+ power-domains = <&rpmhpd SDM845_CX>;
+ operating-points-v2 = <&mdp_opp_table>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dpu_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dpu_intf2_out: endpoint {
+ remote-endpoint = <&dsi1_in>;
+ };
+ };
+ };
+ };
+
+ dsi@ae94000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0ae94000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <4>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE0_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC0_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK_SRC>;
+ assigned-clock-parents = <&dsi0_phy 0>, <&dsi0_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmhpd SDM845_CX>;
+
+ phys = <&dsi0_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&dpu_intf1_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ };
+ };
+ };
+ };
+
+ dsi0_phy: phy@ae94400 {
+ compatible = "qcom,dsi-phy-10nm";
+ reg = <0x0ae94400 0x200>,
+ <0x0ae94600 0x280>,
+ <0x0ae94a00 0x1e0>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "iface", "ref";
+ vdds-supply = <&vreg_dsi_phy>;
+ };
+
+ dsi@ae96000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0ae96000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <5>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE1_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE1_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK1_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC1_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE1_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_PCLK1_CLK_SRC>;
+ assigned-clock-parents = <&dsi1_phy 0>, <&dsi1_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmhpd SDM845_CX>;
+
+ phys = <&dsi1_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi1_in: endpoint {
+ remote-endpoint = <&dpu_intf2_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi1_out: endpoint {
+ };
+ };
+ };
+ };
+
+ dsi1_phy: phy@ae96400 {
+ compatible = "qcom,dsi-phy-10nm";
+ reg = <0x0ae96400 0x200>,
+ <0x0ae96600 0x280>,
+ <0x0ae96a00 0x10e>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "iface", "ref";
+ vdds-supply = <&vreg_dsi_phy>;
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sm6115-dpu.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sm6115-dpu.yaml
new file mode 100644
index 0000000..4a39a30
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sm6115-dpu.yaml
@@ -0,0 +1,94 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sm6115-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Display DPU dt properties for SM6115 target
+
+maintainers:
+ - Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
+
+$ref: /schemas/display/msm/dpu-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,sm6115-dpu
+
+ reg:
+ items:
+ - description: MDP register set
+ - description: VBIF register set
+
+ reg-names:
+ items:
+ - const: mdp
+ - const: vbif
+
+ clocks:
+ items:
+ - description: Display AXI
+ - description: Display AHB
+ - description: Display core
+ - description: Display lut
+ - description: Display rotator
+ - description: Display vsync
+
+ clock-names:
+ items:
+ - const: bus
+ - const: iface
+ - const: core
+ - const: lut
+ - const: rot
+ - const: vsync
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - clocks
+ - clock-names
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,sm6115-dispcc.h>
+ #include <dt-bindings/clock/qcom,gcc-sm6115.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-controller@5e01000 {
+ compatible = "qcom,sm6115-dpu";
+ reg = <0x05e01000 0x8f000>,
+ <0x05eb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_ROT_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus", "iface", "core", "lut", "rot", "vsync";
+
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmpd SM6115_VDDCX>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sm6115-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sm6115-mdss.yaml
new file mode 100644
index 0000000..a86d7f5
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sm6115-mdss.yaml
@@ -0,0 +1,182 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sm6115-mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SM6115 Display MDSS
+
+maintainers:
+ - Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
+ sub-blocks like DPU display controller and DSI. Device tree bindings of MDSS
+ are mentioned for SM6115 target.
+
+$ref: /schemas/display/msm/mdss-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,sm6115-mdss
+
+ clocks:
+ items:
+ - description: Display AHB clock from gcc
+ - description: Display AXI clock
+ - description: Display core clock
+
+ iommus:
+ maxItems: 2
+
+patternProperties:
+ "^display-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sm6115-dpu
+
+ "^dsi@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-ctrl-6g-qcm2290
+
+ "^phy@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-phy-14nm-2290
+
+required:
+ - compatible
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,sm6115-dispcc.h>
+ #include <dt-bindings/clock/qcom,gcc-sm6115.h>
+ #include <dt-bindings/clock/qcom,rpmcc.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ mdss@5e00000 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "qcom,sm6115-mdss";
+ reg = <0x05e00000 0x1000>;
+ reg-names = "mdss";
+ power-domains = <&dispcc MDSS_GDSC>;
+ clocks = <&gcc GCC_DISP_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>;
+
+ interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ iommus = <&apps_smmu 0x420 0x2>,
+ <&apps_smmu 0x421 0x0>;
+ ranges;
+
+ display-controller@5e01000 {
+ compatible = "qcom,sm6115-dpu";
+ reg = <0x05e01000 0x8f000>,
+ <0x05eb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_LUT_CLK>,
+ <&dispcc DISP_CC_MDSS_ROT_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "bus", "iface", "core", "lut", "rot", "vsync";
+
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmpd SM6115_VDDCX>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dpu_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+ };
+ };
+
+ dsi@5e94000 {
+ compatible = "qcom,dsi-ctrl-6g-qcm2290";
+ reg = <0x05e94000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <4>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE0_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC0_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK_SRC>, <&dispcc DISP_CC_MDSS_PCLK0_CLK_SRC>;
+ assigned-clock-parents = <&dsi0_phy 0>, <&dsi0_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmpd SM6115_VDDCX>;
+ phys = <&dsi0_phy>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&dpu_intf1_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ };
+ };
+ };
+ };
+
+ dsi0_phy: phy@5e94400 {
+ compatible = "qcom,dsi-phy-14nm-2290";
+ reg = <0x05e94400 0x100>,
+ <0x05e94500 0x300>,
+ <0x05e94800 0x188>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>, <&rpmcc RPM_SMD_XO_CLK_SRC>;
+ clock-names = "iface", "ref";
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sm8250-dpu.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sm8250-dpu.yaml
new file mode 100644
index 0000000..9ff8a26
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sm8250-dpu.yaml
@@ -0,0 +1,92 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sm8250-dpu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SM8250 Display DPU
+
+maintainers:
+ - Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
+
+$ref: /schemas/display/msm/dpu-common.yaml#
+
+properties:
+ compatible:
+ const: qcom,sm8250-dpu
+
+ reg:
+ items:
+ - description: Address offset and size for mdp register set
+ - description: Address offset and size for vbif register set
+
+ reg-names:
+ items:
+ - const: mdp
+ - const: vbif
+
+ clocks:
+ items:
+ - description: Display ahb clock
+ - description: Display hf axi clock
+ - description: Display core clock
+ - description: Display vsync clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: bus
+ - const: core
+ - const: vsync
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sm8250.h>
+ #include <dt-bindings/clock/qcom,gcc-sm8250.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interconnect/qcom,sm8250.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-controller@ae01000 {
+ compatible = "qcom,sm8250-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "iface", "bus", "core", "vsync";
+
+ assigned-clocks = <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ assigned-clock-rates = <19200000>;
+
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmhpd SM8250_MMCX>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ endpoint {
+ remote-endpoint = <&dsi1_in>;
+ };
+ };
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sm8250-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sm8250-mdss.yaml
new file mode 100644
index 0000000..0d3be53
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/msm/qcom,sm8250-mdss.yaml
@@ -0,0 +1,330 @@
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/msm/qcom,sm8250-mdss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SM8250 Display MDSS
+
+maintainers:
+ - Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
+
+description:
+ Device tree bindings for MSM Mobile Display Subsystem(MDSS) that encapsulates
+ sub-blocks like DPU display controller, DSI and DP interfaces etc. Device tree
+ bindings of MDSS are mentioned for SM8250 target.
+
+$ref: /schemas/display/msm/mdss-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - const: qcom,sm8250-mdss
+
+ clocks:
+ items:
+ - description: Display AHB clock from gcc
+ - description: Display hf axi clock
+ - description: Display sf axi clock
+ - description: Display core clock
+
+ clock-names:
+ items:
+ - const: iface
+ - const: bus
+ - const: nrt_bus
+ - const: core
+
+ iommus:
+ maxItems: 1
+
+ interconnects:
+ maxItems: 2
+
+ interconnect-names:
+ maxItems: 2
+
+patternProperties:
+ "^display-controller@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,sm8250-dpu
+
+ "^dsi@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,mdss-dsi-ctrl
+
+ "^phy@[0-9a-f]+$":
+ type: object
+ properties:
+ compatible:
+ const: qcom,dsi-phy-7nm
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,dispcc-sm8250.h>
+ #include <dt-bindings/clock/qcom,gcc-sm8250.h>
+ #include <dt-bindings/clock/qcom,rpmh.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interconnect/qcom,sm8250.h>
+ #include <dt-bindings/power/qcom-rpmpd.h>
+
+ display-subsystem@ae00000 {
+ compatible = "qcom,sm8250-mdss";
+ reg = <0x0ae00000 0x1000>;
+ reg-names = "mdss";
+
+ interconnects = <&mmss_noc MASTER_MDP_PORT0 &mc_virt SLAVE_EBI_CH0>,
+ <&mmss_noc MASTER_MDP_PORT1 &mc_virt SLAVE_EBI_CH0>;
+ interconnect-names = "mdp0-mem", "mdp1-mem";
+
+ power-domains = <&dispcc MDSS_GDSC>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&gcc GCC_DISP_SF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>;
+ clock-names = "iface", "bus", "nrt_bus", "core";
+
+ interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ iommus = <&apps_smmu 0x820 0x402>;
+
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ display-controller@ae01000 {
+ compatible = "qcom,sm8250-dpu";
+ reg = <0x0ae01000 0x8f000>,
+ <0x0aeb0000 0x2008>;
+ reg-names = "mdp", "vbif";
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>,
+ <&dispcc DISP_CC_MDSS_MDP_CLK>,
+ <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ clock-names = "iface", "bus", "core", "vsync";
+
+ assigned-clocks = <&dispcc DISP_CC_MDSS_VSYNC_CLK>;
+ assigned-clock-rates = <19200000>;
+
+ operating-points-v2 = <&mdp_opp_table>;
+ power-domains = <&rpmhpd SM8250_MMCX>;
+
+ interrupt-parent = <&mdss>;
+ interrupts = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dpu_intf1_out: endpoint {
+ remote-endpoint = <&dsi0_in>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dpu_intf2_out: endpoint {
+ remote-endpoint = <&dsi1_in>;
+ };
+ };
+ };
+
+ mdp_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-345000000 {
+ opp-hz = /bits/ 64 <345000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>;
+ };
+
+ opp-460000000 {
+ opp-hz = /bits/ 64 <460000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ };
+ };
+ };
+
+ dsi@ae94000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0ae94000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <4>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE0_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC0_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE0_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_PCLK0_CLK_SRC>;
+ assigned-clock-parents = <&dsi0_phy 0>, <&dsi0_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmhpd SM8250_MMCX>;
+
+ phys = <&dsi0_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi0_in: endpoint {
+ remote-endpoint = <&dpu_intf1_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi0_out: endpoint {
+ };
+ };
+ };
+
+ dsi_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ opp-187500000 {
+ opp-hz = /bits/ 64 <187500000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ };
+
+ opp-300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ required-opps = <&rpmhpd_opp_svs>;
+ };
+
+ opp-358000000 {
+ opp-hz = /bits/ 64 <358000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>;
+ };
+ };
+ };
+
+ dsi0_phy: phy@ae94400 {
+ compatible = "qcom,dsi-phy-7nm";
+ reg = <0x0ae94400 0x200>,
+ <0x0ae94600 0x280>,
+ <0x0ae94900 0x260>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "iface", "ref";
+ vdds-supply = <&vreg_dsi_phy>;
+ };
+
+ dsi@ae96000 {
+ compatible = "qcom,mdss-dsi-ctrl";
+ reg = <0x0ae96000 0x400>;
+ reg-names = "dsi_ctrl";
+
+ interrupt-parent = <&mdss>;
+ interrupts = <5>;
+
+ clocks = <&dispcc DISP_CC_MDSS_BYTE1_CLK>,
+ <&dispcc DISP_CC_MDSS_BYTE1_INTF_CLK>,
+ <&dispcc DISP_CC_MDSS_PCLK1_CLK>,
+ <&dispcc DISP_CC_MDSS_ESC1_CLK>,
+ <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&gcc GCC_DISP_HF_AXI_CLK>;
+ clock-names = "byte",
+ "byte_intf",
+ "pixel",
+ "core",
+ "iface",
+ "bus";
+
+ assigned-clocks = <&dispcc DISP_CC_MDSS_BYTE1_CLK_SRC>,
+ <&dispcc DISP_CC_MDSS_PCLK1_CLK_SRC>;
+ assigned-clock-parents = <&dsi1_phy 0>, <&dsi1_phy 1>;
+
+ operating-points-v2 = <&dsi_opp_table>;
+ power-domains = <&rpmhpd SM8250_MMCX>;
+
+ phys = <&dsi1_phy>;
+ phy-names = "dsi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ dsi1_in: endpoint {
+ remote-endpoint = <&dpu_intf2_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ dsi1_out: endpoint {
+ };
+ };
+ };
+ };
+
+ dsi1_phy: phy@ae96400 {
+ compatible = "qcom,dsi-phy-7nm";
+ reg = <0x0ae96400 0x200>,
+ <0x0ae96600 0x280>,
+ <0x0ae96900 0x260>;
+ reg-names = "dsi_phy",
+ "dsi_phy_lane",
+ "dsi_pll";
+
+ #clock-cells = <1>;
+ #phy-cells = <0>;
+
+ clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>,
+ <&rpmhcc RPMH_CXO_CLK>;
+ clock-names = "iface", "ref";
+ vdds-supply = <&vreg_dsi_phy>;
+ };
+ };
+...
diff --git a/Documentation/devicetree/bindings/display/panel/jadard,jd9365da-h3.yaml b/Documentation/devicetree/bindings/display/panel/jadard,jd9365da-h3.yaml
new file mode 100644
index 0000000..c06902e
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/jadard,jd9365da-h3.yaml
@@ -0,0 +1,70 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/panel/jadard,jd9365da-h3.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Jadard JD9365DA-HE WXGA DSI panel
+
+maintainers:
+ - Jagan Teki <jagan@edgeble.ai>
+
+allOf:
+ - $ref: panel-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - chongzhou,cz101b4001
+ - const: jadard,jd9365da-h3
+
+ reg: true
+
+ vdd-supply:
+ description: supply regulator for VDD, usually 3.3V
+
+ vccio-supply:
+ description: supply regulator for VCCIO, usually 1.8V
+
+ reset-gpios: true
+
+ backlight: true
+
+ port: true
+
+required:
+ - compatible
+ - reg
+ - vdd-supply
+ - vccio-supply
+ - reset-gpios
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+ #include <dt-bindings/pinctrl/rockchip.h>
+
+ dsi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ panel@0 {
+ compatible = "chongzhou,cz101b4001", "jadard,jd9365da-h3";
+ reg = <0>;
+ vdd-supply = <&lcd_3v3>;
+ vccio-supply = <&vcca_1v8>;
+ reset-gpios = <&gpio1 RK_PC2 GPIO_ACTIVE_HIGH>;
+ backlight = <&backlight>;
+
+ port {
+ mipi_in_panel: endpoint {
+ remote-endpoint = <&mipi_out_panel>;
+ };
+ };
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/display/panel/newvision,nv3051d.yaml b/Documentation/devicetree/bindings/display/panel/newvision,nv3051d.yaml
new file mode 100644
index 0000000..116c1b6
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/newvision,nv3051d.yaml
@@ -0,0 +1,63 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/display/panel/newvision,nv3051d.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NewVision NV3051D based LCD panel
+
+description: |
+ The NewVision NV3051D is a driver chip used to drive DSI panels. For now,
+ this driver only supports the 640x480 panels found in the Anbernic RG353
+ based devices.
+
+maintainers:
+ - Chris Morgan <macromorgan@hotmail.com>
+
+allOf:
+ - $ref: panel-common.yaml#
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - anbernic,rg353p-panel
+ - anbernic,rg353v-panel
+ - const: newvision,nv3051d
+
+ reg: true
+ backlight: true
+ port: true
+ reset-gpios:
+ description: Active low reset GPIO
+ vdd-supply: true
+
+required:
+ - compatible
+ - reg
+ - backlight
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+ dsi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ panel@0 {
+ compatible = "anbernic,rg353p-panel", "newvision,nv3051d";
+ reg = <0>;
+ backlight = <&backlight>;
+ reset-gpios = <&gpio4 0 GPIO_ACTIVE_LOW>;
+ vdd-supply = <&vcc3v3_lcd>;
+
+ port {
+ mipi_in_panel: endpoint {
+ remote-endpoint = <&mipi_out_panel>;
+ };
+ };
+ };
+ };
+
+...
diff --git a/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml b/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml
index 10ec78ca..554f9d5 100644
--- a/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml
+++ b/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml
@@ -117,6 +117,45 @@
- const: dp-phy0
- const: dp-phy1
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+ description: |
+ Connections to the programmable logic and the DisplayPort PHYs. Each port
+ shall have a single endpoint.
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: The live video input from the programmable logic
+
+ port@1:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: The live graphics input from the programmable logic
+
+ port@2:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: The live audio input from the programmable logic
+
+ port@3:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: The blended video output to the programmable logic
+
+ port@4:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: The mixed audio output to the programmable logic
+
+ port@5:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: The DisplayPort output
+
+ required:
+ - port@0
+ - port@1
+ - port@2
+ - port@3
+ - port@4
+ - port@5
+
required:
- compatible
- reg
@@ -130,6 +169,7 @@
- dma-names
- phys
- phy-names
+ - ports
additionalProperties: false
@@ -164,6 +204,33 @@
<&psgtr 0 PHY_TYPE_DP 1 3>;
phy-names = "dp-phy0", "dp-phy1";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ };
+ port@1 {
+ reg = <1>;
+ };
+ port@2 {
+ reg = <2>;
+ };
+ port@3 {
+ reg = <3>;
+ };
+ port@4 {
+ reg = <4>;
+ };
+ port@5 {
+ reg = <5>;
+ dpsub_dp_out: endpoint {
+ remote-endpoint = <&dp_connector>;
+ };
+ };
+ };
};
...
diff --git a/Documentation/devicetree/bindings/vendor-prefixes.yaml b/Documentation/devicetree/bindings/vendor-prefixes.yaml
index cbe12da..83ba9c4 100644
--- a/Documentation/devicetree/bindings/vendor-prefixes.yaml
+++ b/Documentation/devicetree/bindings/vendor-prefixes.yaml
@@ -248,6 +248,8 @@
description: ChipOne
"^chipspark,.*":
description: ChipSPARK
+ "^chongzhou,.*":
+ description: Shenzhen Chongzhou Electronic Technology Co., Ltd
"^chrontel,.*":
description: Chrontel, Inc.
"^chrp,.*":
@@ -645,6 +647,8 @@
description: ITian Corporation
"^iwave,.*":
description: iWave Systems Technologies Pvt. Ltd.
+ "^jadard,.*":
+ description: Jadard Technology Inc.
"^jdi,.*":
description: Japan Display Inc.
"^jedec,.*":
@@ -889,6 +893,8 @@
description: Shenzhen Netxeon Technology CO., LTD
"^neweast,.*":
description: Guangdong Neweast Optoelectronics CO., LTD
+ "^newvision,.*":
+ description: New Vision Display (Shenzhen) Co., Ltd.
"^nexbox,.*":
description: Nexbox
"^nextthing,.*":
diff --git a/Documentation/driver-api/dma-buf.rst b/Documentation/driver-api/dma-buf.rst
index 36a76cb..622b815 100644
--- a/Documentation/driver-api/dma-buf.rst
+++ b/Documentation/driver-api/dma-buf.rst
@@ -119,6 +119,12 @@
.. kernel-doc:: include/uapi/linux/dma-buf.h
+DMA-BUF locking convention
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. kernel-doc:: drivers/dma-buf/dma-buf.c
+ :doc: locking convention
+
Kernel Functions and Structures Reference
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/Documentation/fb/modedb.rst b/Documentation/fb/modedb.rst
index 4d2411e..e533750 100644
--- a/Documentation/fb/modedb.rst
+++ b/Documentation/fb/modedb.rst
@@ -26,6 +26,11 @@
with <xres>, <yres>, <bpp> and <refresh> decimal numbers and <name> a string.
Things between square brackets are optional.
+Valid names are::
+
+ - NSTC: 480i output, with the CCIR System-M TV mode and NTSC color encoding
+ - PAL: 576i output, with the CCIR System-B TV mode and PAL color encoding
+
If 'M' is specified in the mode_option argument (after <yres> and before
<bpp> and <refresh>, if specified) the timings will be calculated using
VESA(TM) Coordinated Video Timings instead of looking up the mode from a table.
diff --git a/Documentation/gpu/amdgpu/amdgpu-glossary.rst b/Documentation/gpu/amdgpu/amdgpu-glossary.rst
index 326896e..00a47eb 100644
--- a/Documentation/gpu/amdgpu/amdgpu-glossary.rst
+++ b/Documentation/gpu/amdgpu/amdgpu-glossary.rst
@@ -30,12 +30,35 @@
EOP
End Of Pipe/Pipeline
+ GART
+ Graphics Address Remapping Table. This is the name we use for the GPUVM
+ page table used by the GPU kernel driver. It remaps system resources
+ (memory or MMIO space) into the GPU's address space so the GPU can access
+ them. The name GART harkens back to the days of AGP when the platform
+ provided an MMU that the GPU could use to get a contiguous view of
+ scattered pages for DMA. The MMU has since moved on to the GPU, but the
+ name stuck.
+
GC
Graphics and Compute
GMC
Graphic Memory Controller
+ GPUVM
+ GPU Virtual Memory. This is the GPU's MMU. The GPU supports multiple
+ virtual address spaces that can be in flight at any given time. These
+ allow the GPU to remap VRAM and system resources into GPU virtual address
+ spaces for use by the GPU kernel driver and applications using the GPU.
+ These provide memory protection for different applications using the GPU.
+
+ GTT
+ Graphics Translation Tables. This is a memory pool managed through TTM
+ which provides access to system resources (memory or MMIO space) for
+ use by the GPU. These addresses can be mapped into the "GART" GPUVM page
+ table for use by the kernel driver or into per process GPUVM page tables
+ for application usage.
+
IH
Interrupt Handler
diff --git a/Documentation/gpu/amdgpu/driver-core.rst b/Documentation/gpu/amdgpu/driver-core.rst
index ebf5932..467e684 100644
--- a/Documentation/gpu/amdgpu/driver-core.rst
+++ b/Documentation/gpu/amdgpu/driver-core.rst
@@ -148,10 +148,10 @@
MMU Notifier
============
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c
:doc: MMU Notifier
-.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c
:internal:
AMDGPU Virtual Memory
diff --git a/Documentation/gpu/amdgpu/index.rst b/Documentation/gpu/amdgpu/index.rst
index a24e1cf..03c2966 100644
--- a/Documentation/gpu/amdgpu/index.rst
+++ b/Documentation/gpu/amdgpu/index.rst
@@ -3,7 +3,7 @@
==========================
The drm/amdgpu driver supports all AMD Radeon GPUs based on the Graphics Core
-Next (GCN) architecture.
+Next (GCN), Radeon DNA (RDNA), and Compute DNA (CDNA) architectures.
.. toctree::
diff --git a/Documentation/gpu/drm-kms-helpers.rst b/Documentation/gpu/drm-kms-helpers.rst
index dbc85fd..a4860ff 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -116,6 +116,9 @@
.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
:export:
+.. kernel-doc:: drivers/gpu/drm/drm_fbdev_generic.c
+ :export:
+
format Helper Functions Reference
=================================
diff --git a/Documentation/gpu/drm-usage-stats.rst b/Documentation/gpu/drm-usage-stats.rst
index 92c5117..b463273 100644
--- a/Documentation/gpu/drm-usage-stats.rst
+++ b/Documentation/gpu/drm-usage-stats.rst
@@ -126,7 +126,6 @@
time active without considering what frequency the engine is operating as a
percentage of it's maximum frequency.
-===============================
Driver specific implementations
===============================
diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
index 4e59db1c..60ea217 100644
--- a/Documentation/gpu/i915.rst
+++ b/Documentation/gpu/i915.rst
@@ -494,7 +494,7 @@
WOPCM Layout
~~~~~~~~~~~~
-.. kernel-doc:: drivers/gpu/drm/i915/intel_wopcm.c
+.. kernel-doc:: drivers/gpu/drm/i915/gt/intel_wopcm.c
:doc: WOPCM Layout
GuC
diff --git a/Documentation/gpu/todo.rst b/Documentation/gpu/todo.rst
index 393d218..b2c6aaf 100644
--- a/Documentation/gpu/todo.rst
+++ b/Documentation/gpu/todo.rst
@@ -651,17 +651,6 @@
Contact: Harry Wentland, Alex Deucher
-vmwgfx: Replace hashtable with Linux' implementation
-----------------------------------------------------
-
-The vmwgfx driver uses its own hashtable implementation. Replace the
-code with Linux' implementation and update the callers. It's mostly a
-refactoring task, but the interfaces are different.
-
-Contact: Zack Rusin, Thomas Zimmermann <tzimmermann@suse.de>
-
-Level: Intermediate
-
Bootsplash
==========
diff --git a/Documentation/subsystem-apis.rst b/Documentation/subsystem-apis.rst
index af65004..b51f385 100644
--- a/Documentation/subsystem-apis.rst
+++ b/Documentation/subsystem-apis.rst
@@ -43,6 +43,7 @@
input/index
hwmon/index
gpu/index
+ accel/index
security/index
sound/index
crypto/index
diff --git a/MAINTAINERS b/MAINTAINERS
index 658db9f..363a2ca 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6539,6 +6539,12 @@
F: drivers/gpu/drm/i810/
F: include/uapi/drm/i810_drm.h
+DRM DRIVER FOR JADARD JD9365DA-H3 MIPI-DSI LCD PANELS
+M: Jagan Teki <jagan@edgeble.ai>
+S: Maintained
+F: Documentation/devicetree/bindings/display/panel/jadard,jd9365da-h3.yaml
+F: drivers/gpu/drm/panel/panel-jadard-jd9365da-h3.c
+
DRM DRIVER FOR LOGICVC DISPLAY CONTROLLER
M: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
S: Supported
@@ -6728,10 +6734,13 @@
S: Maintained
T: git git://anongit.freedesktop.org/drm/drm-misc
F: drivers/gpu/drm/drm_aperture.c
+F: drivers/gpu/drm/tiny/ofdrm.c
F: drivers/gpu/drm/tiny/simpledrm.c
F: drivers/video/aperture.c
+F: drivers/video/nomodeset.c
F: include/drm/drm_aperture.h
F: include/linux/aperture.h
+F: include/video/nomodeset.h
DRM DRIVER FOR SIS VIDEO CARDS
S: Orphan / Obsolete
@@ -6860,6 +6869,15 @@
F: include/linux/vga*
F: include/uapi/drm/drm*
+DRM COMPUTE ACCELERATORS DRIVERS AND FRAMEWORK
+M: Oded Gabbay <ogabbay@kernel.org>
+L: dri-devel@lists.freedesktop.org
+S: Maintained
+C: irc://irc.oftc.net/dri-devel
+T: git https://git.kernel.org/pub/scm/linux/kernel/git/ogabbay/accel.git
+F: Documentation/accel/
+F: drivers/accel/
+
DRM DRIVERS FOR ALLWINNER A10
M: Maxime Ripard <mripard@kernel.org>
M: Chen-Yu Tsai <wens@csie.org>
@@ -7148,7 +7166,7 @@
F: include/drm/ttm/
DRM GPU SCHEDULER
-M: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
+M: Luben Tuikov <luben.tuikov@amd.com>
L: dri-devel@lists.freedesktop.org
S: Maintained
T: git git://anongit.freedesktop.org/drm/drm-misc
@@ -10273,6 +10291,7 @@
B: https://gitlab.freedesktop.org/drm/intel/-/wikis/How-to-file-i915-bugs
C: irc://irc.oftc.net/intel-gfx
T: git git://anongit.freedesktop.org/drm-intel
+F: Documentation/ABI/testing/sysfs-driver-intel-i915-hwmon
F: Documentation/gpu/i915.rst
F: drivers/gpu/drm/i915/
F: include/drm/i915*
diff --git a/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts b/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts
index e2dd72f..24a2523 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts
+++ b/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts
@@ -150,6 +150,18 @@ refhdmi: refhdmi {
#clock-cells = <0>;
clock-frequency = <114285000>;
};
+
+ dpcon {
+ compatible = "dp-connector";
+ label = "P11";
+ type = "full-size";
+
+ port {
+ dpcon_in: endpoint {
+ remote-endpoint = <&dpsub_dp_out>;
+ };
+ };
+ };
};
&can1 {
@@ -1015,4 +1027,12 @@ &zynqmp_dpsub {
phy-names = "dp-phy0", "dp-phy1";
phys = <&psgtr 1 PHY_TYPE_DP 0 3>,
<&psgtr 0 PHY_TYPE_DP 1 3>;
+
+ ports {
+ port@5 {
+ dpsub_dp_out: endpoint {
+ remote-endpoint = <&dpcon_in>;
+ };
+ };
+ };
};
diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
index a549265..307c76c 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
+++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
@@ -930,6 +930,30 @@ zynqmp_dpsub: display@fd4a0000 {
<&zynqmp_dpdma ZYNQMP_DPDMA_VIDEO1>,
<&zynqmp_dpdma ZYNQMP_DPDMA_VIDEO2>,
<&zynqmp_dpdma ZYNQMP_DPDMA_GRAPHICS>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ };
+ port@1 {
+ reg = <1>;
+ };
+ port@2 {
+ reg = <2>;
+ };
+ port@3 {
+ reg = <3>;
+ };
+ port@4 {
+ reg = <4>;
+ };
+ port@5 {
+ reg = <5>;
+ };
+ };
};
};
};
diff --git a/drivers/Kconfig b/drivers/Kconfig
index 19ee995..968bd0a 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -99,6 +99,8 @@
source "drivers/video/Kconfig"
+source "drivers/accel/Kconfig"
+
source "sound/Kconfig"
source "drivers/hid/Kconfig"
diff --git a/drivers/accel/Kconfig b/drivers/accel/Kconfig
new file mode 100644
index 0000000..c9ce849
--- /dev/null
+++ b/drivers/accel/Kconfig
@@ -0,0 +1,24 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Compute Acceleration device configuration
+#
+# This framework provides support for compute acceleration devices, such
+# as, but not limited to, Machine-Learning and Deep-Learning acceleration
+# devices
+#
+menuconfig DRM_ACCEL
+ bool "Compute Acceleration Framework"
+ depends on DRM
+ help
+ Framework for device drivers of compute acceleration devices, such
+ as, but not limited to, Machine-Learning and Deep-Learning
+ acceleration devices.
+ If you say Y here, you need to select the module that's right for
+ your acceleration device from the list below.
+ This framework is integrated with the DRM subsystem as compute
+ accelerators and GPUs share a lot in common and can use almost the
+ same infrastructure code.
+ Having said that, acceleration devices will have a different
+ major number than GPUs, and will be exposed to user-space using
+ different device files, called accel/accel* (in /dev, sysfs
+ and debugfs).
diff --git a/drivers/accel/drm_accel.c b/drivers/accel/drm_accel.c
new file mode 100644
index 0000000..a5ee84a
--- /dev/null
+++ b/drivers/accel/drm_accel.c
@@ -0,0 +1,323 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Copyright 2022 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ *
+ */
+
+#include <linux/debugfs.h>
+#include <linux/device.h>
+#include <linux/idr.h>
+
+#include <drm/drm_accel.h>
+#include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_file.h>
+#include <drm/drm_ioctl.h>
+#include <drm/drm_print.h>
+
+static DEFINE_SPINLOCK(accel_minor_lock);
+static struct idr accel_minors_idr;
+
+static struct dentry *accel_debugfs_root;
+static struct class *accel_class;
+
+static struct device_type accel_sysfs_device_minor = {
+ .name = "accel_minor"
+};
+
+static char *accel_devnode(struct device *dev, umode_t *mode)
+{
+ return kasprintf(GFP_KERNEL, "accel/%s", dev_name(dev));
+}
+
+static int accel_sysfs_init(void)
+{
+ accel_class = class_create(THIS_MODULE, "accel");
+ if (IS_ERR(accel_class))
+ return PTR_ERR(accel_class);
+
+ accel_class->devnode = accel_devnode;
+
+ return 0;
+}
+
+static void accel_sysfs_destroy(void)
+{
+ if (IS_ERR_OR_NULL(accel_class))
+ return;
+ class_destroy(accel_class);
+ accel_class = NULL;
+}
+
+static int accel_name_info(struct seq_file *m, void *data)
+{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_minor *minor = node->minor;
+ struct drm_device *dev = minor->dev;
+ struct drm_master *master;
+
+ mutex_lock(&dev->master_mutex);
+ master = dev->master;
+ seq_printf(m, "%s", dev->driver->name);
+ if (dev->dev)
+ seq_printf(m, " dev=%s", dev_name(dev->dev));
+ if (master && master->unique)
+ seq_printf(m, " master=%s", master->unique);
+ if (dev->unique)
+ seq_printf(m, " unique=%s", dev->unique);
+ seq_puts(m, "\n");
+ mutex_unlock(&dev->master_mutex);
+
+ return 0;
+}
+
+static const struct drm_info_list accel_debugfs_list[] = {
+ {"name", accel_name_info, 0}
+};
+#define ACCEL_DEBUGFS_ENTRIES ARRAY_SIZE(accel_debugfs_list)
+
+/**
+ * accel_debugfs_init() - Initialize debugfs for accel minor
+ * @minor: Pointer to the drm_minor instance.
+ * @minor_id: The minor's id
+ *
+ * This function initializes the drm minor's debugfs members and creates
+ * a root directory for the minor in debugfs. It also creates common files
+ * for accelerators and calls the driver's debugfs init callback.
+ */
+void accel_debugfs_init(struct drm_minor *minor, int minor_id)
+{
+ struct drm_device *dev = minor->dev;
+ char name[64];
+
+ INIT_LIST_HEAD(&minor->debugfs_list);
+ mutex_init(&minor->debugfs_lock);
+ sprintf(name, "%d", minor_id);
+ minor->debugfs_root = debugfs_create_dir(name, accel_debugfs_root);
+
+ drm_debugfs_create_files(accel_debugfs_list, ACCEL_DEBUGFS_ENTRIES,
+ minor->debugfs_root, minor);
+
+ if (dev->driver->debugfs_init)
+ dev->driver->debugfs_init(minor);
+}
+
+/**
+ * accel_set_device_instance_params() - Set some device parameters for accel device
+ * @kdev: Pointer to the device instance.
+ * @index: The minor's index
+ *
+ * This function creates the dev_t of the device using the accel major and
+ * the device's minor number. In addition, it sets the class and type of the
+ * device instance to the accel sysfs class and device type, respectively.
+ */
+void accel_set_device_instance_params(struct device *kdev, int index)
+{
+ kdev->devt = MKDEV(ACCEL_MAJOR, index);
+ kdev->class = accel_class;
+ kdev->type = &accel_sysfs_device_minor;
+}
+
+/**
+ * accel_minor_alloc() - Allocates a new accel minor
+ *
+ * This function access the accel minors idr and allocates from it
+ * a new id to represent a new accel minor
+ *
+ * Return: A new id on success or error code in case idr_alloc failed
+ */
+int accel_minor_alloc(void)
+{
+ unsigned long flags;
+ int r;
+
+ spin_lock_irqsave(&accel_minor_lock, flags);
+ r = idr_alloc(&accel_minors_idr, NULL, 0, ACCEL_MAX_MINORS, GFP_NOWAIT);
+ spin_unlock_irqrestore(&accel_minor_lock, flags);
+
+ return r;
+}
+
+/**
+ * accel_minor_remove() - Remove an accel minor
+ * @index: The minor id to remove.
+ *
+ * This function access the accel minors idr and removes from
+ * it the member with the id that is passed to this function.
+ */
+void accel_minor_remove(int index)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&accel_minor_lock, flags);
+ idr_remove(&accel_minors_idr, index);
+ spin_unlock_irqrestore(&accel_minor_lock, flags);
+}
+
+/**
+ * accel_minor_replace() - Replace minor pointer in accel minors idr.
+ * @minor: Pointer to the new minor.
+ * @index: The minor id to replace.
+ *
+ * This function access the accel minors idr structure and replaces the pointer
+ * that is associated with an existing id. Because the minor pointer can be
+ * NULL, we need to explicitly pass the index.
+ *
+ * Return: 0 for success, negative value for error
+ */
+void accel_minor_replace(struct drm_minor *minor, int index)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&accel_minor_lock, flags);
+ idr_replace(&accel_minors_idr, minor, index);
+ spin_unlock_irqrestore(&accel_minor_lock, flags);
+}
+
+/*
+ * Looks up the given minor-ID and returns the respective DRM-minor object. The
+ * refence-count of the underlying device is increased so you must release this
+ * object with accel_minor_release().
+ *
+ * The object can be only a drm_minor that represents an accel device.
+ *
+ * As long as you hold this minor, it is guaranteed that the object and the
+ * minor->dev pointer will stay valid! However, the device may get unplugged and
+ * unregistered while you hold the minor.
+ */
+static struct drm_minor *accel_minor_acquire(unsigned int minor_id)
+{
+ struct drm_minor *minor;
+ unsigned long flags;
+
+ spin_lock_irqsave(&accel_minor_lock, flags);
+ minor = idr_find(&accel_minors_idr, minor_id);
+ if (minor)
+ drm_dev_get(minor->dev);
+ spin_unlock_irqrestore(&accel_minor_lock, flags);
+
+ if (!minor) {
+ return ERR_PTR(-ENODEV);
+ } else if (drm_dev_is_unplugged(minor->dev)) {
+ drm_dev_put(minor->dev);
+ return ERR_PTR(-ENODEV);
+ }
+
+ return minor;
+}
+
+static void accel_minor_release(struct drm_minor *minor)
+{
+ drm_dev_put(minor->dev);
+}
+
+/**
+ * accel_open - open method for ACCEL file
+ * @inode: device inode
+ * @filp: file pointer.
+ *
+ * This function must be used by drivers as their &file_operations.open method.
+ * It looks up the correct ACCEL device and instantiates all the per-file
+ * resources for it. It also calls the &drm_driver.open driver callback.
+ *
+ * Return: 0 on success or negative errno value on failure.
+ */
+int accel_open(struct inode *inode, struct file *filp)
+{
+ struct drm_device *dev;
+ struct drm_minor *minor;
+ int retcode;
+
+ minor = accel_minor_acquire(iminor(inode));
+ if (IS_ERR(minor))
+ return PTR_ERR(minor);
+
+ dev = minor->dev;
+
+ atomic_fetch_inc(&dev->open_count);
+
+ /* share address_space across all char-devs of a single device */
+ filp->f_mapping = dev->anon_inode->i_mapping;
+
+ retcode = drm_open_helper(filp, minor);
+ if (retcode)
+ goto err_undo;
+
+ return 0;
+
+err_undo:
+ atomic_dec(&dev->open_count);
+ accel_minor_release(minor);
+ return retcode;
+}
+EXPORT_SYMBOL_GPL(accel_open);
+
+static int accel_stub_open(struct inode *inode, struct file *filp)
+{
+ const struct file_operations *new_fops;
+ struct drm_minor *minor;
+ int err;
+
+ minor = accel_minor_acquire(iminor(inode));
+ if (IS_ERR(minor))
+ return PTR_ERR(minor);
+
+ new_fops = fops_get(minor->dev->driver->fops);
+ if (!new_fops) {
+ err = -ENODEV;
+ goto out;
+ }
+
+ replace_fops(filp, new_fops);
+ if (filp->f_op->open)
+ err = filp->f_op->open(inode, filp);
+ else
+ err = 0;
+
+out:
+ accel_minor_release(minor);
+
+ return err;
+}
+
+static const struct file_operations accel_stub_fops = {
+ .owner = THIS_MODULE,
+ .open = accel_stub_open,
+ .llseek = noop_llseek,
+};
+
+void accel_core_exit(void)
+{
+ unregister_chrdev(ACCEL_MAJOR, "accel");
+ debugfs_remove(accel_debugfs_root);
+ accel_sysfs_destroy();
+ idr_destroy(&accel_minors_idr);
+}
+
+int __init accel_core_init(void)
+{
+ int ret;
+
+ idr_init(&accel_minors_idr);
+
+ ret = accel_sysfs_init();
+ if (ret < 0) {
+ DRM_ERROR("Cannot create ACCEL class: %d\n", ret);
+ goto error;
+ }
+
+ accel_debugfs_root = debugfs_create_dir("accel", NULL);
+
+ ret = register_chrdev(ACCEL_MAJOR, "accel", &accel_stub_fops);
+ if (ret < 0)
+ DRM_ERROR("Cannot register ACCEL major: %d\n", ret);
+
+error:
+ /*
+ * Any cleanup due to errors will be done in drm_core_exit() that
+ * will call accel_core_exit()
+ */
+ return ret;
+}
diff --git a/drivers/clk/bcm/clk-raspberrypi.c b/drivers/clk/bcm/clk-raspberrypi.c
index 679f464..ce2f934 100644
--- a/drivers/clk/bcm/clk-raspberrypi.c
+++ b/drivers/clk/bcm/clk-raspberrypi.c
@@ -18,25 +18,6 @@
#include <soc/bcm2835/raspberrypi-firmware.h>
-enum rpi_firmware_clk_id {
- RPI_FIRMWARE_EMMC_CLK_ID = 1,
- RPI_FIRMWARE_UART_CLK_ID,
- RPI_FIRMWARE_ARM_CLK_ID,
- RPI_FIRMWARE_CORE_CLK_ID,
- RPI_FIRMWARE_V3D_CLK_ID,
- RPI_FIRMWARE_H264_CLK_ID,
- RPI_FIRMWARE_ISP_CLK_ID,
- RPI_FIRMWARE_SDRAM_CLK_ID,
- RPI_FIRMWARE_PIXEL_CLK_ID,
- RPI_FIRMWARE_PWM_CLK_ID,
- RPI_FIRMWARE_HEVC_CLK_ID,
- RPI_FIRMWARE_EMMC2_CLK_ID,
- RPI_FIRMWARE_M2MC_CLK_ID,
- RPI_FIRMWARE_PIXEL_BVB_CLK_ID,
- RPI_FIRMWARE_VEC_CLK_ID,
- RPI_FIRMWARE_NUM_CLK_ID,
-};
-
static char *rpi_firmware_clk_names[] = {
[RPI_FIRMWARE_EMMC_CLK_ID] = "emmc",
[RPI_FIRMWARE_UART_CLK_ID] = "uart",
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 639d254..981a5ac 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -130,6 +130,7 @@ static struct file_system_type dma_buf_fs_type = {
static int dma_buf_mmap_internal(struct file *file, struct vm_area_struct *vma)
{
struct dma_buf *dmabuf;
+ int ret;
if (!is_dma_buf_file(file))
return -EINVAL;
@@ -145,7 +146,11 @@ static int dma_buf_mmap_internal(struct file *file, struct vm_area_struct *vma)
dmabuf->size >> PAGE_SHIFT)
return -EINVAL;
- return dmabuf->ops->mmap(dmabuf, vma);
+ dma_resv_lock(dmabuf->resv, NULL);
+ ret = dmabuf->ops->mmap(dmabuf, vma);
+ dma_resv_unlock(dmabuf->resv);
+
+ return ret;
}
static loff_t dma_buf_llseek(struct file *file, loff_t offset, int whence)
@@ -668,7 +673,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
dmabuf->file = file;
- mutex_init(&dmabuf->lock);
INIT_LIST_HEAD(&dmabuf->attachments);
mutex_lock(&db_list.lock);
@@ -807,6 +811,70 @@ static struct sg_table * __map_dma_buf(struct dma_buf_attachment *attach,
}
/**
+ * DOC: locking convention
+ *
+ * In order to avoid deadlock situations between dma-buf exports and importers,
+ * all dma-buf API users must follow the common dma-buf locking convention.
+ *
+ * Convention for importers
+ *
+ * 1. Importers must hold the dma-buf reservation lock when calling these
+ * functions:
+ *
+ * - dma_buf_pin()
+ * - dma_buf_unpin()
+ * - dma_buf_map_attachment()
+ * - dma_buf_unmap_attachment()
+ * - dma_buf_vmap()
+ * - dma_buf_vunmap()
+ *
+ * 2. Importers must not hold the dma-buf reservation lock when calling these
+ * functions:
+ *
+ * - dma_buf_attach()
+ * - dma_buf_dynamic_attach()
+ * - dma_buf_detach()
+ * - dma_buf_export(
+ * - dma_buf_fd()
+ * - dma_buf_get()
+ * - dma_buf_put()
+ * - dma_buf_mmap()
+ * - dma_buf_begin_cpu_access()
+ * - dma_buf_end_cpu_access()
+ * - dma_buf_map_attachment_unlocked()
+ * - dma_buf_unmap_attachment_unlocked()
+ * - dma_buf_vmap_unlocked()
+ * - dma_buf_vunmap_unlocked()
+ *
+ * Convention for exporters
+ *
+ * 1. These &dma_buf_ops callbacks are invoked with unlocked dma-buf
+ * reservation and exporter can take the lock:
+ *
+ * - &dma_buf_ops.attach()
+ * - &dma_buf_ops.detach()
+ * - &dma_buf_ops.release()
+ * - &dma_buf_ops.begin_cpu_access()
+ * - &dma_buf_ops.end_cpu_access()
+ *
+ * 2. These &dma_buf_ops callbacks are invoked with locked dma-buf
+ * reservation and exporter can't take the lock:
+ *
+ * - &dma_buf_ops.pin()
+ * - &dma_buf_ops.unpin()
+ * - &dma_buf_ops.map_dma_buf()
+ * - &dma_buf_ops.unmap_dma_buf()
+ * - &dma_buf_ops.mmap()
+ * - &dma_buf_ops.vmap()
+ * - &dma_buf_ops.vunmap()
+ *
+ * 3. Exporters must hold the dma-buf reservation lock when calling these
+ * functions:
+ *
+ * - dma_buf_move_notify()
+ */
+
+/**
* dma_buf_dynamic_attach - Add the device to dma_buf's attachments list
* @dmabuf: [in] buffer to attach device to.
* @dev: [in] device to be attached.
@@ -870,8 +938,8 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
dma_buf_is_dynamic(dmabuf)) {
struct sg_table *sgt;
+ dma_resv_lock(attach->dmabuf->resv, NULL);
if (dma_buf_is_dynamic(attach->dmabuf)) {
- dma_resv_lock(attach->dmabuf->resv, NULL);
ret = dmabuf->ops->pin(attach);
if (ret)
goto err_unlock;
@@ -884,8 +952,7 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
ret = PTR_ERR(sgt);
goto err_unpin;
}
- if (dma_buf_is_dynamic(attach->dmabuf))
- dma_resv_unlock(attach->dmabuf->resv);
+ dma_resv_unlock(attach->dmabuf->resv);
attach->sgt = sgt;
attach->dir = DMA_BIDIRECTIONAL;
}
@@ -901,8 +968,7 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
dmabuf->ops->unpin(attach);
err_unlock:
- if (dma_buf_is_dynamic(attach->dmabuf))
- dma_resv_unlock(attach->dmabuf->resv);
+ dma_resv_unlock(attach->dmabuf->resv);
dma_buf_detach(dmabuf, attach);
return ERR_PTR(ret);
@@ -945,24 +1011,22 @@ static void __unmap_dma_buf(struct dma_buf_attachment *attach,
*/
void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
{
- if (WARN_ON(!dmabuf || !attach))
+ if (WARN_ON(!dmabuf || !attach || dmabuf != attach->dmabuf))
return;
+ dma_resv_lock(dmabuf->resv, NULL);
+
if (attach->sgt) {
- if (dma_buf_is_dynamic(attach->dmabuf))
- dma_resv_lock(attach->dmabuf->resv, NULL);
__unmap_dma_buf(attach, attach->sgt, attach->dir);
- if (dma_buf_is_dynamic(attach->dmabuf)) {
+ if (dma_buf_is_dynamic(attach->dmabuf))
dmabuf->ops->unpin(attach);
- dma_resv_unlock(attach->dmabuf->resv);
- }
}
-
- dma_resv_lock(dmabuf->resv, NULL);
list_del(&attach->node);
+
dma_resv_unlock(dmabuf->resv);
+
if (dmabuf->ops->detach)
dmabuf->ops->detach(dmabuf, attach);
@@ -1053,8 +1117,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
if (WARN_ON(!attach || !attach->dmabuf))
return ERR_PTR(-EINVAL);
- if (dma_buf_attachment_is_dynamic(attach))
- dma_resv_assert_held(attach->dmabuf->resv);
+ dma_resv_assert_held(attach->dmabuf->resv);
if (attach->sgt) {
/*
@@ -1069,7 +1132,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
}
if (dma_buf_is_dynamic(attach->dmabuf)) {
- dma_resv_assert_held(attach->dmabuf->resv);
if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
r = attach->dmabuf->ops->pin(attach);
if (r)
@@ -1112,6 +1174,34 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
/**
+ * dma_buf_map_attachment_unlocked - Returns the scatterlist table of the attachment;
+ * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
+ * dma_buf_ops.
+ * @attach: [in] attachment whose scatterlist is to be returned
+ * @direction: [in] direction of DMA transfer
+ *
+ * Unlocked variant of dma_buf_map_attachment().
+ */
+struct sg_table *
+dma_buf_map_attachment_unlocked(struct dma_buf_attachment *attach,
+ enum dma_data_direction direction)
+{
+ struct sg_table *sg_table;
+
+ might_sleep();
+
+ if (WARN_ON(!attach || !attach->dmabuf))
+ return ERR_PTR(-EINVAL);
+
+ dma_resv_lock(attach->dmabuf->resv, NULL);
+ sg_table = dma_buf_map_attachment(attach, direction);
+ dma_resv_unlock(attach->dmabuf->resv);
+
+ return sg_table;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_unlocked, DMA_BUF);
+
+/**
* dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
* deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
* dma_buf_ops.
@@ -1130,15 +1220,11 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
return;
- if (dma_buf_attachment_is_dynamic(attach))
- dma_resv_assert_held(attach->dmabuf->resv);
+ dma_resv_assert_held(attach->dmabuf->resv);
if (attach->sgt == sg_table)
return;
- if (dma_buf_is_dynamic(attach->dmabuf))
- dma_resv_assert_held(attach->dmabuf->resv);
-
__unmap_dma_buf(attach, sg_table, direction);
if (dma_buf_is_dynamic(attach->dmabuf) &&
@@ -1148,6 +1234,31 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
/**
+ * dma_buf_unmap_attachment_unlocked - unmaps and decreases usecount of the buffer;might
+ * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
+ * dma_buf_ops.
+ * @attach: [in] attachment to unmap buffer from
+ * @sg_table: [in] scatterlist info of the buffer to unmap
+ * @direction: [in] direction of DMA transfer
+ *
+ * Unlocked variant of dma_buf_unmap_attachment().
+ */
+void dma_buf_unmap_attachment_unlocked(struct dma_buf_attachment *attach,
+ struct sg_table *sg_table,
+ enum dma_data_direction direction)
+{
+ might_sleep();
+
+ if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
+ return;
+
+ dma_resv_lock(attach->dmabuf->resv, NULL);
+ dma_buf_unmap_attachment(attach, sg_table, direction);
+ dma_resv_unlock(attach->dmabuf->resv);
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_unlocked, DMA_BUF);
+
+/**
* dma_buf_move_notify - notify attachments that DMA-buf is moving
*
* @dmabuf: [in] buffer which is moving
@@ -1397,6 +1508,8 @@ EXPORT_SYMBOL_GPL(dma_buf_end_cpu_access_partial);
int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
unsigned long pgoff)
{
+ int ret;
+
if (WARN_ON(!dmabuf || !vma))
return -EINVAL;
@@ -1417,7 +1530,11 @@ int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
vma_set_file(vma, dmabuf->file);
vma->vm_pgoff = pgoff;
- return dmabuf->ops->mmap(dmabuf, vma);
+ dma_resv_lock(dmabuf->resv, NULL);
+ ret = dmabuf->ops->mmap(dmabuf, vma);
+ dma_resv_unlock(dmabuf->resv);
+
+ return ret;
}
EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
@@ -1440,42 +1557,68 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
{
struct iosys_map ptr;
- int ret = 0;
+ int ret;
iosys_map_clear(map);
if (WARN_ON(!dmabuf))
return -EINVAL;
+ dma_resv_assert_held(dmabuf->resv);
+
if (!dmabuf->ops->vmap)
return -EINVAL;
- mutex_lock(&dmabuf->lock);
if (dmabuf->vmapping_counter) {
dmabuf->vmapping_counter++;
BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
*map = dmabuf->vmap_ptr;
- goto out_unlock;
+ return 0;
}
BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
ret = dmabuf->ops->vmap(dmabuf, &ptr);
if (WARN_ON_ONCE(ret))
- goto out_unlock;
+ return ret;
dmabuf->vmap_ptr = ptr;
dmabuf->vmapping_counter = 1;
*map = dmabuf->vmap_ptr;
-out_unlock:
- mutex_unlock(&dmabuf->lock);
- return ret;
+ return 0;
}
EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
/**
+ * dma_buf_vmap_unlocked - Create virtual mapping for the buffer object into kernel
+ * address space. Same restrictions as for vmap and friends apply.
+ * @dmabuf: [in] buffer to vmap
+ * @map: [out] returns the vmap pointer
+ *
+ * Unlocked version of dma_buf_vmap()
+ *
+ * Returns 0 on success, or a negative errno code otherwise.
+ */
+int dma_buf_vmap_unlocked(struct dma_buf *dmabuf, struct iosys_map *map)
+{
+ int ret;
+
+ iosys_map_clear(map);
+
+ if (WARN_ON(!dmabuf))
+ return -EINVAL;
+
+ dma_resv_lock(dmabuf->resv, NULL);
+ ret = dma_buf_vmap(dmabuf, map);
+ dma_resv_unlock(dmabuf->resv);
+
+ return ret;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_vmap_unlocked, DMA_BUF);
+
+/**
* dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
* @dmabuf: [in] buffer to vunmap
* @map: [in] vmap pointer to vunmap
@@ -1485,20 +1628,36 @@ void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
if (WARN_ON(!dmabuf))
return;
+ dma_resv_assert_held(dmabuf->resv);
+
BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
BUG_ON(dmabuf->vmapping_counter == 0);
BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
- mutex_lock(&dmabuf->lock);
if (--dmabuf->vmapping_counter == 0) {
if (dmabuf->ops->vunmap)
dmabuf->ops->vunmap(dmabuf, map);
iosys_map_clear(&dmabuf->vmap_ptr);
}
- mutex_unlock(&dmabuf->lock);
}
EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
+/**
+ * dma_buf_vunmap_unlocked - Unmap a vmap obtained by dma_buf_vmap.
+ * @dmabuf: [in] buffer to vunmap
+ * @map: [in] vmap pointer to vunmap
+ */
+void dma_buf_vunmap_unlocked(struct dma_buf *dmabuf, struct iosys_map *map)
+{
+ if (WARN_ON(!dmabuf))
+ return;
+
+ dma_resv_lock(dmabuf->resv, NULL);
+ dma_buf_vunmap(dmabuf, map);
+ dma_resv_unlock(dmabuf->resv);
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap_unlocked, DMA_BUF);
+
int dma_buf_get_flags(struct dma_buf *dmabuf, unsigned long *flags)
{
int ret = 0;
diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c
index 3da34c9..aeb4544 100644
--- a/drivers/dma-buf/heaps/cma_heap.c
+++ b/drivers/dma-buf/heaps/cma_heap.c
@@ -13,6 +13,7 @@
#include <linux/dma-buf.h>
#include <linux/dma-heap.h>
#include <linux/dma-map-ops.h>
+#include <linux/dma-resv.h>
#include <linux/err.h>
#include <linux/highmem.h>
#include <linux/io.h>
@@ -184,6 +185,8 @@ static int cma_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
{
struct cma_heap_buffer *buffer = dmabuf->priv;
+ dma_resv_assert_held(dmabuf->resv);
+
if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) == 0)
return -EINVAL;
diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
index 41f7aaf2..3eb3ee3 100644
--- a/drivers/dma-buf/heaps/system_heap.c
+++ b/drivers/dma-buf/heaps/system_heap.c
@@ -13,6 +13,7 @@
#include <linux/dma-buf.h>
#include <linux/dma-mapping.h>
#include <linux/dma-heap.h>
+#include <linux/dma-resv.h>
#include <linux/err.h>
#include <linux/highmem.h>
#include <linux/mm.h>
@@ -217,6 +218,8 @@ static int system_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
struct sg_page_iter piter;
int ret;
+ dma_resv_assert_held(dmabuf->resv);
+
if (buffer->uncached)
vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c
index 2bcdb93..283816f 100644
--- a/drivers/dma-buf/udmabuf.c
+++ b/drivers/dma-buf/udmabuf.c
@@ -2,6 +2,7 @@
#include <linux/cred.h>
#include <linux/device.h>
#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
#include <linux/highmem.h>
#include <linux/init.h>
#include <linux/kernel.h>
@@ -49,6 +50,8 @@ static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma)
{
struct udmabuf *ubuf = buf->priv;
+ dma_resv_assert_held(buf->resv);
+
if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) == 0)
return -EINVAL;
diff --git a/drivers/firmware/raspberrypi.c b/drivers/firmware/raspberrypi.c
index a9d7aaf..ec07bf2 100644
--- a/drivers/firmware/raspberrypi.c
+++ b/drivers/firmware/raspberrypi.c
@@ -228,6 +228,26 @@ static void rpi_register_clk_driver(struct device *dev)
-1, NULL, 0);
}
+unsigned int rpi_firmware_clk_get_max_rate(struct rpi_firmware *fw, unsigned int id)
+{
+ struct rpi_firmware_clk_rate_request msg =
+ RPI_FIRMWARE_CLK_RATE_REQUEST(id);
+ int ret;
+
+ ret = rpi_firmware_property(fw, RPI_FIRMWARE_GET_MAX_CLOCK_RATE,
+ &msg, sizeof(msg));
+ if (ret)
+ /*
+ * If our firmware doesn't support that operation, or fails, we
+ * assume the maximum clock rate is absolute maximum we can
+ * store over our type.
+ */
+ return UINT_MAX;
+
+ return le32_to_cpu(msg.rate);
+}
+EXPORT_SYMBOL_GPL(rpi_firmware_clk_get_max_rate);
+
static void rpi_firmware_delete(struct kref *kref)
{
struct rpi_firmware *fw = container_of(kref, struct rpi_firmware,
@@ -308,6 +328,18 @@ static int rpi_firmware_remove(struct platform_device *pdev)
return 0;
}
+static const struct of_device_id rpi_firmware_of_match[] = {
+ { .compatible = "raspberrypi,bcm2835-firmware", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, rpi_firmware_of_match);
+
+struct device_node *rpi_firmware_find_node(void)
+{
+ return of_find_matching_node(NULL, rpi_firmware_of_match);
+}
+EXPORT_SYMBOL_GPL(rpi_firmware_find_node);
+
/**
* rpi_firmware_get - Get pointer to rpi_firmware structure.
* @firmware_node: Pointer to the firmware Device Tree node.
@@ -363,12 +395,6 @@ struct rpi_firmware *devm_rpi_firmware_get(struct device *dev,
}
EXPORT_SYMBOL_GPL(devm_rpi_firmware_get);
-static const struct of_device_id rpi_firmware_of_match[] = {
- { .compatible = "raspberrypi,bcm2835-firmware", },
- {},
-};
-MODULE_DEVICE_TABLE(of, rpi_firmware_of_match);
-
static struct platform_driver rpi_firmware_driver = {
.driver = {
.name = "raspberrypi-firmware",
diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index 34f5a09..315cbdf 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -8,7 +8,6 @@
menuconfig DRM
tristate "Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)"
depends on (AGP || AGP=n) && !EMULATED_CMPXCHG && HAS_DMA
- select DRM_NOMODESET
select DRM_PANEL_ORIENTATION_QUIRKS
select HDMI
select FB_CMDLINE
@@ -19,6 +18,7 @@
# gallium uses SYS_kcmp for os_same_file_description() to de-duplicate
# device and dmabuf fd. Let's make sure that is available for our userspace.
select KCMP
+ select VIDEO_NOMODESET
help
Kernel-level support for the Direct Rendering Infrastructure (DRI)
introduced in XFree86 4.0. If you say Y here, you need to select
@@ -233,64 +233,8 @@
source "drivers/gpu/drm/arm/Kconfig"
-config DRM_RADEON
- tristate "ATI Radeon"
- depends on DRM && PCI && MMU
- depends on AGP || !AGP
- select FW_LOADER
- select DRM_DISPLAY_DP_HELPER
- select DRM_DISPLAY_HELPER
- select DRM_KMS_HELPER
- select DRM_TTM
- select DRM_TTM_HELPER
- select POWER_SUPPLY
- select HWMON
- select BACKLIGHT_CLASS_DEVICE
- select INTERVAL_TREE
- # radeon depends on ACPI_VIDEO when ACPI is enabled, for select to work
- # ACPI_VIDEO's dependencies must also be selected.
- select INPUT if ACPI
- select ACPI_VIDEO if ACPI
- # On x86 ACPI_VIDEO also needs ACPI_WMI
- select X86_PLATFORM_DEVICES if ACPI && X86
- select ACPI_WMI if ACPI && X86
- help
- Choose this option if you have an ATI Radeon graphics card. There
- are both PCI and AGP versions. You don't need to choose this to
- run the Radeon in plain VGA mode.
-
- If M is selected, the module will be called radeon.
-
source "drivers/gpu/drm/radeon/Kconfig"
-config DRM_AMDGPU
- tristate "AMD GPU"
- depends on DRM && PCI && MMU
- select FW_LOADER
- select DRM_DISPLAY_DP_HELPER
- select DRM_DISPLAY_HDMI_HELPER
- select DRM_DISPLAY_HELPER
- select DRM_KMS_HELPER
- select DRM_SCHED
- select DRM_TTM
- select DRM_TTM_HELPER
- select POWER_SUPPLY
- select HWMON
- select BACKLIGHT_CLASS_DEVICE
- select INTERVAL_TREE
- select DRM_BUDDY
- # amdgpu depends on ACPI_VIDEO when ACPI is enabled, for select to work
- # ACPI_VIDEO's dependencies must also be selected.
- select INPUT if ACPI
- select ACPI_VIDEO if ACPI
- # On x86 ACPI_VIDEO also needs ACPI_WMI
- select X86_PLATFORM_DEVICES if ACPI && X86
- select ACPI_WMI if ACPI && X86
- help
- Choose this option if you have a recent AMD Radeon graphics card.
-
- If M is selected, the module will be called amdgpu.
-
source "drivers/gpu/drm/amd/amdgpu/Kconfig"
source "drivers/gpu/drm/nouveau/Kconfig"
@@ -514,11 +458,6 @@
config DRM_PANEL_ORIENTATION_QUIRKS
tristate
-# Separate option because nomodeset parameter is global and expected built-in
-config DRM_NOMODESET
- bool
- default n
-
config DRM_LIB_RANDOM
bool
default n
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 0b283e4..cc63734 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -5,35 +5,74 @@
CFLAGS-$(CONFIG_DRM_USE_DYNAMIC_DEBUG) += -DDYNAMIC_DEBUG_MODULE
-drm-y := drm_aperture.o drm_auth.o drm_cache.o \
- drm_file.o drm_gem.o drm_ioctl.o \
- drm_drv.o \
- drm_sysfs.o drm_mm.o \
- drm_crtc.o drm_fourcc.o drm_modes.o drm_edid.o drm_displayid.o \
- drm_trace_points.o drm_prime.o \
- drm_vma_manager.o \
- drm_modeset_lock.o drm_atomic.o drm_bridge.o \
- drm_framebuffer.o drm_connector.o drm_blend.o \
- drm_encoder.o drm_mode_object.o drm_property.o \
- drm_plane.o drm_color_mgmt.o drm_print.o \
- drm_dumb_buffers.o drm_mode_config.o drm_vblank.o \
- drm_syncobj.o drm_lease.o drm_writeback.o drm_client.o \
- drm_client_modeset.o drm_atomic_uapi.o \
- drm_managed.o drm_vblank_work.o
-drm-$(CONFIG_DRM_LEGACY) += drm_agpsupport.o drm_bufs.o drm_context.o drm_dma.o \
- drm_hashtab.o drm_irq.o drm_legacy_misc.o drm_lock.o \
- drm_memory.o drm_scatter.o drm_vm.o
+drm-y := \
+ drm_aperture.o \
+ drm_atomic.o \
+ drm_atomic_uapi.o \
+ drm_auth.o \
+ drm_blend.o \
+ drm_bridge.o \
+ drm_cache.o \
+ drm_client.o \
+ drm_client_modeset.o \
+ drm_color_mgmt.o \
+ drm_connector.o \
+ drm_crtc.o \
+ drm_displayid.o \
+ drm_drv.o \
+ drm_dumb_buffers.o \
+ drm_edid.o \
+ drm_encoder.o \
+ drm_file.o \
+ drm_fourcc.o \
+ drm_framebuffer.o \
+ drm_gem.o \
+ drm_ioctl.o \
+ drm_lease.o \
+ drm_managed.o \
+ drm_mm.o \
+ drm_mode_config.o \
+ drm_mode_object.o \
+ drm_modes.o \
+ drm_modeset_lock.o \
+ drm_plane.o \
+ drm_prime.o \
+ drm_print.o \
+ drm_property.o \
+ drm_syncobj.o \
+ drm_sysfs.o \
+ drm_trace_points.o \
+ drm_vblank.o \
+ drm_vblank_work.o \
+ drm_vma_manager.o \
+ drm_writeback.o
+drm-$(CONFIG_DRM_LEGACY) += \
+ drm_agpsupport.o \
+ drm_bufs.o \
+ drm_context.o \
+ drm_dma.o \
+ drm_hashtab.o \
+ drm_irq.o \
+ drm_legacy_misc.o \
+ drm_lock.o \
+ drm_memory.o \
+ drm_scatter.o \
+ drm_vm.o
drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o
drm-$(CONFIG_COMPAT) += drm_ioc32.o
drm-$(CONFIG_DRM_PANEL) += drm_panel.o
drm-$(CONFIG_OF) += drm_of.o
drm-$(CONFIG_PCI) += drm_pci.o
-drm-$(CONFIG_DEBUG_FS) += drm_debugfs.o drm_debugfs_crc.o
+drm-$(CONFIG_DEBUG_FS) += \
+ drm_debugfs.o \
+ drm_debugfs_crc.o
drm-$(CONFIG_DRM_LOAD_EDID_FIRMWARE) += drm_edid_load.o
-drm-$(CONFIG_DRM_PRIVACY_SCREEN) += drm_privacy_screen.o drm_privacy_screen_x86.o
+drm-$(CONFIG_DRM_PRIVACY_SCREEN) += \
+ drm_privacy_screen.o \
+ drm_privacy_screen_x86.o
+drm-$(CONFIG_DRM_ACCEL) += ../../accel/drm_accel.o
obj-$(CONFIG_DRM) += drm.o
-obj-$(CONFIG_DRM_NOMODESET) += drm_nomodeset.o
obj-$(CONFIG_DRM_PANEL_ORIENTATION_QUIRKS) += drm_panel_orientation_quirks.o
#
@@ -59,18 +98,28 @@
# Modesetting helpers
#
-drm_kms_helper-y := drm_bridge_connector.o drm_crtc_helper.o \
- drm_encoder_slave.o drm_flip_work.o \
- drm_probe_helper.o \
- drm_plane_helper.o drm_atomic_helper.o \
- drm_kms_helper_common.o \
- drm_simple_kms_helper.o drm_modeset_helper.o \
- drm_gem_atomic_helper.o \
- drm_gem_framebuffer_helper.o \
- drm_atomic_state_helper.o drm_damage_helper.o \
- drm_format_helper.o drm_self_refresh_helper.o drm_rect.o
+drm_kms_helper-y := \
+ drm_atomic_helper.o \
+ drm_atomic_state_helper.o \
+ drm_bridge_connector.o \
+ drm_crtc_helper.o \
+ drm_damage_helper.o \
+ drm_encoder_slave.o \
+ drm_flip_work.o \
+ drm_format_helper.o \
+ drm_gem_atomic_helper.o \
+ drm_gem_framebuffer_helper.o \
+ drm_kms_helper_common.o \
+ drm_modeset_helper.o \
+ drm_plane_helper.o \
+ drm_probe_helper.o \
+ drm_rect.o \
+ drm_self_refresh_helper.o \
+ drm_simple_kms_helper.o
drm_kms_helper-$(CONFIG_DRM_PANEL_BRIDGE) += bridge/panel.o
-drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o
+drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += \
+ drm_fbdev_generic.o \
+ drm_fb_helper.o
obj-$(CONFIG_DRM_KMS_HELPER) += drm_kms_helper.o
#
diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 7777d55..5fcd510 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -1,4 +1,33 @@
# SPDX-License-Identifier: MIT
+
+config DRM_AMDGPU
+ tristate "AMD GPU"
+ depends on DRM && PCI && MMU
+ select FW_LOADER
+ select DRM_DISPLAY_DP_HELPER
+ select DRM_DISPLAY_HDMI_HELPER
+ select DRM_DISPLAY_HELPER
+ select DRM_KMS_HELPER
+ select DRM_SCHED
+ select DRM_TTM
+ select DRM_TTM_HELPER
+ select POWER_SUPPLY
+ select HWMON
+ select BACKLIGHT_CLASS_DEVICE
+ select INTERVAL_TREE
+ select DRM_BUDDY
+ # amdgpu depends on ACPI_VIDEO when ACPI is enabled, for select to work
+ # ACPI_VIDEO's dependencies must also be selected.
+ select INPUT if ACPI
+ select ACPI_VIDEO if ACPI
+ # On x86 ACPI_VIDEO also needs ACPI_WMI
+ select X86_PLATFORM_DEVICES if ACPI && X86
+ select ACPI_WMI if ACPI && X86
+ help
+ Choose this option if you have a recent AMD Radeon graphics card.
+
+ If M is selected, the module will be called amdgpu.
+
config DRM_AMDGPU_SI
bool "Enable amdgpu support for SI parts"
depends on DRM_AMDGPU
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index 6ad39cf..798d0e9 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -58,7 +58,8 @@
amdgpu_vm_sdma.o amdgpu_discovery.o amdgpu_ras_eeprom.o amdgpu_nbio.o \
amdgpu_umc.o smu_v11_0_i2c.o amdgpu_fru_eeprom.o amdgpu_rap.o \
amdgpu_fw_attestation.o amdgpu_securedisplay.o \
- amdgpu_eeprom.o amdgpu_mca.o amdgpu_psp_ta.o amdgpu_lsdma.o
+ amdgpu_eeprom.o amdgpu_mca.o amdgpu_psp_ta.o amdgpu_lsdma.o \
+ amdgpu_ring_mux.o
amdgpu-$(CONFIG_PROC_FS) += amdgpu_fdinfo.o
@@ -250,7 +251,7 @@
amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_hmm.o
include $(FULL_AMD_PATH)/pm/Makefile
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 2eca582..6b74df4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -82,7 +82,6 @@
#include "amdgpu_vce.h"
#include "amdgpu_vcn.h"
#include "amdgpu_jpeg.h"
-#include "amdgpu_mn.h"
#include "amdgpu_gmc.h"
#include "amdgpu_gfx.h"
#include "amdgpu_sdma.h"
@@ -219,10 +218,12 @@ extern int amdgpu_use_xgmi_p2p;
extern int sched_policy;
extern bool debug_evictions;
extern bool no_system_mem_limit;
+extern int halt_if_hws_hang;
#else
static const int __maybe_unused sched_policy = KFD_SCHED_POLICY_HWS;
static const bool __maybe_unused debug_evictions; /* = false */
static const bool __maybe_unused no_system_mem_limit;
+static const int __maybe_unused halt_if_hws_hang;
#endif
#ifdef CONFIG_HSA_AMD_P2P
extern bool pcie_p2p;
@@ -675,7 +676,7 @@ enum amd_hw_ip_block_type {
MAX_HWIP
};
-#define HWIP_MAX_INSTANCE 11
+#define HWIP_MAX_INSTANCE 28
#define HW_ID_MAX 300
#define IP_VERSION(mj, mn, rv) (((mj) << 16) | ((mn) << 8) | (rv))
@@ -1063,6 +1064,7 @@ struct amdgpu_device {
struct work_struct reset_work;
bool job_hang;
+ bool dc_enabled;
};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
@@ -1120,6 +1122,8 @@ void amdgpu_device_indirect_wreg64(struct amdgpu_device *adev,
bool amdgpu_device_asic_has_dc_support(enum amd_asic_type asic_type);
bool amdgpu_device_has_dc_support(struct amdgpu_device *adev);
+void amdgpu_device_set_sriov_virtual_display(struct amdgpu_device *adev);
+
int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
struct amdgpu_reset_context *reset_context);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
index b14800a..57b5e114 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
@@ -847,7 +847,7 @@ int amdgpu_acpi_init(struct amdgpu_device *adev)
struct amdgpu_atif *atif = &amdgpu_acpi_priv.atif;
if (atif->notifications.brightness_change) {
- if (amdgpu_device_has_dc_support(adev)) {
+ if (adev->dc_enabled) {
#if defined(CONFIG_DRM_AMD_DC)
struct amdgpu_display_manager *dm = &adev->dm;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
index 5d9a346..f99d487 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -195,7 +195,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)
}
adev->kfd.init_complete = kgd2kfd_device_init(adev->kfd.dev,
- adev_to_drm(adev), &gpu_resources);
+ &gpu_resources);
amdgpu_amdkfd_total_mem_size += adev->gmc.real_vram_size;
@@ -673,7 +673,7 @@ int amdgpu_amdkfd_submit_ib(struct amdgpu_device *adev,
goto err;
}
- ret = amdgpu_job_alloc(adev, 1, &job, NULL);
+ ret = amdgpu_job_alloc(adev, NULL, NULL, NULL, 1, &job);
if (ret)
goto err;
@@ -760,9 +760,7 @@ bool amdgpu_amdkfd_have_atomics_support(struct amdgpu_device *adev)
void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, bool reset)
{
- struct ras_err_data err_data = {0, 0, 0, NULL};
-
- amdgpu_umc_poison_handler(adev, &err_data, reset);
+ amdgpu_umc_poison_handler(adev, reset);
}
bool amdgpu_amdkfd_ras_query_utcl2_poison_status(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 647220a..f50e3ba 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -353,7 +353,6 @@ int kgd2kfd_init(void);
void kgd2kfd_exit(void);
struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf);
bool kgd2kfd_device_init(struct kfd_dev *kfd,
- struct drm_device *ddev,
const struct kgd2kfd_shared_resources *gpu_resources);
void kgd2kfd_device_exit(struct kfd_dev *kfd);
void kgd2kfd_suspend(struct kfd_dev *kfd, bool run_pm);
@@ -381,7 +380,7 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
}
static inline
-bool kgd2kfd_device_init(struct kfd_dev *kfd, struct drm_device *ddev,
+bool kgd2kfd_device_init(struct kfd_dev *kfd,
const struct kgd2kfd_shared_resources *gpu_resources)
{
return false;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
index 81e3b52..e92b935 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
@@ -787,7 +787,7 @@ void kgd_gfx_v9_get_cu_occupancy(struct amdgpu_device *adev, int pasid,
for (se_idx = 0; se_idx < se_cnt; se_idx++) {
for (sh_idx = 0; sh_idx < sh_cnt; sh_idx++) {
- gfx_v9_0_select_se_sh(adev, se_idx, sh_idx, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, se_idx, sh_idx, 0xffffffff);
queue_map = RREG32_SOC15(GC, 0, mmSPI_CSQ_WF_ACTIVE_STATUS);
/*
@@ -820,7 +820,7 @@ void kgd_gfx_v9_get_cu_occupancy(struct amdgpu_device *adev, int pasid,
}
}
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
soc15_grbm_select(adev, 0, 0, 0, 0);
unlock_spi_csq_mutexes(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index 1f76e27..8782916 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -29,6 +29,7 @@
#include "amdgpu_object.h"
#include "amdgpu_gem.h"
#include "amdgpu_vm.h"
+#include "amdgpu_hmm.h"
#include "amdgpu_amdkfd.h"
#include "amdgpu_dma_buf.h"
#include <uapi/linux/kfd_ioctl.h>
@@ -403,63 +404,15 @@ static int vm_update_pds(struct amdgpu_vm *vm, struct amdgpu_sync *sync)
static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem)
{
- struct amdgpu_device *bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev);
- bool coherent = mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_COHERENT;
- bool uncached = mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_UNCACHED;
- uint32_t mapping_flags;
- uint64_t pte_flags;
- bool snoop = false;
+ uint32_t mapping_flags = AMDGPU_VM_PAGE_READABLE |
+ AMDGPU_VM_MTYPE_DEFAULT;
- mapping_flags = AMDGPU_VM_PAGE_READABLE;
if (mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE)
mapping_flags |= AMDGPU_VM_PAGE_WRITEABLE;
if (mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_EXECUTABLE)
mapping_flags |= AMDGPU_VM_PAGE_EXECUTABLE;
- switch (adev->asic_type) {
- case CHIP_ARCTURUS:
- case CHIP_ALDEBARAN:
- if (mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) {
- if (bo_adev == adev) {
- if (uncached)
- mapping_flags |= AMDGPU_VM_MTYPE_UC;
- else if (coherent)
- mapping_flags |= AMDGPU_VM_MTYPE_CC;
- else
- mapping_flags |= AMDGPU_VM_MTYPE_RW;
- if (adev->asic_type == CHIP_ALDEBARAN &&
- adev->gmc.xgmi.connected_to_cpu)
- snoop = true;
- } else {
- if (uncached || coherent)
- mapping_flags |= AMDGPU_VM_MTYPE_UC;
- else
- mapping_flags |= AMDGPU_VM_MTYPE_NC;
- if (amdgpu_xgmi_same_hive(adev, bo_adev))
- snoop = true;
- }
- } else {
- if (uncached || coherent)
- mapping_flags |= AMDGPU_VM_MTYPE_UC;
- else
- mapping_flags |= AMDGPU_VM_MTYPE_NC;
- snoop = true;
- }
- break;
- default:
- if (uncached || coherent)
- mapping_flags |= AMDGPU_VM_MTYPE_UC;
- else
- mapping_flags |= AMDGPU_VM_MTYPE_NC;
-
- if (!(mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM))
- snoop = true;
- }
-
- pte_flags = amdgpu_gem_va_map_flags(adev, mapping_flags);
- pte_flags |= snoop ? AMDGPU_PTE_SNOOPED : 0;
-
- return pte_flags;
+ return amdgpu_gem_va_map_flags(adev, mapping_flags);
}
/**
@@ -997,7 +950,7 @@ static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr,
goto out;
}
- ret = amdgpu_mn_register(bo, user_addr);
+ ret = amdgpu_hmm_register(bo, user_addr);
if (ret) {
pr_err("%s: Failed to register MMU notifier: %d\n",
__func__, ret);
@@ -1037,7 +990,7 @@ static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr,
amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm, range);
unregister_out:
if (ret)
- amdgpu_mn_unregister(bo);
+ amdgpu_hmm_unregister(bo);
out:
mutex_unlock(&process_info->lock);
return ret;
@@ -1672,6 +1625,11 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
}
}
+ if (flags & KFD_IOC_ALLOC_MEM_FLAGS_COHERENT)
+ alloc_flags |= AMDGPU_GEM_CREATE_COHERENT;
+ if (flags & KFD_IOC_ALLOC_MEM_FLAGS_UNCACHED)
+ alloc_flags |= AMDGPU_GEM_CREATE_UNCACHED;
+
*mem = kzalloc(sizeof(struct kgd_mem), GFP_KERNEL);
if (!*mem) {
ret = -ENOMEM;
@@ -1816,7 +1774,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
mutex_unlock(&process_info->lock);
/* No more MMU notifiers */
- amdgpu_mn_unregister(mem->bo);
+ amdgpu_hmm_unregister(mem->bo);
ret = reserve_bo_and_cond_vms(mem, NULL, BO_VM_ALL, &ctx);
if (unlikely(ret))
@@ -1906,16 +1864,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu(
*/
mutex_lock(&mem->process_info->lock);
- /* Lock mmap-sem. If we find an invalid userptr BO, we can be
- * sure that the MMU notifier is no longer running
- * concurrently and the queues are actually stopped
- */
- if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm)) {
- mmap_write_lock(current->mm);
- is_invalid_userptr = atomic_read(&mem->invalid);
- mmap_write_unlock(current->mm);
- }
-
mutex_lock(&mem->lock);
domain = mem->domain;
@@ -2256,7 +2204,7 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev,
ret = drm_vma_node_allow(&obj->vma_node, drm_priv);
if (ret) {
- kfree(mem);
+ kfree(*mem);
return ret;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
index b81b77a..ac6fe0a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
@@ -101,39 +101,101 @@ void amdgpu_atomfirmware_scratch_regs_init(struct amdgpu_device *adev)
}
}
+static int amdgpu_atomfirmware_allocate_fb_v2_1(struct amdgpu_device *adev,
+ struct vram_usagebyfirmware_v2_1 *fw_usage, int *usage_bytes)
+{
+ u32 start_addr, fw_size, drv_size;
+
+ start_addr = le32_to_cpu(fw_usage->start_address_in_kb);
+ fw_size = le16_to_cpu(fw_usage->used_by_firmware_in_kb);
+ drv_size = le16_to_cpu(fw_usage->used_by_driver_in_kb);
+
+ DRM_DEBUG("atom firmware v2_1 requested %08x %dkb fw %dkb drv\n",
+ start_addr,
+ fw_size,
+ drv_size);
+
+ if ((start_addr & ATOM_VRAM_OPERATION_FLAGS_MASK) ==
+ (u32)(ATOM_VRAM_BLOCK_SRIOV_MSG_SHARE_RESERVATION <<
+ ATOM_VRAM_OPERATION_FLAGS_SHIFT)) {
+ /* Firmware request VRAM reservation for SR-IOV */
+ adev->mman.fw_vram_usage_start_offset = (start_addr &
+ (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
+ adev->mman.fw_vram_usage_size = fw_size << 10;
+ /* Use the default scratch size */
+ *usage_bytes = 0;
+ } else {
+ *usage_bytes = drv_size << 10;
+ }
+ return 0;
+}
+
+static int amdgpu_atomfirmware_allocate_fb_v2_2(struct amdgpu_device *adev,
+ struct vram_usagebyfirmware_v2_2 *fw_usage, int *usage_bytes)
+{
+ u32 fw_start_addr, fw_size, drv_start_addr, drv_size;
+
+ fw_start_addr = le32_to_cpu(fw_usage->fw_region_start_address_in_kb);
+ fw_size = le16_to_cpu(fw_usage->used_by_firmware_in_kb);
+
+ drv_start_addr = le32_to_cpu(fw_usage->driver_region0_start_address_in_kb);
+ drv_size = le32_to_cpu(fw_usage->used_by_driver_region0_in_kb);
+
+ DRM_DEBUG("atom requested fw start at %08x %dkb and drv start at %08x %dkb\n",
+ fw_start_addr,
+ fw_size,
+ drv_start_addr,
+ drv_size);
+
+ if (amdgpu_sriov_vf(adev) &&
+ ((fw_start_addr & (ATOM_VRAM_BLOCK_NEEDS_NO_RESERVATION <<
+ ATOM_VRAM_OPERATION_FLAGS_SHIFT)) == 0)) {
+ /* Firmware request VRAM reservation for SR-IOV */
+ adev->mman.fw_vram_usage_start_offset = (fw_start_addr &
+ (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
+ adev->mman.fw_vram_usage_size = fw_size << 10;
+ }
+
+ if (amdgpu_sriov_vf(adev) &&
+ ((drv_start_addr & (ATOM_VRAM_BLOCK_NEEDS_NO_RESERVATION <<
+ ATOM_VRAM_OPERATION_FLAGS_SHIFT)) == 0)) {
+ /* driver request VRAM reservation for SR-IOV */
+ adev->mman.drv_vram_usage_start_offset = (drv_start_addr &
+ (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
+ adev->mman.drv_vram_usage_size = drv_size << 10;
+ }
+
+ *usage_bytes = 0;
+ return 0;
+}
+
int amdgpu_atomfirmware_allocate_fb_scratch(struct amdgpu_device *adev)
{
struct atom_context *ctx = adev->mode_info.atom_context;
int index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1,
vram_usagebyfirmware);
- struct vram_usagebyfirmware_v2_1 *firmware_usage;
- uint32_t start_addr, size;
- uint16_t data_offset;
+ struct vram_usagebyfirmware_v2_1 *fw_usage_v2_1;
+ struct vram_usagebyfirmware_v2_2 *fw_usage_v2_2;
+ u16 data_offset;
+ u8 frev, crev;
int usage_bytes = 0;
- if (amdgpu_atom_parse_data_header(ctx, index, NULL, NULL, NULL, &data_offset)) {
- firmware_usage = (struct vram_usagebyfirmware_v2_1 *)(ctx->bios + data_offset);
- DRM_DEBUG("atom firmware requested %08x %dkb fw %dkb drv\n",
- le32_to_cpu(firmware_usage->start_address_in_kb),
- le16_to_cpu(firmware_usage->used_by_firmware_in_kb),
- le16_to_cpu(firmware_usage->used_by_driver_in_kb));
-
- start_addr = le32_to_cpu(firmware_usage->start_address_in_kb);
- size = le16_to_cpu(firmware_usage->used_by_firmware_in_kb);
-
- if ((uint32_t)(start_addr & ATOM_VRAM_OPERATION_FLAGS_MASK) ==
- (uint32_t)(ATOM_VRAM_BLOCK_SRIOV_MSG_SHARE_RESERVATION <<
- ATOM_VRAM_OPERATION_FLAGS_SHIFT)) {
- /* Firmware request VRAM reservation for SR-IOV */
- adev->mman.fw_vram_usage_start_offset = (start_addr &
- (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
- adev->mman.fw_vram_usage_size = size << 10;
- /* Use the default scratch size */
- usage_bytes = 0;
- } else {
- usage_bytes = le16_to_cpu(firmware_usage->used_by_driver_in_kb) << 10;
+ if (amdgpu_atom_parse_data_header(ctx, index, NULL, &frev, &crev, &data_offset)) {
+ if (frev == 2 && crev == 1) {
+ fw_usage_v2_1 =
+ (struct vram_usagebyfirmware_v2_1 *)(ctx->bios + data_offset);
+ amdgpu_atomfirmware_allocate_fb_v2_1(adev,
+ fw_usage_v2_1,
+ &usage_bytes);
+ } else if (frev >= 2 && crev >= 2) {
+ fw_usage_v2_2 =
+ (struct vram_usagebyfirmware_v2_2 *)(ctx->bios + data_offset);
+ amdgpu_atomfirmware_allocate_fb_v2_2(adev,
+ fw_usage_v2_2,
+ &usage_bytes);
}
}
+
ctx->scratch_size_bytes = 0;
if (usage_bytes == 0)
usage_bytes = 20 * 1024;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
index e363f56..30c28a6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
@@ -317,6 +317,7 @@ static bool amdgpu_atrm_get_bios(struct amdgpu_device *adev)
if (!found)
return false;
+ pci_dev_put(pdev);
adev->bios = kmalloc(size, GFP_KERNEL);
if (!adev->bios) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
index cfb2629..2ebbc63 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
@@ -26,7 +26,6 @@
#include <drm/display/drm_dp_helper.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_probe_helper.h>
#include <drm/amdgpu_drm.h>
#include "amdgpu.h"
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 365e3fb..8516c81 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -294,12 +294,8 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
}
for (i = 0; i < p->gang_size; ++i) {
- ret = amdgpu_job_alloc(p->adev, num_ibs[i], &p->jobs[i], vm);
- if (ret)
- goto free_all_kdata;
-
- ret = drm_sched_job_init(&p->jobs[i]->base, p->entities[i],
- &fpriv->vm);
+ ret = amdgpu_job_alloc(p->adev, vm, p->entities[i], vm,
+ num_ibs[i], &p->jobs[i]);
if (ret)
goto free_all_kdata;
}
@@ -433,7 +429,7 @@ static int amdgpu_cs_p2_dependencies(struct amdgpu_cs_parser *p,
dma_fence_put(old);
}
- r = amdgpu_sync_fence(&p->gang_leader->sync, fence);
+ r = amdgpu_sync_fence(&p->sync, fence);
dma_fence_put(fence);
if (r)
return r;
@@ -455,9 +451,20 @@ static int amdgpu_syncobj_lookup_and_add(struct amdgpu_cs_parser *p,
return r;
}
- r = amdgpu_sync_fence(&p->gang_leader->sync, fence);
- dma_fence_put(fence);
+ r = amdgpu_sync_fence(&p->sync, fence);
+ if (r)
+ goto error;
+ /*
+ * When we have an explicit dependency it might be necessary to insert a
+ * pipeline sync to make sure that all caches etc are flushed and the
+ * next job actually sees the results from the previous one.
+ */
+ if (fence->context == p->gang_leader->base.entity->fence_context)
+ r = amdgpu_sync_fence(&p->gang_leader->explicit_sync, fence);
+
+error:
+ dma_fence_put(fence);
return r;
}
@@ -1106,7 +1113,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
if (r)
return r;
- r = amdgpu_sync_fence(&job->sync, fpriv->prt_va->last_pt_update);
+ r = amdgpu_sync_fence(&p->sync, fpriv->prt_va->last_pt_update);
if (r)
return r;
@@ -1117,7 +1124,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
if (r)
return r;
- r = amdgpu_sync_fence(&job->sync, bo_va->last_pt_update);
+ r = amdgpu_sync_fence(&p->sync, bo_va->last_pt_update);
if (r)
return r;
}
@@ -1136,7 +1143,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
if (r)
return r;
- r = amdgpu_sync_fence(&job->sync, bo_va->last_pt_update);
+ r = amdgpu_sync_fence(&p->sync, bo_va->last_pt_update);
if (r)
return r;
}
@@ -1149,7 +1156,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
if (r)
return r;
- r = amdgpu_sync_fence(&job->sync, vm->last_update);
+ r = amdgpu_sync_fence(&p->sync, vm->last_update);
if (r)
return r;
@@ -1181,7 +1188,6 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
static int amdgpu_cs_sync_rings(struct amdgpu_cs_parser *p)
{
struct amdgpu_fpriv *fpriv = p->filp->driver_priv;
- struct amdgpu_job *leader = p->gang_leader;
struct amdgpu_bo_list_entry *e;
unsigned int i;
int r;
@@ -1193,17 +1199,14 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_parser *p)
sync_mode = amdgpu_bo_explicit_sync(bo) ?
AMDGPU_SYNC_EXPLICIT : AMDGPU_SYNC_NE_OWNER;
- r = amdgpu_sync_resv(p->adev, &leader->sync, resv, sync_mode,
+ r = amdgpu_sync_resv(p->adev, &p->sync, resv, sync_mode,
&fpriv->vm);
if (r)
return r;
}
for (i = 0; i < p->gang_size; ++i) {
- if (p->jobs[i] == leader)
- continue;
-
- r = amdgpu_sync_clone(&leader->sync, &p->jobs[i]->sync);
+ r = amdgpu_sync_push_to_job(&p->sync, p->jobs[i]);
if (r)
return r;
}
@@ -1251,7 +1254,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
continue;
fence = &p->jobs[i]->base.s_fence->scheduled;
- r = amdgpu_sync_fence(&leader->sync, fence);
+ r = drm_sched_job_add_dependency(&leader->base, fence);
if (r)
goto error_cleanup;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h
index f80adf9..113f395 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h
@@ -76,6 +76,8 @@ struct amdgpu_cs_parser {
unsigned num_post_deps;
struct amdgpu_cs_post_dep *post_deps;
+
+ struct amdgpu_sync sync;
};
int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
index de61a85..0f16d3c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
@@ -1969,7 +1969,7 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev)
amdgpu_ta_if_debugfs_init(adev);
#if defined(CONFIG_DRM_AMD_DC)
- if (amdgpu_device_has_dc_support(adev))
+ if (adev->dc_enabled)
dtn_debugfs_init(adev);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index f1e9663..cfa411c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -37,6 +37,7 @@
#include <linux/pci-p2pdma.h>
#include <drm/drm_atomic_helper.h>
+#include <drm/drm_fb_helper.h>
#include <drm/drm_probe_helper.h>
#include <drm/amdgpu_drm.h>
#include <linux/vgaarb.h>
@@ -1568,7 +1569,7 @@ static int amdgpu_device_check_arguments(struct amdgpu_device *adev)
* @pdev: pci dev pointer
* @state: vga_switcheroo state
*
- * Callback for the switcheroo driver. Suspends or resumes the
+ * Callback for the switcheroo driver. Suspends or resumes
* the asics before or after it is powered up using ACPI methods.
*/
static void amdgpu_switcheroo_set_state(struct pci_dev *pdev,
@@ -1915,6 +1916,16 @@ static void amdgpu_device_enable_virtual_display(struct amdgpu_device *adev)
}
}
+void amdgpu_device_set_sriov_virtual_display(struct amdgpu_device *adev)
+{
+ if (amdgpu_sriov_vf(adev) && !adev->enable_virtual_display) {
+ adev->mode_info.num_crtc = 1;
+ adev->enable_virtual_display = true;
+ DRM_INFO("virtual_display:%d, num_crtc:%d\n",
+ adev->enable_virtual_display, adev->mode_info.num_crtc);
+ }
+}
+
/**
* amdgpu_device_parse_gpu_info_fw - parse gpu info firmware
*
@@ -2397,7 +2408,7 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev)
adev->ip_blocks[i].status.hw = true;
/* right after GMC hw init, we create CSA */
- if (amdgpu_mcbp || amdgpu_sriov_vf(adev)) {
+ if (amdgpu_mcbp) {
r = amdgpu_allocate_static_csa(adev, &adev->virt.csa_obj,
AMDGPU_GEM_DOMAIN_VRAM,
AMDGPU_CSA_SIZE);
@@ -2462,6 +2473,11 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev)
if (!amdgpu_sriov_vf(adev)) {
struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev);
+ if (WARN_ON(!hive)) {
+ r = -ENOENT;
+ goto init_failed;
+ }
+
if (!hive->reset_domain ||
!amdgpu_reset_get_reset_domain(hive->reset_domain)) {
r = -ENOENT;
@@ -3347,8 +3363,7 @@ bool amdgpu_device_asic_has_dc_support(enum amd_asic_type asic_type)
*/
bool amdgpu_device_has_dc_support(struct amdgpu_device *adev)
{
- if (amdgpu_sriov_vf(adev) ||
- adev->enable_virtual_display ||
+ if (adev->enable_virtual_display ||
(adev->harvest_ip_mask & AMD_HARVEST_IP_DMU_MASK))
return false;
@@ -4171,21 +4186,15 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
r = amdgpu_device_ip_resume(adev);
- /* no matter what r is, always need to properly release full GPU */
- if (amdgpu_sriov_vf(adev)) {
- amdgpu_virt_init_data_exchange(adev);
- amdgpu_virt_release_full_gpu(adev, true);
- }
-
if (r) {
dev_err(adev->dev, "amdgpu_device_ip_resume failed (%d).\n", r);
- return r;
+ goto exit;
}
amdgpu_fence_driver_hw_init(adev);
r = amdgpu_device_ip_late_init(adev);
if (r)
- return r;
+ goto exit;
queue_delayed_work(system_wq, &adev->delayed_init_work,
msecs_to_jiffies(AMDGPU_RESUME_MS));
@@ -4193,9 +4202,18 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
if (!adev->in_s0ix) {
r = amdgpu_amdkfd_resume(adev, adev->in_runpm);
if (r)
- return r;
+ goto exit;
}
+exit:
+ if (amdgpu_sriov_vf(adev)) {
+ amdgpu_virt_init_data_exchange(adev);
+ amdgpu_virt_release_full_gpu(adev, true);
+ }
+
+ if (r)
+ return r;
+
/* Make sure IB tests flushed */
flush_delayed_work(&adev->delayed_init_work);
@@ -4213,25 +4231,27 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
amdgpu_ras_resume(adev);
- /*
- * Most of the connector probing functions try to acquire runtime pm
- * refs to ensure that the GPU is powered on when connector polling is
- * performed. Since we're calling this from a runtime PM callback,
- * trying to acquire rpm refs will cause us to deadlock.
- *
- * Since we're guaranteed to be holding the rpm lock, it's safe to
- * temporarily disable the rpm helpers so this doesn't deadlock us.
- */
+ if (adev->mode_info.num_crtc) {
+ /*
+ * Most of the connector probing functions try to acquire runtime pm
+ * refs to ensure that the GPU is powered on when connector polling is
+ * performed. Since we're calling this from a runtime PM callback,
+ * trying to acquire rpm refs will cause us to deadlock.
+ *
+ * Since we're guaranteed to be holding the rpm lock, it's safe to
+ * temporarily disable the rpm helpers so this doesn't deadlock us.
+ */
#ifdef CONFIG_PM
- dev->dev->power.disable_depth++;
+ dev->dev->power.disable_depth++;
#endif
- if (!amdgpu_device_has_dc_support(adev))
- drm_helper_hpd_irq_event(dev);
- else
- drm_kms_helper_hotplug_event(dev);
+ if (!adev->dc_enabled)
+ drm_helper_hpd_irq_event(dev);
+ else
+ drm_kms_helper_hotplug_event(dev);
#ifdef CONFIG_PM
- dev->dev->power.disable_depth--;
+ dev->dev->power.disable_depth--;
#endif
+ }
adev->in_suspend = false;
if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DEV_D0))
@@ -4580,6 +4600,10 @@ bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev)
if (amdgpu_gpu_recovery == 0)
goto disabled;
+ /* Skip soft reset check in fatal error mode */
+ if (!amdgpu_ras_is_poison_mode_supported(adev))
+ return true;
+
if (!amdgpu_device_ip_check_soft_reset(adev)) {
dev_info(adev->dev,"Timeout, but no hardware hang detected.\n");
return false;
@@ -5027,6 +5051,8 @@ static void amdgpu_device_resume_display_audio(struct amdgpu_device *adev)
pm_runtime_enable(&(p->dev));
pm_runtime_resume(&(p->dev));
}
+
+ pci_dev_put(p);
}
static int amdgpu_device_suspend_display_audio(struct amdgpu_device *adev)
@@ -5065,6 +5091,7 @@ static int amdgpu_device_suspend_display_audio(struct amdgpu_device *adev)
if (expires < ktime_get_mono_fast_ns()) {
dev_warn(adev->dev, "failed to suspend display audio\n");
+ pci_dev_put(p);
/* TODO: abort the succeeding gpu reset? */
return -ETIMEDOUT;
}
@@ -5072,97 +5099,10 @@ static int amdgpu_device_suspend_display_audio(struct amdgpu_device *adev)
pm_runtime_disable(&(p->dev));
+ pci_dev_put(p);
return 0;
}
-static void amdgpu_device_recheck_guilty_jobs(
- struct amdgpu_device *adev, struct list_head *device_list_handle,
- struct amdgpu_reset_context *reset_context)
-{
- int i, r = 0;
-
- for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
- struct amdgpu_ring *ring = adev->rings[i];
- int ret = 0;
- struct drm_sched_job *s_job;
-
- if (!ring || !ring->sched.thread)
- continue;
-
- s_job = list_first_entry_or_null(&ring->sched.pending_list,
- struct drm_sched_job, list);
- if (s_job == NULL)
- continue;
-
- /* clear job's guilty and depend the folowing step to decide the real one */
- drm_sched_reset_karma(s_job);
- drm_sched_resubmit_jobs_ext(&ring->sched, 1);
-
- if (!s_job->s_fence->parent) {
- DRM_WARN("Failed to get a HW fence for job!");
- continue;
- }
-
- ret = dma_fence_wait_timeout(s_job->s_fence->parent, false, ring->sched.timeout);
- if (ret == 0) { /* timeout */
- DRM_ERROR("Found the real bad job! ring:%s, job_id:%llx\n",
- ring->sched.name, s_job->id);
-
-
- amdgpu_fence_driver_isr_toggle(adev, true);
-
- /* Clear this failed job from fence array */
- amdgpu_fence_driver_clear_job_fences(ring);
-
- amdgpu_fence_driver_isr_toggle(adev, false);
-
- /* Since the job won't signal and we go for
- * another resubmit drop this parent pointer
- */
- dma_fence_put(s_job->s_fence->parent);
- s_job->s_fence->parent = NULL;
-
- /* set guilty */
- drm_sched_increase_karma(s_job);
- amdgpu_reset_prepare_hwcontext(adev, reset_context);
-retry:
- /* do hw reset */
- if (amdgpu_sriov_vf(adev)) {
- amdgpu_virt_fini_data_exchange(adev);
- r = amdgpu_device_reset_sriov(adev, false);
- if (r)
- adev->asic_reset_res = r;
- } else {
- clear_bit(AMDGPU_SKIP_HW_RESET,
- &reset_context->flags);
- r = amdgpu_do_asic_reset(device_list_handle,
- reset_context);
- if (r && r == -EAGAIN)
- goto retry;
- }
-
- /*
- * add reset counter so that the following
- * resubmitted job could flush vmid
- */
- atomic_inc(&adev->gpu_reset_counter);
- continue;
- }
-
- /* got the hw fence, signal finished fence */
- atomic_dec(ring->sched.score);
- dma_fence_get(&s_job->s_fence->finished);
- dma_fence_signal(&s_job->s_fence->finished);
- dma_fence_put(&s_job->s_fence->finished);
-
- /* remove node from list and free the job */
- spin_lock(&ring->sched.job_list_lock);
- list_del_init(&s_job->list);
- spin_unlock(&ring->sched.job_list_lock);
- ring->sched.ops->free_job(s_job);
- }
-}
-
static inline void amdgpu_device_stop_pending_resets(struct amdgpu_device *adev)
{
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
@@ -5183,7 +5123,6 @@ static inline void amdgpu_device_stop_pending_resets(struct amdgpu_device *adev)
}
-
/**
* amdgpu_device_gpu_recover - reset the asic and recover scheduler
*
@@ -5206,7 +5145,6 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
int i, r = 0;
bool need_emergency_restart = false;
bool audio_suspended = false;
- int tmp_vram_lost_counter;
bool gpu_reset_for_dev_remove = false;
gpu_reset_for_dev_remove =
@@ -5352,7 +5290,6 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
amdgpu_device_stop_pending_resets(tmp_adev);
}
- tmp_vram_lost_counter = atomic_read(&((adev)->vram_lost_counter));
/* Actual ASIC resets if needed.*/
/* Host driver will handle XGMI hive reset for SRIOV */
if (amdgpu_sriov_vf(adev)) {
@@ -5377,29 +5314,13 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
/* Post ASIC reset for all devs .*/
list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
- /*
- * Sometimes a later bad compute job can block a good gfx job as gfx
- * and compute ring share internal GC HW mutually. We add an additional
- * guilty jobs recheck step to find the real guilty job, it synchronously
- * submits and pends for the first job being signaled. If it gets timeout,
- * we identify it as a real guilty job.
- */
- if (amdgpu_gpu_recovery == 2 &&
- !(tmp_vram_lost_counter < atomic_read(&adev->vram_lost_counter)))
- amdgpu_device_recheck_guilty_jobs(
- tmp_adev, device_list_handle, reset_context);
-
for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
struct amdgpu_ring *ring = tmp_adev->rings[i];
if (!ring || !ring->sched.thread)
continue;
- /* No point to resubmit jobs if we didn't HW reset*/
- if (!tmp_adev->asic_reset_res && !job_signaled)
- drm_sched_resubmit_jobs(&ring->sched);
-
- drm_sched_start(&ring->sched, !tmp_adev->asic_reset_res);
+ drm_sched_start(&ring->sched, true);
}
if (adev->enable_mes && adev->ip_versions[GC_HWIP][0] != IP_VERSION(11, 0, 3))
@@ -5441,6 +5362,8 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
amdgpu_device_resume_display_audio(tmp_adev);
amdgpu_device_unset_mp1_state(tmp_adev);
+
+ amdgpu_ras_set_error_query_ready(tmp_adev, true);
}
recover_end:
@@ -5852,8 +5775,6 @@ void amdgpu_pci_resume(struct pci_dev *pdev)
if (!ring || !ring->sched.thread)
continue;
-
- drm_sched_resubmit_jobs(&ring->sched);
drm_sched_start(&ring->sched, true);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
index 3993e61..1bbd560 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
@@ -305,8 +305,13 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev)
goto out;
}
- if (!amdgpu_discovery_verify_binary_signature(adev->mman.discovery_bin)) {
- dev_warn(adev->dev, "get invalid ip discovery binary signature from vram\n");
+ if (!amdgpu_discovery_verify_binary_signature(adev->mman.discovery_bin) || amdgpu_discovery == 2) {
+ /* ignore the discovery binary from vram if discovery=2 in kernel module parameter */
+ if (amdgpu_discovery == 2)
+ dev_info(adev->dev,"force read ip discovery binary from file");
+ else
+ dev_warn(adev->dev, "get invalid ip discovery binary signature from vram\n");
+
/* retry read ip discovery binary from file */
r = amdgpu_discovery_read_binary_from_file(adev, adev->mman.discovery_bin);
if (r) {
@@ -1507,6 +1512,7 @@ static int amdgpu_discovery_set_common_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
amdgpu_device_ip_block_add(adev, &soc21_common_ip_block);
break;
default:
@@ -1551,6 +1557,7 @@ static int amdgpu_discovery_set_gmc_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
amdgpu_device_ip_block_add(adev, &gmc_v11_0_ip_block);
break;
default:
@@ -1636,6 +1643,7 @@ static int amdgpu_discovery_set_psp_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(13, 0, 7):
case IP_VERSION(13, 0, 8):
case IP_VERSION(13, 0, 10):
+ case IP_VERSION(13, 0, 11):
amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block);
break;
case IP_VERSION(13, 0, 4):
@@ -1686,6 +1694,7 @@ static int amdgpu_discovery_set_smu_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(13, 0, 7):
case IP_VERSION(13, 0, 8):
case IP_VERSION(13, 0, 10):
+ case IP_VERSION(13, 0, 11):
amdgpu_device_ip_block_add(adev, &smu_v13_0_ip_block);
break;
default:
@@ -1697,9 +1706,17 @@ static int amdgpu_discovery_set_smu_ip_blocks(struct amdgpu_device *adev)
return 0;
}
+#if defined(CONFIG_DRM_AMD_DC)
+static void amdgpu_discovery_set_sriov_display(struct amdgpu_device *adev)
+{
+ amdgpu_device_set_sriov_virtual_display(adev);
+ amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block);
+}
+#endif
+
static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev)
{
- if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) {
+ if (adev->enable_virtual_display) {
amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block);
return 0;
}
@@ -1727,7 +1744,10 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(3, 1, 6):
case IP_VERSION(3, 2, 0):
case IP_VERSION(3, 2, 1):
- amdgpu_device_ip_block_add(adev, &dm_ip_block);
+ if (amdgpu_sriov_vf(adev))
+ amdgpu_discovery_set_sriov_display(adev);
+ else
+ amdgpu_device_ip_block_add(adev, &dm_ip_block);
break;
default:
dev_err(adev->dev,
@@ -1740,7 +1760,10 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(12, 0, 0):
case IP_VERSION(12, 0, 1):
case IP_VERSION(12, 1, 0):
- amdgpu_device_ip_block_add(adev, &dm_ip_block);
+ if (amdgpu_sriov_vf(adev))
+ amdgpu_discovery_set_sriov_display(adev);
+ else
+ amdgpu_device_ip_block_add(adev, &dm_ip_block);
break;
default:
dev_err(adev->dev,
@@ -1785,6 +1808,7 @@ static int amdgpu_discovery_set_gc_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
amdgpu_device_ip_block_add(adev, &gfx_v11_0_ip_block);
break;
default:
@@ -1948,6 +1972,7 @@ static int amdgpu_discovery_set_mes_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
amdgpu_device_ip_block_add(adev, &mes_v11_0_ip_block);
adev->enable_mes = true;
adev->enable_mes_kiq = true;
@@ -2161,6 +2186,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev)
break;
case IP_VERSION(10, 3, 1):
adev->family = AMDGPU_FAMILY_VGH;
+ adev->apu_flags |= AMD_APU_IS_VANGOGH;
break;
case IP_VERSION(10, 3, 3):
adev->family = AMDGPU_FAMILY_YC;
@@ -2177,6 +2203,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev)
adev->family = AMDGPU_FAMILY_GC_11_0_0;
break;
case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
adev->family = AMDGPU_FAMILY_GC_11_0_1;
break;
default:
@@ -2194,6 +2221,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev)
case IP_VERSION(10, 3, 6):
case IP_VERSION(10, 3, 7):
case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
adev->flags |= AMD_IS_APU;
break;
default:
@@ -2250,6 +2278,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev)
adev->nbio.hdp_flush_reg = &nbio_v4_3_hdp_flush_reg;
break;
case IP_VERSION(7, 7, 0):
+ case IP_VERSION(7, 7, 1):
adev->nbio.funcs = &nbio_v7_7_funcs;
adev->nbio.hdp_flush_reg = &nbio_v7_7_hdp_flush_reg;
break;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
index 1a06b8d..b22471b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
@@ -39,11 +39,46 @@
#include <linux/pm_runtime.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_edid.h>
-#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_fb_helper.h>
+#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_vblank.h>
+/**
+ * amdgpu_display_hotplug_work_func - work handler for display hotplug event
+ *
+ * @work: work struct pointer
+ *
+ * This is the hotplug event work handler (all ASICs).
+ * The work gets scheduled from the IRQ handler if there
+ * was a hotplug interrupt. It walks through the connector table
+ * and calls hotplug handler for each connector. After this, it sends
+ * a DRM hotplug event to alert userspace.
+ *
+ * This design approach is required in order to defer hotplug event handling
+ * from the IRQ handler to a work handler because hotplug handler has to use
+ * mutexes which cannot be locked in an IRQ handler (since &mutex_lock may
+ * sleep).
+ */
+void amdgpu_display_hotplug_work_func(struct work_struct *work)
+{
+ struct amdgpu_device *adev = container_of(work, struct amdgpu_device,
+ hotplug_work);
+ struct drm_device *dev = adev_to_drm(adev);
+ struct drm_mode_config *mode_config = &dev->mode_config;
+ struct drm_connector *connector;
+ struct drm_connector_list_iter iter;
+
+ mutex_lock(&mode_config->mutex);
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter)
+ amdgpu_connector_hotplug(connector);
+ drm_connector_list_iter_end(&iter);
+ mutex_unlock(&mode_config->mutex);
+ /* Just fire off a uevent and let userspace tell us what to do */
+ drm_helper_hpd_irq_event(dev);
+}
+
static int amdgpu_display_framebuffer_init(struct drm_device *dev,
struct amdgpu_framebuffer *rfb,
const struct drm_mode_fb_cmd2 *mode_cmd,
@@ -514,7 +549,7 @@ uint32_t amdgpu_display_supported_domains(struct amdgpu_device *adev,
*/
if ((bo_flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC) &&
amdgpu_bo_support_uswc(bo_flags) &&
- amdgpu_device_asic_has_dc_support(adev->asic_type) &&
+ adev->dc_enabled &&
adev->mode_info.gpu_vm_support)
domain |= AMDGPU_GEM_DOMAIN_GTT;
#endif
@@ -1214,7 +1249,6 @@ amdgpu_display_user_framebuffer_create(struct drm_device *dev,
const struct drm_mode_config_funcs amdgpu_mode_funcs = {
.fb_create = amdgpu_display_user_framebuffer_create,
- .output_poll_changed = drm_fb_helper_output_poll_changed,
};
static const struct drm_prop_enum_list amdgpu_underscan_enum_list[] =
@@ -1281,7 +1315,7 @@ int amdgpu_display_modeset_create_props(struct amdgpu_device *adev)
"dither",
amdgpu_dither_enum_list, sz);
- if (amdgpu_device_has_dc_support(adev)) {
+ if (adev->dc_enabled) {
adev->mode_info.abm_level_property =
drm_property_create_range(adev_to_drm(adev), 0,
"abm level", 0, 4);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h
index 560352f..9d19940 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h
@@ -35,6 +35,7 @@
#define amdgpu_display_add_encoder(adev, e, s, c) (adev)->mode_info.funcs->add_encoder((adev), (e), (s), (c))
#define amdgpu_display_add_connector(adev, ci, sd, ct, ib, coi, h, r) (adev)->mode_info.funcs->add_connector((adev), (ci), (sd), (ct), (ib), (coi), (h), (r))
+void amdgpu_display_hotplug_work_func(struct work_struct *work);
void amdgpu_display_update_priority(struct amdgpu_device *adev);
uint32_t amdgpu_display_supported_domains(struct amdgpu_device *adev,
uint64_t bo_flags);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index 7bd8e33..271e30e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -328,7 +328,9 @@ amdgpu_dma_buf_create_obj(struct drm_device *dev, struct dma_buf *dma_buf)
if (dma_buf->ops == &amdgpu_dmabuf_ops) {
struct amdgpu_bo *other = gem_to_amdgpu_bo(dma_buf->priv);
- flags |= other->flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC;
+ flags |= other->flags & (AMDGPU_GEM_CREATE_CPU_GTT_USWC |
+ AMDGPU_GEM_CREATE_COHERENT |
+ AMDGPU_GEM_CREATE_UNCACHED);
}
ret = amdgpu_gem_object_create(adev, dma_buf->size, PAGE_SIZE,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index bf2d50c..7383272 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -25,6 +25,7 @@
#include <drm/amdgpu_drm.h>
#include <drm/drm_aperture.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem.h>
#include <drm/drm_vblank.h>
#include <drm/drm_managed.h>
@@ -230,17 +231,18 @@ module_param_named(vis_vramlimit, amdgpu_vis_vram_limit, int, 0444);
/**
* DOC: gartsize (uint)
- * Restrict the size of GART in Mib (32, 64, etc.) for testing. The default is -1 (The size depends on asic).
+ * Restrict the size of GART (for kernel use) in Mib (32, 64, etc.) for testing.
+ * The default is -1 (The size depends on asic).
*/
-MODULE_PARM_DESC(gartsize, "Size of GART to setup in megabytes (32, 64, etc., -1=auto)");
+MODULE_PARM_DESC(gartsize, "Size of kernel GART to setup in megabytes (32, 64, etc., -1=auto)");
module_param_named(gartsize, amdgpu_gart_size, uint, 0600);
/**
* DOC: gttsize (int)
- * Restrict the size of GTT domain in MiB for testing. The default is -1 (It's VRAM size if 3GB < VRAM < 3/4 RAM,
- * otherwise 3/4 RAM size).
+ * Restrict the size of GTT domain (for userspace use) in MiB for testing.
+ * The default is -1 (Use 1/2 RAM, minimum value is 3GB).
*/
-MODULE_PARM_DESC(gttsize, "Size of the GTT domain in megabytes (-1 = auto)");
+MODULE_PARM_DESC(gttsize, "Size of the GTT userspace domain in megabytes (-1 = auto)");
module_param_named(gttsize, amdgpu_gtt_size, int, 0600);
/**
@@ -533,7 +535,7 @@ module_param_named(compute_multipipe, amdgpu_compute_multipipe, int, 0444);
* DOC: gpu_recovery (int)
* Set to enable GPU recovery mechanism (1 = enable, 0 = disable). The default is -1 (auto, disabled except SRIOV).
*/
-MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism, (2 = advanced tdr mode, 1 = enable, 0 = disable, -1 = auto)");
+MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism, (1 = enable, 0 = disable, -1 = auto)");
module_param_named(gpu_recovery, amdgpu_gpu_recovery, int, 0444);
/**
@@ -1924,9 +1926,6 @@ static const struct pci_device_id pciidlist[] = {
{0x1002, 0x73AF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
{0x1002, 0x73BF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
- /* Van Gogh */
- {0x1002, 0x163F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_VANGOGH|AMD_IS_APU},
-
/* Yellow Carp */
{0x1002, 0x164D, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_YELLOW_CARP|AMD_IS_APU},
{0x1002, 0x1681, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_YELLOW_CARP|AMD_IS_APU},
@@ -2471,7 +2470,7 @@ static int amdgpu_runtime_idle_check_display(struct device *dev)
if (ret)
return ret;
- if (amdgpu_device_has_dc_support(adev)) {
+ if (adev->dc_enabled) {
struct drm_crtc *crtc;
drm_for_each_crtc(crtc, drm_dev) {
@@ -2572,6 +2571,8 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev)
amdgpu_device_baco_enter(drm_dev);
}
+ dev_dbg(&pdev->dev, "asic/device is runtime suspended\n");
+
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_eeprom.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_eeprom.c
index 4d9eb01..7d2a908 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_eeprom.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_eeprom.c
@@ -79,13 +79,15 @@
* That is, for an I2C EEPROM driver everything is controlled by
* the "eeprom_addr".
*
+ * See also top of amdgpu_ras_eeprom.c.
+ *
* P.S. If you need to write, lock and read the Identification Page,
* (M24M02-DR device only, which we do not use), change the "7" to
* "0xF" in the macro below, and let the client set bit 20 to 1 in
* "eeprom_addr", and set A10 to 0 to write into it, and A10 and A1 to
* 1 to lock it permanently.
*/
-#define MAKE_I2C_ADDR(_aa) ((0xA << 3) | (((_aa) >> 16) & 7))
+#define MAKE_I2C_ADDR(_aa) ((0xA << 3) | (((_aa) >> 16) & 0xF))
static int __amdgpu_eeprom_xfer(struct i2c_adapter *i2c_adap, u32 eeprom_addr,
u8 *eeprom_buf, u16 buf_size, bool read)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index d0d99ed..0044420 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -55,6 +55,7 @@ struct amdgpu_fence {
/* RB, DMA, etc. */
struct amdgpu_ring *ring;
+ ktime_t start_timestamp;
};
static struct kmem_cache *amdgpu_fence_slab;
@@ -199,6 +200,8 @@ int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, struct amd
}
}
+ to_amdgpu_fence(fence)->start_timestamp = ktime_get();
+
/* This function can't be called concurrently anyway, otherwise
* emitting the fence would mess up the hardware ring buffer.
*/
@@ -407,6 +410,57 @@ unsigned amdgpu_fence_count_emitted(struct amdgpu_ring *ring)
}
/**
+ * amdgpu_fence_last_unsignaled_time_us - the time fence emitted until now
+ * @ring: ring the fence is associated with
+ *
+ * Find the earliest fence unsignaled until now, calculate the time delta
+ * between the time fence emitted and now.
+ */
+u64 amdgpu_fence_last_unsignaled_time_us(struct amdgpu_ring *ring)
+{
+ struct amdgpu_fence_driver *drv = &ring->fence_drv;
+ struct dma_fence *fence;
+ uint32_t last_seq, sync_seq;
+
+ last_seq = atomic_read(&ring->fence_drv.last_seq);
+ sync_seq = READ_ONCE(ring->fence_drv.sync_seq);
+ if (last_seq == sync_seq)
+ return 0;
+
+ ++last_seq;
+ last_seq &= drv->num_fences_mask;
+ fence = drv->fences[last_seq];
+ if (!fence)
+ return 0;
+
+ return ktime_us_delta(ktime_get(),
+ to_amdgpu_fence(fence)->start_timestamp);
+}
+
+/**
+ * amdgpu_fence_update_start_timestamp - update the timestamp of the fence
+ * @ring: ring the fence is associated with
+ * @seq: the fence seq number to update.
+ * @timestamp: the start timestamp to update.
+ *
+ * The function called at the time the fence and related ib is about to
+ * resubmit to gpu in MCBP scenario. Thus we do not consider race condition
+ * with amdgpu_fence_process to modify the same fence.
+ */
+void amdgpu_fence_update_start_timestamp(struct amdgpu_ring *ring, uint32_t seq, ktime_t timestamp)
+{
+ struct amdgpu_fence_driver *drv = &ring->fence_drv;
+ struct dma_fence *fence;
+
+ seq &= drv->num_fences_mask;
+ fence = drv->fences[seq];
+ if (!fence)
+ return;
+
+ to_amdgpu_fence(fence)->start_timestamp = timestamp;
+}
+
+/**
* amdgpu_fence_driver_start_ring - make the fence driver
* ready for use on the requested ring.
*
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c
index e325150..2c38ac7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c
@@ -29,9 +29,10 @@
#include "amdgpu_fru_eeprom.h"
#include "amdgpu_eeprom.h"
-#define FRU_EEPROM_MADDR 0x60000
+#define FRU_EEPROM_MADDR_6 0x60000
+#define FRU_EEPROM_MADDR_8 0x80000
-static bool is_fru_eeprom_supported(struct amdgpu_device *adev)
+static bool is_fru_eeprom_supported(struct amdgpu_device *adev, u32 *fru_addr)
{
/* Only server cards have the FRU EEPROM
* TODO: See if we can figure this out dynamically instead of
@@ -45,6 +46,11 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev)
if (amdgpu_sriov_vf(adev))
return false;
+ /* The default I2C EEPROM address of the FRU.
+ */
+ if (fru_addr)
+ *fru_addr = FRU_EEPROM_MADDR_8;
+
/* VBIOS is of the format ###-DXXXYYYY-##. For SKU identification,
* we can use just the "DXXX" portion. If there were more models, we
* could convert the 3 characters to a hex integer and use a switch
@@ -57,21 +63,29 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev)
if (strnstr(atom_ctx->vbios_version, "D161",
sizeof(atom_ctx->vbios_version)) ||
strnstr(atom_ctx->vbios_version, "D163",
- sizeof(atom_ctx->vbios_version)))
+ sizeof(atom_ctx->vbios_version))) {
+ *fru_addr = FRU_EEPROM_MADDR_6;
return true;
- else
+ } else {
return false;
+ }
case CHIP_ALDEBARAN:
- /* All Aldebaran SKUs have the FRU */
+ /* All Aldebaran SKUs have an FRU */
+ if (!strnstr(atom_ctx->vbios_version, "D673",
+ sizeof(atom_ctx->vbios_version)))
+ if (fru_addr)
+ *fru_addr = FRU_EEPROM_MADDR_6;
return true;
case CHIP_SIENNA_CICHLID:
if (strnstr(atom_ctx->vbios_version, "D603",
- sizeof(atom_ctx->vbios_version))) {
+ sizeof(atom_ctx->vbios_version))) {
if (strnstr(atom_ctx->vbios_version, "D603GLXE",
- sizeof(atom_ctx->vbios_version)))
+ sizeof(atom_ctx->vbios_version))) {
return false;
- else
+ } else {
+ *fru_addr = FRU_EEPROM_MADDR_6;
return true;
+ }
} else {
return false;
}
@@ -80,41 +94,14 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev)
}
}
-static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr,
- unsigned char *buf, size_t buf_size)
-{
- int ret;
- u8 size;
-
- ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr, buf, 1);
- if (ret < 1) {
- DRM_WARN("FRU: Failed to get size field");
- return ret;
- }
-
- /* The size returned by the i2c requires subtraction of 0xC0 since the
- * size apparently always reports as 0xC0+actual size.
- */
- size = buf[0] & 0x3F;
- size = min_t(size_t, size, buf_size);
-
- ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr + 1,
- buf, size);
- if (ret < 1) {
- DRM_WARN("FRU: Failed to get data field");
- return ret;
- }
-
- return size;
-}
-
int amdgpu_fru_get_product_info(struct amdgpu_device *adev)
{
- unsigned char buf[AMDGPU_PRODUCT_NAME_LEN];
- u32 addrptr;
+ unsigned char buf[8], *pia;
+ u32 addr, fru_addr;
int size, len;
+ u8 csum;
- if (!is_fru_eeprom_supported(adev))
+ if (!is_fru_eeprom_supported(adev, &fru_addr))
return 0;
/* If algo exists, it means that the i2c_adapter's initialized */
@@ -123,88 +110,102 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev)
return -ENODEV;
}
- /* There's a lot of repetition here. This is due to the FRU having
- * variable-length fields. To get the information, we have to find the
- * size of each field, and then keep reading along and reading along
- * until we get all of the data that we want. We use addrptr to track
- * the address as we go
+ /* Read the IPMI Common header */
+ len = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, fru_addr, buf,
+ sizeof(buf));
+ if (len != 8) {
+ DRM_ERROR("Couldn't read the IPMI Common Header: %d", len);
+ return len < 0 ? len : -EIO;
+ }
+
+ if (buf[0] != 1) {
+ DRM_ERROR("Bad IPMI Common Header version: 0x%02x", buf[0]);
+ return -EIO;
+ }
+
+ for (csum = 0; len > 0; len--)
+ csum += buf[len - 1];
+ if (csum) {
+ DRM_ERROR("Bad IPMI Common Header checksum: 0x%02x", csum);
+ return -EIO;
+ }
+
+ /* Get the offset to the Product Info Area (PIA). */
+ addr = buf[4] * 8;
+ if (!addr)
+ return 0;
+
+ /* Get the absolute address to the PIA. */
+ addr += fru_addr;
+
+ /* Read the header of the PIA. */
+ len = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addr, buf, 3);
+ if (len != 3) {
+ DRM_ERROR("Couldn't read the Product Info Area header: %d", len);
+ return len < 0 ? len : -EIO;
+ }
+
+ if (buf[0] != 1) {
+ DRM_ERROR("Bad IPMI Product Info Area version: 0x%02x", buf[0]);
+ return -EIO;
+ }
+
+ size = buf[1] * 8;
+ pia = kzalloc(size, GFP_KERNEL);
+ if (!pia)
+ return -ENOMEM;
+
+ /* Read the whole PIA. */
+ len = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addr, pia, size);
+ if (len != size) {
+ kfree(pia);
+ DRM_ERROR("Couldn't read the Product Info Area: %d", len);
+ return len < 0 ? len : -EIO;
+ }
+
+ for (csum = 0; size > 0; size--)
+ csum += pia[size - 1];
+ if (csum) {
+ DRM_ERROR("Bad Product Info Area checksum: 0x%02x", csum);
+ return -EIO;
+ }
+
+ /* Now extract useful information from the PIA.
+ *
+ * Skip the Manufacturer Name at [3] and go directly to
+ * the Product Name field.
*/
+ addr = 3 + 1 + (pia[3] & 0x3F);
+ if (addr + 1 >= len)
+ goto Out;
+ memcpy(adev->product_name, pia + addr + 1,
+ min_t(size_t,
+ sizeof(adev->product_name),
+ pia[addr] & 0x3F));
+ adev->product_name[sizeof(adev->product_name) - 1] = '\0';
- /* The first fields are all of size 1-byte, from 0-7 are offsets that
- * contain information that isn't useful to us.
- * Bytes 8-a are all 1-byte and refer to the size of the entire struct,
- * and the language field, so just start from 0xb, manufacturer size
- */
- addrptr = FRU_EEPROM_MADDR + 0xb;
- size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
- if (size < 1) {
- DRM_ERROR("Failed to read FRU Manufacturer, ret:%d", size);
- return -EINVAL;
- }
+ /* Go to the Product Part/Model Number field. */
+ addr += 1 + (pia[addr] & 0x3F);
+ if (addr + 1 >= len)
+ goto Out;
+ memcpy(adev->product_number, pia + addr + 1,
+ min_t(size_t,
+ sizeof(adev->product_number),
+ pia[addr] & 0x3F));
+ adev->product_number[sizeof(adev->product_number) - 1] = '\0';
- /* Increment the addrptr by the size of the field, and 1 due to the
- * size field being 1 byte. This pattern continues below.
- */
- addrptr += size + 1;
- size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
- if (size < 1) {
- DRM_ERROR("Failed to read FRU product name, ret:%d", size);
- return -EINVAL;
- }
+ /* Go to the Product Version field. */
+ addr += 1 + (pia[addr] & 0x3F);
- len = size;
- if (len >= AMDGPU_PRODUCT_NAME_LEN) {
- DRM_WARN("FRU Product Name is larger than %d characters. This is likely a mistake",
- AMDGPU_PRODUCT_NAME_LEN);
- len = AMDGPU_PRODUCT_NAME_LEN - 1;
- }
- memcpy(adev->product_name, buf, len);
- adev->product_name[len] = '\0';
-
- addrptr += size + 1;
- size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
- if (size < 1) {
- DRM_ERROR("Failed to read FRU product number, ret:%d", size);
- return -EINVAL;
- }
-
- len = size;
- /* Product number should only be 16 characters. Any more,
- * and something could be wrong. Cap it at 16 to be safe
- */
- if (len >= sizeof(adev->product_number)) {
- DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake");
- len = sizeof(adev->product_number) - 1;
- }
- memcpy(adev->product_number, buf, len);
- adev->product_number[len] = '\0';
-
- addrptr += size + 1;
- size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
-
- if (size < 1) {
- DRM_ERROR("Failed to read FRU product version, ret:%d", size);
- return -EINVAL;
- }
-
- addrptr += size + 1;
- size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
-
- if (size < 1) {
- DRM_ERROR("Failed to read FRU serial number, ret:%d", size);
- return -EINVAL;
- }
-
- len = size;
- /* Serial number should only be 16 characters. Any more,
- * and something could be wrong. Cap it at 16 to be safe
- */
- if (len >= sizeof(adev->serial)) {
- DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake");
- len = sizeof(adev->serial) - 1;
- }
- memcpy(adev->serial, buf, len);
- adev->serial[len] = '\0';
-
+ /* Go to the Product Serial Number field. */
+ addr += 1 + (pia[addr] & 0x3F);
+ if (addr + 1 >= len)
+ goto Out;
+ memcpy(adev->serial, pia + addr + 1, min_t(size_t,
+ sizeof(adev->serial),
+ pia[addr] & 0x3F));
+ adev->serial[sizeof(adev->serial) - 1] = '\0';
+Out:
+ kfree(pia);
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 91571b1..62e98f1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -38,6 +38,7 @@
#include "amdgpu.h"
#include "amdgpu_display.h"
#include "amdgpu_dma_buf.h"
+#include "amdgpu_hmm.h"
#include "amdgpu_xgmi.h"
static const struct drm_gem_object_funcs amdgpu_gem_object_funcs;
@@ -87,7 +88,7 @@ static void amdgpu_gem_object_free(struct drm_gem_object *gobj)
struct amdgpu_bo *robj = gem_to_amdgpu_bo(gobj);
if (robj) {
- amdgpu_mn_unregister(robj);
+ amdgpu_hmm_unregister(robj);
amdgpu_bo_unref(&robj);
}
}
@@ -112,7 +113,7 @@ int amdgpu_gem_object_create(struct amdgpu_device *adev, unsigned long size,
bp.resv = resv;
bp.preferred_domain = initial_domain;
bp.flags = flags;
- bp.domain = initial_domain;
+ bp.domain = initial_domain | AMDGPU_GEM_DOMAIN_CPU;
bp.bo_ptr_size = sizeof(struct amdgpu_bo);
r = amdgpu_bo_create_user(adev, &bp, &ubo);
@@ -331,20 +332,10 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data,
}
initial_domain = (u32)(0xffffffff & args->in.domains);
-retry:
r = amdgpu_gem_object_create(adev, size, args->in.alignment,
- initial_domain,
- flags, ttm_bo_type_device, resv, &gobj);
+ initial_domain, flags, ttm_bo_type_device,
+ resv, &gobj);
if (r && r != -ERESTARTSYS) {
- if (flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED) {
- flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
- goto retry;
- }
-
- if (initial_domain == AMDGPU_GEM_DOMAIN_VRAM) {
- initial_domain |= AMDGPU_GEM_DOMAIN_GTT;
- goto retry;
- }
DRM_DEBUG("Failed to allocate GEM object (%llu, %d, %llu, %d)\n",
size, initial_domain, args->in.alignment, r);
}
@@ -414,7 +405,7 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data,
if (r)
goto release_object;
- r = amdgpu_mn_register(bo, args->addr);
+ r = amdgpu_hmm_register(bo, args->addr);
if (r)
goto release_object;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
index 9546adc..23692e5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
@@ -583,10 +583,14 @@ void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable)
if (adev->gfx.gfx_off_req_count == 0 &&
!adev->gfx.gfx_off_state) {
/* If going to s2idle, no need to wait */
- if (adev->in_s0ix)
- delay = GFX_OFF_NO_DELAY;
- schedule_delayed_work(&adev->gfx.gfx_off_delay_work,
+ if (adev->in_s0ix) {
+ if (!amdgpu_dpm_set_powergating_by_smu(adev,
+ AMD_IP_BLOCK_TYPE_GFX, true))
+ adev->gfx.gfx_off_state = true;
+ } else {
+ schedule_delayed_work(&adev->gfx.gfx_off_delay_work,
delay);
+ }
}
} else {
if (adev->gfx.gfx_off_req_count == 0) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
index 832b380..b3df4787 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
@@ -33,6 +33,7 @@
#include "amdgpu_imu.h"
#include "soc15.h"
#include "amdgpu_ras.h"
+#include "amdgpu_ring_mux.h"
/* GFX current status */
#define AMDGPU_GFX_NORMAL_MODE 0x00000000L
@@ -352,6 +353,9 @@ struct amdgpu_gfx {
struct amdgpu_gfx_ras *ras;
bool is_poweron;
+
+ struct amdgpu_ring sw_gfx_ring[AMDGPU_MAX_SW_GFX_RINGS];
+ struct amdgpu_ring_mux muxer;
};
#define amdgpu_gfx_get_gpu_clock_counter(adev) (adev)->gfx.funcs->get_gpu_clock_counter((adev))
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index 28612e5..02a4c93 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -548,6 +548,8 @@ void amdgpu_gmc_tmz_set(struct amdgpu_device *adev)
case IP_VERSION(10, 3, 1):
/* YELLOW_CARP*/
case IP_VERSION(10, 3, 3):
+ case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
/* Don't enable it by default yet.
*/
if (amdgpu_tmz < 1) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
index 1f3302a..44367f03 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
@@ -144,7 +144,7 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man,
node->base.start = node->mm_nodes[0].start;
} else {
node->mm_nodes[0].start = 0;
- node->mm_nodes[0].size = node->base.num_pages;
+ node->mm_nodes[0].size = PFN_UP(node->base.size);
node->base.start = AMDGPU_BO_INVALID_OFFSET;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c
similarity index 71%
rename from drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
rename to drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c
index b86c0b8..65715cb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c
@@ -49,9 +49,12 @@
#include "amdgpu.h"
#include "amdgpu_amdkfd.h"
+#include "amdgpu_hmm.h"
+
+#define MAX_WALK_BYTE (2UL << 30)
/**
- * amdgpu_mn_invalidate_gfx - callback to notify about mm change
+ * amdgpu_hmm_invalidate_gfx - callback to notify about mm change
*
* @mni: the range (mm) is about to update
* @range: details on the invalidation
@@ -60,9 +63,9 @@
* Block for operations on BOs to finish and mark pages as accessed and
* potentially dirty.
*/
-static bool amdgpu_mn_invalidate_gfx(struct mmu_interval_notifier *mni,
- const struct mmu_notifier_range *range,
- unsigned long cur_seq)
+static bool amdgpu_hmm_invalidate_gfx(struct mmu_interval_notifier *mni,
+ const struct mmu_notifier_range *range,
+ unsigned long cur_seq)
{
struct amdgpu_bo *bo = container_of(mni, struct amdgpu_bo, notifier);
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
@@ -83,12 +86,12 @@ static bool amdgpu_mn_invalidate_gfx(struct mmu_interval_notifier *mni,
return true;
}
-static const struct mmu_interval_notifier_ops amdgpu_mn_gfx_ops = {
- .invalidate = amdgpu_mn_invalidate_gfx,
+static const struct mmu_interval_notifier_ops amdgpu_hmm_gfx_ops = {
+ .invalidate = amdgpu_hmm_invalidate_gfx,
};
/**
- * amdgpu_mn_invalidate_hsa - callback to notify about mm change
+ * amdgpu_hmm_invalidate_hsa - callback to notify about mm change
*
* @mni: the range (mm) is about to update
* @range: details on the invalidation
@@ -97,9 +100,9 @@ static const struct mmu_interval_notifier_ops amdgpu_mn_gfx_ops = {
* We temporarily evict the BO attached to this range. This necessitates
* evicting all user-mode queues of the process.
*/
-static bool amdgpu_mn_invalidate_hsa(struct mmu_interval_notifier *mni,
- const struct mmu_notifier_range *range,
- unsigned long cur_seq)
+static bool amdgpu_hmm_invalidate_hsa(struct mmu_interval_notifier *mni,
+ const struct mmu_notifier_range *range,
+ unsigned long cur_seq)
{
struct amdgpu_bo *bo = container_of(mni, struct amdgpu_bo, notifier);
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
@@ -117,12 +120,12 @@ static bool amdgpu_mn_invalidate_hsa(struct mmu_interval_notifier *mni,
return true;
}
-static const struct mmu_interval_notifier_ops amdgpu_mn_hsa_ops = {
- .invalidate = amdgpu_mn_invalidate_hsa,
+static const struct mmu_interval_notifier_ops amdgpu_hmm_hsa_ops = {
+ .invalidate = amdgpu_hmm_invalidate_hsa,
};
/**
- * amdgpu_mn_register - register a BO for notifier updates
+ * amdgpu_hmm_register - register a BO for notifier updates
*
* @bo: amdgpu buffer object
* @addr: userptr addr we should monitor
@@ -130,25 +133,25 @@ static const struct mmu_interval_notifier_ops amdgpu_mn_hsa_ops = {
* Registers a mmu_notifier for the given BO at the specified address.
* Returns 0 on success, -ERRNO if anything goes wrong.
*/
-int amdgpu_mn_register(struct amdgpu_bo *bo, unsigned long addr)
+int amdgpu_hmm_register(struct amdgpu_bo *bo, unsigned long addr)
{
if (bo->kfd_bo)
return mmu_interval_notifier_insert(&bo->notifier, current->mm,
addr, amdgpu_bo_size(bo),
- &amdgpu_mn_hsa_ops);
+ &amdgpu_hmm_hsa_ops);
return mmu_interval_notifier_insert(&bo->notifier, current->mm, addr,
amdgpu_bo_size(bo),
- &amdgpu_mn_gfx_ops);
+ &amdgpu_hmm_gfx_ops);
}
/**
- * amdgpu_mn_unregister - unregister a BO for notifier updates
+ * amdgpu_hmm_unregister - unregister a BO for notifier updates
*
* @bo: amdgpu buffer object
*
* Remove any registration of mmu notifier updates from the buffer object.
*/
-void amdgpu_mn_unregister(struct amdgpu_bo *bo)
+void amdgpu_hmm_unregister(struct amdgpu_bo *bo)
{
if (!bo->notifier.mm)
return;
@@ -157,12 +160,12 @@ void amdgpu_mn_unregister(struct amdgpu_bo *bo)
}
int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier,
- struct mm_struct *mm, struct page **pages,
- uint64_t start, uint64_t npages,
- struct hmm_range **phmm_range, bool readonly,
- bool mmap_locked, void *owner)
+ uint64_t start, uint64_t npages, bool readonly,
+ void *owner, struct page **pages,
+ struct hmm_range **phmm_range)
{
struct hmm_range *hmm_range;
+ unsigned long end;
unsigned long timeout;
unsigned long i;
unsigned long *pfns;
@@ -184,32 +187,42 @@ int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier,
hmm_range->default_flags |= HMM_PFN_REQ_WRITE;
hmm_range->hmm_pfns = pfns;
hmm_range->start = start;
- hmm_range->end = start + npages * PAGE_SIZE;
+ end = start + npages * PAGE_SIZE;
hmm_range->dev_private_owner = owner;
- /* Assuming 512MB takes maxmium 1 second to fault page address */
- timeout = max(npages >> 17, 1ULL) * HMM_RANGE_DEFAULT_TIMEOUT;
- timeout = jiffies + msecs_to_jiffies(timeout);
+ do {
+ hmm_range->end = min(hmm_range->start + MAX_WALK_BYTE, end);
+
+ pr_debug("hmm range: start = 0x%lx, end = 0x%lx",
+ hmm_range->start, hmm_range->end);
+
+ /* Assuming 512MB takes maxmium 1 second to fault page address */
+ timeout = max((hmm_range->end - hmm_range->start) >> 29, 1UL);
+ timeout *= HMM_RANGE_DEFAULT_TIMEOUT;
+ timeout = jiffies + msecs_to_jiffies(timeout);
retry:
- hmm_range->notifier_seq = mmu_interval_read_begin(notifier);
+ hmm_range->notifier_seq = mmu_interval_read_begin(notifier);
+ r = hmm_range_fault(hmm_range);
+ if (unlikely(r)) {
+ /*
+ * FIXME: This timeout should encompass the retry from
+ * mmu_interval_read_retry() as well.
+ */
+ if (r == -EBUSY && !time_after(jiffies, timeout))
+ goto retry;
+ goto out_free_pfns;
+ }
- if (likely(!mmap_locked))
- mmap_read_lock(mm);
+ if (hmm_range->end == end)
+ break;
+ hmm_range->hmm_pfns += MAX_WALK_BYTE >> PAGE_SHIFT;
+ hmm_range->start = hmm_range->end;
+ schedule();
+ } while (hmm_range->end < end);
- r = hmm_range_fault(hmm_range);
-
- if (likely(!mmap_locked))
- mmap_read_unlock(mm);
- if (unlikely(r)) {
- /*
- * FIXME: This timeout should encompass the retry from
- * mmu_interval_read_retry() as well.
- */
- if (r == -EBUSY && !time_after(jiffies, timeout))
- goto retry;
- goto out_free_pfns;
- }
+ hmm_range->start = start;
+ hmm_range->hmm_pfns = pfns;
/*
* Due to default_flags, all pages are HMM_PFN_VALID or
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.h
similarity index 78%
rename from drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h
rename to drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.h
index 14a3c18..13ed94d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.h
@@ -31,23 +31,22 @@
#include <linux/interval_tree.h>
int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier,
- struct mm_struct *mm, struct page **pages,
- uint64_t start, uint64_t npages,
- struct hmm_range **phmm_range, bool readonly,
- bool mmap_locked, void *owner);
+ uint64_t start, uint64_t npages, bool readonly,
+ void *owner, struct page **pages,
+ struct hmm_range **phmm_range);
int amdgpu_hmm_range_get_pages_done(struct hmm_range *hmm_range);
#if defined(CONFIG_HMM_MIRROR)
-int amdgpu_mn_register(struct amdgpu_bo *bo, unsigned long addr);
-void amdgpu_mn_unregister(struct amdgpu_bo *bo);
+int amdgpu_hmm_register(struct amdgpu_bo *bo, unsigned long addr);
+void amdgpu_hmm_unregister(struct amdgpu_bo *bo);
#else
-static inline int amdgpu_mn_register(struct amdgpu_bo *bo, unsigned long addr)
+static inline int amdgpu_hmm_register(struct amdgpu_bo *bo, unsigned long addr)
{
DRM_WARN_ONCE("HMM_MIRROR kernel config option is not enabled, "
"add CONFIG_ZONE_DEVICE=y in config file to fix this\n");
return -ENODEV;
}
-static inline void amdgpu_mn_unregister(struct amdgpu_bo *bo) {}
+static inline void amdgpu_hmm_unregister(struct amdgpu_bo *bo) {}
#endif
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
index 258cffe..bcccc34 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
@@ -182,7 +182,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
need_ctx_switch = ring->current_ctx != fence_ctx;
if (ring->funcs->emit_pipeline_sync && job &&
- ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
+ ((tmp = amdgpu_sync_get_fence(&job->explicit_sync)) ||
(amdgpu_sriov_vf(adev) && need_ctx_switch) ||
amdgpu_vm_need_pipeline_sync(ring, job))) {
need_pipe_sync = true;
@@ -211,6 +211,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
}
}
+ amdgpu_ring_ib_begin(ring);
if (job && ring->funcs->init_cond_exec)
patch_offset = amdgpu_ring_init_cond_exec(ring);
@@ -285,6 +286,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
ring->funcs->emit_wave_limit(ring, false);
+ amdgpu_ring_ib_end(ring);
amdgpu_ring_commit(ring);
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
index 03d115d..2a9a259 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
@@ -170,26 +170,27 @@ bool amdgpu_vmid_had_gpu_reset(struct amdgpu_device *adev,
*
* @vm: vm to allocate id for
* @ring: ring we want to submit job to
- * @sync: sync object where we add dependencies
* @idle: resulting idle VMID
+ * @fence: fence to wait for if no id could be grabbed
*
* Try to find an idle VMID, if none is idle add a fence to wait to the sync
* object. Returns -ENOMEM when we are out of memory.
*/
static int amdgpu_vmid_grab_idle(struct amdgpu_vm *vm,
struct amdgpu_ring *ring,
- struct amdgpu_sync *sync,
- struct amdgpu_vmid **idle)
+ struct amdgpu_vmid **idle,
+ struct dma_fence **fence)
{
struct amdgpu_device *adev = ring->adev;
unsigned vmhub = ring->funcs->vmhub;
struct amdgpu_vmid_mgr *id_mgr = &adev->vm_manager.id_mgr[vmhub];
struct dma_fence **fences;
unsigned i;
- int r;
- if (!dma_fence_is_signaled(ring->vmid_wait))
- return amdgpu_sync_fence(sync, ring->vmid_wait);
+ if (!dma_fence_is_signaled(ring->vmid_wait)) {
+ *fence = dma_fence_get(ring->vmid_wait);
+ return 0;
+ }
fences = kmalloc_array(id_mgr->num_ids, sizeof(void *), GFP_KERNEL);
if (!fences)
@@ -228,10 +229,10 @@ static int amdgpu_vmid_grab_idle(struct amdgpu_vm *vm,
return -ENOMEM;
}
- r = amdgpu_sync_fence(sync, &array->base);
+ *fence = dma_fence_get(&array->base);
dma_fence_put(ring->vmid_wait);
ring->vmid_wait = &array->base;
- return r;
+ return 0;
}
kfree(fences);
@@ -243,19 +244,17 @@ static int amdgpu_vmid_grab_idle(struct amdgpu_vm *vm,
*
* @vm: vm to allocate id for
* @ring: ring we want to submit job to
- * @sync: sync object where we add dependencies
- * @fence: fence protecting ID from reuse
* @job: job who wants to use the VMID
* @id: resulting VMID
+ * @fence: fence to wait for if no id could be grabbed
*
* Try to assign a reserved VMID.
*/
static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm,
struct amdgpu_ring *ring,
- struct amdgpu_sync *sync,
- struct dma_fence *fence,
struct amdgpu_job *job,
- struct amdgpu_vmid **id)
+ struct amdgpu_vmid **id,
+ struct dma_fence **fence)
{
struct amdgpu_device *adev = ring->adev;
unsigned vmhub = ring->funcs->vmhub;
@@ -282,7 +281,8 @@ static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm,
tmp = amdgpu_sync_peek_fence(&(*id)->active, ring);
if (tmp) {
*id = NULL;
- return amdgpu_sync_fence(sync, tmp);
+ *fence = dma_fence_get(tmp);
+ return 0;
}
needs_flush = true;
}
@@ -290,7 +290,7 @@ static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm,
/* Good we can use this VMID. Remember this submission as
* user of the VMID.
*/
- r = amdgpu_sync_fence(&(*id)->active, fence);
+ r = amdgpu_sync_fence(&(*id)->active, &job->base.s_fence->finished);
if (r)
return r;
@@ -304,19 +304,17 @@ static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm,
*
* @vm: vm to allocate id for
* @ring: ring we want to submit job to
- * @sync: sync object where we add dependencies
- * @fence: fence protecting ID from reuse
* @job: job who wants to use the VMID
* @id: resulting VMID
+ * @fence: fence to wait for if no id could be grabbed
*
* Try to reuse a VMID for this submission.
*/
static int amdgpu_vmid_grab_used(struct amdgpu_vm *vm,
struct amdgpu_ring *ring,
- struct amdgpu_sync *sync,
- struct dma_fence *fence,
struct amdgpu_job *job,
- struct amdgpu_vmid **id)
+ struct amdgpu_vmid **id,
+ struct dma_fence **fence)
{
struct amdgpu_device *adev = ring->adev;
unsigned vmhub = ring->funcs->vmhub;
@@ -352,7 +350,8 @@ static int amdgpu_vmid_grab_used(struct amdgpu_vm *vm,
/* Good, we can use this VMID. Remember this submission as
* user of the VMID.
*/
- r = amdgpu_sync_fence(&(*id)->active, fence);
+ r = amdgpu_sync_fence(&(*id)->active,
+ &job->base.s_fence->finished);
if (r)
return r;
@@ -370,15 +369,13 @@ static int amdgpu_vmid_grab_used(struct amdgpu_vm *vm,
*
* @vm: vm to allocate id for
* @ring: ring we want to submit job to
- * @sync: sync object where we add dependencies
- * @fence: fence protecting ID from reuse
* @job: job who wants to use the VMID
+ * @fence: fence to wait for if no id could be grabbed
*
* Allocate an id for the vm, adding fences to the sync obj as necessary.
*/
int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
- struct amdgpu_sync *sync, struct dma_fence *fence,
- struct amdgpu_job *job)
+ struct amdgpu_job *job, struct dma_fence **fence)
{
struct amdgpu_device *adev = ring->adev;
unsigned vmhub = ring->funcs->vmhub;
@@ -388,16 +385,16 @@ int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
int r = 0;
mutex_lock(&id_mgr->lock);
- r = amdgpu_vmid_grab_idle(vm, ring, sync, &idle);
+ r = amdgpu_vmid_grab_idle(vm, ring, &idle, fence);
if (r || !idle)
goto error;
if (vm->reserved_vmid[vmhub]) {
- r = amdgpu_vmid_grab_reserved(vm, ring, sync, fence, job, &id);
+ r = amdgpu_vmid_grab_reserved(vm, ring, job, &id, fence);
if (r || !id)
goto error;
} else {
- r = amdgpu_vmid_grab_used(vm, ring, sync, fence, job, &id);
+ r = amdgpu_vmid_grab_used(vm, ring, job, &id, fence);
if (r)
goto error;
@@ -406,7 +403,8 @@ int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
id = idle;
/* Remember this submission as user of the VMID */
- r = amdgpu_sync_fence(&id->active, fence);
+ r = amdgpu_sync_fence(&id->active,
+ &job->base.s_fence->finished);
if (r)
goto error;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h
index 06c8a00..57efe61 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h
@@ -84,8 +84,7 @@ void amdgpu_vmid_free_reserved(struct amdgpu_device *adev,
struct amdgpu_vm *vm,
unsigned vmhub);
int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
- struct amdgpu_sync *sync, struct dma_fence *fence,
- struct amdgpu_job *job);
+ struct amdgpu_job *job, struct dma_fence **fence);
void amdgpu_vmid_reset(struct amdgpu_device *adev, unsigned vmhub,
unsigned vmid);
void amdgpu_vmid_reset_all(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index 89011ba..a6aef48 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -101,41 +101,6 @@ const char *soc15_ih_clientid_name[] = {
};
/**
- * amdgpu_hotplug_work_func - work handler for display hotplug event
- *
- * @work: work struct pointer
- *
- * This is the hotplug event work handler (all ASICs).
- * The work gets scheduled from the IRQ handler if there
- * was a hotplug interrupt. It walks through the connector table
- * and calls hotplug handler for each connector. After this, it sends
- * a DRM hotplug event to alert userspace.
- *
- * This design approach is required in order to defer hotplug event handling
- * from the IRQ handler to a work handler because hotplug handler has to use
- * mutexes which cannot be locked in an IRQ handler (since &mutex_lock may
- * sleep).
- */
-static void amdgpu_hotplug_work_func(struct work_struct *work)
-{
- struct amdgpu_device *adev = container_of(work, struct amdgpu_device,
- hotplug_work);
- struct drm_device *dev = adev_to_drm(adev);
- struct drm_mode_config *mode_config = &dev->mode_config;
- struct drm_connector *connector;
- struct drm_connector_list_iter iter;
-
- mutex_lock(&mode_config->mutex);
- drm_connector_list_iter_begin(dev, &iter);
- drm_for_each_connector_iter(connector, &iter)
- amdgpu_connector_hotplug(connector);
- drm_connector_list_iter_end(&iter);
- mutex_unlock(&mode_config->mutex);
- /* Just fire off a uevent and let userspace tell us what to do */
- drm_helper_hpd_irq_event(dev);
-}
-
-/**
* amdgpu_irq_disable_all - disable *all* interrupts
*
* @adev: amdgpu device pointer
@@ -317,21 +282,6 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
}
}
- if (!amdgpu_device_has_dc_support(adev)) {
- if (!adev->enable_virtual_display)
- /* Disable vblank IRQs aggressively for power-saving */
- /* XXX: can this be enabled for DC? */
- adev_to_drm(adev)->vblank_disable_immediate = true;
-
- r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc);
- if (r)
- return r;
-
- /* Pre-DCE11 */
- INIT_WORK(&adev->hotplug_work,
- amdgpu_hotplug_work_func);
- }
-
INIT_WORK(&adev->irq.ih1_work, amdgpu_irq_handle_ih1);
INIT_WORK(&adev->irq.ih2_work, amdgpu_irq_handle_ih2);
INIT_WORK(&adev->irq.ih_soft_work, amdgpu_irq_handle_ih_soft);
@@ -345,11 +295,8 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
/* PCI devices require shared interrupts. */
r = request_irq(irq, amdgpu_irq_handler, IRQF_SHARED, adev_to_drm(adev)->driver->name,
adev_to_drm(adev));
- if (r) {
- if (!amdgpu_device_has_dc_support(adev))
- flush_work(&adev->hotplug_work);
+ if (r)
return r;
- }
adev->irq.installed = true;
adev->irq.irq = irq;
adev_to_drm(adev)->max_vblank_count = 0x00ffffff;
@@ -366,9 +313,6 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
adev->irq.installed = false;
if (adev->irq.msi_enabled)
pci_free_irq_vectors(adev->pdev);
-
- if (!amdgpu_device_has_dc_support(adev))
- flush_work(&adev->hotplug_work);
}
amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index adac650..9e54992 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -88,8 +88,9 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
return DRM_GPU_SCHED_STAT_NOMINAL;
}
-int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
- struct amdgpu_job **job, struct amdgpu_vm *vm)
+int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm,
+ struct drm_sched_entity *entity, void *owner,
+ unsigned int num_ibs, struct amdgpu_job **job)
{
if (num_ibs == 0)
return -EINVAL;
@@ -105,28 +106,34 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
(*job)->base.sched = &adev->rings[0]->sched;
(*job)->vm = vm;
- amdgpu_sync_create(&(*job)->sync);
- amdgpu_sync_create(&(*job)->sched_sync);
+ amdgpu_sync_create(&(*job)->explicit_sync);
(*job)->vram_lost_counter = atomic_read(&adev->vram_lost_counter);
(*job)->vm_pd_addr = AMDGPU_BO_INVALID_OFFSET;
- return 0;
+ if (!entity)
+ return 0;
+
+ return drm_sched_job_init(&(*job)->base, entity, owner);
}
-int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, unsigned size,
- enum amdgpu_ib_pool_type pool_type,
- struct amdgpu_job **job)
+int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev,
+ struct drm_sched_entity *entity, void *owner,
+ size_t size, enum amdgpu_ib_pool_type pool_type,
+ struct amdgpu_job **job)
{
int r;
- r = amdgpu_job_alloc(adev, 1, job, NULL);
+ r = amdgpu_job_alloc(adev, NULL, entity, owner, 1, job);
if (r)
return r;
(*job)->num_ibs = 1;
r = amdgpu_ib_get(adev, NULL, size, pool_type, &(*job)->ibs[0]);
- if (r)
+ if (r) {
+ if (entity)
+ drm_sched_job_cleanup(&(*job)->base);
kfree(*job);
+ }
return r;
}
@@ -166,8 +173,7 @@ static void amdgpu_job_free_cb(struct drm_sched_job *s_job)
drm_sched_job_cleanup(s_job);
- amdgpu_sync_free(&job->sync);
- amdgpu_sync_free(&job->sched_sync);
+ amdgpu_sync_free(&job->explicit_sync);
/* only put the hw fence if has embedded fence */
if (!job->hw_fence.ops)
@@ -194,9 +200,11 @@ void amdgpu_job_set_gang_leader(struct amdgpu_job *job,
void amdgpu_job_free(struct amdgpu_job *job)
{
+ if (job->base.entity)
+ drm_sched_job_cleanup(&job->base);
+
amdgpu_job_free_resources(job);
- amdgpu_sync_free(&job->sync);
- amdgpu_sync_free(&job->sched_sync);
+ amdgpu_sync_free(&job->explicit_sync);
if (job->gang_submit != &job->base.s_fence->scheduled)
dma_fence_put(job->gang_submit);
@@ -206,25 +214,16 @@ void amdgpu_job_free(struct amdgpu_job *job)
dma_fence_put(&job->hw_fence);
}
-int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
- void *owner, struct dma_fence **f)
+struct dma_fence *amdgpu_job_submit(struct amdgpu_job *job)
{
- int r;
-
- if (!f)
- return -EINVAL;
-
- r = drm_sched_job_init(&job->base, entity, owner);
- if (r)
- return r;
+ struct dma_fence *f;
drm_sched_job_arm(&job->base);
-
- *f = dma_fence_get(&job->base.s_fence->finished);
+ f = dma_fence_get(&job->base.s_fence->finished);
amdgpu_job_free_resources(job);
drm_sched_entity_push_job(&job->base);
- return 0;
+ return f;
}
int amdgpu_job_submit_direct(struct amdgpu_job *job, struct amdgpu_ring *ring,
@@ -242,33 +241,22 @@ int amdgpu_job_submit_direct(struct amdgpu_job *job, struct amdgpu_ring *ring,
return 0;
}
-static struct dma_fence *amdgpu_job_dependency(struct drm_sched_job *sched_job,
- struct drm_sched_entity *s_entity)
+static struct dma_fence *
+amdgpu_job_prepare_job(struct drm_sched_job *sched_job,
+ struct drm_sched_entity *s_entity)
{
struct amdgpu_ring *ring = to_amdgpu_ring(s_entity->rq->sched);
struct amdgpu_job *job = to_amdgpu_job(sched_job);
- struct amdgpu_vm *vm = job->vm;
- struct dma_fence *fence;
+ struct dma_fence *fence = NULL;
int r;
- fence = amdgpu_sync_get_fence(&job->sync);
- if (fence && drm_sched_dependency_optimized(fence, s_entity)) {
- r = amdgpu_sync_fence(&job->sched_sync, fence);
- if (r)
- DRM_ERROR("Error adding fence (%d)\n", r);
- }
-
if (!fence && job->gang_submit)
fence = amdgpu_device_switch_gang(ring->adev, job->gang_submit);
- while (fence == NULL && vm && !job->vmid) {
- r = amdgpu_vmid_grab(vm, ring, &job->sync,
- &job->base.s_fence->finished,
- job);
+ while (!fence && job->vm && !job->vmid) {
+ r = amdgpu_vmid_grab(job->vm, ring, job, &fence);
if (r)
DRM_ERROR("Error getting VM ID (%d)\n", r);
-
- fence = amdgpu_sync_get_fence(&job->sync);
}
return fence;
@@ -285,8 +273,6 @@ static struct dma_fence *amdgpu_job_run(struct drm_sched_job *sched_job)
job = to_amdgpu_job(sched_job);
finished = &job->base.s_fence->finished;
- BUG_ON(amdgpu_sync_peek_fence(&job->sync, NULL));
-
trace_amdgpu_sched_run_job(job);
/* Skip job if VRAM is lost and never resubmit gangs */
@@ -345,7 +331,7 @@ void amdgpu_job_stop_all_jobs_on_sched(struct drm_gpu_scheduler *sched)
}
const struct drm_sched_backend_ops amdgpu_sched_ops = {
- .dependency = amdgpu_job_dependency,
+ .prepare_job = amdgpu_job_prepare_job,
.run_job = amdgpu_job_run,
.timedout_job = amdgpu_job_timedout,
.free_job = amdgpu_job_free_cb
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h
index ab7b150..a372802 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h
@@ -47,8 +47,7 @@ enum amdgpu_ib_pool_type;
struct amdgpu_job {
struct drm_sched_job base;
struct amdgpu_vm *vm;
- struct amdgpu_sync sync;
- struct amdgpu_sync sched_sync;
+ struct amdgpu_sync explicit_sync;
struct dma_fence hw_fence;
struct dma_fence *gang_submit;
uint32_t preamble_status;
@@ -78,18 +77,20 @@ static inline struct amdgpu_ring *amdgpu_job_ring(struct amdgpu_job *job)
return to_amdgpu_ring(job->base.entity->rq->sched);
}
-int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
- struct amdgpu_job **job, struct amdgpu_vm *vm);
-int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, unsigned size,
- enum amdgpu_ib_pool_type pool, struct amdgpu_job **job);
+int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm,
+ struct drm_sched_entity *entity, void *owner,
+ unsigned int num_ibs, struct amdgpu_job **job);
+int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev,
+ struct drm_sched_entity *entity, void *owner,
+ size_t size, enum amdgpu_ib_pool_type pool_type,
+ struct amdgpu_job **job);
void amdgpu_job_set_resources(struct amdgpu_job *job, struct amdgpu_bo *gds,
struct amdgpu_bo *gws, struct amdgpu_bo *oa);
void amdgpu_job_free_resources(struct amdgpu_job *job);
void amdgpu_job_set_gang_leader(struct amdgpu_job *job,
struct amdgpu_job *leader);
void amdgpu_job_free(struct amdgpu_job *job);
-int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
- void *owner, struct dma_fence **f);
+struct dma_fence *amdgpu_job_submit(struct amdgpu_job *job);
int amdgpu_job_submit_direct(struct amdgpu_job *job, struct amdgpu_ring *ring,
struct dma_fence **fence);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c
index 518eb0e..6f81ed4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c
@@ -150,14 +150,15 @@ static int amdgpu_jpeg_dec_set_reg(struct amdgpu_ring *ring, uint32_t handle,
const unsigned ib_size_dw = 16;
int i, r;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4,
+ AMDGPU_IB_POOL_DIRECT, &job);
if (r)
return r;
ib = &job->ibs[0];
- ib->ptr[0] = PACKETJ(adev->jpeg.internal.jpeg_pitch, 0, 0, PACKETJ_TYPE0);
+ ib->ptr[0] = PACKETJ(adev->jpeg.internal.jpeg_pitch, 0, 0,
+ PACKETJ_TYPE0);
ib->ptr[1] = 0xDEADBEEF;
for (i = 2; i < 16; i += 2) {
ib->ptr[i] = PACKETJ(0, 0, 0, PACKETJ_TYPE6);
@@ -234,3 +235,20 @@ int amdgpu_jpeg_process_poison_irq(struct amdgpu_device *adev,
return 0;
}
+
+void jpeg_set_ras_funcs(struct amdgpu_device *adev)
+{
+ if (!adev->jpeg.ras)
+ return;
+
+ amdgpu_ras_register_ras_block(adev, &adev->jpeg.ras->ras_block);
+
+ strcpy(adev->jpeg.ras->ras_block.ras_comm.name, "jpeg");
+ adev->jpeg.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__JPEG;
+ adev->jpeg.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__POISON;
+ adev->jpeg.ras_if = &adev->jpeg.ras->ras_block.ras_comm;
+
+ /* If don't define special ras_late_init function, use default ras_late_init */
+ if (!adev->jpeg.ras->ras_block.ras_late_init)
+ adev->jpeg.ras->ras_block.ras_late_init = amdgpu_ras_block_late_init;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.h
index 635dca5..e8ca3e3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.h
@@ -72,5 +72,6 @@ int amdgpu_jpeg_dec_ring_test_ib(struct amdgpu_ring *ring, long timeout);
int amdgpu_jpeg_process_poison_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry);
+void jpeg_set_ras_funcs(struct amdgpu_device *adev);
#endif /*__AMDGPU_JPEG_H__*/
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index 4e42dcb..7aa7e52 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -29,6 +29,7 @@
#include "amdgpu.h"
#include <drm/amdgpu_drm.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fb_helper.h>
#include "amdgpu_uvd.h"
#include "amdgpu_vce.h"
#include "atom.h"
@@ -430,7 +431,7 @@ static int amdgpu_hw_ip_info(struct amdgpu_device *adev,
case AMDGPU_HW_IP_VCN_DEC:
type = AMD_IP_BLOCK_TYPE_VCN;
for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
- if (adev->uvd.harvest_config & (1 << i))
+ if (adev->vcn.harvest_config & (1 << i))
continue;
if (adev->vcn.inst[i].ring_dec.sched.ready)
@@ -442,7 +443,7 @@ static int amdgpu_hw_ip_info(struct amdgpu_device *adev,
case AMDGPU_HW_IP_VCN_ENC:
type = AMD_IP_BLOCK_TYPE_VCN;
for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
- if (adev->uvd.harvest_config & (1 << i))
+ if (adev->vcn.harvest_config & (1 << i))
continue;
for (j = 0; j < adev->vcn.num_enc_rings; j++)
@@ -796,7 +797,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
dev_info->ids_flags = 0;
if (adev->flags & AMD_IS_APU)
dev_info->ids_flags |= AMDGPU_IDS_FLAGS_FUSION;
- if (amdgpu_mcbp || amdgpu_sriov_vf(adev))
+ if (amdgpu_mcbp)
dev_info->ids_flags |= AMDGPU_IDS_FLAGS_PREEMPTION;
if (amdgpu_is_tmz(adev))
dev_info->ids_flags |= AMDGPU_IDS_FLAGS_TMZ;
@@ -1172,7 +1173,7 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
goto error_vm;
}
- if (amdgpu_mcbp || amdgpu_sriov_vf(adev)) {
+ if (amdgpu_mcbp) {
uint64_t csa_addr = amdgpu_csa_vaddr(adev) & AMDGPU_GMC_HOLE_MASK;
r = amdgpu_map_static_csa(adev, &fpriv->vm, adev->virt.csa_obj,
@@ -1236,7 +1237,7 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
if (amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_VCE) != NULL)
amdgpu_vce_free_handles(adev, file_priv);
- if (amdgpu_mcbp || amdgpu_sriov_vf(adev)) {
+ if (amdgpu_mcbp) {
/* TODO: how to handle reserve failure */
BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, true));
amdgpu_vm_bo_del(adev, fpriv->csa_va);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h
index ad980f4..97c05d08 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h
@@ -91,14 +91,12 @@ struct amdgpu_mes {
struct amdgpu_bo *ucode_fw_obj[AMDGPU_MAX_MES_PIPES];
uint64_t ucode_fw_gpu_addr[AMDGPU_MAX_MES_PIPES];
uint32_t *ucode_fw_ptr[AMDGPU_MAX_MES_PIPES];
- uint32_t ucode_fw_version[AMDGPU_MAX_MES_PIPES];
uint64_t uc_start_addr[AMDGPU_MAX_MES_PIPES];
/* mes ucode data */
struct amdgpu_bo *data_fw_obj[AMDGPU_MAX_MES_PIPES];
uint64_t data_fw_gpu_addr[AMDGPU_MAX_MES_PIPES];
uint32_t *data_fw_ptr[AMDGPU_MAX_MES_PIPES];
- uint32_t data_fw_version[AMDGPU_MAX_MES_PIPES];
uint64_t data_start_addr[AMDGPU_MAX_MES_PIPES];
/* eop gpu obj */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
index 37322550..8a39300 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
@@ -36,7 +36,6 @@
#include <drm/drm_encoder.h>
#include <drm/drm_fixed.h>
#include <drm/drm_crtc_helper.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_probe_helper.h>
#include <linux/i2c.h>
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 2e8f6cd..919bbea 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -542,6 +542,7 @@ int amdgpu_bo_create(struct amdgpu_device *adev,
/* GWS and OA don't need any alignment. */
page_align = bp->byte_align;
size <<= PAGE_SHIFT;
+
} else if (bp->domain & AMDGPU_GEM_DOMAIN_GDS) {
/* Both size and alignment must be a multiple of 4. */
page_align = ALIGN(bp->byte_align, 4);
@@ -580,11 +581,7 @@ int amdgpu_bo_create(struct amdgpu_device *adev,
bo->flags |= AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE;
bo->tbo.bdev = &adev->mman.bdev;
- if (bp->domain & (AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA |
- AMDGPU_GEM_DOMAIN_GDS))
- amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU);
- else
- amdgpu_bo_placement_from_domain(bo, bp->domain);
+ amdgpu_bo_placement_from_domain(bo, bp->domain);
if (bp->type == ttm_bo_type_kernel)
bo->tbo.priority = 1;
@@ -776,7 +773,7 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr)
return 0;
}
- r = ttm_bo_kmap(&bo->tbo, 0, bo->tbo.resource->num_pages, &bo->kmap);
+ r = ttm_bo_kmap(&bo->tbo, 0, PFN_UP(bo->tbo.base.size), &bo->kmap);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 7978307..7a2fc92 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -52,6 +52,32 @@ static int psp_load_smu_fw(struct psp_context *psp);
static int psp_rap_terminate(struct psp_context *psp);
static int psp_securedisplay_terminate(struct psp_context *psp);
+static int psp_ring_init(struct psp_context *psp,
+ enum psp_ring_type ring_type)
+{
+ int ret = 0;
+ struct psp_ring *ring;
+ struct amdgpu_device *adev = psp->adev;
+
+ ring = &psp->km_ring;
+
+ ring->ring_type = ring_type;
+
+ /* allocate 4k Page of Local Frame Buffer memory for ring */
+ ring->ring_size = 0x1000;
+ ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
+ AMDGPU_GEM_DOMAIN_VRAM,
+ &adev->firmware.rbuf,
+ &ring->ring_mem_mc_addr,
+ (void **)&ring->ring_mem);
+ if (ret) {
+ ring->ring_size = 0;
+ return ret;
+ }
+
+ return 0;
+}
+
/*
* Due to DF Cstate management centralized to PMFW, the firmware
* loading sequence will be updated as below:
@@ -139,6 +165,7 @@ static int psp_early_init(void *handle)
case IP_VERSION(13, 0, 5):
case IP_VERSION(13, 0, 8):
case IP_VERSION(13, 0, 10):
+ case IP_VERSION(13, 0, 11):
psp_v13_0_set_psp_funcs(psp);
psp->autoload_supported = true;
break;
@@ -486,26 +513,22 @@ static int psp_sw_fini(void *handle)
struct psp_gfx_cmd_resp *cmd = psp->cmd;
psp_memory_training_fini(psp);
- if (psp->sos_fw) {
- release_firmware(psp->sos_fw);
- psp->sos_fw = NULL;
- }
- if (psp->asd_fw) {
- release_firmware(psp->asd_fw);
- psp->asd_fw = NULL;
- }
- if (psp->ta_fw) {
- release_firmware(psp->ta_fw);
- psp->ta_fw = NULL;
- }
- if (psp->cap_fw) {
- release_firmware(psp->cap_fw);
- psp->cap_fw = NULL;
- }
- if (psp->toc_fw) {
- release_firmware(psp->toc_fw);
- psp->toc_fw = NULL;
- }
+
+ release_firmware(psp->sos_fw);
+ psp->sos_fw = NULL;
+
+ release_firmware(psp->asd_fw);
+ psp->asd_fw = NULL;
+
+ release_firmware(psp->ta_fw);
+ psp->ta_fw = NULL;
+
+ release_firmware(psp->cap_fw);
+ psp->cap_fw = NULL;
+
+ release_firmware(psp->toc_fw);
+ psp->toc_fw = NULL;
+
if (adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 0) ||
adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 7))
psp_sysfs_fini(adev);
@@ -835,7 +858,7 @@ static int psp_tmr_unload(struct psp_context *psp)
struct psp_gfx_cmd_resp *cmd = acquire_psp_cmd_buf(psp);
psp_prep_tmr_unload_cmd_buf(psp, cmd);
- dev_info(psp->adev->dev, "free PSP TMR buffer\n");
+ dev_dbg(psp->adev->dev, "free PSP TMR buffer\n");
ret = psp_cmd_submit_buf(psp, NULL, cmd,
psp->fence_buf_mc_addr);
@@ -992,6 +1015,8 @@ int psp_ta_unload(struct psp_context *psp, struct ta_context *context)
ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr);
+ context->resp_status = cmd->resp.status;
+
release_psp_cmd_buf(psp);
return ret;
@@ -1073,42 +1098,6 @@ int psp_ta_init_shared_buf(struct psp_context *psp,
&mem_ctx->shared_buf);
}
-static void psp_prep_ta_invoke_indirect_cmd_buf(struct psp_gfx_cmd_resp *cmd,
- uint32_t ta_cmd_id,
- struct ta_context *context)
-{
- cmd->cmd_id = GFX_CMD_ID_INVOKE_CMD;
- cmd->cmd.cmd_invoke_cmd.session_id = context->session_id;
- cmd->cmd.cmd_invoke_cmd.ta_cmd_id = ta_cmd_id;
-
- cmd->cmd.cmd_invoke_cmd.buf.num_desc = 1;
- cmd->cmd.cmd_invoke_cmd.buf.total_size = context->mem_context.shared_mem_size;
- cmd->cmd.cmd_invoke_cmd.buf.buf_desc[0].buf_size = context->mem_context.shared_mem_size;
- cmd->cmd.cmd_invoke_cmd.buf.buf_desc[0].buf_phy_addr_lo =
- lower_32_bits(context->mem_context.shared_mc_addr);
- cmd->cmd.cmd_invoke_cmd.buf.buf_desc[0].buf_phy_addr_hi =
- upper_32_bits(context->mem_context.shared_mc_addr);
-}
-
-int psp_ta_invoke_indirect(struct psp_context *psp,
- uint32_t ta_cmd_id,
- struct ta_context *context)
-{
- int ret;
- struct psp_gfx_cmd_resp *cmd = acquire_psp_cmd_buf(psp);
-
- psp_prep_ta_invoke_indirect_cmd_buf(cmd, ta_cmd_id, context);
-
- ret = psp_cmd_submit_buf(psp, NULL, cmd,
- psp->fence_buf_mc_addr);
-
- context->resp_status = cmd->resp.status;
-
- release_psp_cmd_buf(psp);
-
- return ret;
-}
-
static void psp_prep_ta_invoke_cmd_buf(struct psp_gfx_cmd_resp *cmd,
uint32_t ta_cmd_id,
uint32_t session_id)
@@ -1551,7 +1540,7 @@ int psp_ras_terminate(struct psp_context *psp)
return ret;
}
-static int psp_ras_initialize(struct psp_context *psp)
+int psp_ras_initialize(struct psp_context *psp)
{
int ret;
uint32_t boot_cfg = 0xFF;
@@ -1614,7 +1603,7 @@ static int psp_ras_initialize(struct psp_context *psp)
psp->ras_context.context.mem_context.shared_mem_size = PSP_RAS_SHARED_MEM_SIZE;
psp->ras_context.context.ta_load_type = GFX_CMD_ID_LOAD_TA;
- if (!psp->ras_context.context.initialized) {
+ if (!psp->ras_context.context.mem_context.shared_buf) {
ret = psp_ta_init_shared_buf(psp, &psp->ras_context.context.mem_context);
if (ret)
return ret;
@@ -1635,7 +1624,9 @@ static int psp_ras_initialize(struct psp_context *psp)
else {
if (ras_cmd->ras_status)
dev_warn(psp->adev->dev, "RAS Init Status: 0x%X\n", ras_cmd->ras_status);
- amdgpu_ras_fini(psp->adev);
+
+ /* fail to load RAS TA */
+ psp->ras_context.context.initialized = false;
}
return ret;
@@ -1942,10 +1933,15 @@ static int psp_securedisplay_initialize(struct psp_context *psp)
} else
return ret;
+ mutex_lock(&psp->securedisplay_context.mutex);
+
psp_prep_securedisplay_cmd_buf(psp, &securedisplay_cmd,
TA_SECUREDISPLAY_COMMAND__QUERY_TA);
ret = psp_securedisplay_invoke(psp, TA_SECUREDISPLAY_COMMAND__QUERY_TA);
+
+ mutex_unlock(&psp->securedisplay_context.mutex);
+
if (ret) {
psp_securedisplay_terminate(psp);
/* free securedisplay shared memory */
@@ -1994,12 +1990,8 @@ int psp_securedisplay_invoke(struct psp_context *psp, uint32_t ta_cmd_id)
ta_cmd_id != TA_SECUREDISPLAY_COMMAND__SEND_ROI_CRC)
return -EINVAL;
- mutex_lock(&psp->securedisplay_context.mutex);
-
ret = psp_ta_invoke(psp, ta_cmd_id, &psp->securedisplay_context.context);
- mutex_unlock(&psp->securedisplay_context.mutex);
-
return ret;
}
/* SECUREDISPLAY end */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
index 58ce3eb..cf4f60c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
@@ -118,7 +118,6 @@ struct psp_funcs
int (*bootloader_load_dbg_drv)(struct psp_context *psp);
int (*bootloader_load_ras_drv)(struct psp_context *psp);
int (*bootloader_load_sos)(struct psp_context *psp);
- int (*ring_init)(struct psp_context *psp, enum psp_ring_type ring_type);
int (*ring_create)(struct psp_context *psp,
enum psp_ring_type ring_type);
int (*ring_stop)(struct psp_context *psp,
@@ -136,6 +135,12 @@ struct psp_funcs
int (*vbflash_stat)(struct psp_context *psp);
};
+struct ta_funcs {
+ int (*fn_ta_initialize)(struct psp_context *psp);
+ int (*fn_ta_invoke)(struct psp_context *psp, uint32_t ta_cmd_id);
+ int (*fn_ta_terminate)(struct psp_context *psp);
+};
+
#define AMDGPU_XGMI_MAX_CONNECTED_NODES 64
struct psp_xgmi_node_info {
uint64_t node_id;
@@ -309,6 +314,7 @@ struct psp_context
struct psp_gfx_cmd_resp *cmd;
const struct psp_funcs *funcs;
+ const struct ta_funcs *ta_funcs;
/* firmware buffer */
struct amdgpu_bo *fw_pri_bo;
@@ -389,7 +395,6 @@ struct amdgpu_psp_funcs {
};
-#define psp_ring_init(psp, type) (psp)->funcs->ring_init((psp), (type))
#define psp_ring_create(psp, type) (psp)->funcs->ring_create((psp), (type))
#define psp_ring_stop(psp, type) (psp)->funcs->ring_stop((psp), (type))
#define psp_ring_destroy(psp, type) ((psp)->funcs->ring_destroy((psp), (type)))
@@ -463,9 +468,6 @@ int psp_ta_load(struct psp_context *psp, struct ta_context *context);
int psp_ta_invoke(struct psp_context *psp,
uint32_t ta_cmd_id,
struct ta_context *context);
-int psp_ta_invoke_indirect(struct psp_context *psp,
- uint32_t ta_cmd_id,
- struct ta_context *context);
int psp_xgmi_initialize(struct psp_context *psp, bool set_extended_data, bool load_ta);
int psp_xgmi_terminate(struct psp_context *psp);
@@ -479,7 +481,7 @@ int psp_xgmi_get_topology_info(struct psp_context *psp,
int psp_xgmi_set_topology_info(struct psp_context *psp,
int number_devices,
struct psp_xgmi_topology_info *topology);
-
+int psp_ras_initialize(struct psp_context *psp);
int psp_ras_invoke(struct psp_context *psp, uint32_t ta_cmd_id);
int psp_ras_enable_features(struct psp_context *psp,
union ta_ras_cmd_input *info, bool enable);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.c
index 0988e00..468a67b30 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.c
@@ -41,30 +41,46 @@ static uint32_t get_bin_version(const uint8_t *bin)
return hdr->ucode_version;
}
-static void prep_ta_mem_context(struct psp_context *psp,
- struct ta_context *context,
+static int prep_ta_mem_context(struct ta_mem_context *mem_context,
uint8_t *shared_buf,
uint32_t shared_buf_len)
{
- context->mem_context.shared_mem_size = PAGE_ALIGN(shared_buf_len);
- psp_ta_init_shared_buf(psp, &context->mem_context);
+ if (mem_context->shared_mem_size < shared_buf_len)
+ return -EINVAL;
+ memset(mem_context->shared_buf, 0, mem_context->shared_mem_size);
+ memcpy((void *)mem_context->shared_buf, shared_buf, shared_buf_len);
- memcpy((void *)context->mem_context.shared_buf, shared_buf, shared_buf_len);
+ return 0;
}
static bool is_ta_type_valid(enum ta_type_id ta_type)
{
- bool ret = false;
-
switch (ta_type) {
case TA_TYPE_RAS:
- ret = true;
+ return true;
+ default:
+ return false;
+ }
+}
+
+static const struct ta_funcs ras_ta_funcs = {
+ .fn_ta_initialize = psp_ras_initialize,
+ .fn_ta_invoke = psp_ras_invoke,
+ .fn_ta_terminate = psp_ras_terminate
+};
+
+static void set_ta_context_funcs(struct psp_context *psp,
+ enum ta_type_id ta_type,
+ struct ta_context **pcontext)
+{
+ switch (ta_type) {
+ case TA_TYPE_RAS:
+ *pcontext = &psp->ras_context.context;
+ psp->ta_funcs = &ras_ta_funcs;
break;
default:
break;
}
-
- return ret;
}
static const struct file_operations ta_load_debugfs_fops = {
@@ -85,8 +101,7 @@ static const struct file_operations ta_invoke_debugfs_fops = {
.owner = THIS_MODULE
};
-
-/**
+/*
* DOC: AMDGPU TA debugfs interfaces
*
* Three debugfs interfaces can be opened by a program to
@@ -111,15 +126,18 @@ static const struct file_operations ta_invoke_debugfs_fops = {
*
* - For TA invoke debugfs interface:
* Transmit buffer:
+ * - TA type (4bytes)
* - TA ID (4bytes)
* - TA CMD ID (4bytes)
- * - TA shard buf length (4bytes)
+ * - TA shard buf length
+ * (4bytes, value not beyond TA shared memory size)
* - TA shared buf
* Receive buffer:
* - TA shared buf
*
* - For TA unload debugfs interface:
* Transmit buffer:
+ * - TA type (4bytes)
* - TA ID (4bytes)
*/
@@ -131,59 +149,92 @@ static ssize_t ta_if_load_debugfs_write(struct file *fp, const char *buf, size_t
uint32_t copy_pos = 0;
int ret = 0;
- struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(fp)->i_private;
- struct psp_context *psp = &adev->psp;
- struct ta_context context = {0};
+ struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(fp)->i_private;
+ struct psp_context *psp = &adev->psp;
+ struct ta_context *context = NULL;
if (!buf)
return -EINVAL;
ret = copy_from_user((void *)&ta_type, &buf[copy_pos], sizeof(uint32_t));
if (ret || (!is_ta_type_valid(ta_type)))
- return -EINVAL;
+ return -EFAULT;
copy_pos += sizeof(uint32_t);
ret = copy_from_user((void *)&ta_bin_len, &buf[copy_pos], sizeof(uint32_t));
if (ret)
- return -EINVAL;
+ return -EFAULT;
copy_pos += sizeof(uint32_t);
ta_bin = kzalloc(ta_bin_len, GFP_KERNEL);
if (!ta_bin)
- ret = -ENOMEM;
+ return -ENOMEM;
if (copy_from_user((void *)ta_bin, &buf[copy_pos], ta_bin_len)) {
ret = -EFAULT;
goto err_free_bin;
}
- ret = psp_ras_terminate(psp);
- if (ret) {
- dev_err(adev->dev, "Failed to unload embedded RAS TA\n");
+ /* Set TA context and functions */
+ set_ta_context_funcs(psp, ta_type, &context);
+
+ if (!psp->ta_funcs || !psp->ta_funcs->fn_ta_terminate) {
+ dev_err(adev->dev, "Unsupported function to terminate TA\n");
+ ret = -EOPNOTSUPP;
goto err_free_bin;
}
- context.ta_type = ta_type;
- context.ta_load_type = GFX_CMD_ID_LOAD_TA;
- context.bin_desc.fw_version = get_bin_version(ta_bin);
- context.bin_desc.size_bytes = ta_bin_len;
- context.bin_desc.start_addr = ta_bin;
+ /*
+ * Allocate TA shared buf in case shared buf was freed
+ * due to loading TA failed before.
+ */
+ if (!context->mem_context.shared_buf) {
+ ret = psp_ta_init_shared_buf(psp, &context->mem_context);
+ if (ret) {
+ ret = -ENOMEM;
+ goto err_free_bin;
+ }
+ }
- ret = psp_ta_load(psp, &context);
-
- if (ret || context.resp_status) {
- dev_err(adev->dev, "TA load via debugfs failed (%d) status %d\n",
- ret, context.resp_status);
+ ret = psp_fn_ta_terminate(psp);
+ if (ret || context->resp_status) {
+ dev_err(adev->dev,
+ "Failed to unload embedded TA (%d) and status (0x%X)\n",
+ ret, context->resp_status);
if (!ret)
ret = -EINVAL;
- goto err_free_bin;
+ goto err_free_ta_shared_buf;
}
- context.initialized = true;
- if (copy_to_user((char *)buf, (void *)&context.session_id, sizeof(uint32_t)))
+ /* Prepare TA context for TA initialization */
+ context->ta_type = ta_type;
+ context->bin_desc.fw_version = get_bin_version(ta_bin);
+ context->bin_desc.size_bytes = ta_bin_len;
+ context->bin_desc.start_addr = ta_bin;
+
+ if (!psp->ta_funcs->fn_ta_initialize) {
+ dev_err(adev->dev, "Unsupported function to initialize TA\n");
+ ret = -EOPNOTSUPP;
+ goto err_free_ta_shared_buf;
+ }
+
+ ret = psp_fn_ta_initialize(psp);
+ if (ret || context->resp_status) {
+ dev_err(adev->dev, "Failed to load TA via debugfs (%d) and status (0x%X)\n",
+ ret, context->resp_status);
+ if (!ret)
+ ret = -EINVAL;
+ goto err_free_ta_shared_buf;
+ }
+
+ if (copy_to_user((char *)buf, (void *)&context->session_id, sizeof(uint32_t)))
ret = -EFAULT;
+err_free_ta_shared_buf:
+ /* Only free TA shared buf when returns error code */
+ if (ret && context->mem_context.shared_buf)
+ psp_ta_free_shared_buf(&context->mem_context);
err_free_bin:
kfree(ta_bin);
@@ -192,58 +243,85 @@ static ssize_t ta_if_load_debugfs_write(struct file *fp, const char *buf, size_t
static ssize_t ta_if_unload_debugfs_write(struct file *fp, const char *buf, size_t len, loff_t *off)
{
- uint32_t ta_id = 0;
- int ret = 0;
+ uint32_t ta_type = 0;
+ uint32_t ta_id = 0;
+ uint32_t copy_pos = 0;
+ int ret = 0;
- struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(fp)->i_private;
- struct psp_context *psp = &adev->psp;
- struct ta_context context = {0};
+ struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(fp)->i_private;
+ struct psp_context *psp = &adev->psp;
+ struct ta_context *context = NULL;
if (!buf)
return -EINVAL;
- ret = copy_from_user((void *)&ta_id, buf, sizeof(uint32_t));
+ ret = copy_from_user((void *)&ta_type, &buf[copy_pos], sizeof(uint32_t));
+ if (ret || (!is_ta_type_valid(ta_type)))
+ return -EFAULT;
+
+ copy_pos += sizeof(uint32_t);
+
+ ret = copy_from_user((void *)&ta_id, &buf[copy_pos], sizeof(uint32_t));
if (ret)
- return -EINVAL;
+ return -EFAULT;
- context.session_id = ta_id;
+ set_ta_context_funcs(psp, ta_type, &context);
+ context->session_id = ta_id;
- ret = psp_ta_unload(psp, &context);
- if (!ret)
- context.initialized = false;
+ if (!psp->ta_funcs || !psp->ta_funcs->fn_ta_terminate) {
+ dev_err(adev->dev, "Unsupported function to terminate TA\n");
+ return -EOPNOTSUPP;
+ }
+
+ ret = psp_fn_ta_terminate(psp);
+ if (ret || context->resp_status) {
+ dev_err(adev->dev, "Failed to unload TA via debugfs (%d) and status (0x%X)\n",
+ ret, context->resp_status);
+ if (!ret)
+ ret = -EINVAL;
+ }
+
+ if (context->mem_context.shared_buf)
+ psp_ta_free_shared_buf(&context->mem_context);
return ret;
}
static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size_t len, loff_t *off)
{
+ uint32_t ta_type = 0;
uint32_t ta_id = 0;
uint32_t cmd_id = 0;
uint32_t shared_buf_len = 0;
- uint8_t *shared_buf = NULL;
+ uint8_t *shared_buf = NULL;
uint32_t copy_pos = 0;
int ret = 0;
- struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(fp)->i_private;
- struct psp_context *psp = &adev->psp;
- struct ta_context context = {0};
+ struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(fp)->i_private;
+ struct psp_context *psp = &adev->psp;
+ struct ta_context *context = NULL;
if (!buf)
return -EINVAL;
+ ret = copy_from_user((void *)&ta_type, &buf[copy_pos], sizeof(uint32_t));
+ if (ret)
+ return -EFAULT;
+ copy_pos += sizeof(uint32_t);
+
ret = copy_from_user((void *)&ta_id, &buf[copy_pos], sizeof(uint32_t));
if (ret)
- return -EINVAL;
+ return -EFAULT;
copy_pos += sizeof(uint32_t);
ret = copy_from_user((void *)&cmd_id, &buf[copy_pos], sizeof(uint32_t));
if (ret)
- return -EINVAL;
+ return -EFAULT;
copy_pos += sizeof(uint32_t);
ret = copy_from_user((void *)&shared_buf_len, &buf[copy_pos], sizeof(uint32_t));
if (ret)
- return -EINVAL;
+ return -EFAULT;
copy_pos += sizeof(uint32_t);
shared_buf = kzalloc(shared_buf_len, GFP_KERNEL);
@@ -254,25 +332,38 @@ static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size
goto err_free_shared_buf;
}
- context.session_id = ta_id;
+ set_ta_context_funcs(psp, ta_type, &context);
- prep_ta_mem_context(psp, &context, shared_buf, shared_buf_len);
-
- ret = psp_ta_invoke_indirect(psp, cmd_id, &context);
-
- if (ret || context.resp_status) {
- dev_err(adev->dev, "TA invoke via debugfs failed (%d) status %d\n",
- ret, context.resp_status);
- if (!ret)
- ret = -EINVAL;
- goto err_free_ta_shared_buf;
+ if (!context->initialized) {
+ dev_err(adev->dev, "TA is not initialized\n");
+ ret = -EINVAL;
+ goto err_free_shared_buf;
}
- if (copy_to_user((char *)buf, context.mem_context.shared_buf, shared_buf_len))
- ret = -EFAULT;
+ if (!psp->ta_funcs || !psp->ta_funcs->fn_ta_invoke) {
+ dev_err(adev->dev, "Unsupported function to invoke TA\n");
+ ret = -EOPNOTSUPP;
+ goto err_free_shared_buf;
+ }
-err_free_ta_shared_buf:
- psp_ta_free_shared_buf(&context.mem_context);
+ context->session_id = ta_id;
+
+ ret = prep_ta_mem_context(&context->mem_context, shared_buf, shared_buf_len);
+ if (ret)
+ goto err_free_shared_buf;
+
+ ret = psp_fn_ta_invoke(psp, cmd_id);
+ if (ret || context->resp_status) {
+ dev_err(adev->dev, "Failed to invoke TA via debugfs (%d) and status (0x%X)\n",
+ ret, context->resp_status);
+ if (!ret) {
+ ret = -EINVAL;
+ goto err_free_shared_buf;
+ }
+ }
+
+ if (copy_to_user((char *)buf, context->mem_context.shared_buf, shared_buf_len))
+ ret = -EFAULT;
err_free_shared_buf:
kfree(shared_buf);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.h
index cfc1542..14cd1c8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.h
@@ -24,6 +24,11 @@
#ifndef __AMDGPU_PSP_TA_H__
#define __AMDGPU_PSP_TA_H__
+/* Calling set_ta_context_funcs is required before using the following macros */
+#define psp_fn_ta_initialize(psp) ((psp)->ta_funcs->fn_ta_initialize((psp)))
+#define psp_fn_ta_invoke(psp, ta_cmd_id) ((psp)->ta_funcs->fn_ta_invoke((psp), (ta_cmd_id)))
+#define psp_fn_ta_terminate(psp) ((psp)->ta_funcs->fn_ta_terminate((psp)))
+
void amdgpu_ta_if_debugfs_init(struct amdgpu_device *adev);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index a4b47e1..ad490c1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -1267,7 +1267,7 @@ static ssize_t amdgpu_ras_sysfs_features_read(struct device *dev,
struct amdgpu_ras *con =
container_of(attr, struct amdgpu_ras, features_attr);
- return scnprintf(buf, PAGE_SIZE, "feature mask: 0x%x\n", con->features);
+ return sysfs_emit(buf, "feature mask: 0x%x\n", con->features);
}
static void amdgpu_ras_sysfs_remove_bad_page_node(struct amdgpu_device *adev)
@@ -1561,7 +1561,6 @@ static void amdgpu_ras_interrupt_poison_consumption_handler(struct ras_manager *
{
bool poison_stat = false;
struct amdgpu_device *adev = obj->adev;
- struct ras_err_data err_data = {0, 0, 0, NULL};
struct amdgpu_ras_block_object *block_obj =
amdgpu_ras_get_ras_block(adev, obj->head.block, 0);
@@ -1584,7 +1583,7 @@ static void amdgpu_ras_interrupt_poison_consumption_handler(struct ras_manager *
}
if (!adev->gmc.xgmi.connected_to_cpu)
- amdgpu_umc_poison_handler(adev, &err_data, false);
+ amdgpu_umc_poison_handler(adev, false);
if (block_obj->hw_ops->handle_poison_consumption)
poison_stat = block_obj->hw_ops->handle_poison_consumption(adev);
@@ -1949,7 +1948,12 @@ static void amdgpu_ras_do_recovery(struct work_struct *work)
reset_context.method = AMD_RESET_METHOD_NONE;
reset_context.reset_req_dev = adev;
- clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+
+ /* Perform full reset in fatal error mode */
+ if (!amdgpu_ras_is_poison_mode_supported(ras->adev))
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+ else
+ clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
amdgpu_device_gpu_recover(ras->adev, NULL, &reset_context);
}
@@ -2344,7 +2348,8 @@ static void amdgpu_ras_check_supported(struct amdgpu_device *adev)
adev->ras_hw_enabled |= ~(1 << AMDGPU_RAS_BLOCK__UMC |
1 << AMDGPU_RAS_BLOCK__DF);
- if (adev->ip_versions[VCN_HWIP][0] == IP_VERSION(2, 6, 0))
+ if (adev->ip_versions[VCN_HWIP][0] == IP_VERSION(2, 6, 0) ||
+ adev->ip_versions[VCN_HWIP][0] == IP_VERSION(4, 0, 0))
adev->ras_hw_enabled |= (1 << AMDGPU_RAS_BLOCK__VCN |
1 << AMDGPU_RAS_BLOCK__JPEG);
else
@@ -2848,7 +2853,6 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb,
struct amdgpu_device *adev = NULL;
uint32_t gpu_id = 0;
uint32_t umc_inst = 0, ch_inst = 0;
- struct ras_err_data err_data = {0, 0, 0, NULL};
/*
* If the error was generated in UMC_V2, which belongs to GPU UMCs,
@@ -2887,31 +2891,10 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb,
dev_info(adev->dev, "Uncorrectable error detected in UMC inst: %d, chan_idx: %d",
umc_inst, ch_inst);
- err_data.err_addr =
- kcalloc(adev->umc.max_ras_err_cnt_per_query,
- sizeof(struct eeprom_table_record), GFP_KERNEL);
- if (!err_data.err_addr) {
- dev_warn(adev->dev,
- "Failed to alloc memory for umc error record in mca notifier!\n");
+ if (!amdgpu_umc_page_retirement_mca(adev, m->addr, ch_inst, umc_inst))
+ return NOTIFY_OK;
+ else
return NOTIFY_DONE;
- }
-
- /*
- * Translate UMC channel address to Physical address
- */
- if (adev->umc.ras &&
- adev->umc.ras->convert_ras_error_address)
- adev->umc.ras->convert_ras_error_address(adev,
- &err_data, m->addr, ch_inst, umc_inst);
-
- if (amdgpu_bad_page_threshold != 0) {
- amdgpu_ras_add_bad_pages(adev, err_data.err_addr,
- err_data.err_addr_cnt);
- amdgpu_ras_save_bad_pages(adev);
- }
-
- kfree(err_data.err_addr);
- return NOTIFY_OK;
}
static struct notifier_block amdgpu_bad_page_nb = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c
index 84c241b..2d9f3f4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c
@@ -33,12 +33,29 @@
#include "amdgpu_reset.h"
-#define EEPROM_I2C_MADDR_VEGA20 0x0
-#define EEPROM_I2C_MADDR_ARCTURUS 0x40000
-#define EEPROM_I2C_MADDR_ARCTURUS_D342 0x0
-#define EEPROM_I2C_MADDR_SIENNA_CICHLID 0x0
-#define EEPROM_I2C_MADDR_ALDEBARAN 0x0
-#define EEPROM_I2C_MADDR_SMU_13_0_0 (0x54UL << 16)
+/* These are memory addresses as would be seen by one or more EEPROM
+ * chips strung on the I2C bus, usually by manipulating pins 1-3 of a
+ * set of EEPROM devices. They form a continuous memory space.
+ *
+ * The I2C device address includes the device type identifier, 1010b,
+ * which is a reserved value and indicates that this is an I2C EEPROM
+ * device. It also includes the top 3 bits of the 19 bit EEPROM memory
+ * address, namely bits 18, 17, and 16. This makes up the 7 bit
+ * address sent on the I2C bus with bit 0 being the direction bit,
+ * which is not represented here, and sent by the hardware directly.
+ *
+ * For instance,
+ * 50h = 1010000b => device type identifier 1010b, bits 18:16 = 000b, address 0.
+ * 54h = 1010100b => --"--, bits 18:16 = 100b, address 40000h.
+ * 56h = 1010110b => --"--, bits 18:16 = 110b, address 60000h.
+ * Depending on the size of the I2C EEPROM device(s), bits 18:16 may
+ * address memory in a device or a device on the I2C bus, depending on
+ * the status of pins 1-3. See top of amdgpu_eeprom.c.
+ *
+ * The RAS table lives either at address 0 or address 40000h of EEPROM.
+ */
+#define EEPROM_I2C_MADDR_0 0x0
+#define EEPROM_I2C_MADDR_4 0x40000
/*
* The 2 macros bellow represent the actual size in bytes that
@@ -90,6 +107,16 @@
static bool __is_ras_eeprom_supported(struct amdgpu_device *adev)
{
+ if (adev->asic_type == CHIP_IP_DISCOVERY) {
+ switch (adev->ip_versions[MP1_HWIP][0]) {
+ case IP_VERSION(13, 0, 0):
+ case IP_VERSION(13, 0, 10):
+ return true;
+ default:
+ return false;
+ }
+ }
+
return adev->asic_type == CHIP_VEGA20 ||
adev->asic_type == CHIP_ARCTURUS ||
adev->asic_type == CHIP_SIENNA_CICHLID ||
@@ -107,16 +134,30 @@ static bool __get_eeprom_i2c_addr_arct(struct amdgpu_device *adev,
if (strnstr(atom_ctx->vbios_version,
"D342",
sizeof(atom_ctx->vbios_version)))
- control->i2c_address = EEPROM_I2C_MADDR_ARCTURUS_D342;
+ control->i2c_address = EEPROM_I2C_MADDR_0;
else
- control->i2c_address = EEPROM_I2C_MADDR_ARCTURUS;
+ control->i2c_address = EEPROM_I2C_MADDR_4;
return true;
}
+static bool __get_eeprom_i2c_addr_ip_discovery(struct amdgpu_device *adev,
+ struct amdgpu_ras_eeprom_control *control)
+{
+ switch (adev->ip_versions[MP1_HWIP][0]) {
+ case IP_VERSION(13, 0, 0):
+ case IP_VERSION(13, 0, 10):
+ control->i2c_address = EEPROM_I2C_MADDR_4;
+ return true;
+ default:
+ return false;
+ }
+}
+
static bool __get_eeprom_i2c_addr(struct amdgpu_device *adev,
struct amdgpu_ras_eeprom_control *control)
{
+ struct atom_context *atom_ctx = adev->mode_info.atom_context;
u8 i2c_addr;
if (!control)
@@ -139,27 +180,34 @@ static bool __get_eeprom_i2c_addr(struct amdgpu_device *adev,
switch (adev->asic_type) {
case CHIP_VEGA20:
- control->i2c_address = EEPROM_I2C_MADDR_VEGA20;
+ control->i2c_address = EEPROM_I2C_MADDR_0;
break;
case CHIP_ARCTURUS:
return __get_eeprom_i2c_addr_arct(adev, control);
case CHIP_SIENNA_CICHLID:
- control->i2c_address = EEPROM_I2C_MADDR_SIENNA_CICHLID;
+ control->i2c_address = EEPROM_I2C_MADDR_0;
break;
case CHIP_ALDEBARAN:
- control->i2c_address = EEPROM_I2C_MADDR_ALDEBARAN;
+ if (strnstr(atom_ctx->vbios_version, "D673",
+ sizeof(atom_ctx->vbios_version)))
+ control->i2c_address = EEPROM_I2C_MADDR_4;
+ else
+ control->i2c_address = EEPROM_I2C_MADDR_0;
break;
+ case CHIP_IP_DISCOVERY:
+ return __get_eeprom_i2c_addr_ip_discovery(adev, control);
+
default:
return false;
}
switch (adev->ip_versions[MP1_HWIP][0]) {
case IP_VERSION(13, 0, 0):
- control->i2c_address = EEPROM_I2C_MADDR_SMU_13_0_0;
+ control->i2c_address = EEPROM_I2C_MADDR_4;
break;
default:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
index 6546552..5c4f93e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
@@ -62,7 +62,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
if (!res)
goto fallback;
- BUG_ON(start + size > res->num_pages << PAGE_SHIFT);
+ BUG_ON(start + size > res->size);
cur->mem_type = res->mem_type;
@@ -110,7 +110,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
cur->size = size;
cur->remaining = size;
cur->node = NULL;
- WARN_ON(res && start + size > res->num_pages << PAGE_SHIFT);
+ WARN_ON(res && start + size > res->size);
return;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
index d3558c3..dc474b8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
@@ -569,3 +569,15 @@ int amdgpu_ring_init_mqd(struct amdgpu_ring *ring)
return mqd_mgr->init_mqd(adev, ring->mqd_ptr, &prop);
}
+
+void amdgpu_ring_ib_begin(struct amdgpu_ring *ring)
+{
+ if (ring->is_sw_ring)
+ amdgpu_sw_ring_ib_begin(ring);
+}
+
+void amdgpu_ring_ib_end(struct amdgpu_ring *ring)
+{
+ if (ring->is_sw_ring)
+ amdgpu_sw_ring_ib_end(ring);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
index 82c178a..f752c7a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
@@ -39,6 +39,7 @@ struct amdgpu_vm;
#define AMDGPU_MAX_RINGS 28
#define AMDGPU_MAX_HWIP_RINGS 8
#define AMDGPU_MAX_GFX_RINGS 2
+#define AMDGPU_MAX_SW_GFX_RINGS 2
#define AMDGPU_MAX_COMPUTE_RINGS 8
#define AMDGPU_MAX_VCE_RINGS 3
#define AMDGPU_MAX_UVD_ENC_RINGS 2
@@ -59,6 +60,7 @@ enum amdgpu_ring_priority_level {
#define AMDGPU_FENCE_FLAG_64BIT (1 << 0)
#define AMDGPU_FENCE_FLAG_INT (1 << 1)
#define AMDGPU_FENCE_FLAG_TC_WB_ONLY (1 << 2)
+#define AMDGPU_FENCE_FLAG_EXEC (1 << 3)
#define to_amdgpu_ring(s) container_of((s), struct amdgpu_ring, sched)
@@ -143,8 +145,13 @@ signed long amdgpu_fence_wait_polling(struct amdgpu_ring *ring,
uint32_t wait_seq,
signed long timeout);
unsigned amdgpu_fence_count_emitted(struct amdgpu_ring *ring);
+
void amdgpu_fence_driver_isr_toggle(struct amdgpu_device *adev, bool stop);
+u64 amdgpu_fence_last_unsignaled_time_us(struct amdgpu_ring *ring);
+void amdgpu_fence_update_start_timestamp(struct amdgpu_ring *ring, uint32_t seq,
+ ktime_t timestamp);
+
/*
* Rings.
*/
@@ -279,6 +286,10 @@ struct amdgpu_ring {
bool is_mes_queue;
uint32_t hw_queue_id;
struct amdgpu_mes_ctx_data *mes_ctx;
+
+ bool is_sw_ring;
+ unsigned int entry_index;
+
};
#define amdgpu_ring_parse_cs(r, p, job, ib) ((r)->funcs->parse_cs((p), (job), (ib)))
@@ -307,6 +318,9 @@ struct amdgpu_ring {
#define amdgpu_ring_preempt_ib(r) (r)->funcs->preempt_ib(r)
int amdgpu_ring_alloc(struct amdgpu_ring *ring, unsigned ndw);
+void amdgpu_ring_ib_begin(struct amdgpu_ring *ring);
+void amdgpu_ring_ib_end(struct amdgpu_ring *ring);
+
void amdgpu_ring_insert_nop(struct amdgpu_ring *ring, uint32_t count);
void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib);
void amdgpu_ring_commit(struct amdgpu_ring *ring);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.c
new file mode 100644
index 0000000..62079f0
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.c
@@ -0,0 +1,514 @@
+/*
+ * Copyright 2022 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include <linux/slab.h>
+#include <drm/drm_print.h>
+
+#include "amdgpu_ring_mux.h"
+#include "amdgpu_ring.h"
+#include "amdgpu.h"
+
+#define AMDGPU_MUX_RESUBMIT_JIFFIES_TIMEOUT (HZ / 2)
+#define AMDGPU_MAX_LAST_UNSIGNALED_THRESHOLD_US 10000
+
+static const struct ring_info {
+ unsigned int hw_pio;
+ const char *ring_name;
+} sw_ring_info[] = {
+ { AMDGPU_RING_PRIO_DEFAULT, "gfx_low"},
+ { AMDGPU_RING_PRIO_2, "gfx_high"},
+};
+
+static struct kmem_cache *amdgpu_mux_chunk_slab;
+
+static inline struct amdgpu_mux_entry *amdgpu_ring_mux_sw_entry(struct amdgpu_ring_mux *mux,
+ struct amdgpu_ring *ring)
+{
+ return ring->entry_index < mux->ring_entry_size ?
+ &mux->ring_entry[ring->entry_index] : NULL;
+}
+
+/* copy packages on sw ring range[begin, end) */
+static void amdgpu_ring_mux_copy_pkt_from_sw_ring(struct amdgpu_ring_mux *mux,
+ struct amdgpu_ring *ring,
+ u64 s_start, u64 s_end)
+{
+ u64 start, end;
+ struct amdgpu_ring *real_ring = mux->real_ring;
+
+ start = s_start & ring->buf_mask;
+ end = s_end & ring->buf_mask;
+
+ if (start == end) {
+ DRM_ERROR("no more data copied from sw ring\n");
+ return;
+ }
+ if (start > end) {
+ amdgpu_ring_alloc(real_ring, (ring->ring_size >> 2) + end - start);
+ amdgpu_ring_write_multiple(real_ring, (void *)&ring->ring[start],
+ (ring->ring_size >> 2) - start);
+ amdgpu_ring_write_multiple(real_ring, (void *)&ring->ring[0], end);
+ } else {
+ amdgpu_ring_alloc(real_ring, end - start);
+ amdgpu_ring_write_multiple(real_ring, (void *)&ring->ring[start], end - start);
+ }
+}
+
+static void amdgpu_mux_resubmit_chunks(struct amdgpu_ring_mux *mux)
+{
+ struct amdgpu_mux_entry *e = NULL;
+ struct amdgpu_mux_chunk *chunk;
+ uint32_t seq, last_seq;
+ int i;
+
+ /*find low priority entries:*/
+ if (!mux->s_resubmit)
+ return;
+
+ for (i = 0; i < mux->num_ring_entries; i++) {
+ if (mux->ring_entry[i].ring->hw_prio <= AMDGPU_RING_PRIO_DEFAULT) {
+ e = &mux->ring_entry[i];
+ break;
+ }
+ }
+
+ if (!e) {
+ DRM_ERROR("%s no low priority ring found\n", __func__);
+ return;
+ }
+
+ last_seq = atomic_read(&e->ring->fence_drv.last_seq);
+ seq = mux->seqno_to_resubmit;
+ if (last_seq < seq) {
+ /*resubmit all the fences between (last_seq, seq]*/
+ list_for_each_entry(chunk, &e->list, entry) {
+ if (chunk->sync_seq > last_seq && chunk->sync_seq <= seq) {
+ amdgpu_fence_update_start_timestamp(e->ring,
+ chunk->sync_seq,
+ ktime_get());
+ amdgpu_ring_mux_copy_pkt_from_sw_ring(mux, e->ring,
+ chunk->start,
+ chunk->end);
+ mux->wptr_resubmit = chunk->end;
+ amdgpu_ring_commit(mux->real_ring);
+ }
+ }
+ }
+
+ del_timer(&mux->resubmit_timer);
+ mux->s_resubmit = false;
+}
+
+static void amdgpu_ring_mux_schedule_resubmit(struct amdgpu_ring_mux *mux)
+{
+ mod_timer(&mux->resubmit_timer, jiffies + AMDGPU_MUX_RESUBMIT_JIFFIES_TIMEOUT);
+}
+
+static void amdgpu_mux_resubmit_fallback(struct timer_list *t)
+{
+ struct amdgpu_ring_mux *mux = from_timer(mux, t, resubmit_timer);
+
+ if (!spin_trylock(&mux->lock)) {
+ amdgpu_ring_mux_schedule_resubmit(mux);
+ DRM_ERROR("reschedule resubmit\n");
+ return;
+ }
+ amdgpu_mux_resubmit_chunks(mux);
+ spin_unlock(&mux->lock);
+}
+
+int amdgpu_ring_mux_init(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring,
+ unsigned int entry_size)
+{
+ mux->real_ring = ring;
+ mux->num_ring_entries = 0;
+
+ mux->ring_entry = kcalloc(entry_size, sizeof(struct amdgpu_mux_entry), GFP_KERNEL);
+ if (!mux->ring_entry)
+ return -ENOMEM;
+
+ mux->ring_entry_size = entry_size;
+ mux->s_resubmit = false;
+
+ amdgpu_mux_chunk_slab = kmem_cache_create("amdgpu_mux_chunk",
+ sizeof(struct amdgpu_mux_chunk), 0,
+ SLAB_HWCACHE_ALIGN, NULL);
+ if (!amdgpu_mux_chunk_slab) {
+ DRM_ERROR("create amdgpu_mux_chunk cache failed\n");
+ return -ENOMEM;
+ }
+
+ spin_lock_init(&mux->lock);
+ timer_setup(&mux->resubmit_timer, amdgpu_mux_resubmit_fallback, 0);
+
+ return 0;
+}
+
+void amdgpu_ring_mux_fini(struct amdgpu_ring_mux *mux)
+{
+ struct amdgpu_mux_entry *e;
+ struct amdgpu_mux_chunk *chunk, *chunk2;
+ int i;
+
+ for (i = 0; i < mux->num_ring_entries; i++) {
+ e = &mux->ring_entry[i];
+ list_for_each_entry_safe(chunk, chunk2, &e->list, entry) {
+ list_del(&chunk->entry);
+ kmem_cache_free(amdgpu_mux_chunk_slab, chunk);
+ }
+ }
+ kmem_cache_destroy(amdgpu_mux_chunk_slab);
+ kfree(mux->ring_entry);
+ mux->ring_entry = NULL;
+ mux->num_ring_entries = 0;
+ mux->ring_entry_size = 0;
+}
+
+int amdgpu_ring_mux_add_sw_ring(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring)
+{
+ struct amdgpu_mux_entry *e;
+
+ if (mux->num_ring_entries >= mux->ring_entry_size) {
+ DRM_ERROR("add sw ring exceeding max entry size\n");
+ return -ENOENT;
+ }
+
+ e = &mux->ring_entry[mux->num_ring_entries];
+ ring->entry_index = mux->num_ring_entries;
+ e->ring = ring;
+
+ INIT_LIST_HEAD(&e->list);
+ mux->num_ring_entries += 1;
+ return 0;
+}
+
+void amdgpu_ring_mux_set_wptr(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring, u64 wptr)
+{
+ struct amdgpu_mux_entry *e;
+
+ spin_lock(&mux->lock);
+
+ if (ring->hw_prio <= AMDGPU_RING_PRIO_DEFAULT)
+ amdgpu_mux_resubmit_chunks(mux);
+
+ e = amdgpu_ring_mux_sw_entry(mux, ring);
+ if (!e) {
+ DRM_ERROR("cannot find entry for sw ring\n");
+ spin_unlock(&mux->lock);
+ return;
+ }
+
+ /* We could skip this set wptr as preemption in process. */
+ if (ring->hw_prio <= AMDGPU_RING_PRIO_DEFAULT && mux->pending_trailing_fence_signaled) {
+ spin_unlock(&mux->lock);
+ return;
+ }
+
+ e->sw_cptr = e->sw_wptr;
+ /* Update cptr if the package already copied in resubmit functions */
+ if (ring->hw_prio <= AMDGPU_RING_PRIO_DEFAULT && e->sw_cptr < mux->wptr_resubmit)
+ e->sw_cptr = mux->wptr_resubmit;
+ e->sw_wptr = wptr;
+ e->start_ptr_in_hw_ring = mux->real_ring->wptr;
+
+ /* Skip copying for the packages already resubmitted.*/
+ if (ring->hw_prio > AMDGPU_RING_PRIO_DEFAULT || mux->wptr_resubmit < wptr) {
+ amdgpu_ring_mux_copy_pkt_from_sw_ring(mux, ring, e->sw_cptr, wptr);
+ e->end_ptr_in_hw_ring = mux->real_ring->wptr;
+ amdgpu_ring_commit(mux->real_ring);
+ } else {
+ e->end_ptr_in_hw_ring = mux->real_ring->wptr;
+ }
+ spin_unlock(&mux->lock);
+}
+
+u64 amdgpu_ring_mux_get_wptr(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring)
+{
+ struct amdgpu_mux_entry *e;
+
+ e = amdgpu_ring_mux_sw_entry(mux, ring);
+ if (!e) {
+ DRM_ERROR("cannot find entry for sw ring\n");
+ return 0;
+ }
+
+ return e->sw_wptr;
+}
+
+/**
+ * amdgpu_ring_mux_get_rptr - get the readptr of the software ring
+ * @mux: the multiplexer the software rings attach to
+ * @ring: the software ring of which we calculate the readptr
+ *
+ * The return value of the readptr is not precise while the other rings could
+ * write data onto the real ring buffer.After overwriting on the real ring, we
+ * can not decide if our packages have been excuted or not read yet. However,
+ * this function is only called by the tools such as umr to collect the latest
+ * packages for the hang analysis. We assume the hang happens near our latest
+ * submit. Thus we could use the following logic to give the clue:
+ * If the readptr is between start and end, then we return the copy pointer
+ * plus the distance from start to readptr. If the readptr is before start, we
+ * return the copy pointer. Lastly, if the readptr is past end, we return the
+ * write pointer.
+ */
+u64 amdgpu_ring_mux_get_rptr(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring)
+{
+ struct amdgpu_mux_entry *e;
+ u64 readp, offset, start, end;
+
+ e = amdgpu_ring_mux_sw_entry(mux, ring);
+ if (!e) {
+ DRM_ERROR("no sw entry found!\n");
+ return 0;
+ }
+
+ readp = amdgpu_ring_get_rptr(mux->real_ring);
+
+ start = e->start_ptr_in_hw_ring & mux->real_ring->buf_mask;
+ end = e->end_ptr_in_hw_ring & mux->real_ring->buf_mask;
+ if (start > end) {
+ if (readp <= end)
+ readp += mux->real_ring->ring_size >> 2;
+ end += mux->real_ring->ring_size >> 2;
+ }
+
+ if (start <= readp && readp <= end) {
+ offset = readp - start;
+ e->sw_rptr = (e->sw_cptr + offset) & ring->buf_mask;
+ } else if (readp < start) {
+ e->sw_rptr = e->sw_cptr;
+ } else {
+ /* end < readptr */
+ e->sw_rptr = e->sw_wptr;
+ }
+
+ return e->sw_rptr;
+}
+
+u64 amdgpu_sw_ring_get_rptr_gfx(struct amdgpu_ring *ring)
+{
+ struct amdgpu_device *adev = ring->adev;
+ struct amdgpu_ring_mux *mux = &adev->gfx.muxer;
+
+ WARN_ON(!ring->is_sw_ring);
+ return amdgpu_ring_mux_get_rptr(mux, ring);
+}
+
+u64 amdgpu_sw_ring_get_wptr_gfx(struct amdgpu_ring *ring)
+{
+ struct amdgpu_device *adev = ring->adev;
+ struct amdgpu_ring_mux *mux = &adev->gfx.muxer;
+
+ WARN_ON(!ring->is_sw_ring);
+ return amdgpu_ring_mux_get_wptr(mux, ring);
+}
+
+void amdgpu_sw_ring_set_wptr_gfx(struct amdgpu_ring *ring)
+{
+ struct amdgpu_device *adev = ring->adev;
+ struct amdgpu_ring_mux *mux = &adev->gfx.muxer;
+
+ WARN_ON(!ring->is_sw_ring);
+ amdgpu_ring_mux_set_wptr(mux, ring, ring->wptr);
+}
+
+/* Override insert_nop to prevent emitting nops to the software rings */
+void amdgpu_sw_ring_insert_nop(struct amdgpu_ring *ring, uint32_t count)
+{
+ WARN_ON(!ring->is_sw_ring);
+}
+
+const char *amdgpu_sw_ring_name(int idx)
+{
+ return idx < ARRAY_SIZE(sw_ring_info) ?
+ sw_ring_info[idx].ring_name : NULL;
+}
+
+unsigned int amdgpu_sw_ring_priority(int idx)
+{
+ return idx < ARRAY_SIZE(sw_ring_info) ?
+ sw_ring_info[idx].hw_pio : AMDGPU_RING_PRIO_DEFAULT;
+}
+
+/*Scan on low prio rings to have unsignaled fence and high ring has no fence.*/
+static int amdgpu_mcbp_scan(struct amdgpu_ring_mux *mux)
+{
+ struct amdgpu_ring *ring;
+ int i, need_preempt;
+
+ need_preempt = 0;
+ for (i = 0; i < mux->num_ring_entries; i++) {
+ ring = mux->ring_entry[i].ring;
+ if (ring->hw_prio > AMDGPU_RING_PRIO_DEFAULT &&
+ amdgpu_fence_count_emitted(ring) > 0)
+ return 0;
+ if (ring->hw_prio <= AMDGPU_RING_PRIO_DEFAULT &&
+ amdgpu_fence_last_unsignaled_time_us(ring) >
+ AMDGPU_MAX_LAST_UNSIGNALED_THRESHOLD_US)
+ need_preempt = 1;
+ }
+ return need_preempt && !mux->s_resubmit;
+}
+
+/* Trigger Mid-Command Buffer Preemption (MCBP) and find if we need to resubmit. */
+static int amdgpu_mcbp_trigger_preempt(struct amdgpu_ring_mux *mux)
+{
+ int r;
+
+ spin_lock(&mux->lock);
+ mux->pending_trailing_fence_signaled = true;
+ r = amdgpu_ring_preempt_ib(mux->real_ring);
+ spin_unlock(&mux->lock);
+ return r;
+}
+
+void amdgpu_sw_ring_ib_begin(struct amdgpu_ring *ring)
+{
+ struct amdgpu_device *adev = ring->adev;
+ struct amdgpu_ring_mux *mux = &adev->gfx.muxer;
+
+ WARN_ON(!ring->is_sw_ring);
+ if (ring->hw_prio > AMDGPU_RING_PRIO_DEFAULT) {
+ if (amdgpu_mcbp_scan(mux) > 0)
+ amdgpu_mcbp_trigger_preempt(mux);
+ return;
+ }
+
+ amdgpu_ring_mux_start_ib(mux, ring);
+}
+
+void amdgpu_sw_ring_ib_end(struct amdgpu_ring *ring)
+{
+ struct amdgpu_device *adev = ring->adev;
+ struct amdgpu_ring_mux *mux = &adev->gfx.muxer;
+
+ WARN_ON(!ring->is_sw_ring);
+ if (ring->hw_prio > AMDGPU_RING_PRIO_DEFAULT)
+ return;
+ amdgpu_ring_mux_end_ib(mux, ring);
+}
+
+void amdgpu_ring_mux_start_ib(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring)
+{
+ struct amdgpu_mux_entry *e;
+ struct amdgpu_mux_chunk *chunk;
+
+ spin_lock(&mux->lock);
+ amdgpu_mux_resubmit_chunks(mux);
+ spin_unlock(&mux->lock);
+
+ e = amdgpu_ring_mux_sw_entry(mux, ring);
+ if (!e) {
+ DRM_ERROR("cannot find entry!\n");
+ return;
+ }
+
+ chunk = kmem_cache_alloc(amdgpu_mux_chunk_slab, GFP_KERNEL);
+ if (!chunk) {
+ DRM_ERROR("alloc amdgpu_mux_chunk_slab failed\n");
+ return;
+ }
+
+ chunk->start = ring->wptr;
+ list_add_tail(&chunk->entry, &e->list);
+}
+
+static void scan_and_remove_signaled_chunk(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring)
+{
+ uint32_t last_seq = 0;
+ struct amdgpu_mux_entry *e;
+ struct amdgpu_mux_chunk *chunk, *tmp;
+
+ e = amdgpu_ring_mux_sw_entry(mux, ring);
+ if (!e) {
+ DRM_ERROR("cannot find entry!\n");
+ return;
+ }
+
+ last_seq = atomic_read(&ring->fence_drv.last_seq);
+
+ list_for_each_entry_safe(chunk, tmp, &e->list, entry) {
+ if (chunk->sync_seq <= last_seq) {
+ list_del(&chunk->entry);
+ kmem_cache_free(amdgpu_mux_chunk_slab, chunk);
+ }
+ }
+}
+
+void amdgpu_ring_mux_end_ib(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring)
+{
+ struct amdgpu_mux_entry *e;
+ struct amdgpu_mux_chunk *chunk;
+
+ e = amdgpu_ring_mux_sw_entry(mux, ring);
+ if (!e) {
+ DRM_ERROR("cannot find entry!\n");
+ return;
+ }
+
+ chunk = list_last_entry(&e->list, struct amdgpu_mux_chunk, entry);
+ if (!chunk) {
+ DRM_ERROR("cannot find chunk!\n");
+ return;
+ }
+
+ chunk->end = ring->wptr;
+ chunk->sync_seq = READ_ONCE(ring->fence_drv.sync_seq);
+
+ scan_and_remove_signaled_chunk(mux, ring);
+}
+
+bool amdgpu_mcbp_handle_trailing_fence_irq(struct amdgpu_ring_mux *mux)
+{
+ struct amdgpu_mux_entry *e;
+ struct amdgpu_ring *ring = NULL;
+ int i;
+
+ if (!mux->pending_trailing_fence_signaled)
+ return false;
+
+ if (mux->real_ring->trail_seq != le32_to_cpu(*mux->real_ring->trail_fence_cpu_addr))
+ return false;
+
+ for (i = 0; i < mux->num_ring_entries; i++) {
+ e = &mux->ring_entry[i];
+ if (e->ring->hw_prio <= AMDGPU_RING_PRIO_DEFAULT) {
+ ring = e->ring;
+ break;
+ }
+ }
+
+ if (!ring) {
+ DRM_ERROR("cannot find low priority ring\n");
+ return false;
+ }
+
+ amdgpu_fence_process(ring);
+ if (amdgpu_fence_count_emitted(ring) > 0) {
+ mux->s_resubmit = true;
+ mux->seqno_to_resubmit = ring->fence_drv.sync_seq;
+ amdgpu_ring_mux_schedule_resubmit(mux);
+ }
+
+ mux->pending_trailing_fence_signaled = false;
+ return true;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.h
new file mode 100644
index 0000000..4be45fc
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.h
@@ -0,0 +1,103 @@
+/*
+ * Copyright 2022 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_RING_MUX__
+#define __AMDGPU_RING_MUX__
+
+#include <linux/timer.h>
+#include <linux/spinlock.h>
+#include "amdgpu_ring.h"
+
+struct amdgpu_ring;
+
+/**
+ * struct amdgpu_mux_entry - the entry recording software rings copying information.
+ * @ring: the pointer to the software ring.
+ * @start_ptr_in_hw_ring: last start location copied to in the hardware ring.
+ * @end_ptr_in_hw_ring: last end location copied to in the hardware ring.
+ * @sw_cptr: the position of the copy pointer in the sw ring.
+ * @sw_rptr: the read pointer in software ring.
+ * @sw_wptr: the write pointer in software ring.
+ * @list: list head for amdgpu_mux_chunk
+ */
+struct amdgpu_mux_entry {
+ struct amdgpu_ring *ring;
+ u64 start_ptr_in_hw_ring;
+ u64 end_ptr_in_hw_ring;
+ u64 sw_cptr;
+ u64 sw_rptr;
+ u64 sw_wptr;
+ struct list_head list;
+};
+
+struct amdgpu_ring_mux {
+ struct amdgpu_ring *real_ring;
+
+ struct amdgpu_mux_entry *ring_entry;
+ unsigned int num_ring_entries;
+ unsigned int ring_entry_size;
+ /*the lock for copy data from different software rings*/
+ spinlock_t lock;
+ bool s_resubmit;
+ uint32_t seqno_to_resubmit;
+ u64 wptr_resubmit;
+ struct timer_list resubmit_timer;
+
+ bool pending_trailing_fence_signaled;
+};
+
+/**
+ * struct amdgpu_mux_chunk - save the location of indirect buffer's package on softare rings.
+ * @entry: the list entry.
+ * @sync_seq: the fence seqno related with the saved IB.
+ * @start:- start location on the software ring.
+ * @end:- end location on the software ring.
+ */
+struct amdgpu_mux_chunk {
+ struct list_head entry;
+ uint32_t sync_seq;
+ u64 start;
+ u64 end;
+};
+
+int amdgpu_ring_mux_init(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring,
+ unsigned int entry_size);
+void amdgpu_ring_mux_fini(struct amdgpu_ring_mux *mux);
+int amdgpu_ring_mux_add_sw_ring(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring);
+void amdgpu_ring_mux_set_wptr(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring, u64 wptr);
+u64 amdgpu_ring_mux_get_wptr(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring);
+u64 amdgpu_ring_mux_get_rptr(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring);
+void amdgpu_ring_mux_start_ib(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring);
+void amdgpu_ring_mux_end_ib(struct amdgpu_ring_mux *mux, struct amdgpu_ring *ring);
+bool amdgpu_mcbp_handle_trailing_fence_irq(struct amdgpu_ring_mux *mux);
+
+u64 amdgpu_sw_ring_get_rptr_gfx(struct amdgpu_ring *ring);
+u64 amdgpu_sw_ring_get_wptr_gfx(struct amdgpu_ring *ring);
+void amdgpu_sw_ring_set_wptr_gfx(struct amdgpu_ring *ring);
+void amdgpu_sw_ring_insert_nop(struct amdgpu_ring *ring, uint32_t count);
+void amdgpu_sw_ring_ib_begin(struct amdgpu_ring *ring);
+void amdgpu_sw_ring_ib_end(struct amdgpu_ring *ring);
+const char *amdgpu_sw_ring_name(int idx);
+unsigned int amdgpu_sw_ring_priority(int idx);
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c
index cc7597a..2c1d82f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c
@@ -121,6 +121,7 @@ static ssize_t amdgpu_securedisplay_debugfs_write(struct file *f, const char __u
switch (op) {
case 1:
+ mutex_lock(&psp->securedisplay_context.mutex);
psp_prep_securedisplay_cmd_buf(psp, &securedisplay_cmd,
TA_SECUREDISPLAY_COMMAND__QUERY_TA);
ret = psp_securedisplay_invoke(psp, TA_SECUREDISPLAY_COMMAND__QUERY_TA);
@@ -131,8 +132,10 @@ static ssize_t amdgpu_securedisplay_debugfs_write(struct file *f, const char __u
else
psp_securedisplay_parse_resp_status(psp, securedisplay_cmd->status);
}
+ mutex_unlock(&psp->securedisplay_context.mutex);
break;
case 2:
+ mutex_lock(&psp->securedisplay_context.mutex);
psp_prep_securedisplay_cmd_buf(psp, &securedisplay_cmd,
TA_SECUREDISPLAY_COMMAND__SEND_ROI_CRC);
securedisplay_cmd->securedisplay_in_message.send_roi_crc.phy_id = phy_id;
@@ -146,6 +149,7 @@ static ssize_t amdgpu_securedisplay_debugfs_write(struct file *f, const char __u
psp_securedisplay_parse_resp_status(psp, securedisplay_cmd->status);
}
}
+ mutex_unlock(&psp->securedisplay_context.mutex);
break;
default:
dev_err(adev->dev, "Invalid input: %s\n", str);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index 090e66a..bac7976 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -259,6 +259,14 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct amdgpu_sync *sync,
return 0;
}
+/* Free the entry back to the slab */
+static void amdgpu_sync_entry_free(struct amdgpu_sync_entry *e)
+{
+ hash_del(&e->node);
+ dma_fence_put(e->fence);
+ kmem_cache_free(amdgpu_sync_slab, e);
+}
+
/**
* amdgpu_sync_peek_fence - get the next fence not signaled yet
*
@@ -280,9 +288,7 @@ struct dma_fence *amdgpu_sync_peek_fence(struct amdgpu_sync *sync,
struct drm_sched_fence *s_fence = to_drm_sched_fence(f);
if (dma_fence_is_signaled(f)) {
- hash_del(&e->node);
- dma_fence_put(f);
- kmem_cache_free(amdgpu_sync_slab, e);
+ amdgpu_sync_entry_free(e);
continue;
}
if (ring && s_fence) {
@@ -355,15 +361,42 @@ int amdgpu_sync_clone(struct amdgpu_sync *source, struct amdgpu_sync *clone)
if (r)
return r;
} else {
- hash_del(&e->node);
- dma_fence_put(f);
- kmem_cache_free(amdgpu_sync_slab, e);
+ amdgpu_sync_entry_free(e);
}
}
return 0;
}
+/**
+ * amdgpu_sync_push_to_job - push fences into job
+ * @sync: sync object to get the fences from
+ * @job: job to push the fences into
+ *
+ * Add all unsignaled fences from sync to job.
+ */
+int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job)
+{
+ struct amdgpu_sync_entry *e;
+ struct hlist_node *tmp;
+ struct dma_fence *f;
+ int i, r;
+
+ hash_for_each_safe(sync->fences, i, tmp, e, node) {
+ f = e->fence;
+ if (dma_fence_is_signaled(f)) {
+ amdgpu_sync_entry_free(e);
+ continue;
+ }
+
+ dma_fence_get(f);
+ r = drm_sched_job_add_dependency(&job->base, f);
+ if (r)
+ return r;
+ }
+ return 0;
+}
+
int amdgpu_sync_wait(struct amdgpu_sync *sync, bool intr)
{
struct amdgpu_sync_entry *e;
@@ -375,9 +408,7 @@ int amdgpu_sync_wait(struct amdgpu_sync *sync, bool intr)
if (r)
return r;
- hash_del(&e->node);
- dma_fence_put(e->fence);
- kmem_cache_free(amdgpu_sync_slab, e);
+ amdgpu_sync_entry_free(e);
}
return 0;
@@ -396,11 +427,8 @@ void amdgpu_sync_free(struct amdgpu_sync *sync)
struct hlist_node *tmp;
unsigned int i;
- hash_for_each_safe(sync->fences, i, tmp, e, node) {
- hash_del(&e->node);
- dma_fence_put(e->fence);
- kmem_cache_free(amdgpu_sync_slab, e);
- }
+ hash_for_each_safe(sync->fences, i, tmp, e, node)
+ amdgpu_sync_entry_free(e);
}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.h
index 2d5c613..cf1e9e8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.h
@@ -30,6 +30,7 @@ struct dma_fence;
struct dma_resv;
struct amdgpu_device;
struct amdgpu_ring;
+struct amdgpu_job;
enum amdgpu_sync_mode {
AMDGPU_SYNC_ALWAYS,
@@ -54,6 +55,7 @@ struct dma_fence *amdgpu_sync_peek_fence(struct amdgpu_sync *sync,
struct amdgpu_ring *ring);
struct dma_fence *amdgpu_sync_get_fence(struct amdgpu_sync *sync);
int amdgpu_sync_clone(struct amdgpu_sync *source, struct amdgpu_sync *clone);
+int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job);
int amdgpu_sync_wait(struct amdgpu_sync *sync, bool intr);
void amdgpu_sync_free(struct amdgpu_sync *sync);
int amdgpu_sync_init(void);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h
index 5e6ddc7e..677ad20 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h
@@ -127,7 +127,7 @@ TRACE_EVENT(amdgpu_bo_create,
TP_fast_assign(
__entry->bo = bo;
- __entry->pages = bo->tbo.resource->num_pages;
+ __entry->pages = PFN_UP(bo->tbo.resource->size);
__entry->type = bo->tbo.resource->mem_type;
__entry->prefer = bo->preferred_domains;
__entry->allow = bo->allowed_domains;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index b64938e..b423657 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -58,6 +58,7 @@
#include "amdgpu_amdkfd.h"
#include "amdgpu_sdma.h"
#include "amdgpu_ras.h"
+#include "amdgpu_hmm.h"
#include "amdgpu_atomfirmware.h"
#include "amdgpu_res_cursor.h"
#include "bif/bif_4_1_d.h"
@@ -189,7 +190,6 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,
struct amdgpu_device *adev = ring->adev;
unsigned offset, num_pages, num_dw, num_bytes;
uint64_t src_addr, dst_addr;
- struct dma_fence *fence;
struct amdgpu_job *job;
void *cpu_addr;
uint64_t flags;
@@ -229,7 +229,9 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,
num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
num_bytes = num_pages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE;
- r = amdgpu_job_alloc_with_ib(adev, num_dw * 4 + num_bytes,
+ r = amdgpu_job_alloc_with_ib(adev, &adev->mman.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ num_dw * 4 + num_bytes,
AMDGPU_IB_POOL_DELAYED, &job);
if (r)
return r;
@@ -269,18 +271,8 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,
}
}
- r = amdgpu_job_submit(job, &adev->mman.entity,
- AMDGPU_FENCE_OWNER_UNDEFINED, &fence);
- if (r)
- goto error_free;
-
- dma_fence_put(fence);
-
- return r;
-
-error_free:
- amdgpu_job_free(job);
- return r;
+ dma_fence_put(amdgpu_job_submit(job));
+ return 0;
}
/**
@@ -381,7 +373,7 @@ static int amdgpu_move_blit(struct ttm_buffer_object *bo,
dst.offset = 0;
r = amdgpu_ttm_copy_mem_to_mem(adev, &src, &dst,
- new_mem->num_pages << PAGE_SHIFT,
+ new_mem->size,
amdgpu_bo_encrypted(abo),
bo->base.resv, &fence);
if (r)
@@ -424,7 +416,7 @@ static int amdgpu_move_blit(struct ttm_buffer_object *bo,
static bool amdgpu_mem_visible(struct amdgpu_device *adev,
struct ttm_resource *mem)
{
- u64 mem_size = (u64)mem->num_pages << PAGE_SHIFT;
+ u64 mem_size = (u64)mem->size;
struct amdgpu_res_cursor cursor;
u64 end;
@@ -571,7 +563,7 @@ static int amdgpu_ttm_io_mem_reserve(struct ttm_device *bdev,
struct ttm_resource *mem)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
- size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+ size_t bus_size = (size_t)mem->size;
switch (mem->mem_type) {
case TTM_PL_SYSTEM:
@@ -691,9 +683,8 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages,
}
readonly = amdgpu_ttm_tt_is_readonly(ttm);
- r = amdgpu_hmm_range_get_pages(&bo->notifier, mm, pages, start,
- ttm->num_pages, range, readonly,
- true, NULL);
+ r = amdgpu_hmm_range_get_pages(&bo->notifier, start, ttm->num_pages,
+ readonly, NULL, pages, range);
out_unlock:
mmap_read_unlock(mm);
if (r)
@@ -1154,8 +1145,9 @@ int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo,
* @addr: The address in the current tasks VM space to use
* @flags: Requirements of userptr object.
*
- * Called by amdgpu_gem_userptr_ioctl() to bind userptr pages
- * to current task
+ * Called by amdgpu_gem_userptr_ioctl() and kfd_ioctl_alloc_memory_of_gpu() to
+ * bind userptr pages to current task and by kfd_ioctl_acquire_vm() to
+ * initialize GPU VM for a KFD process.
*/
int amdgpu_ttm_tt_set_userptr(struct ttm_buffer_object *bo,
uint64_t addr, uint32_t flags)
@@ -1394,7 +1386,8 @@ static void amdgpu_ttm_vram_mm_access(struct amdgpu_device *adev, loff_t pos,
}
static int amdgpu_ttm_access_memory_sdma(struct ttm_buffer_object *bo,
- unsigned long offset, void *buf, int len, int write)
+ unsigned long offset, void *buf,
+ int len, int write)
{
struct amdgpu_bo *abo = ttm_to_amdgpu_bo(bo);
struct amdgpu_device *adev = amdgpu_ttm_adev(abo->tbo.bdev);
@@ -1418,26 +1411,27 @@ static int amdgpu_ttm_access_memory_sdma(struct ttm_buffer_object *bo,
memcpy(adev->mman.sdma_access_ptr, buf, len);
num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
- r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, AMDGPU_IB_POOL_DELAYED, &job);
+ r = amdgpu_job_alloc_with_ib(adev, &adev->mman.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ num_dw * 4, AMDGPU_IB_POOL_DELAYED,
+ &job);
if (r)
goto out;
amdgpu_res_first(abo->tbo.resource, offset, len, &src_mm);
- src_addr = amdgpu_ttm_domain_start(adev, bo->resource->mem_type) + src_mm.start;
+ src_addr = amdgpu_ttm_domain_start(adev, bo->resource->mem_type) +
+ src_mm.start;
dst_addr = amdgpu_bo_gpu_offset(adev->mman.sdma_access_bo);
if (write)
swap(src_addr, dst_addr);
- amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr, dst_addr, PAGE_SIZE, false);
+ amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr, dst_addr,
+ PAGE_SIZE, false);
amdgpu_ring_pad_ib(adev->mman.buffer_funcs_ring, &job->ibs[0]);
WARN_ON(job->ibs[0].length_dw > num_dw);
- r = amdgpu_job_submit(job, &adev->mman.entity, AMDGPU_FENCE_OWNER_UNDEFINED, &fence);
- if (r) {
- amdgpu_job_free(job);
- goto out;
- }
+ fence = amdgpu_job_submit(job);
if (!dma_fence_wait_timeout(fence, false, adev->sdma_timeout))
r = -ETIMEDOUT;
@@ -1537,6 +1531,23 @@ static void amdgpu_ttm_fw_reserve_vram_fini(struct amdgpu_device *adev)
NULL, &adev->mman.fw_vram_usage_va);
}
+/*
+ * Driver Reservation functions
+ */
+/**
+ * amdgpu_ttm_drv_reserve_vram_fini - free drv reserved vram
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * free drv reserved vram if it has been reserved.
+ */
+static void amdgpu_ttm_drv_reserve_vram_fini(struct amdgpu_device *adev)
+{
+ amdgpu_bo_free_kernel(&adev->mman.drv_vram_usage_reserved_bo,
+ NULL,
+ &adev->mman.drv_vram_usage_va);
+}
+
/**
* amdgpu_ttm_fw_reserve_vram_init - create bo vram reservation from fw
*
@@ -1563,6 +1574,32 @@ static int amdgpu_ttm_fw_reserve_vram_init(struct amdgpu_device *adev)
&adev->mman.fw_vram_usage_va);
}
+/**
+ * amdgpu_ttm_drv_reserve_vram_init - create bo vram reservation from driver
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * create bo vram reservation from drv.
+ */
+static int amdgpu_ttm_drv_reserve_vram_init(struct amdgpu_device *adev)
+{
+ u64 vram_size = adev->gmc.visible_vram_size;
+
+ adev->mman.drv_vram_usage_va = NULL;
+ adev->mman.drv_vram_usage_reserved_bo = NULL;
+
+ if (adev->mman.drv_vram_usage_size == 0 ||
+ adev->mman.drv_vram_usage_size > vram_size)
+ return 0;
+
+ return amdgpu_bo_create_kernel_at(adev,
+ adev->mman.drv_vram_usage_start_offset,
+ adev->mman.drv_vram_usage_size,
+ AMDGPU_GEM_DOMAIN_VRAM,
+ &adev->mman.drv_vram_usage_reserved_bo,
+ &adev->mman.drv_vram_usage_va);
+}
+
/*
* Memoy training reservation functions
*/
@@ -1731,6 +1768,14 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
}
/*
+ *The reserved vram for driver must be pinned to the specified
+ *place on the VRAM, so reserve it early.
+ */
+ r = amdgpu_ttm_drv_reserve_vram_init(adev);
+ if (r)
+ return r;
+
+ /*
* only NAVI10 and onwards ASIC support for IP discovery.
* If IP discovery enabled, a block of memory should be
* reserved for IP discovey.
@@ -1855,6 +1900,7 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev)
amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL,
&adev->mman.sdma_access_ptr);
amdgpu_ttm_fw_reserve_vram_fini(adev);
+ amdgpu_ttm_drv_reserve_vram_fini(adev);
if (drm_dev_enter(adev_to_drm(adev), &idx)) {
@@ -1936,7 +1982,9 @@ static int amdgpu_ttm_prepare_job(struct amdgpu_device *adev,
AMDGPU_IB_POOL_DELAYED;
int r;
- r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, pool, job);
+ r = amdgpu_job_alloc_with_ib(adev, &adev->mman.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ num_dw * 4, pool, job);
if (r)
return r;
@@ -1946,17 +1994,11 @@ static int amdgpu_ttm_prepare_job(struct amdgpu_device *adev,
adev->gart.bo);
(*job)->vm_needs_flush = true;
}
- if (resv) {
- r = amdgpu_sync_resv(adev, &(*job)->sync, resv,
- AMDGPU_SYNC_ALWAYS,
- AMDGPU_FENCE_OWNER_UNDEFINED);
- if (r) {
- DRM_ERROR("sync failed (%d).\n", r);
- amdgpu_job_free(*job);
- return r;
- }
- }
- return 0;
+ if (!resv)
+ return 0;
+
+ return drm_sched_job_add_resv_dependencies(&(*job)->base, resv,
+ DMA_RESV_USAGE_BOOKKEEP);
}
int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset,
@@ -2001,8 +2043,7 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset,
if (direct_submit)
r = amdgpu_job_submit_direct(job, ring, fence);
else
- r = amdgpu_job_submit(job, &adev->mman.entity,
- AMDGPU_FENCE_OWNER_UNDEFINED, fence);
+ *fence = amdgpu_job_submit(job);
if (r)
goto error_free;
@@ -2047,16 +2088,8 @@ static int amdgpu_ttm_fill_mem(struct amdgpu_ring *ring, uint32_t src_data,
amdgpu_ring_pad_ib(ring, &job->ibs[0]);
WARN_ON(job->ibs[0].length_dw > num_dw);
- r = amdgpu_job_submit(job, &adev->mman.entity,
- AMDGPU_FENCE_OWNER_UNDEFINED, fence);
- if (r)
- goto error_free;
-
+ *fence = amdgpu_job_submit(job);
return 0;
-
-error_free:
- amdgpu_job_free(job);
- return r;
}
int amdgpu_fill_buffer(struct amdgpu_bo *bo,
@@ -2272,9 +2305,9 @@ static ssize_t amdgpu_iomem_read(struct file *f, char __user *buf,
if (p->mapping != adev->mman.bdev.dev_mapping)
return -EPERM;
- ptr = kmap(p);
+ ptr = kmap_local_page(p);
r = copy_to_user(buf, ptr + off, bytes);
- kunmap(p);
+ kunmap_local(ptr);
if (r)
return -EFAULT;
@@ -2323,9 +2356,9 @@ static ssize_t amdgpu_iomem_write(struct file *f, const char __user *buf,
if (p->mapping != adev->mman.bdev.dev_mapping)
return -EPERM;
- ptr = kmap(p);
+ ptr = kmap_local_page(p);
r = copy_from_user(ptr + off, buf, bytes);
- kunmap(p);
+ kunmap_local(ptr);
if (r)
return -EFAULT;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
index a372070..b4d8ba2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
@@ -86,6 +86,12 @@ struct amdgpu_mman {
struct amdgpu_bo *fw_vram_usage_reserved_bo;
void *fw_vram_usage_va;
+ /* driver VRAM reservation */
+ u64 drv_vram_usage_start_offset;
+ u64 drv_vram_usage_size;
+ struct amdgpu_bo *drv_vram_usage_reserved_bo;
+ void *drv_vram_usage_va;
+
/* PAGE_SIZE'd BO for process memory r/w over SDMA. */
struct amdgpu_bo *sdma_access_bo;
void *sdma_access_ptr;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h
index 1c36235..552e069 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h
@@ -125,6 +125,7 @@ enum psp_fw_type {
PSP_FW_TYPE_PSP_INTF_DRV,
PSP_FW_TYPE_PSP_DBG_DRV,
PSP_FW_TYPE_PSP_RAS_DRV,
+ PSP_FW_TYPE_MAX_INDEX,
};
/* version_major=2, version_minor=0 */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
index aad3c8b..f76c19f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
@@ -22,6 +22,59 @@
*/
#include "amdgpu.h"
+#include "umc_v6_7.h"
+
+static int amdgpu_umc_convert_error_address(struct amdgpu_device *adev,
+ struct ras_err_data *err_data, uint64_t err_addr,
+ uint32_t ch_inst, uint32_t umc_inst)
+{
+ switch (adev->ip_versions[UMC_HWIP][0]) {
+ case IP_VERSION(6, 7, 0):
+ umc_v6_7_convert_error_address(adev,
+ err_data, err_addr, ch_inst, umc_inst);
+ break;
+ default:
+ dev_warn(adev->dev,
+ "UMC address to Physical address translation is not supported\n");
+ return AMDGPU_RAS_FAIL;
+ }
+
+ return AMDGPU_RAS_SUCCESS;
+}
+
+int amdgpu_umc_page_retirement_mca(struct amdgpu_device *adev,
+ uint64_t err_addr, uint32_t ch_inst, uint32_t umc_inst)
+{
+ struct ras_err_data err_data = {0, 0, 0, NULL};
+ int ret = AMDGPU_RAS_FAIL;
+
+ err_data.err_addr =
+ kcalloc(adev->umc.max_ras_err_cnt_per_query,
+ sizeof(struct eeprom_table_record), GFP_KERNEL);
+ if (!err_data.err_addr) {
+ dev_warn(adev->dev,
+ "Failed to alloc memory for umc error record in MCA notifier!\n");
+ return AMDGPU_RAS_FAIL;
+ }
+
+ /*
+ * Translate UMC channel address to Physical address
+ */
+ ret = amdgpu_umc_convert_error_address(adev, &err_data, err_addr,
+ ch_inst, umc_inst);
+ if (ret)
+ goto out;
+
+ if (amdgpu_bad_page_threshold != 0) {
+ amdgpu_ras_add_bad_pages(adev, err_data.err_addr,
+ err_data.err_addr_cnt);
+ amdgpu_ras_save_bad_pages(adev);
+ }
+
+out:
+ kfree(err_data.err_addr);
+ return ret;
+}
static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev,
void *ras_error_status,
@@ -112,23 +165,29 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev,
return AMDGPU_RAS_SUCCESS;
}
-int amdgpu_umc_poison_handler(struct amdgpu_device *adev,
- void *ras_error_status,
- bool reset)
+int amdgpu_umc_poison_handler(struct amdgpu_device *adev, bool reset)
{
- int ret;
- struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
- struct ras_common_if head = {
- .block = AMDGPU_RAS_BLOCK__UMC,
- };
- struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head);
+ int ret = AMDGPU_RAS_SUCCESS;
- ret =
- amdgpu_umc_do_page_retirement(adev, ras_error_status, NULL, reset);
+ if (!adev->gmc.xgmi.connected_to_cpu) {
+ struct ras_err_data err_data = {0, 0, 0, NULL};
+ struct ras_common_if head = {
+ .block = AMDGPU_RAS_BLOCK__UMC,
+ };
+ struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head);
- if (ret == AMDGPU_RAS_SUCCESS && obj) {
- obj->err_data.ue_count += err_data->ue_count;
- obj->err_data.ce_count += err_data->ce_count;
+ ret = amdgpu_umc_do_page_retirement(adev, &err_data, NULL, reset);
+
+ if (ret == AMDGPU_RAS_SUCCESS && obj) {
+ obj->err_data.ue_count += err_data.ue_count;
+ obj->err_data.ce_count += err_data.ce_count;
+ }
+ } else if (reset) {
+ /* MCA poison handler is only responsible for GPU reset,
+ * let MCA notifier do page retirement.
+ */
+ kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
+ amdgpu_ras_reset_gpu(adev);
}
return ret;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
index e464392..a695116 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
@@ -51,9 +51,6 @@ struct amdgpu_umc_ras {
struct amdgpu_ras_block_object ras_block;
void (*err_cnt_init)(struct amdgpu_device *adev);
bool (*query_ras_poison_mode)(struct amdgpu_device *adev);
- void (*convert_ras_error_address)(struct amdgpu_device *adev,
- struct ras_err_data *err_data, uint64_t err_addr,
- uint32_t ch_inst, uint32_t umc_inst);
void (*ecc_info_query_ras_error_count)(struct amdgpu_device *adev,
void *ras_error_status);
void (*ecc_info_query_ras_error_address)(struct amdgpu_device *adev,
@@ -86,9 +83,7 @@ struct amdgpu_umc {
};
int amdgpu_umc_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block);
-int amdgpu_umc_poison_handler(struct amdgpu_device *adev,
- void *ras_error_status,
- bool reset);
+int amdgpu_umc_poison_handler(struct amdgpu_device *adev, bool reset);
int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry);
@@ -101,4 +96,6 @@ void amdgpu_umc_fill_error_record(struct ras_err_data *err_data,
int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
void *ras_error_status,
struct amdgpu_iv_entry *entry);
+int amdgpu_umc_page_retirement_mca(struct amdgpu_device *adev,
+ uint64_t err_addr, uint32_t ch_inst, uint32_t umc_inst);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
index 6eac649..e00bb65 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
@@ -1132,7 +1132,9 @@ static int amdgpu_uvd_send_msg(struct amdgpu_ring *ring, struct amdgpu_bo *bo,
unsigned offset_idx = 0;
unsigned offset[3] = { UVD_BASE_SI, 0, 0 };
- r = amdgpu_job_alloc_with_ib(adev, 64, direct ? AMDGPU_IB_POOL_DIRECT :
+ r = amdgpu_job_alloc_with_ib(ring->adev, &adev->uvd.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ 64, direct ? AMDGPU_IB_POOL_DIRECT :
AMDGPU_IB_POOL_DELAYED, &job);
if (r)
return r;
@@ -1175,16 +1177,13 @@ static int amdgpu_uvd_send_msg(struct amdgpu_ring *ring, struct amdgpu_bo *bo,
if (r)
goto err_free;
} else {
- r = amdgpu_sync_resv(adev, &job->sync, bo->tbo.base.resv,
- AMDGPU_SYNC_ALWAYS,
- AMDGPU_FENCE_OWNER_UNDEFINED);
+ r = drm_sched_job_add_resv_dependencies(&job->base,
+ bo->tbo.base.resv,
+ DMA_RESV_USAGE_KERNEL);
if (r)
goto err_free;
- r = amdgpu_job_submit(job, &adev->uvd.entity,
- AMDGPU_FENCE_OWNER_UNDEFINED, &f);
- if (r)
- goto err_free;
+ f = amdgpu_job_submit(job);
}
amdgpu_bo_reserve(bo, true);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
index 02cb3a1..b239e87 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
@@ -450,8 +450,10 @@ static int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle,
uint64_t addr;
int i, r;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, &ring->adev->vce.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT,
+ &job);
if (r)
return r;
@@ -538,7 +540,9 @@ static int amdgpu_vce_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle,
struct dma_fence *f = NULL;
int i, r;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
+ r = amdgpu_job_alloc_with_ib(ring->adev, &ring->adev->vce.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ ib_size_dw * 4,
direct ? AMDGPU_IB_POOL_DIRECT :
AMDGPU_IB_POOL_DELAYED, &job);
if (r)
@@ -570,8 +574,7 @@ static int amdgpu_vce_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle,
if (direct)
r = amdgpu_job_submit_direct(job, ring, &f);
else
- r = amdgpu_job_submit(job, &ring->adev->vce.entity,
- AMDGPU_FENCE_OWNER_UNDEFINED, &f);
+ f = amdgpu_job_submit(job);
if (r)
goto err;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
index ce64ca1c..b1622ac 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
@@ -603,15 +603,16 @@ static int amdgpu_vcn_dec_send_msg(struct amdgpu_ring *ring,
struct amdgpu_ib *ib_msg,
struct dma_fence **fence)
{
+ u64 addr = AMDGPU_GPU_PAGE_ALIGN(ib_msg->gpu_addr);
struct amdgpu_device *adev = ring->adev;
struct dma_fence *f = NULL;
struct amdgpu_job *job;
struct amdgpu_ib *ib;
- uint64_t addr = AMDGPU_GPU_PAGE_ALIGN(ib_msg->gpu_addr);
int i, r;
- r = amdgpu_job_alloc_with_ib(adev, 64,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL,
+ 64, AMDGPU_IB_POOL_DIRECT,
+ &job);
if (r)
goto err;
@@ -790,8 +791,9 @@ static int amdgpu_vcn_dec_sw_send_msg(struct amdgpu_ring *ring,
if (sq)
ib_size_dw += 8;
- r = amdgpu_job_alloc_with_ib(adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL,
+ ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT,
+ &job);
if (r)
goto err;
@@ -919,8 +921,9 @@ static int amdgpu_vcn_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t hand
if (sq)
ib_size_dw += 8;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL,
+ ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT,
+ &job);
if (r)
return r;
@@ -985,8 +988,9 @@ static int amdgpu_vcn_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t han
if (sq)
ib_size_dw += 8;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL,
+ ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT,
+ &job);
if (r)
return r;
@@ -1251,3 +1255,20 @@ int amdgpu_vcn_process_poison_irq(struct amdgpu_device *adev,
return 0;
}
+
+void amdgpu_vcn_set_ras_funcs(struct amdgpu_device *adev)
+{
+ if (!adev->vcn.ras)
+ return;
+
+ amdgpu_ras_register_ras_block(adev, &adev->vcn.ras->ras_block);
+
+ strcpy(adev->vcn.ras->ras_block.ras_comm.name, "vcn");
+ adev->vcn.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__VCN;
+ adev->vcn.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__POISON;
+ adev->vcn.ras_if = &adev->vcn.ras->ras_block.ras_comm;
+
+ /* If don't define special ras_late_init function, use default ras_late_init */
+ if (!adev->vcn.ras->ras_block.ras_late_init)
+ adev->vcn.ras->ras_block.ras_late_init = amdgpu_ras_block_late_init;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
index 253ea6b..dbb8d68 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
@@ -399,5 +399,6 @@ void amdgpu_debugfs_vcn_fwlog_init(struct amdgpu_device *adev,
int amdgpu_vcn_process_poison_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry);
+void amdgpu_vcn_set_ras_funcs(struct amdgpu_device *adev);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index c73abe5..15544f2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -64,6 +64,10 @@ void amdgpu_virt_init_setting(struct amdgpu_device *adev)
ddev->driver_features &= ~DRIVER_ATOMIC;
adev->cg_flags = 0;
adev->pg_flags = 0;
+
+ /* enable mcbp for sriov asic_type before soc21 */
+ amdgpu_mcbp = (adev->asic_type < CHIP_IP_DISCOVERY) ? 1 : 0;
+
}
void amdgpu_virt_kiq_reg_write_reg_wait(struct amdgpu_device *adev,
@@ -424,11 +428,17 @@ static void amdgpu_virt_add_bad_page(struct amdgpu_device *adev,
struct eeprom_table_record bp;
uint64_t retired_page;
uint32_t bp_idx, bp_cnt;
+ void *vram_usage_va = NULL;
+
+ if (adev->mman.fw_vram_usage_va)
+ vram_usage_va = adev->mman.fw_vram_usage_va;
+ else
+ vram_usage_va = adev->mman.drv_vram_usage_va;
if (bp_block_size) {
bp_cnt = bp_block_size / sizeof(uint64_t);
for (bp_idx = 0; bp_idx < bp_cnt; bp_idx++) {
- retired_page = *(uint64_t *)(adev->mman.fw_vram_usage_va +
+ retired_page = *(uint64_t *)(vram_usage_va +
bp_block_offset + bp_idx * sizeof(uint64_t));
bp.retired_page = retired_page;
@@ -639,7 +649,9 @@ void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev)
adev->virt.fw_reserve.p_vf2pf = NULL;
adev->virt.vf2pf_update_interval_ms = 0;
- if (adev->mman.fw_vram_usage_va != NULL) {
+ if (adev->mman.fw_vram_usage_va && adev->mman.drv_vram_usage_va) {
+ DRM_WARN("Currently fw_vram and drv_vram should not have values at the same time!");
+ } else if (adev->mman.fw_vram_usage_va || adev->mman.drv_vram_usage_va) {
/* go through this logic in ip_init and reset to init workqueue*/
amdgpu_virt_exchange_data(adev);
@@ -662,32 +674,40 @@ void amdgpu_virt_exchange_data(struct amdgpu_device *adev)
uint32_t bp_block_size = 0;
struct amd_sriov_msg_pf2vf_info *pf2vf_v2 = NULL;
- if (adev->mman.fw_vram_usage_va != NULL) {
-
- adev->virt.fw_reserve.p_pf2vf =
- (struct amd_sriov_msg_pf2vf_info_header *)
- (adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
- adev->virt.fw_reserve.p_vf2pf =
- (struct amd_sriov_msg_vf2pf_info_header *)
- (adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_VF2PF_OFFSET_KB << 10));
+ if (adev->mman.fw_vram_usage_va || adev->mman.drv_vram_usage_va) {
+ if (adev->mman.fw_vram_usage_va) {
+ adev->virt.fw_reserve.p_pf2vf =
+ (struct amd_sriov_msg_pf2vf_info_header *)
+ (adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
+ adev->virt.fw_reserve.p_vf2pf =
+ (struct amd_sriov_msg_vf2pf_info_header *)
+ (adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_VF2PF_OFFSET_KB << 10));
+ } else if (adev->mman.drv_vram_usage_va) {
+ adev->virt.fw_reserve.p_pf2vf =
+ (struct amd_sriov_msg_pf2vf_info_header *)
+ (adev->mman.drv_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
+ adev->virt.fw_reserve.p_vf2pf =
+ (struct amd_sriov_msg_vf2pf_info_header *)
+ (adev->mman.drv_vram_usage_va + (AMD_SRIOV_MSG_VF2PF_OFFSET_KB << 10));
+ }
amdgpu_virt_read_pf2vf_data(adev);
amdgpu_virt_write_vf2pf_data(adev);
/* bad page handling for version 2 */
if (adev->virt.fw_reserve.p_pf2vf->version == 2) {
- pf2vf_v2 = (struct amd_sriov_msg_pf2vf_info *)adev->virt.fw_reserve.p_pf2vf;
+ pf2vf_v2 = (struct amd_sriov_msg_pf2vf_info *)adev->virt.fw_reserve.p_pf2vf;
- bp_block_offset = ((uint64_t)pf2vf_v2->bp_block_offset_low & 0xFFFFFFFF) |
- ((((uint64_t)pf2vf_v2->bp_block_offset_high) << 32) & 0xFFFFFFFF00000000);
- bp_block_size = pf2vf_v2->bp_block_size;
+ bp_block_offset = ((uint64_t)pf2vf_v2->bp_block_offset_low & 0xFFFFFFFF) |
+ ((((uint64_t)pf2vf_v2->bp_block_offset_high) << 32) & 0xFFFFFFFF00000000);
+ bp_block_size = pf2vf_v2->bp_block_size;
- if (bp_block_size && !adev->virt.ras_init_done)
- amdgpu_virt_init_ras_err_handler_data(adev);
+ if (bp_block_size && !adev->virt.ras_init_done)
+ amdgpu_virt_init_ras_err_handler_data(adev);
- if (adev->virt.ras_init_done)
- amdgpu_virt_add_bad_page(adev, bp_block_offset, bp_block_size);
- }
+ if (adev->virt.ras_init_done)
+ amdgpu_virt_add_bad_page(adev, bp_block_offset, bp_block_size);
+ }
}
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
index 49c4347d..2b9d806 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
@@ -75,6 +75,8 @@ struct amdgpu_vf_error_buffer {
uint64_t data[AMDGPU_VF_ERROR_ENTRY_SIZE];
};
+enum idh_request;
+
/**
* struct amdgpu_virt_ops - amdgpu device virt operations
*/
@@ -84,7 +86,8 @@ struct amdgpu_virt_ops {
int (*req_init_data)(struct amdgpu_device *adev);
int (*reset_gpu)(struct amdgpu_device *adev);
int (*wait_reset)(struct amdgpu_device *adev);
- void (*trans_msg)(struct amdgpu_device *adev, u32 req, u32 data1, u32 data2, u32 data3);
+ void (*trans_msg)(struct amdgpu_device *adev, enum idh_request req,
+ u32 data1, u32 data2, u32 data3);
};
/*
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c
index 500a1dc..53ff91fc6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.c
@@ -498,7 +498,7 @@ static int amdgpu_vkms_sw_init(void *handle)
adev_to_drm(adev)->mode_config.preferred_depth = 24;
adev_to_drm(adev)->mode_config.prefer_shadow = 1;
- adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
+ adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
@@ -513,6 +513,10 @@ static int amdgpu_vkms_sw_init(void *handle)
return r;
}
+ r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc);
+ if (r)
+ return r;
+
drm_kms_helper_poll_init(adev_to_drm(adev));
adev->mode_info.mode_config_initialized = true;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 003aa9e..c05cff9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -45,22 +45,43 @@
/**
* DOC: GPUVM
*
- * GPUVM is similar to the legacy gart on older asics, however
- * rather than there being a single global gart table
- * for the entire GPU, there are multiple VM page tables active
- * at any given time. The VM page tables can contain a mix
- * vram pages and system memory pages and system memory pages
+ * GPUVM is the MMU functionality provided on the GPU.
+ * GPUVM is similar to the legacy GART on older asics, however
+ * rather than there being a single global GART table
+ * for the entire GPU, there can be multiple GPUVM page tables active
+ * at any given time. The GPUVM page tables can contain a mix
+ * VRAM pages and system pages (both memory and MMIO) and system pages
* can be mapped as snooped (cached system pages) or unsnooped
* (uncached system pages).
- * Each VM has an ID associated with it and there is a page table
- * associated with each VMID. When executing a command buffer,
- * the kernel tells the ring what VMID to use for that command
+ *
+ * Each active GPUVM has an ID associated with it and there is a page table
+ * linked with each VMID. When executing a command buffer,
+ * the kernel tells the engine what VMID to use for that command
* buffer. VMIDs are allocated dynamically as commands are submitted.
* The userspace drivers maintain their own address space and the kernel
* sets up their pages tables accordingly when they submit their
* command buffers and a VMID is assigned.
- * Cayman/Trinity support up to 8 active VMs at any given time;
- * SI supports 16.
+ * The hardware supports up to 16 active GPUVMs at any given time.
+ *
+ * Each GPUVM is represented by a 1-2 or 1-5 level page table, depending
+ * on the ASIC family. GPUVM supports RWX attributes on each page as well
+ * as other features such as encryption and caching attributes.
+ *
+ * VMID 0 is special. It is the GPUVM used for the kernel driver. In
+ * addition to an aperture managed by a page table, VMID 0 also has
+ * several other apertures. There is an aperture for direct access to VRAM
+ * and there is a legacy AGP aperture which just forwards accesses directly
+ * to the matching system physical addresses (or IOVAs when an IOMMU is
+ * present). These apertures provide direct access to these memories without
+ * incurring the overhead of a page table. VMID 0 is used by the kernel
+ * driver for tasks like memory management.
+ *
+ * GPU clients (i.e., engines on the GPU) use GPUVM VMIDs to access memory.
+ * For user applications, each application can have their own unique GPUVM
+ * address space. The application manages the address space and the kernel
+ * driver manages the GPUVM page tables for each process. If an GPU client
+ * accesses an invalid page, it will generate a GPU page fault, similar to
+ * accessing an invalid page on a CPU.
*/
#define START(node) ((node)->start)
@@ -541,6 +562,7 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
if (!vm_flush_needed && !gds_switch_needed && !need_pipe_sync)
return 0;
+ amdgpu_ring_ib_begin(ring);
if (ring->funcs->init_cond_exec)
patch_offset = amdgpu_ring_init_cond_exec(ring);
@@ -601,6 +623,7 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
amdgpu_ring_emit_switch_buffer(ring);
amdgpu_ring_emit_switch_buffer(ring);
}
+ amdgpu_ring_ib_end(ring);
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index 69e105f..59cf642 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -47,6 +47,32 @@ static int amdgpu_vm_sdma_map_table(struct amdgpu_bo_vm *table)
return r;
}
+/* Allocate a new job for @count PTE updates */
+static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p,
+ unsigned int count)
+{
+ enum amdgpu_ib_pool_type pool = p->immediate ? AMDGPU_IB_POOL_IMMEDIATE
+ : AMDGPU_IB_POOL_DELAYED;
+ struct drm_sched_entity *entity = p->immediate ? &p->vm->immediate
+ : &p->vm->delayed;
+ unsigned int ndw;
+ int r;
+
+ /* estimate how many dw we need */
+ ndw = AMDGPU_VM_SDMA_MIN_NUM_DW;
+ if (p->pages_addr)
+ ndw += count * 2;
+ ndw = min(ndw, AMDGPU_VM_SDMA_MAX_NUM_DW);
+
+ r = amdgpu_job_alloc_with_ib(p->adev, entity, AMDGPU_FENCE_OWNER_VM,
+ ndw * 4, pool, &p->job);
+ if (r)
+ return r;
+
+ p->num_dw_left = ndw;
+ return 0;
+}
+
/**
* amdgpu_vm_sdma_prepare - prepare SDMA command submission
*
@@ -61,21 +87,22 @@ static int amdgpu_vm_sdma_prepare(struct amdgpu_vm_update_params *p,
struct dma_resv *resv,
enum amdgpu_sync_mode sync_mode)
{
- enum amdgpu_ib_pool_type pool = p->immediate ? AMDGPU_IB_POOL_IMMEDIATE
- : AMDGPU_IB_POOL_DELAYED;
- unsigned int ndw = AMDGPU_VM_SDMA_MIN_NUM_DW;
+ struct amdgpu_sync sync;
int r;
- r = amdgpu_job_alloc_with_ib(p->adev, ndw * 4, pool, &p->job);
+ r = amdgpu_vm_sdma_alloc_job(p, 0);
if (r)
return r;
- p->num_dw_left = ndw;
-
if (!resv)
return 0;
- return amdgpu_sync_resv(p->adev, &p->job->sync, resv, sync_mode, p->vm);
+ amdgpu_sync_create(&sync);
+ r = amdgpu_sync_resv(p->adev, &sync, resv, sync_mode, p->vm);
+ if (!r)
+ r = amdgpu_sync_push_to_job(&sync, p->job);
+ amdgpu_sync_free(&sync);
+ return r;
}
/**
@@ -91,20 +118,16 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
struct dma_fence **fence)
{
struct amdgpu_ib *ib = p->job->ibs;
- struct drm_sched_entity *entity;
struct amdgpu_ring *ring;
struct dma_fence *f;
- int r;
- entity = p->immediate ? &p->vm->immediate : &p->vm->delayed;
- ring = container_of(entity->rq->sched, struct amdgpu_ring, sched);
+ ring = container_of(p->vm->delayed.rq->sched, struct amdgpu_ring,
+ sched);
WARN_ON(ib->length_dw == 0);
amdgpu_ring_pad_ib(ring, ib);
WARN_ON(ib->length_dw > p->num_dw_left);
- r = amdgpu_job_submit(p->job, entity, AMDGPU_FENCE_OWNER_VM, &f);
- if (r)
- goto error;
+ f = amdgpu_job_submit(p->job);
if (p->unlocked) {
struct dma_fence *tmp = dma_fence_get(f);
@@ -127,10 +150,6 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
}
dma_fence_put(f);
return 0;
-
-error:
- amdgpu_job_free(p->job);
- return r;
}
/**
@@ -210,8 +229,6 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
uint64_t flags)
{
struct amdgpu_bo *bo = &vmbo->bo;
- enum amdgpu_ib_pool_type pool = p->immediate ? AMDGPU_IB_POOL_IMMEDIATE
- : AMDGPU_IB_POOL_DELAYED;
struct dma_resv_iter cursor;
unsigned int i, ndw, nptes;
struct dma_fence *fence;
@@ -221,7 +238,7 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
/* Wait for PD/PT moves to be completed */
dma_resv_iter_begin(&cursor, bo->tbo.base.resv, DMA_RESV_USAGE_KERNEL);
dma_resv_for_each_fence_unlocked(&cursor, fence) {
- r = amdgpu_sync_fence(&p->job->sync, fence);
+ r = drm_sched_job_add_dependency(&p->job->base, fence);
if (r) {
dma_resv_iter_end(&cursor);
return r;
@@ -238,19 +255,9 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
if (r)
return r;
- /* estimate how many dw we need */
- ndw = 32;
- if (p->pages_addr)
- ndw += count * 2;
- ndw = max(ndw, AMDGPU_VM_SDMA_MIN_NUM_DW);
- ndw = min(ndw, AMDGPU_VM_SDMA_MAX_NUM_DW);
-
- r = amdgpu_job_alloc_with_ib(p->adev, ndw * 4, pool,
- &p->job);
+ r = amdgpu_vm_sdma_alloc_job(p, count);
if (r)
return r;
-
- p->num_dw_left = ndw;
}
if (!p->pages_addr) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
index 80dd134..faa12146 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
@@ -439,7 +439,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
/* Allocate blocks in desired range */
vres->flags |= DRM_BUDDY_RANGE_ALLOCATION;
- remaining_size = (u64)vres->base.num_pages << PAGE_SHIFT;
+ remaining_size = (u64)vres->base.size;
mutex_lock(&mgr->lock);
while (remaining_size) {
@@ -498,7 +498,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
LIST_HEAD(temp);
trim_list = &vres->blocks;
- original_size = (u64)vres->base.num_pages << PAGE_SHIFT;
+ original_size = (u64)vres->base.size;
/*
* If size value is rounded up to min_block_size, trim the last
@@ -533,8 +533,8 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
amdgpu_vram_mgr_block_size(block);
start >>= PAGE_SHIFT;
- if (start > vres->base.num_pages)
- start -= vres->base.num_pages;
+ if (start > PFN_UP(vres->base.size))
+ start -= PFN_UP(vres->base.size);
else
start = 0;
vres->base.start = max(vres->base.start, start);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
index 47159e9..4b9e7b0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
@@ -386,7 +386,6 @@ struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev)
if (ret) {
dev_err(adev->dev, "XGMI: failed initializing kobject for xgmi hive\n");
kobject_put(&hive->kobj);
- kfree(hive);
hive = NULL;
goto pro_end;
}
@@ -410,7 +409,6 @@ struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev)
dev_err(adev->dev, "XGMI: failed initializing reset domain for xgmi hive\n");
ret = -ENOMEM;
kobject_put(&hive->kobj);
- kfree(hive);
hive = NULL;
goto pro_end;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c b/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
index 6be9ac2..18ae943 100644
--- a/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
+++ b/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
@@ -2081,8 +2081,11 @@ amdgpu_atombios_encoder_get_lcd_info(struct amdgpu_encoder *encoder)
}
}
record += fake_edid_record->ucFakeEDIDLength ?
- fake_edid_record->ucFakeEDIDLength + 2 :
- sizeof(ATOM_FAKE_EDID_PATCH_RECORD);
+ struct_size(fake_edid_record,
+ ucFakeEDIDString,
+ fake_edid_record->ucFakeEDIDLength) :
+ /* empty fake edid record must be 3 bytes long */
+ sizeof(ATOM_FAKE_EDID_PATCH_RECORD) + 1;
break;
case LCD_PANEL_RESOLUTION_RECORD_TYPE:
panel_res_record = (ATOM_PANEL_RESOLUTION_PATCH_RECORD *)record;
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index 288fce7..248f1a4 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -21,6 +21,7 @@
*
*/
+#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_vblank.h>
@@ -2800,8 +2801,6 @@ static int dce_v10_0_sw_init(void *handle)
adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
- adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
-
r = amdgpu_display_modeset_create_props(adev);
if (r)
return r;
@@ -2830,6 +2829,17 @@ static int dce_v10_0_sw_init(void *handle)
if (r)
return r;
+ /* Disable vblank IRQs aggressively for power-saving */
+ /* XXX: can this be enabled for DC? */
+ adev_to_drm(adev)->vblank_disable_immediate = true;
+
+ r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc);
+ if (r)
+ return r;
+
+ INIT_WORK(&adev->hotplug_work,
+ amdgpu_display_hotplug_work_func);
+
drm_kms_helper_poll_init(adev_to_drm(adev));
adev->mode_info.mode_config_initialized = true;
@@ -2892,6 +2902,8 @@ static int dce_v10_0_hw_fini(void *handle)
dce_v10_0_pageflip_interrupt_fini(adev);
+ flush_work(&adev->hotplug_work);
+
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
index cbe5250..cd9c190 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
@@ -21,6 +21,7 @@
*
*/
+#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_vblank.h>
@@ -2918,8 +2919,6 @@ static int dce_v11_0_sw_init(void *handle)
adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
- adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
-
r = amdgpu_display_modeset_create_props(adev);
if (r)
return r;
@@ -2949,6 +2948,17 @@ static int dce_v11_0_sw_init(void *handle)
if (r)
return r;
+ /* Disable vblank IRQs aggressively for power-saving */
+ /* XXX: can this be enabled for DC? */
+ adev_to_drm(adev)->vblank_disable_immediate = true;
+
+ r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc);
+ if (r)
+ return r;
+
+ INIT_WORK(&adev->hotplug_work,
+ amdgpu_display_hotplug_work_func);
+
drm_kms_helper_poll_init(adev_to_drm(adev));
adev->mode_info.mode_config_initialized = true;
@@ -3022,6 +3032,8 @@ static int dce_v11_0_hw_fini(void *handle)
dce_v11_0_pageflip_interrupt_fini(adev);
+ flush_work(&adev->hotplug_work);
+
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
index b1c44fab..76323de 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
@@ -23,6 +23,7 @@
#include <linux/pci.h>
+#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_vblank.h>
@@ -2675,7 +2676,6 @@ static int dce_v6_0_sw_init(void *handle)
adev_to_drm(adev)->mode_config.preferred_depth = 24;
adev_to_drm(adev)->mode_config.prefer_shadow = 1;
adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
- adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
r = amdgpu_display_modeset_create_props(adev);
if (r)
@@ -2706,6 +2706,18 @@ static int dce_v6_0_sw_init(void *handle)
if (r)
return r;
+ /* Disable vblank IRQs aggressively for power-saving */
+ /* XXX: can this be enabled for DC? */
+ adev_to_drm(adev)->vblank_disable_immediate = true;
+
+ r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc);
+ if (r)
+ return r;
+
+ /* Pre-DCE11 */
+ INIT_WORK(&adev->hotplug_work,
+ amdgpu_display_hotplug_work_func);
+
drm_kms_helper_poll_init(adev_to_drm(adev));
return r;
@@ -2764,6 +2776,8 @@ static int dce_v6_0_hw_fini(void *handle)
dce_v6_0_pageflip_interrupt_fini(adev);
+ flush_work(&adev->hotplug_work);
+
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index a22b45c..01cf3ab 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -21,6 +21,7 @@
*
*/
+#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_vblank.h>
@@ -2701,8 +2702,6 @@ static int dce_v8_0_sw_init(void *handle)
adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
- adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
-
r = amdgpu_display_modeset_create_props(adev);
if (r)
return r;
@@ -2731,6 +2730,18 @@ static int dce_v8_0_sw_init(void *handle)
if (r)
return r;
+ /* Disable vblank IRQs aggressively for power-saving */
+ /* XXX: can this be enabled for DC? */
+ adev_to_drm(adev)->vblank_disable_immediate = true;
+
+ r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc);
+ if (r)
+ return r;
+
+ /* Pre-DCE11 */
+ INIT_WORK(&adev->hotplug_work,
+ amdgpu_display_hotplug_work_func);
+
drm_kms_helper_poll_init(adev_to_drm(adev));
adev->mode_info.mode_config_initialized = true;
@@ -2791,6 +2802,8 @@ static int dce_v8_0_hw_fini(void *handle)
dce_v8_0_pageflip_interrupt_fini(adev);
+ flush_work(&adev->hotplug_work);
+
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
index af94ac5..49d34c7 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
@@ -4453,8 +4453,6 @@ static void gfx_v10_0_gpu_early_init(struct amdgpu_device *adev)
{
u32 gb_addr_config;
- adev->gfx.funcs = &gfx_v10_0_gfx_funcs;
-
switch (adev->ip_versions[GC_HWIP][0]) {
case IP_VERSION(10, 1, 10):
case IP_VERSION(10, 1, 1):
@@ -6911,6 +6909,8 @@ static int gfx_v10_0_kiq_init_queue(struct amdgpu_ring *ring)
mutex_unlock(&adev->srbm_mutex);
} else {
memset((void *)mqd, 0, sizeof(*mqd));
+ if (amdgpu_sriov_vf(adev) && adev->in_suspend)
+ amdgpu_ring_clear_ring(ring);
mutex_lock(&adev->srbm_mutex);
nv_grbm_select(adev, ring->me, ring->pipe, ring->queue, 0);
amdgpu_ring_init_mqd(ring);
@@ -7593,6 +7593,8 @@ static int gfx_v10_0_early_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ adev->gfx.funcs = &gfx_v10_0_gfx_funcs;
+
switch (adev->ip_versions[GC_HWIP][0]) {
case IP_VERSION(10, 1, 10):
case IP_VERSION(10, 1, 1):
@@ -8489,7 +8491,7 @@ static void gfx_v10_0_ring_emit_ib_gfx(struct amdgpu_ring *ring,
control |= ib->length_dw | (vmid << 24);
- if ((amdgpu_sriov_vf(ring->adev) || amdgpu_mcbp) && (ib->flags & AMDGPU_IB_FLAG_PREEMPT)) {
+ if (amdgpu_mcbp && (ib->flags & AMDGPU_IB_FLAG_PREEMPT)) {
control |= INDIRECT_BUFFER_PRE_ENB(1);
if (flags & AMDGPU_IB_PREEMPTED)
@@ -8664,7 +8666,7 @@ static void gfx_v10_0_ring_emit_cntxcntl(struct amdgpu_ring *ring,
{
uint32_t dw2 = 0;
- if (amdgpu_mcbp || amdgpu_sriov_vf(ring->adev))
+ if (amdgpu_mcbp)
gfx_v10_0_ring_emit_ce_meta(ring,
(!amdgpu_sriov_vf(ring->adev) && flags & AMDGPU_IB_PREEMPTED) ? true : false);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
index 0fecc5b..a56c6e1 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
@@ -77,6 +77,10 @@ MODULE_FIRMWARE("amdgpu/gc_11_0_3_pfp.bin");
MODULE_FIRMWARE("amdgpu/gc_11_0_3_me.bin");
MODULE_FIRMWARE("amdgpu/gc_11_0_3_mec.bin");
MODULE_FIRMWARE("amdgpu/gc_11_0_3_rlc.bin");
+MODULE_FIRMWARE("amdgpu/gc_11_0_4_pfp.bin");
+MODULE_FIRMWARE("amdgpu/gc_11_0_4_me.bin");
+MODULE_FIRMWARE("amdgpu/gc_11_0_4_mec.bin");
+MODULE_FIRMWARE("amdgpu/gc_11_0_4_rlc.bin");
static const struct soc15_reg_golden golden_settings_gc_11_0_1[] =
{
@@ -262,6 +266,7 @@ static void gfx_v11_0_init_golden_registers(struct amdgpu_device *adev)
{
switch (adev->ip_versions[GC_HWIP][0]) {
case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
soc15_program_register_sequence(adev,
golden_settings_gc_11_0_1,
(const u32)ARRAY_SIZE(golden_settings_gc_11_0_1));
@@ -843,7 +848,6 @@ static const struct amdgpu_gfx_funcs gfx_v11_0_gfx_funcs = {
static int gfx_v11_0_gpu_early_init(struct amdgpu_device *adev)
{
- adev->gfx.funcs = &gfx_v11_0_gfx_funcs;
switch (adev->ip_versions[GC_HWIP][0]) {
case IP_VERSION(11, 0, 0):
@@ -856,6 +860,7 @@ static int gfx_v11_0_gpu_early_init(struct amdgpu_device *adev)
adev->gfx.config.sc_earlyz_tile_fifo_size = 0x4C0;
break;
case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
adev->gfx.config.max_hw_contexts = 8;
adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
@@ -1285,6 +1290,7 @@ static int gfx_v11_0_sw_init(void *handle)
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
adev->gfx.me.num_me = 1;
adev->gfx.me.num_pipe_per_me = 1;
adev->gfx.me.num_queue_per_pipe = 1;
@@ -1626,7 +1632,8 @@ static void gfx_v11_0_constants_init(struct amdgpu_device *adev)
u32 tmp;
int i;
- WREG32_FIELD15_PREREG(GC, 0, GRBM_CNTL, READ_TIMEOUT, 0xff);
+ if (!amdgpu_sriov_vf(adev))
+ WREG32_FIELD15_PREREG(GC, 0, GRBM_CNTL, READ_TIMEOUT, 0xff);
gfx_v11_0_setup_rb(adev);
gfx_v11_0_get_cu_info(adev, &adev->gfx.cu_info);
@@ -2486,7 +2493,8 @@ static int gfx_v11_0_wait_for_rlc_autoload_complete(struct amdgpu_device *adev)
for (i = 0; i < adev->usec_timeout; i++) {
cp_status = RREG32_SOC15(GC, 0, regCP_STAT);
- if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(11, 0, 1))
+ if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(11, 0, 1) ||
+ adev->ip_versions[GC_HWIP][0] == IP_VERSION(11, 0, 4))
bootload_status = RREG32_SOC15(GC, 0,
regRLC_RLCS_BOOTLOAD_STATUS_gc_11_0_1);
else
@@ -4004,6 +4012,8 @@ static int gfx_v11_0_kiq_init_queue(struct amdgpu_ring *ring)
mutex_unlock(&adev->srbm_mutex);
} else {
memset((void *)mqd, 0, sizeof(*mqd));
+ if (amdgpu_sriov_vf(adev) && adev->in_suspend)
+ amdgpu_ring_clear_ring(ring);
mutex_lock(&adev->srbm_mutex);
soc21_grbm_select(adev, ring->me, ring->pipe, ring->queue, 0);
amdgpu_ring_init_mqd(ring);
@@ -4390,7 +4400,6 @@ static int gfx_v11_0_hw_fini(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
int r;
- uint32_t tmp;
amdgpu_irq_put(adev, &adev->gfx.priv_reg_irq, 0);
amdgpu_irq_put(adev, &adev->gfx.priv_inst_irq, 0);
@@ -4409,15 +4418,14 @@ static int gfx_v11_0_hw_fini(void *handle)
amdgpu_mes_kiq_hw_fini(adev);
}
- if (amdgpu_sriov_vf(adev)) {
- gfx_v11_0_cp_gfx_enable(adev, false);
- /* Program KIQ position of RLC_CP_SCHEDULERS during destroy */
- tmp = RREG32_SOC15(GC, 0, regRLC_CP_SCHEDULERS);
- tmp &= 0xffffff00;
- WREG32_SOC15(GC, 0, regRLC_CP_SCHEDULERS, tmp);
-
+ if (amdgpu_sriov_vf(adev))
+ /* Remove the steps disabling CPG and clearing KIQ position,
+ * so that CP could perform IDLE-SAVE during switch. Those
+ * steps are necessary to avoid a DMAR error in gfx9 but it is
+ * not reproduced on gfx11.
+ */
return 0;
- }
+
gfx_v11_0_cp_enable(adev, false);
gfx_v11_0_enable_gui_idle_interrupt(adev, false);
@@ -4656,6 +4664,8 @@ static int gfx_v11_0_early_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ adev->gfx.funcs = &gfx_v11_0_gfx_funcs;
+
adev->gfx.num_gfx_rings = GFX11_NUM_GFX_RINGS;
adev->gfx.num_compute_rings = min(amdgpu_gfx_get_num_kcq(adev),
AMDGPU_MAX_COMPUTE_RINGS);
@@ -4673,6 +4683,26 @@ static int gfx_v11_0_early_init(void *handle)
return 0;
}
+static int gfx_v11_0_ras_late_init(void *handle)
+{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct ras_common_if *gfx_common_if;
+ int ret;
+
+ gfx_common_if = kzalloc(sizeof(struct ras_common_if), GFP_KERNEL);
+ if (!gfx_common_if)
+ return -ENOMEM;
+
+ gfx_common_if->block = AMDGPU_RAS_BLOCK__GFX;
+
+ ret = amdgpu_ras_feature_enable(adev, gfx_common_if, true);
+ if (ret)
+ dev_warn(adev->dev, "Failed to enable gfx11 ras feature\n");
+
+ kfree(gfx_common_if);
+ return 0;
+}
+
static int gfx_v11_0_late_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
@@ -4686,6 +4716,12 @@ static int gfx_v11_0_late_init(void *handle)
if (r)
return r;
+ if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(11, 0, 3)) {
+ r = gfx_v11_0_ras_late_init(handle);
+ if (r)
+ return r;
+ }
+
return 0;
}
@@ -5022,6 +5058,7 @@ static void gfx_v11_cntl_power_gating(struct amdgpu_device *adev, bool enable)
if (enable && (adev->pg_flags & AMD_PG_SUPPORT_GFX_PG)) {
switch (adev->ip_versions[GC_HWIP][0]) {
case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
WREG32_SOC15(GC, 0, regRLC_PG_DELAY_3, RLC_PG_DELAY_3_DEFAULT_GC_11_0_1);
break;
default:
@@ -5055,6 +5092,7 @@ static int gfx_v11_0_set_powergating_state(void *handle,
amdgpu_gfx_off_ctrl(adev, enable);
break;
case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
gfx_v11_cntl_pg(adev, enable);
amdgpu_gfx_off_ctrl(adev, enable);
break;
@@ -5078,6 +5116,7 @@ static int gfx_v11_0_set_clockgating_state(void *handle,
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
gfx_v11_0_update_gfx_clock_gating(adev,
state == AMD_CG_STATE_GATE);
break;
@@ -5299,7 +5338,7 @@ static void gfx_v11_0_ring_emit_ib_gfx(struct amdgpu_ring *ring,
control |= ib->length_dw | (vmid << 24);
- if ((amdgpu_sriov_vf(ring->adev) || amdgpu_mcbp) && (ib->flags & AMDGPU_IB_FLAG_PREEMPT)) {
+ if (amdgpu_mcbp && (ib->flags & AMDGPU_IB_FLAG_PREEMPT)) {
control |= INDIRECT_BUFFER_PRE_ENB(1);
if (flags & AMDGPU_IB_PREEMPTED)
@@ -6060,6 +6099,7 @@ static const struct amdgpu_ring_funcs gfx_v11_0_ring_funcs_gfx = {
.align_mask = 0xff,
.nop = PACKET3(PACKET3_NOP, 0x3FFF),
.support_64bit_ptrs = true,
+ .secure_submission_supported = true,
.vmhub = AMDGPU_GFXHUB_0,
.get_rptr = gfx_v11_0_ring_get_rptr_gfx,
.get_wptr = gfx_v11_0_ring_get_wptr_gfx,
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
index 7f0b18b..d471356 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
@@ -4643,6 +4643,8 @@ static int gfx_v8_0_kiq_init_queue(struct amdgpu_ring *ring)
memset((void *)mqd, 0, sizeof(struct vi_mqd_allocation));
((struct vi_mqd_allocation *)mqd)->dynamic_cu_mask = 0xFFFFFFFF;
((struct vi_mqd_allocation *)mqd)->dynamic_rb_mask = 0xFFFFFFFF;
+ if (amdgpu_sriov_vf(adev) && adev->in_suspend)
+ amdgpu_ring_clear_ring(ring);
mutex_lock(&adev->srbm_mutex);
vi_srbm_select(adev, ring->me, ring->pipe, ring->queue, 0);
gfx_v8_0_mqd_init(ring);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
index 0320be4..f202b45 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
@@ -47,6 +47,7 @@
#include "amdgpu_ras.h"
+#include "amdgpu_ring_mux.h"
#include "gfx_v9_4.h"
#include "gfx_v9_0.h"
#include "gfx_v9_4_2.h"
@@ -56,6 +57,7 @@
#include "asic_reg/gc/gc_9_0_default.h"
#define GFX9_NUM_GFX_RINGS 1
+#define GFX9_NUM_SW_GFX_RINGS 2
#define GFX9_MEC_HPD_SIZE 4096
#define RLCG_UCODE_LOADING_START_ADDRESS 0x00002000L
#define RLC_SAVE_RESTORE_ADDR_STARTING_OFFSET 0x00000000L
@@ -753,7 +755,7 @@ static void gfx_v9_0_set_rlc_funcs(struct amdgpu_device *adev);
static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev,
struct amdgpu_cu_info *cu_info);
static uint64_t gfx_v9_0_get_gpu_clock_counter(struct amdgpu_device *adev);
-static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring);
+static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring, bool resume);
static u64 gfx_v9_0_ring_get_rptr_compute(struct amdgpu_ring *ring);
static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev,
void *ras_error_status);
@@ -826,9 +828,10 @@ static void gfx_v9_0_kiq_unmap_queues(struct amdgpu_ring *kiq_ring,
PACKET3_UNMAP_QUEUES_DOORBELL_OFFSET0(ring->doorbell_index));
if (action == PREEMPT_QUEUES_NO_UNMAP) {
- amdgpu_ring_write(kiq_ring, lower_32_bits(gpu_addr));
- amdgpu_ring_write(kiq_ring, upper_32_bits(gpu_addr));
- amdgpu_ring_write(kiq_ring, seq);
+ amdgpu_ring_write(kiq_ring, lower_32_bits(ring->wptr & ring->buf_mask));
+ amdgpu_ring_write(kiq_ring, 0);
+ amdgpu_ring_write(kiq_ring, 0);
+
} else {
amdgpu_ring_write(kiq_ring, 0);
amdgpu_ring_write(kiq_ring, 0);
@@ -1564,7 +1567,7 @@ static void gfx_v9_0_init_always_on_cu_mask(struct amdgpu_device *adev)
mask = 1;
cu_bitmap = 0;
counter = 0;
- gfx_v9_0_select_se_sh(adev, i, j, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, i, j, 0xffffffff);
for (k = 0; k < adev->gfx.config.max_cu_per_sh; k ++) {
if (cu_info->bitmap[i][j] & mask) {
@@ -1583,7 +1586,7 @@ static void gfx_v9_0_init_always_on_cu_mask(struct amdgpu_device *adev)
cu_info->ao_cu_bitmap[i][j] = cu_bitmap;
}
}
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
}
@@ -1605,7 +1608,7 @@ static void gfx_v9_0_init_lbpw(struct amdgpu_device *adev)
mutex_lock(&adev->grbm_idx_mutex);
/* set mmRLC_LB_INIT_CU_MASK thru broadcast mode to enable all SE/SH*/
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
WREG32_SOC15(GC, 0, mmRLC_LB_INIT_CU_MASK, 0xffffffff);
/* set mmRLC_LB_PARAMS = 0x003F_1006 */
@@ -1654,7 +1657,7 @@ static void gfx_v9_4_init_lbpw(struct amdgpu_device *adev)
mutex_lock(&adev->grbm_idx_mutex);
/* set mmRLC_LB_INIT_CU_MASK thru broadcast mode to enable all SE/SH*/
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
WREG32_SOC15(GC, 0, mmRLC_LB_INIT_CU_MASK, 0xffffffff);
/* set mmRLC_LB_PARAMS = 0x003F_1006 */
@@ -1919,8 +1922,6 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)
u32 gb_addr_config;
int err;
- adev->gfx.funcs = &gfx_v9_0_gfx_funcs;
-
switch (adev->ip_versions[GC_HWIP][0]) {
case IP_VERSION(9, 0, 1):
adev->gfx.config.max_hw_contexts = 8;
@@ -2105,6 +2106,7 @@ static int gfx_v9_0_sw_init(void *handle)
struct amdgpu_ring *ring;
struct amdgpu_kiq *kiq;
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ unsigned int hw_prio;
switch (adev->ip_versions[GC_HWIP][0]) {
case IP_VERSION(9, 0, 1):
@@ -2188,6 +2190,9 @@ static int gfx_v9_0_sw_init(void *handle)
sprintf(ring->name, "gfx_%d", i);
ring->use_doorbell = true;
ring->doorbell_index = adev->doorbell_index.gfx_ring0 << 1;
+
+ /* disable scheduler on the real ring */
+ ring->no_scheduler = true;
r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
AMDGPU_RING_PRIO_DEFAULT, NULL);
@@ -2195,6 +2200,41 @@ static int gfx_v9_0_sw_init(void *handle)
return r;
}
+ /* set up the software rings */
+ if (adev->gfx.num_gfx_rings) {
+ for (i = 0; i < GFX9_NUM_SW_GFX_RINGS; i++) {
+ ring = &adev->gfx.sw_gfx_ring[i];
+ ring->ring_obj = NULL;
+ sprintf(ring->name, amdgpu_sw_ring_name(i));
+ ring->use_doorbell = true;
+ ring->doorbell_index = adev->doorbell_index.gfx_ring0 << 1;
+ ring->is_sw_ring = true;
+ hw_prio = amdgpu_sw_ring_priority(i);
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq,
+ AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP, hw_prio,
+ NULL);
+ if (r)
+ return r;
+ ring->wptr = 0;
+ }
+
+ /* init the muxer and add software rings */
+ r = amdgpu_ring_mux_init(&adev->gfx.muxer, &adev->gfx.gfx_ring[0],
+ GFX9_NUM_SW_GFX_RINGS);
+ if (r) {
+ DRM_ERROR("amdgpu_ring_mux_init failed(%d)\n", r);
+ return r;
+ }
+ for (i = 0; i < GFX9_NUM_SW_GFX_RINGS; i++) {
+ r = amdgpu_ring_mux_add_sw_ring(&adev->gfx.muxer,
+ &adev->gfx.sw_gfx_ring[i]);
+ if (r) {
+ DRM_ERROR("amdgpu_ring_mux_add_sw_ring failed(%d)\n", r);
+ return r;
+ }
+ }
+ }
+
/* set up the compute queues - allocate horizontally across pipes */
ring_id = 0;
for (i = 0; i < adev->gfx.mec.num_mec; ++i) {
@@ -2245,6 +2285,12 @@ static int gfx_v9_0_sw_fini(void *handle)
int i;
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ if (adev->gfx.num_gfx_rings) {
+ for (i = 0; i < GFX9_NUM_SW_GFX_RINGS; i++)
+ amdgpu_ring_fini(&adev->gfx.sw_gfx_ring[i]);
+ amdgpu_ring_mux_fini(&adev->gfx.muxer);
+ }
+
for (i = 0; i < adev->gfx.num_gfx_rings; i++)
amdgpu_ring_fini(&adev->gfx.gfx_ring[i]);
for (i = 0; i < adev->gfx.num_compute_rings; i++)
@@ -2324,13 +2370,13 @@ static void gfx_v9_0_setup_rb(struct amdgpu_device *adev)
mutex_lock(&adev->grbm_idx_mutex);
for (i = 0; i < adev->gfx.config.max_shader_engines; i++) {
for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) {
- gfx_v9_0_select_se_sh(adev, i, j, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, i, j, 0xffffffff);
data = gfx_v9_0_get_rb_active_bitmap(adev);
active_rbs |= data << ((i * adev->gfx.config.max_sh_per_se + j) *
rb_bitmap_width_per_sh);
}
}
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
adev->gfx.config.backend_enable_mask = active_rbs;
@@ -2467,14 +2513,14 @@ static void gfx_v9_0_wait_for_rlc_serdes(struct amdgpu_device *adev)
mutex_lock(&adev->grbm_idx_mutex);
for (i = 0; i < adev->gfx.config.max_shader_engines; i++) {
for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) {
- gfx_v9_0_select_se_sh(adev, i, j, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, i, j, 0xffffffff);
for (k = 0; k < adev->usec_timeout; k++) {
if (RREG32_SOC15(GC, 0, mmRLC_SERDES_CU_MASTER_BUSY) == 0)
break;
udelay(1);
}
if (k == adev->usec_timeout) {
- gfx_v9_0_select_se_sh(adev, 0xffffffff,
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff,
0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
DRM_INFO("Timeout wait for RLC serdes %u,%u\n",
@@ -2483,7 +2529,7 @@ static void gfx_v9_0_wait_for_rlc_serdes(struct amdgpu_device *adev)
}
}
}
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
mask = RLC_SERDES_NONCU_MASTER_BUSY__SE_MASTER_BUSY_MASK |
@@ -3583,6 +3629,8 @@ static int gfx_v9_0_kiq_init_queue(struct amdgpu_ring *ring)
memset((void *)mqd, 0, sizeof(struct v9_mqd_allocation));
((struct v9_mqd_allocation *)mqd)->dynamic_cu_mask = 0xFFFFFFFF;
((struct v9_mqd_allocation *)mqd)->dynamic_rb_mask = 0xFFFFFFFF;
+ if (amdgpu_sriov_vf(adev) && adev->in_suspend)
+ amdgpu_ring_clear_ring(ring);
mutex_lock(&adev->srbm_mutex);
soc15_grbm_select(adev, ring->me, ring->pipe, ring->queue, 0);
gfx_v9_0_mqd_init(ring);
@@ -4539,6 +4587,8 @@ static int gfx_v9_0_early_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ adev->gfx.funcs = &gfx_v9_0_gfx_funcs;
+
if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 1) ||
adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 2))
adev->gfx.num_gfx_rings = 0;
@@ -5155,11 +5205,17 @@ static void gfx_v9_0_ring_emit_ib_gfx(struct amdgpu_ring *ring,
control |= ib->length_dw | (vmid << 24);
- if (amdgpu_sriov_vf(ring->adev) && (ib->flags & AMDGPU_IB_FLAG_PREEMPT)) {
+ if (ib->flags & AMDGPU_IB_FLAG_PREEMPT) {
control |= INDIRECT_BUFFER_PRE_ENB(1);
+ if (flags & AMDGPU_IB_PREEMPTED)
+ control |= INDIRECT_BUFFER_PRE_RESUME(1);
+
if (!(ib->flags & AMDGPU_IB_FLAG_CE) && vmid)
- gfx_v9_0_ring_emit_de_meta(ring);
+ gfx_v9_0_ring_emit_de_meta(ring,
+ (!amdgpu_sriov_vf(ring->adev) &&
+ flags & AMDGPU_IB_PREEMPTED) ?
+ true : false);
}
amdgpu_ring_write(ring, header);
@@ -5214,17 +5270,24 @@ static void gfx_v9_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr,
bool write64bit = flags & AMDGPU_FENCE_FLAG_64BIT;
bool int_sel = flags & AMDGPU_FENCE_FLAG_INT;
bool writeback = flags & AMDGPU_FENCE_FLAG_TC_WB_ONLY;
+ bool exec = flags & AMDGPU_FENCE_FLAG_EXEC;
+ uint32_t dw2 = 0;
/* RELEASE_MEM - flush caches, send int */
amdgpu_ring_write(ring, PACKET3(PACKET3_RELEASE_MEM, 6));
- amdgpu_ring_write(ring, ((writeback ? (EOP_TC_WB_ACTION_EN |
- EOP_TC_NC_ACTION_EN) :
- (EOP_TCL1_ACTION_EN |
- EOP_TC_ACTION_EN |
- EOP_TC_WB_ACTION_EN |
- EOP_TC_MD_ACTION_EN)) |
- EVENT_TYPE(CACHE_FLUSH_AND_INV_TS_EVENT) |
- EVENT_INDEX(5)));
+
+ if (writeback) {
+ dw2 = EOP_TC_NC_ACTION_EN;
+ } else {
+ dw2 = EOP_TCL1_ACTION_EN | EOP_TC_ACTION_EN |
+ EOP_TC_MD_ACTION_EN;
+ }
+ dw2 |= EOP_TC_WB_ACTION_EN | EVENT_TYPE(CACHE_FLUSH_AND_INV_TS_EVENT) |
+ EVENT_INDEX(5);
+ if (exec)
+ dw2 |= EOP_EXEC;
+
+ amdgpu_ring_write(ring, dw2);
amdgpu_ring_write(ring, DATA_SEL(write64bit ? 2 : 1) | INT_SEL(int_sel ? 2 : 0));
/*
@@ -5329,33 +5392,135 @@ static void gfx_v9_ring_emit_sb(struct amdgpu_ring *ring)
amdgpu_ring_write(ring, 0);
}
-static void gfx_v9_0_ring_emit_ce_meta(struct amdgpu_ring *ring)
+static void gfx_v9_0_ring_emit_ce_meta(struct amdgpu_ring *ring, bool resume)
{
+ struct amdgpu_device *adev = ring->adev;
struct v9_ce_ib_state ce_payload = {0};
- uint64_t csa_addr;
+ uint64_t offset, ce_payload_gpu_addr;
+ void *ce_payload_cpu_addr;
int cnt;
cnt = (sizeof(ce_payload) >> 2) + 4 - 2;
- csa_addr = amdgpu_csa_vaddr(ring->adev);
+
+ if (ring->is_mes_queue) {
+ offset = offsetof(struct amdgpu_mes_ctx_meta_data,
+ gfx[0].gfx_meta_data) +
+ offsetof(struct v9_gfx_meta_data, ce_payload);
+ ce_payload_gpu_addr =
+ amdgpu_mes_ctx_get_offs_gpu_addr(ring, offset);
+ ce_payload_cpu_addr =
+ amdgpu_mes_ctx_get_offs_cpu_addr(ring, offset);
+ } else {
+ offset = offsetof(struct v9_gfx_meta_data, ce_payload);
+ ce_payload_gpu_addr = amdgpu_csa_vaddr(ring->adev) + offset;
+ ce_payload_cpu_addr = adev->virt.csa_cpu_addr + offset;
+ }
amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, cnt));
amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(2) |
WRITE_DATA_DST_SEL(8) |
WR_CONFIRM) |
WRITE_DATA_CACHE_POLICY(0));
- amdgpu_ring_write(ring, lower_32_bits(csa_addr + offsetof(struct v9_gfx_meta_data, ce_payload)));
- amdgpu_ring_write(ring, upper_32_bits(csa_addr + offsetof(struct v9_gfx_meta_data, ce_payload)));
- amdgpu_ring_write_multiple(ring, (void *)&ce_payload, sizeof(ce_payload) >> 2);
+ amdgpu_ring_write(ring, lower_32_bits(ce_payload_gpu_addr));
+ amdgpu_ring_write(ring, upper_32_bits(ce_payload_gpu_addr));
+
+ if (resume)
+ amdgpu_ring_write_multiple(ring, ce_payload_cpu_addr,
+ sizeof(ce_payload) >> 2);
+ else
+ amdgpu_ring_write_multiple(ring, (void *)&ce_payload,
+ sizeof(ce_payload) >> 2);
}
-static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring)
+static int gfx_v9_0_ring_preempt_ib(struct amdgpu_ring *ring)
{
+ int i, r = 0;
+ struct amdgpu_device *adev = ring->adev;
+ struct amdgpu_kiq *kiq = &adev->gfx.kiq;
+ struct amdgpu_ring *kiq_ring = &kiq->ring;
+ unsigned long flags;
+
+ if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues)
+ return -EINVAL;
+
+ spin_lock_irqsave(&kiq->ring_lock, flags);
+
+ if (amdgpu_ring_alloc(kiq_ring, kiq->pmf->unmap_queues_size)) {
+ spin_unlock_irqrestore(&kiq->ring_lock, flags);
+ return -ENOMEM;
+ }
+
+ /* assert preemption condition */
+ amdgpu_ring_set_preempt_cond_exec(ring, false);
+
+ ring->trail_seq += 1;
+ amdgpu_ring_alloc(ring, 13);
+ gfx_v9_0_ring_emit_fence(ring, ring->trail_fence_gpu_addr,
+ ring->trail_seq, AMDGPU_FENCE_FLAG_EXEC | AMDGPU_FENCE_FLAG_INT);
+ /*reset the CP_VMID_PREEMPT after trailing fence*/
+ amdgpu_ring_emit_wreg(ring,
+ SOC15_REG_OFFSET(GC, 0, mmCP_VMID_PREEMPT),
+ 0x0);
+
+ /* assert IB preemption, emit the trailing fence */
+ kiq->pmf->kiq_unmap_queues(kiq_ring, ring, PREEMPT_QUEUES_NO_UNMAP,
+ ring->trail_fence_gpu_addr,
+ ring->trail_seq);
+
+ amdgpu_ring_commit(kiq_ring);
+ spin_unlock_irqrestore(&kiq->ring_lock, flags);
+
+ /* poll the trailing fence */
+ for (i = 0; i < adev->usec_timeout; i++) {
+ if (ring->trail_seq ==
+ le32_to_cpu(*ring->trail_fence_cpu_addr))
+ break;
+ udelay(1);
+ }
+
+ if (i >= adev->usec_timeout) {
+ r = -EINVAL;
+ DRM_WARN("ring %d timeout to preempt ib\n", ring->idx);
+ }
+
+ amdgpu_ring_commit(ring);
+
+ /* deassert preemption condition */
+ amdgpu_ring_set_preempt_cond_exec(ring, true);
+ return r;
+}
+
+static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring, bool resume)
+{
+ struct amdgpu_device *adev = ring->adev;
struct v9_de_ib_state de_payload = {0};
- uint64_t csa_addr, gds_addr;
+ uint64_t offset, gds_addr, de_payload_gpu_addr;
+ void *de_payload_cpu_addr;
int cnt;
- csa_addr = amdgpu_csa_vaddr(ring->adev);
- gds_addr = csa_addr + 4096;
+ if (ring->is_mes_queue) {
+ offset = offsetof(struct amdgpu_mes_ctx_meta_data,
+ gfx[0].gfx_meta_data) +
+ offsetof(struct v9_gfx_meta_data, de_payload);
+ de_payload_gpu_addr =
+ amdgpu_mes_ctx_get_offs_gpu_addr(ring, offset);
+ de_payload_cpu_addr =
+ amdgpu_mes_ctx_get_offs_cpu_addr(ring, offset);
+
+ offset = offsetof(struct amdgpu_mes_ctx_meta_data,
+ gfx[0].gds_backup) +
+ offsetof(struct v9_gfx_meta_data, de_payload);
+ gds_addr = amdgpu_mes_ctx_get_offs_gpu_addr(ring, offset);
+ } else {
+ offset = offsetof(struct v9_gfx_meta_data, de_payload);
+ de_payload_gpu_addr = amdgpu_csa_vaddr(ring->adev) + offset;
+ de_payload_cpu_addr = adev->virt.csa_cpu_addr + offset;
+
+ gds_addr = ALIGN(amdgpu_csa_vaddr(ring->adev) +
+ AMDGPU_CSA_SIZE - adev->gds.gds_size,
+ PAGE_SIZE);
+ }
+
de_payload.gds_backup_addrlo = lower_32_bits(gds_addr);
de_payload.gds_backup_addrhi = upper_32_bits(gds_addr);
@@ -5365,9 +5530,15 @@ static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring)
WRITE_DATA_DST_SEL(8) |
WR_CONFIRM) |
WRITE_DATA_CACHE_POLICY(0));
- amdgpu_ring_write(ring, lower_32_bits(csa_addr + offsetof(struct v9_gfx_meta_data, de_payload)));
- amdgpu_ring_write(ring, upper_32_bits(csa_addr + offsetof(struct v9_gfx_meta_data, de_payload)));
- amdgpu_ring_write_multiple(ring, (void *)&de_payload, sizeof(de_payload) >> 2);
+ amdgpu_ring_write(ring, lower_32_bits(de_payload_gpu_addr));
+ amdgpu_ring_write(ring, upper_32_bits(de_payload_gpu_addr));
+
+ if (resume)
+ amdgpu_ring_write_multiple(ring, de_payload_cpu_addr,
+ sizeof(de_payload) >> 2);
+ else
+ amdgpu_ring_write_multiple(ring, (void *)&de_payload,
+ sizeof(de_payload) >> 2);
}
static void gfx_v9_0_ring_emit_frame_cntl(struct amdgpu_ring *ring, bool start,
@@ -5383,8 +5554,9 @@ static void gfx_v9_ring_emit_cntxcntl(struct amdgpu_ring *ring, uint32_t flags)
{
uint32_t dw2 = 0;
- if (amdgpu_sriov_vf(ring->adev))
- gfx_v9_0_ring_emit_ce_meta(ring);
+ gfx_v9_0_ring_emit_ce_meta(ring,
+ (!amdgpu_sriov_vf(ring->adev) &&
+ flags & AMDGPU_IB_PREEMPTED) ? true : false);
dw2 |= 0x80000000; /* set load_enable otherwise this package is just NOPs */
if (flags & AMDGPU_HAVE_CTX_SWITCH) {
@@ -5710,7 +5882,12 @@ static int gfx_v9_0_eop_irq(struct amdgpu_device *adev,
switch (me_id) {
case 0:
- amdgpu_fence_process(&adev->gfx.gfx_ring[0]);
+ if (adev->gfx.num_gfx_rings &&
+ !amdgpu_mcbp_handle_trailing_fence_irq(&adev->gfx.muxer)) {
+ /* Fence signals are handled on the software rings*/
+ for (i = 0; i < GFX9_NUM_SW_GFX_RINGS; i++)
+ amdgpu_fence_process(&adev->gfx.sw_gfx_ring[i]);
+ }
break;
case 1:
case 2:
@@ -6482,7 +6659,7 @@ static void gfx_v9_0_reset_ras_error_count(struct amdgpu_device *adev)
for (i = 0; i < ARRAY_SIZE(gfx_v9_0_edc_counter_regs); i++) {
for (j = 0; j < gfx_v9_0_edc_counter_regs[i].se_num; j++) {
for (k = 0; k < gfx_v9_0_edc_counter_regs[i].instance; k++) {
- gfx_v9_0_select_se_sh(adev, j, 0x0, k);
+ amdgpu_gfx_select_se_sh(adev, j, 0x0, k);
RREG32(SOC15_REG_ENTRY_OFFSET(gfx_v9_0_edc_counter_regs[i]));
}
}
@@ -6544,7 +6721,7 @@ static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev,
for (i = 0; i < ARRAY_SIZE(gfx_v9_0_edc_counter_regs); i++) {
for (j = 0; j < gfx_v9_0_edc_counter_regs[i].se_num; j++) {
for (k = 0; k < gfx_v9_0_edc_counter_regs[i].instance; k++) {
- gfx_v9_0_select_se_sh(adev, j, 0, k);
+ amdgpu_gfx_select_se_sh(adev, j, 0, k);
reg_value =
RREG32(SOC15_REG_ENTRY_OFFSET(gfx_v9_0_edc_counter_regs[i]));
if (reg_value)
@@ -6559,7 +6736,7 @@ static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev,
err_data->ce_count += sec_count;
err_data->ue_count += ded_count;
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
gfx_v9_0_query_utc_edc_status(adev, err_data);
@@ -6707,6 +6884,62 @@ static const struct amdgpu_ring_funcs gfx_v9_0_ring_funcs_gfx = {
.emit_cntxcntl = gfx_v9_ring_emit_cntxcntl,
.init_cond_exec = gfx_v9_0_ring_emit_init_cond_exec,
.patch_cond_exec = gfx_v9_0_ring_emit_patch_cond_exec,
+ .preempt_ib = gfx_v9_0_ring_preempt_ib,
+ .emit_frame_cntl = gfx_v9_0_ring_emit_frame_cntl,
+ .emit_wreg = gfx_v9_0_ring_emit_wreg,
+ .emit_reg_wait = gfx_v9_0_ring_emit_reg_wait,
+ .emit_reg_write_reg_wait = gfx_v9_0_ring_emit_reg_write_reg_wait,
+ .soft_recovery = gfx_v9_0_ring_soft_recovery,
+ .emit_mem_sync = gfx_v9_0_emit_mem_sync,
+};
+
+static const struct amdgpu_ring_funcs gfx_v9_0_sw_ring_funcs_gfx = {
+ .type = AMDGPU_RING_TYPE_GFX,
+ .align_mask = 0xff,
+ .nop = PACKET3(PACKET3_NOP, 0x3FFF),
+ .support_64bit_ptrs = true,
+ .secure_submission_supported = true,
+ .vmhub = AMDGPU_GFXHUB_0,
+ .get_rptr = amdgpu_sw_ring_get_rptr_gfx,
+ .get_wptr = amdgpu_sw_ring_get_wptr_gfx,
+ .set_wptr = amdgpu_sw_ring_set_wptr_gfx,
+ .emit_frame_size = /* totally 242 maximum if 16 IBs */
+ 5 + /* COND_EXEC */
+ 7 + /* PIPELINE_SYNC */
+ SOC15_FLUSH_GPU_TLB_NUM_WREG * 5 +
+ SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 7 +
+ 2 + /* VM_FLUSH */
+ 8 + /* FENCE for VM_FLUSH */
+ 20 + /* GDS switch */
+ 4 + /* double SWITCH_BUFFER,
+ * the first COND_EXEC jump to the place just
+ * prior to this double SWITCH_BUFFER
+ */
+ 5 + /* COND_EXEC */
+ 7 + /* HDP_flush */
+ 4 + /* VGT_flush */
+ 14 + /* CE_META */
+ 31 + /* DE_META */
+ 3 + /* CNTX_CTRL */
+ 5 + /* HDP_INVL */
+ 8 + 8 + /* FENCE x2 */
+ 2 + /* SWITCH_BUFFER */
+ 7, /* gfx_v9_0_emit_mem_sync */
+ .emit_ib_size = 4, /* gfx_v9_0_ring_emit_ib_gfx */
+ .emit_ib = gfx_v9_0_ring_emit_ib_gfx,
+ .emit_fence = gfx_v9_0_ring_emit_fence,
+ .emit_pipeline_sync = gfx_v9_0_ring_emit_pipeline_sync,
+ .emit_vm_flush = gfx_v9_0_ring_emit_vm_flush,
+ .emit_gds_switch = gfx_v9_0_ring_emit_gds_switch,
+ .emit_hdp_flush = gfx_v9_0_ring_emit_hdp_flush,
+ .test_ring = gfx_v9_0_ring_test_ring,
+ .test_ib = gfx_v9_0_ring_test_ib,
+ .insert_nop = amdgpu_sw_ring_insert_nop,
+ .pad_ib = amdgpu_ring_generic_pad_ib,
+ .emit_switch_buffer = gfx_v9_ring_emit_sb,
+ .emit_cntxcntl = gfx_v9_ring_emit_cntxcntl,
+ .init_cond_exec = gfx_v9_0_ring_emit_init_cond_exec,
+ .patch_cond_exec = gfx_v9_0_ring_emit_patch_cond_exec,
.emit_frame_cntl = gfx_v9_0_ring_emit_frame_cntl,
.emit_wreg = gfx_v9_0_ring_emit_wreg,
.emit_reg_wait = gfx_v9_0_ring_emit_reg_wait,
@@ -6792,6 +7025,11 @@ static void gfx_v9_0_set_ring_funcs(struct amdgpu_device *adev)
for (i = 0; i < adev->gfx.num_gfx_rings; i++)
adev->gfx.gfx_ring[i].funcs = &gfx_v9_0_ring_funcs_gfx;
+ if (adev->gfx.num_gfx_rings) {
+ for (i = 0; i < GFX9_NUM_SW_GFX_RINGS; i++)
+ adev->gfx.sw_gfx_ring[i].funcs = &gfx_v9_0_sw_ring_funcs_gfx;
+ }
+
for (i = 0; i < adev->gfx.num_compute_rings; i++)
adev->gfx.compute_ring[i].funcs = &gfx_v9_0_ring_funcs_compute;
}
@@ -6963,7 +7201,7 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev,
mask = 1;
ao_bitmap = 0;
counter = 0;
- gfx_v9_0_select_se_sh(adev, i, j, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, i, j, 0xffffffff);
gfx_v9_0_set_user_cu_inactive_bitmap(
adev, disable_masks[i * adev->gfx.config.max_sh_per_se + j]);
bitmap = gfx_v9_0_get_cu_active_bitmap(adev);
@@ -6996,7 +7234,7 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev,
cu_info->ao_cu_bitmap[i % 4][j + i / 4] = ao_bitmap;
}
}
- gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
cu_info->number = active_cu_number;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
index 8cf53e0..3f8676d 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
@@ -397,6 +397,9 @@ static void gfxhub_v2_1_gart_disable(struct amdgpu_device *adev)
ENABLE_ADVANCED_DRIVER_MODEL, 0);
WREG32_SOC15(GC, 0, mmGCMC_VM_MX_L1_TLB_CNTL, tmp);
+ if (amdgpu_sriov_vf(adev))
+ return;
+
/* Setup L2 cache */
WREG32_FIELD15(GC, 0, GCVM_L2_CNTL, ENABLE_L2_CACHE, 0);
WREG32_SOC15(GC, 0, mmGCVM_L2_CNTL3, 0);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
index 5d3fffd..080ff11 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
@@ -154,6 +154,9 @@ static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)
{
uint64_t value;
+ if (amdgpu_sriov_vf(adev))
+ return;
+
/* Disable AGP. */
WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
WREG32_SOC15(GC, 0, regGCMC_VM_AGP_TOP, 0);
@@ -354,18 +357,6 @@ static void gfxhub_v3_0_3_program_invalidation(struct amdgpu_device *adev)
static int gfxhub_v3_0_3_gart_enable(struct amdgpu_device *adev)
{
- if (amdgpu_sriov_vf(adev)) {
- /*
- * GCMC_VM_FB_LOCATION_BASE/TOP is NULL for VF, becuase they are
- * VF copy registers so vbios post doesn't program them, for
- * SRIOV driver need to program them
- */
- WREG32_SOC15(GC, 0, regGCMC_VM_FB_LOCATION_BASE,
- adev->gmc.vram_start >> 24);
- WREG32_SOC15(GC, 0, regGCMC_VM_FB_LOCATION_TOP,
- adev->gmc.vram_end >> 24);
- }
-
/* GART Enable. */
gfxhub_v3_0_3_init_gart_aperture_regs(adev);
gfxhub_v3_0_3_init_system_aperture_regs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
index f513e2c2..21e4681 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
@@ -371,7 +371,9 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
* translation. Avoid this by doing the invalidation from the SDMA
* itself.
*/
- r = amdgpu_job_alloc_with_ib(adev, 16 * 4, AMDGPU_IB_POOL_IMMEDIATE,
+ r = amdgpu_job_alloc_with_ib(ring->adev, &adev->mman.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ 16 * 4, AMDGPU_IB_POOL_IMMEDIATE,
&job);
if (r)
goto error_alloc;
@@ -380,10 +382,7 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
job->vm_needs_flush = true;
job->ibs->ptr[job->ibs->length_dw++] = ring->funcs->nop;
amdgpu_ring_pad_ib(ring, &job->ibs[0]);
- r = amdgpu_job_submit(job, &adev->mman.entity,
- AMDGPU_FENCE_OWNER_UNDEFINED, &fence);
- if (r)
- goto error_submit;
+ fence = amdgpu_job_submit(job);
mutex_unlock(&adev->mman.gtt_window_lock);
@@ -392,9 +391,6 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
return;
-error_submit:
- amdgpu_job_free(job);
-
error_alloc:
mutex_unlock(&adev->mman.gtt_window_lock);
DRM_ERROR("Error flushing GPU TLB using the SDMA (%d)!\n", r);
@@ -612,6 +608,8 @@ static void gmc_v10_0_get_vm_pte(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping,
uint64_t *flags)
{
+ struct amdgpu_bo *bo = mapping->bo_va->base.bo;
+
*flags &= ~AMDGPU_PTE_EXECUTABLE;
*flags |= mapping->flags & AMDGPU_PTE_EXECUTABLE;
@@ -628,6 +626,11 @@ static void gmc_v10_0_get_vm_pte(struct amdgpu_device *adev,
*flags |= AMDGPU_PTE_SYSTEM;
*flags &= ~AMDGPU_PTE_VALID;
}
+
+ if (bo && bo->flags & (AMDGPU_GEM_CREATE_COHERENT |
+ AMDGPU_GEM_CREATE_UNCACHED))
+ *flags = (*flags & ~AMDGPU_PTE_MTYPE_NV10_MASK) |
+ AMDGPU_PTE_MTYPE_NV10(MTYPE_UC);
}
static unsigned gmc_v10_0_get_vbios_fb_size(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
index 66dfb57..4326078 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
@@ -503,6 +503,8 @@ static void gmc_v11_0_get_vm_pte(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping,
uint64_t *flags)
{
+ struct amdgpu_bo *bo = mapping->bo_va->base.bo;
+
*flags &= ~AMDGPU_PTE_EXECUTABLE;
*flags |= mapping->flags & AMDGPU_PTE_EXECUTABLE;
@@ -519,6 +521,11 @@ static void gmc_v11_0_get_vm_pte(struct amdgpu_device *adev,
*flags |= AMDGPU_PTE_SYSTEM;
*flags &= ~AMDGPU_PTE_VALID;
}
+
+ if (bo && bo->flags & (AMDGPU_GEM_CREATE_COHERENT |
+ AMDGPU_GEM_CREATE_UNCACHED))
+ *flags = (*flags & ~AMDGPU_PTE_MTYPE_NV10_MASK) |
+ AMDGPU_PTE_MTYPE_NV10(MTYPE_UC);
}
static unsigned gmc_v11_0_get_vbios_fb_size(struct amdgpu_device *adev)
@@ -551,7 +558,10 @@ static void gmc_v11_0_set_umc_funcs(struct amdgpu_device *adev)
adev->umc.node_inst_num = adev->gmc.num_umc;
adev->umc.max_ras_err_cnt_per_query = UMC_V8_10_TOTAL_CHANNEL_NUM(adev);
adev->umc.channel_offs = UMC_V8_10_PER_CHANNEL_OFFSET;
- adev->umc.channel_idx_tbl = &umc_v8_10_channel_idx_tbl[0][0][0];
+ if (adev->umc.node_inst_num == 4)
+ adev->umc.channel_idx_tbl = &umc_v8_10_channel_idx_tbl_ext0[0][0][0];
+ else
+ adev->umc.channel_idx_tbl = &umc_v8_10_channel_idx_tbl[0][0][0];
adev->umc.ras = &umc_v8_10_ras;
break;
case IP_VERSION(8, 11, 0):
@@ -749,6 +759,7 @@ static int gmc_v11_0_sw_init(void *handle)
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
adev->num_vmhubs = 2;
/*
* To fulfill 4-level page support,
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index 67ca16a..50386eb 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -1113,6 +1113,74 @@ static void gmc_v9_0_get_vm_pde(struct amdgpu_device *adev, int level,
}
}
+static void gmc_v9_0_get_coherence_flags(struct amdgpu_device *adev,
+ struct amdgpu_bo *bo,
+ struct amdgpu_bo_va_mapping *mapping,
+ uint64_t *flags)
+{
+ struct amdgpu_device *bo_adev = amdgpu_ttm_adev(bo->tbo.bdev);
+ bool is_vram = bo->tbo.resource->mem_type == TTM_PL_VRAM;
+ bool coherent = bo->flags & AMDGPU_GEM_CREATE_COHERENT;
+ bool uncached = bo->flags & AMDGPU_GEM_CREATE_UNCACHED;
+ unsigned int mtype;
+ bool snoop = false;
+
+ switch (adev->ip_versions[GC_HWIP][0]) {
+ case IP_VERSION(9, 4, 1):
+ case IP_VERSION(9, 4, 2):
+ if (is_vram) {
+ if (bo_adev == adev) {
+ if (uncached)
+ mtype = MTYPE_UC;
+ else if (coherent)
+ mtype = MTYPE_CC;
+ else
+ mtype = MTYPE_RW;
+ /* FIXME: is this still needed? Or does
+ * amdgpu_ttm_tt_pde_flags already handle this?
+ */
+ if (adev->ip_versions[GC_HWIP][0] ==
+ IP_VERSION(9, 4, 2) &&
+ adev->gmc.xgmi.connected_to_cpu)
+ snoop = true;
+ } else {
+ if (uncached || coherent)
+ mtype = MTYPE_UC;
+ else
+ mtype = MTYPE_NC;
+ if (mapping->bo_va->is_xgmi)
+ snoop = true;
+ }
+ } else {
+ if (uncached || coherent)
+ mtype = MTYPE_UC;
+ else
+ mtype = MTYPE_NC;
+ /* FIXME: is this still needed? Or does
+ * amdgpu_ttm_tt_pde_flags already handle this?
+ */
+ snoop = true;
+ }
+ break;
+ default:
+ if (uncached || coherent)
+ mtype = MTYPE_UC;
+ else
+ mtype = MTYPE_NC;
+
+ /* FIXME: is this still needed? Or does
+ * amdgpu_ttm_tt_pde_flags already handle this?
+ */
+ if (!is_vram)
+ snoop = true;
+ }
+
+ if (mtype != MTYPE_NC)
+ *flags = (*flags & ~AMDGPU_PTE_MTYPE_VG10_MASK) |
+ AMDGPU_PTE_MTYPE_VG10(mtype);
+ *flags |= snoop ? AMDGPU_PTE_SNOOPED : 0;
+}
+
static void gmc_v9_0_get_vm_pte(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping,
uint64_t *flags)
@@ -1128,14 +1196,9 @@ static void gmc_v9_0_get_vm_pte(struct amdgpu_device *adev,
*flags &= ~AMDGPU_PTE_VALID;
}
- if ((adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 1) ||
- adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 2)) &&
- !(*flags & AMDGPU_PTE_SYSTEM) &&
- mapping->bo_va->is_xgmi)
- *flags |= AMDGPU_PTE_SNOOPED;
-
- if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(9, 4, 2))
- *flags |= mapping->flags & AMDGPU_PTE_SNOOPED;
+ if (mapping->bo_va->base.bo)
+ gmc_v9_0_get_coherence_flags(adev, mapping->bo_va->base.bo,
+ mapping, flags);
}
static unsigned gmc_v9_0_get_vbios_fb_size(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
index f87d0f6..f2b743a 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
@@ -807,16 +807,5 @@ static void jpeg_v2_5_set_ras_funcs(struct amdgpu_device *adev)
break;
}
- if (adev->jpeg.ras) {
- amdgpu_ras_register_ras_block(adev, &adev->jpeg.ras->ras_block);
-
- strcpy(adev->jpeg.ras->ras_block.ras_comm.name, "jpeg");
- adev->jpeg.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__JPEG;
- adev->jpeg.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__POISON;
- adev->jpeg.ras_if = &adev->jpeg.ras->ras_block.ras_comm;
-
- /* If don't define special ras_late_init function, use default ras_late_init */
- if (!adev->jpeg.ras->ras_block.ras_late_init)
- adev->jpeg.ras->ras_block.ras_late_init = amdgpu_ras_block_late_init;
- }
+ jpeg_set_ras_funcs(adev);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c
index 63b0d0b..3beb731 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c
@@ -27,6 +27,7 @@
#include "soc15.h"
#include "soc15d.h"
#include "jpeg_v2_0.h"
+#include "jpeg_v4_0.h"
#include "vcn/vcn_4_0_0_offset.h"
#include "vcn/vcn_4_0_0_sh_mask.h"
@@ -38,6 +39,7 @@ static void jpeg_v4_0_set_dec_ring_funcs(struct amdgpu_device *adev);
static void jpeg_v4_0_set_irq_funcs(struct amdgpu_device *adev);
static int jpeg_v4_0_set_powergating_state(void *handle,
enum amd_powergating_state state);
+static void jpeg_v4_0_set_ras_funcs(struct amdgpu_device *adev);
/**
* jpeg_v4_0_early_init - set function pointers
@@ -55,6 +57,7 @@ static int jpeg_v4_0_early_init(void *handle)
jpeg_v4_0_set_dec_ring_funcs(adev);
jpeg_v4_0_set_irq_funcs(adev);
+ jpeg_v4_0_set_ras_funcs(adev);
return 0;
}
@@ -78,6 +81,18 @@ static int jpeg_v4_0_sw_init(void *handle)
if (r)
return r;
+ /* JPEG DJPEG POISON EVENT */
+ r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_VCN,
+ VCN_4_0__SRCID_DJPEG0_POISON, &adev->jpeg.inst->irq);
+ if (r)
+ return r;
+
+ /* JPEG EJPEG POISON EVENT */
+ r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_VCN,
+ VCN_4_0__SRCID_EJPEG0_POISON, &adev->jpeg.inst->irq);
+ if (r)
+ return r;
+
r = amdgpu_jpeg_sw_init(adev);
if (r)
return r;
@@ -167,6 +182,8 @@ static int jpeg_v4_0_hw_fini(void *handle)
RREG32_SOC15(JPEG, 0, regUVD_JRBC_STATUS))
jpeg_v4_0_set_powergating_state(adev, AMD_PG_STATE_GATE);
+ amdgpu_irq_put(adev, &adev->jpeg.inst->irq, 0);
+
return 0;
}
@@ -524,6 +541,10 @@ static int jpeg_v4_0_process_interrupt(struct amdgpu_device *adev,
case VCN_4_0__SRCID__JPEG_DECODE:
amdgpu_fence_process(&adev->jpeg.inst->ring_dec);
break;
+ case VCN_4_0__SRCID_DJPEG0_POISON:
+ case VCN_4_0__SRCID_EJPEG0_POISON:
+ amdgpu_jpeg_process_poison_irq(adev, source, entry);
+ break;
default:
DRM_DEV_ERROR(adev->dev, "Unhandled interrupt: %d %d\n",
entry->src_id, entry->src_data[0]);
@@ -607,3 +628,63 @@ const struct amdgpu_ip_block_version jpeg_v4_0_ip_block = {
.rev = 0,
.funcs = &jpeg_v4_0_ip_funcs,
};
+
+static uint32_t jpeg_v4_0_query_poison_by_instance(struct amdgpu_device *adev,
+ uint32_t instance, uint32_t sub_block)
+{
+ uint32_t poison_stat = 0, reg_value = 0;
+
+ switch (sub_block) {
+ case AMDGPU_JPEG_V4_0_JPEG0:
+ reg_value = RREG32_SOC15(JPEG, instance, regUVD_RAS_JPEG0_STATUS);
+ poison_stat = REG_GET_FIELD(reg_value, UVD_RAS_JPEG0_STATUS, POISONED_PF);
+ break;
+ case AMDGPU_JPEG_V4_0_JPEG1:
+ reg_value = RREG32_SOC15(JPEG, instance, regUVD_RAS_JPEG1_STATUS);
+ poison_stat = REG_GET_FIELD(reg_value, UVD_RAS_JPEG1_STATUS, POISONED_PF);
+ break;
+ default:
+ break;
+ }
+
+ if (poison_stat)
+ dev_info(adev->dev, "Poison detected in JPEG%d sub_block%d\n",
+ instance, sub_block);
+
+ return poison_stat;
+}
+
+static bool jpeg_v4_0_query_ras_poison_status(struct amdgpu_device *adev)
+{
+ uint32_t inst = 0, sub = 0, poison_stat = 0;
+
+ for (inst = 0; inst < adev->jpeg.num_jpeg_inst; inst++)
+ for (sub = 0; sub < AMDGPU_JPEG_V4_0_MAX_SUB_BLOCK; sub++)
+ poison_stat +=
+ jpeg_v4_0_query_poison_by_instance(adev, inst, sub);
+
+ return !!poison_stat;
+}
+
+const struct amdgpu_ras_block_hw_ops jpeg_v4_0_ras_hw_ops = {
+ .query_poison_status = jpeg_v4_0_query_ras_poison_status,
+};
+
+static struct amdgpu_jpeg_ras jpeg_v4_0_ras = {
+ .ras_block = {
+ .hw_ops = &jpeg_v4_0_ras_hw_ops,
+ },
+};
+
+static void jpeg_v4_0_set_ras_funcs(struct amdgpu_device *adev)
+{
+ switch (adev->ip_versions[JPEG_HWIP][0]) {
+ case IP_VERSION(4, 0, 0):
+ adev->jpeg.ras = &jpeg_v4_0_ras;
+ break;
+ default:
+ break;
+ }
+
+ jpeg_set_ras_funcs(adev);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.h b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.h
index f1ed6cc..07d36c2 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.h
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.h
@@ -24,6 +24,13 @@
#ifndef __JPEG_V4_0_H__
#define __JPEG_V4_0_H__
+enum amdgpu_jpeg_v4_0_sub_block {
+ AMDGPU_JPEG_V4_0_JPEG0 = 0,
+ AMDGPU_JPEG_V4_0_JPEG1,
+
+ AMDGPU_JPEG_V4_0_MAX_SUB_BLOCK,
+};
+
extern const struct amdgpu_ip_block_version jpeg_v4_0_ip_block;
#endif /* __JPEG_V4_0_H__ */
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
index 067d100..6143941 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
@@ -121,6 +121,10 @@ static int mes_v10_1_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
if (r < 1) {
DRM_ERROR("MES failed to response msg=%d\n",
x_pkt->header.opcode);
+
+ while (halt_if_hws_hang)
+ schedule();
+
return -ETIMEDOUT;
}
@@ -415,10 +419,6 @@ static int mes_v10_1_init_microcode(struct amdgpu_device *adev,
mes_hdr = (const struct mes_firmware_header_v1_0 *)
adev->mes.fw[pipe]->data;
- adev->mes.ucode_fw_version[pipe] =
- le32_to_cpu(mes_hdr->mes_ucode_version);
- adev->mes.ucode_fw_version[pipe] =
- le32_to_cpu(mes_hdr->mes_ucode_data_version);
adev->mes.uc_start_addr[pipe] =
le32_to_cpu(mes_hdr->mes_uc_start_addr_lo) |
((uint64_t)(le32_to_cpu(mes_hdr->mes_uc_start_addr_hi)) << 32);
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index f141fad..5459366 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -129,6 +129,10 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
if (r < 1) {
DRM_ERROR("MES failed to response msg=%d\n",
x_pkt->header.opcode);
+
+ while (halt_if_hws_hang)
+ schedule();
+
return -ETIMEDOUT;
}
@@ -384,6 +388,7 @@ static int mes_v11_0_set_hw_resources(struct amdgpu_mes *mes)
mes_set_hw_res_pkt.disable_reset = 1;
mes_set_hw_res_pkt.disable_mes_log = 1;
mes_set_hw_res_pkt.use_different_vmid_compute = 1;
+ mes_set_hw_res_pkt.enable_reg_active_poll = 1;
mes_set_hw_res_pkt.oversubscription_timer = 50;
return mes_v11_0_submit_pkt_and_poll_completion(mes,
@@ -485,10 +490,6 @@ static int mes_v11_0_init_microcode(struct amdgpu_device *adev,
mes_hdr = (const struct mes_firmware_header_v1_0 *)
adev->mes.fw[pipe]->data;
- adev->mes.ucode_fw_version[pipe] =
- le32_to_cpu(mes_hdr->mes_ucode_version);
- adev->mes.ucode_fw_version[pipe] =
- le32_to_cpu(mes_hdr->mes_ucode_data_version);
adev->mes.uc_start_addr[pipe] =
le32_to_cpu(mes_hdr->mes_uc_start_addr_lo) |
((uint64_t)(le32_to_cpu(mes_hdr->mes_uc_start_addr_hi)) << 32);
@@ -1253,7 +1254,9 @@ static int mes_v11_0_kiq_hw_fini(struct amdgpu_device *adev)
if (adev->mes.ring.sched.ready)
mes_v11_0_kiq_dequeue_sched(adev);
- mes_v11_0_enable(adev, false);
+ if (!amdgpu_sriov_vf(adev))
+ mes_v11_0_enable(adev, false);
+
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
index 998b5d1..0e664d0 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
@@ -319,7 +319,7 @@ static void mmhub_v2_0_init_cache_regs(struct amdgpu_device *adev)
tmp = mmMMVM_L2_CNTL5_DEFAULT;
tmp = REG_SET_FIELD(tmp, MMVM_L2_CNTL5, L2_CACHE_SMALLK_FRAGMENT_SIZE, 0);
- WREG32_SOC15(GC, 0, mmMMVM_L2_CNTL5, tmp);
+ WREG32_SOC15(MMHUB, 0, mmMMVM_L2_CNTL5, tmp);
}
static void mmhub_v2_0_enable_system_domain(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c
index 1b027d0..4638ea7 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c
@@ -243,7 +243,7 @@ static void mmhub_v2_3_init_cache_regs(struct amdgpu_device *adev)
tmp = mmMMVM_L2_CNTL5_DEFAULT;
tmp = REG_SET_FIELD(tmp, MMVM_L2_CNTL5, L2_CACHE_SMALLK_FRAGMENT_SIZE, 0);
- WREG32_SOC15(GC, 0, mmMMVM_L2_CNTL5, tmp);
+ WREG32_SOC15(MMHUB, 0, mmMMVM_L2_CNTL5, tmp);
}
static void mmhub_v2_3_enable_system_domain(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0.c
index a1d26c4..16cc822 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0.c
@@ -275,7 +275,7 @@ static void mmhub_v3_0_init_cache_regs(struct amdgpu_device *adev)
tmp = regMMVM_L2_CNTL5_DEFAULT;
tmp = REG_SET_FIELD(tmp, MMVM_L2_CNTL5, L2_CACHE_SMALLK_FRAGMENT_SIZE, 0);
- WREG32_SOC15(GC, 0, regMMVM_L2_CNTL5, tmp);
+ WREG32_SOC15(MMHUB, 0, regMMVM_L2_CNTL5, tmp);
}
static void mmhub_v3_0_enable_system_domain(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_1.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_1.c
index e8058ed..6bdf2ef 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_1.c
@@ -269,7 +269,7 @@ static void mmhub_v3_0_1_init_cache_regs(struct amdgpu_device *adev)
tmp = regMMVM_L2_CNTL5_DEFAULT;
tmp = REG_SET_FIELD(tmp, MMVM_L2_CNTL5, L2_CACHE_SMALLK_FRAGMENT_SIZE, 0);
- WREG32_SOC15(GC, 0, regMMVM_L2_CNTL5, tmp);
+ WREG32_SOC15(MMHUB, 0, regMMVM_L2_CNTL5, tmp);
}
static void mmhub_v3_0_1_enable_system_domain(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_2.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_2.c
index 770be0a..45465ac 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_2.c
@@ -268,7 +268,7 @@ static void mmhub_v3_0_2_init_cache_regs(struct amdgpu_device *adev)
tmp = regMMVM_L2_CNTL5_DEFAULT;
tmp = REG_SET_FIELD(tmp, MMVM_L2_CNTL5, L2_CACHE_SMALLK_FRAGMENT_SIZE, 0);
- WREG32_SOC15(GC, 0, regMMVM_L2_CNTL5, tmp);
+ WREG32_SOC15(MMHUB, 0, regMMVM_L2_CNTL5, tmp);
}
static void mmhub_v3_0_2_enable_system_domain(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/nv.c b/drivers/gpu/drm/amd/amdgpu/nv.c
index b3fba8de..6853b93 100644
--- a/drivers/gpu/drm/amd/amdgpu/nv.c
+++ b/drivers/gpu/drm/amd/amdgpu/nv.c
@@ -82,10 +82,10 @@ static const struct amdgpu_video_codecs nv_video_codecs_encode =
/* Navi1x */
static const struct amdgpu_video_codec_info nv_video_codecs_decode_array[] =
{
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4906, 3)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4906, 5)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4906, 4)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4096, 3)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4096, 5)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4096, 4)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 8192, 4352, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 8192, 4352, 0)},
@@ -100,10 +100,10 @@ static const struct amdgpu_video_codecs nv_video_codecs_decode =
/* Sienna Cichlid */
static const struct amdgpu_video_codec_info sc_video_codecs_decode_array[] =
{
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4906, 3)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4906, 5)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4906, 4)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4096, 3)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4096, 5)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4096, 4)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 8192, 4352, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 8192, 4352, 0)},
@@ -125,10 +125,10 @@ static struct amdgpu_video_codec_info sriov_sc_video_codecs_encode_array[] =
static struct amdgpu_video_codec_info sriov_sc_video_codecs_decode_array[] =
{
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4906, 3)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4906, 5)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4906, 4)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4096, 3)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4096, 5)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4096, 4)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 8192, 4352, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 8192, 4352, 0)},
@@ -149,7 +149,7 @@ static struct amdgpu_video_codecs sriov_sc_video_codecs_decode =
/* Beige Goby*/
static const struct amdgpu_video_codec_info bg_video_codecs_decode_array[] = {
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 8192, 4352, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 8192, 4352, 0)},
};
@@ -166,7 +166,7 @@ static const struct amdgpu_video_codecs bg_video_codecs_encode = {
/* Yellow Carp*/
static const struct amdgpu_video_codec_info yc_video_codecs_decode_array[] = {
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 8192, 4352, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 8192, 4352, 0)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v10_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v10_0.c
index ed22936..9de46fa 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v10_0.c
@@ -126,32 +126,6 @@ static int psp_v10_0_init_microcode(struct psp_context *psp)
return err;
}
-static int psp_v10_0_ring_init(struct psp_context *psp,
- enum psp_ring_type ring_type)
-{
- int ret = 0;
- struct psp_ring *ring;
- struct amdgpu_device *adev = psp->adev;
-
- ring = &psp->km_ring;
-
- ring->ring_type = ring_type;
-
- /* allocate 4k Page of Local Frame Buffer memory for ring */
- ring->ring_size = 0x1000;
- ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_VRAM,
- &adev->firmware.rbuf,
- &ring->ring_mem_mc_addr,
- (void **)&ring->ring_mem);
- if (ret) {
- ring->ring_size = 0;
- return ret;
- }
-
- return 0;
-}
-
static int psp_v10_0_ring_create(struct psp_context *psp,
enum psp_ring_type ring_type)
{
@@ -245,7 +219,6 @@ static void psp_v10_0_ring_set_wptr(struct psp_context *psp, uint32_t value)
static const struct psp_funcs psp_v10_0_funcs = {
.init_microcode = psp_v10_0_init_microcode,
- .ring_init = psp_v10_0_ring_init,
.ring_create = psp_v10_0_ring_create,
.ring_stop = psp_v10_0_ring_stop,
.ring_destroy = psp_v10_0_ring_destroy,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
index 9518b43..bd3e3e2 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
@@ -360,32 +360,6 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp)
return ret;
}
-static int psp_v11_0_ring_init(struct psp_context *psp,
- enum psp_ring_type ring_type)
-{
- int ret = 0;
- struct psp_ring *ring;
- struct amdgpu_device *adev = psp->adev;
-
- ring = &psp->km_ring;
-
- ring->ring_type = ring_type;
-
- /* allocate 4k Page of Local Frame Buffer memory for ring */
- ring->ring_size = 0x1000;
- ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_VRAM,
- &adev->firmware.rbuf,
- &ring->ring_mem_mc_addr,
- (void **)&ring->ring_mem);
- if (ret) {
- ring->ring_size = 0;
- return ret;
- }
-
- return 0;
-}
-
static int psp_v11_0_ring_stop(struct psp_context *psp,
enum psp_ring_type ring_type)
{
@@ -779,7 +753,6 @@ static const struct psp_funcs psp_v11_0_funcs = {
.bootloader_load_spl = psp_v11_0_bootloader_load_spl,
.bootloader_load_sysdrv = psp_v11_0_bootloader_load_sysdrv,
.bootloader_load_sos = psp_v11_0_bootloader_load_sos,
- .ring_init = psp_v11_0_ring_init,
.ring_create = psp_v11_0_ring_create,
.ring_stop = psp_v11_0_ring_stop,
.ring_destroy = psp_v11_0_ring_destroy,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0_8.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0_8.c
index ff13e1b..5697760 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0_8.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0_8.c
@@ -28,32 +28,6 @@
#include "mp/mp_11_0_8_offset.h"
-static int psp_v11_0_8_ring_init(struct psp_context *psp,
- enum psp_ring_type ring_type)
-{
- int ret = 0;
- struct psp_ring *ring;
- struct amdgpu_device *adev = psp->adev;
-
- ring = &psp->km_ring;
-
- ring->ring_type = ring_type;
-
- /* allocate 4k Page of Local Frame Buffer memory for ring */
- ring->ring_size = 0x1000;
- ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_VRAM,
- &adev->firmware.rbuf,
- &ring->ring_mem_mc_addr,
- (void **)&ring->ring_mem);
- if (ret) {
- ring->ring_size = 0;
- return ret;
- }
-
- return 0;
-}
-
static int psp_v11_0_8_ring_stop(struct psp_context *psp,
enum psp_ring_type ring_type)
{
@@ -194,7 +168,6 @@ static void psp_v11_0_8_ring_set_wptr(struct psp_context *psp, uint32_t value)
}
static const struct psp_funcs psp_v11_0_8_funcs = {
- .ring_init = psp_v11_0_8_ring_init,
.ring_create = psp_v11_0_8_ring_create,
.ring_stop = psp_v11_0_8_ring_stop,
.ring_destroy = psp_v11_0_8_ring_destroy,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
index 0b2ac418..8ed2281b 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
@@ -236,34 +236,6 @@ static void psp_v12_0_reroute_ih(struct psp_context *psp)
0x80000000, 0x8000FFFF, false);
}
-static int psp_v12_0_ring_init(struct psp_context *psp,
- enum psp_ring_type ring_type)
-{
- int ret = 0;
- struct psp_ring *ring;
- struct amdgpu_device *adev = psp->adev;
-
- psp_v12_0_reroute_ih(psp);
-
- ring = &psp->km_ring;
-
- ring->ring_type = ring_type;
-
- /* allocate 4k Page of Local Frame Buffer memory for ring */
- ring->ring_size = 0x1000;
- ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_VRAM,
- &adev->firmware.rbuf,
- &ring->ring_mem_mc_addr,
- (void **)&ring->ring_mem);
- if (ret) {
- ring->ring_size = 0;
- return ret;
- }
-
- return 0;
-}
-
static int psp_v12_0_ring_create(struct psp_context *psp,
enum psp_ring_type ring_type)
{
@@ -272,6 +244,8 @@ static int psp_v12_0_ring_create(struct psp_context *psp,
struct psp_ring *ring = &psp->km_ring;
struct amdgpu_device *adev = psp->adev;
+ psp_v12_0_reroute_ih(psp);
+
if (amdgpu_sriov_vf(psp->adev)) {
/* Write low address of the ring to C2PMSG_102 */
psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr);
@@ -425,7 +399,6 @@ static const struct psp_funcs psp_v12_0_funcs = {
.init_microcode = psp_v12_0_init_microcode,
.bootloader_load_sysdrv = psp_v12_0_bootloader_load_sysdrv,
.bootloader_load_sos = psp_v12_0_bootloader_load_sos,
- .ring_init = psp_v12_0_ring_init,
.ring_create = psp_v12_0_ring_create,
.ring_stop = psp_v12_0_ring_stop,
.ring_destroy = psp_v12_0_ring_destroy,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v13_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v13_0.c
index 88f9b32..e6a26a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v13_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v13_0.c
@@ -46,6 +46,8 @@ MODULE_FIRMWARE("amdgpu/psp_13_0_7_sos.bin");
MODULE_FIRMWARE("amdgpu/psp_13_0_7_ta.bin");
MODULE_FIRMWARE("amdgpu/psp_13_0_10_sos.bin");
MODULE_FIRMWARE("amdgpu/psp_13_0_10_ta.bin");
+MODULE_FIRMWARE("amdgpu/psp_13_0_11_toc.bin");
+MODULE_FIRMWARE("amdgpu/psp_13_0_11_ta.bin");
/* For large FW files the time to complete can be very long */
#define USBC_PD_POLLING_LIMIT_S 240
@@ -102,6 +104,7 @@ static int psp_v13_0_init_microcode(struct psp_context *psp)
case IP_VERSION(13, 0, 3):
case IP_VERSION(13, 0, 5):
case IP_VERSION(13, 0, 8):
+ case IP_VERSION(13, 0, 11):
err = psp_init_toc_microcode(psp, chip_name);
if (err)
return err;
@@ -268,32 +271,6 @@ static int psp_v13_0_bootloader_load_sos(struct psp_context *psp)
return ret;
}
-static int psp_v13_0_ring_init(struct psp_context *psp,
- enum psp_ring_type ring_type)
-{
- int ret = 0;
- struct psp_ring *ring;
- struct amdgpu_device *adev = psp->adev;
-
- ring = &psp->km_ring;
-
- ring->ring_type = ring_type;
-
- /* allocate 4k Page of Local Frame Buffer memory for ring */
- ring->ring_size = 0x1000;
- ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_VRAM,
- &adev->firmware.rbuf,
- &ring->ring_mem_mc_addr,
- (void **)&ring->ring_mem);
- if (ret) {
- ring->ring_size = 0;
- return ret;
- }
-
- return 0;
-}
-
static int psp_v13_0_ring_stop(struct psp_context *psp,
enum psp_ring_type ring_type)
{
@@ -729,7 +706,6 @@ static const struct psp_funcs psp_v13_0_funcs = {
.bootloader_load_dbg_drv = psp_v13_0_bootloader_load_dbg_drv,
.bootloader_load_ras_drv = psp_v13_0_bootloader_load_ras_drv,
.bootloader_load_sos = psp_v13_0_bootloader_load_sos,
- .ring_init = psp_v13_0_ring_init,
.ring_create = psp_v13_0_ring_create,
.ring_stop = psp_v13_0_ring_stop,
.ring_destroy = psp_v13_0_ring_destroy,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v13_0_4.c b/drivers/gpu/drm/amd/amdgpu/psp_v13_0_4.c
index 321089d..9d4e24e 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v13_0_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v13_0_4.c
@@ -199,32 +199,6 @@ static int psp_v13_0_4_bootloader_load_sos(struct psp_context *psp)
return ret;
}
-static int psp_v13_0_4_ring_init(struct psp_context *psp,
- enum psp_ring_type ring_type)
-{
- int ret = 0;
- struct psp_ring *ring;
- struct amdgpu_device *adev = psp->adev;
-
- ring = &psp->km_ring;
-
- ring->ring_type = ring_type;
-
- /* allocate 4k Page of Local Frame Buffer memory for ring */
- ring->ring_size = 0x1000;
- ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_VRAM,
- &adev->firmware.rbuf,
- &ring->ring_mem_mc_addr,
- (void **)&ring->ring_mem);
- if (ret) {
- ring->ring_size = 0;
- return ret;
- }
-
- return 0;
-}
-
static int psp_v13_0_4_ring_stop(struct psp_context *psp,
enum psp_ring_type ring_type)
{
@@ -373,7 +347,6 @@ static const struct psp_funcs psp_v13_0_4_funcs = {
.bootloader_load_intf_drv = psp_v13_0_4_bootloader_load_intf_drv,
.bootloader_load_dbg_drv = psp_v13_0_4_bootloader_load_dbg_drv,
.bootloader_load_sos = psp_v13_0_4_bootloader_load_sos,
- .ring_init = psp_v13_0_4_ring_init,
.ring_create = psp_v13_0_4_ring_create,
.ring_stop = psp_v13_0_4_ring_stop,
.ring_destroy = psp_v13_0_4_ring_destroy,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
index 01f3bcc..157147c 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
@@ -160,32 +160,6 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp)
return ret;
}
-static int psp_v3_1_ring_init(struct psp_context *psp,
- enum psp_ring_type ring_type)
-{
- int ret = 0;
- struct psp_ring *ring;
- struct amdgpu_device *adev = psp->adev;
-
- ring = &psp->km_ring;
-
- ring->ring_type = ring_type;
-
- /* allocate 4k Page of Local Frame Buffer memory for ring */
- ring->ring_size = 0x1000;
- ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_VRAM,
- &adev->firmware.rbuf,
- &ring->ring_mem_mc_addr,
- (void **)&ring->ring_mem);
- if (ret) {
- ring->ring_size = 0;
- return ret;
- }
-
- return 0;
-}
-
static void psp_v3_1_reroute_ih(struct psp_context *psp)
{
struct amdgpu_device *adev = psp->adev;
@@ -401,7 +375,6 @@ static const struct psp_funcs psp_v3_1_funcs = {
.init_microcode = psp_v3_1_init_microcode,
.bootloader_load_sysdrv = psp_v3_1_bootloader_load_sysdrv,
.bootloader_load_sos = psp_v3_1_bootloader_load_sos,
- .ring_init = psp_v3_1_ring_init,
.ring_create = psp_v3_1_ring_create,
.ring_stop = psp_v3_1_ring_stop,
.ring_destroy = psp_v3_1_ring_destroy,
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c
index da3beb0..049c26a 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c
@@ -455,6 +455,9 @@ static void sdma_v6_0_enable(struct amdgpu_device *adev, bool enable)
sdma_v6_0_rlc_stop(adev);
}
+ if (amdgpu_sriov_vf(adev))
+ return;
+
for (i = 0; i < adev->sdma.num_instances; i++) {
f32_cntl = RREG32_SOC15_IP(GC, sdma_v6_0_get_reg_offset(adev, i, regSDMA0_F32_CNTL));
f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_F32_CNTL, HALT, enable ? 0 : 1);
@@ -1523,6 +1526,7 @@ static const struct amdgpu_ring_funcs sdma_v6_0_ring_funcs = {
.align_mask = 0xf,
.nop = SDMA_PKT_NOP_HEADER_OP(SDMA_OP_NOP),
.support_64bit_ptrs = true,
+ .secure_submission_supported = true,
.vmhub = AMDGPU_GFXHUB_0,
.get_rptr = sdma_v6_0_ring_get_rptr,
.get_wptr = sdma_v6_0_ring_get_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/si_dma.c b/drivers/gpu/drm/amd/amdgpu/si_dma.c
index 4d5e718..abca8b52 100644
--- a/drivers/gpu/drm/amd/amdgpu/si_dma.c
+++ b/drivers/gpu/drm/amd/amdgpu/si_dma.c
@@ -112,14 +112,12 @@ static void si_dma_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 seq,
static void si_dma_stop(struct amdgpu_device *adev)
{
- struct amdgpu_ring *ring;
u32 rb_cntl;
unsigned i;
amdgpu_sdma_unset_buffer_funcs_helper(adev);
for (i = 0; i < adev->sdma.num_instances; i++) {
- ring = &adev->sdma.instance[i].ring;
/* dma0 */
rb_cntl = RREG32(DMA_RB_CNTL + sdma_offsets[i]);
rb_cntl &= ~DMA_RB_ENABLE;
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c
index e3b2b6b..7cd17dd 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.c
@@ -103,10 +103,10 @@ static const struct amdgpu_video_codecs vega_video_codecs_encode =
/* Vega */
static const struct amdgpu_video_codec_info vega_video_codecs_decode_array[] =
{
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4906, 3)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4906, 5)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4906, 4)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4096, 3)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4096, 5)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4096, 4)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 4096, 4096, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
};
@@ -120,10 +120,10 @@ static const struct amdgpu_video_codecs vega_video_codecs_decode =
/* Raven */
static const struct amdgpu_video_codec_info rv_video_codecs_decode_array[] =
{
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4906, 3)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4906, 5)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4906, 4)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4096, 3)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4096, 5)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4096, 4)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 4096, 4096, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 4096, 4096, 0)},
@@ -138,10 +138,10 @@ static const struct amdgpu_video_codecs rv_video_codecs_decode =
/* Renoir, Arcturus */
static const struct amdgpu_video_codec_info rn_video_codecs_decode_array[] =
{
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4906, 3)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4906, 5)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4906, 4)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2, 4096, 4096, 3)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4, 4096, 4096, 5)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1, 4096, 4096, 4)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 8192, 4352, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 8192, 4352, 0)},
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15d.h b/drivers/gpu/drm/amd/amdgpu/soc15d.h
index 799925d..2357ff3 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15d.h
+++ b/drivers/gpu/drm/amd/amdgpu/soc15d.h
@@ -162,6 +162,7 @@
* 2 - Bypass
*/
#define INDIRECT_BUFFER_PRE_ENB(x) ((x) << 21)
+#define INDIRECT_BUFFER_PRE_RESUME(x) ((x) << 30)
#define PACKET3_COPY_DATA 0x40
#define PACKET3_PFP_SYNC_ME 0x42
#define PACKET3_COND_WRITE 0x45
@@ -184,6 +185,7 @@
#define EOP_TC_ACTION_EN (1 << 17) /* L2 */
#define EOP_TC_NC_ACTION_EN (1 << 19)
#define EOP_TC_MD_ACTION_EN (1 << 21) /* L2 metadata */
+#define EOP_EXEC (1 << 28) /* For Trailing Fence */
#define DATA_SEL(x) ((x) << 29)
/* 0 - discard
diff --git a/drivers/gpu/drm/amd/amdgpu/soc21.c b/drivers/gpu/drm/amd/amdgpu/soc21.c
index e080440..7d5fdf4 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc21.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc21.c
@@ -43,6 +43,7 @@
#include "soc15.h"
#include "soc15_common.h"
#include "soc21.h"
+#include "mxgpu_nv.h"
static const struct amd_ip_funcs soc21_common_ip_funcs;
@@ -61,7 +62,7 @@ static const struct amdgpu_video_codecs vcn_4_0_0_video_codecs_encode =
static const struct amdgpu_video_codec_info vcn_4_0_0_video_codecs_decode_array[] =
{
- {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4906, 52)},
+ {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 4096, 52)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 8192, 4352, 186)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG, 4096, 4096, 0)},
{codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9, 8192, 4352, 0)},
@@ -322,8 +323,10 @@ soc21_asic_reset_method(struct amdgpu_device *adev)
switch (adev->ip_versions[MP1_HWIP][0]) {
case IP_VERSION(13, 0, 0):
case IP_VERSION(13, 0, 7):
+ case IP_VERSION(13, 0, 10):
return AMD_RESET_METHOD_MODE1;
case IP_VERSION(13, 0, 4):
+ case IP_VERSION(13, 0, 11):
return AMD_RESET_METHOD_MODE2;
default:
if (amdgpu_dpm_is_baco_supported(adev))
@@ -584,10 +587,6 @@ static int soc21_common_early_init(void *handle)
AMD_PG_SUPPORT_JPEG |
AMD_PG_SUPPORT_ATHUB |
AMD_PG_SUPPORT_MMHUB;
- if (amdgpu_sriov_vf(adev)) {
- adev->cg_flags = 0;
- adev->pg_flags = 0;
- }
adev->external_rev_id = adev->rev_id + 0x1; // TODO: need update
break;
case IP_VERSION(11, 0, 2):
@@ -645,28 +644,63 @@ static int soc21_common_early_init(void *handle)
adev->pg_flags = AMD_PG_SUPPORT_VCN |
AMD_PG_SUPPORT_VCN_DPG |
AMD_PG_SUPPORT_JPEG;
- if (amdgpu_sriov_vf(adev)) {
- /* hypervisor control CG and PG enablement */
- adev->cg_flags = 0;
- adev->pg_flags = 0;
- }
adev->external_rev_id = adev->rev_id + 0x20;
break;
+ case IP_VERSION(11, 0, 4):
+ adev->cg_flags =
+ AMD_CG_SUPPORT_GFX_CGCG |
+ AMD_CG_SUPPORT_GFX_CGLS |
+ AMD_CG_SUPPORT_GFX_MGCG |
+ AMD_CG_SUPPORT_GFX_FGCG |
+ AMD_CG_SUPPORT_REPEATER_FGCG |
+ AMD_CG_SUPPORT_GFX_PERF_CLK |
+ AMD_CG_SUPPORT_MC_MGCG |
+ AMD_CG_SUPPORT_MC_LS |
+ AMD_CG_SUPPORT_HDP_MGCG |
+ AMD_CG_SUPPORT_HDP_LS |
+ AMD_CG_SUPPORT_ATHUB_MGCG |
+ AMD_CG_SUPPORT_ATHUB_LS |
+ AMD_CG_SUPPORT_IH_CG |
+ AMD_CG_SUPPORT_BIF_MGCG |
+ AMD_CG_SUPPORT_BIF_LS |
+ AMD_CG_SUPPORT_VCN_MGCG |
+ AMD_CG_SUPPORT_JPEG_MGCG;
+ adev->pg_flags = AMD_PG_SUPPORT_VCN |
+ AMD_PG_SUPPORT_GFX_PG |
+ AMD_PG_SUPPORT_JPEG;
+ adev->external_rev_id = adev->rev_id + 0x1;
+ break;
+
default:
/* FIXME: not supported yet */
return -EINVAL;
}
+ if (amdgpu_sriov_vf(adev)) {
+ amdgpu_virt_init_setting(adev);
+ xgpu_nv_mailbox_set_irq_funcs(adev);
+ }
+
return 0;
}
static int soc21_common_late_init(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ if (amdgpu_sriov_vf(adev))
+ xgpu_nv_mailbox_get_irq(adev);
+
return 0;
}
static int soc21_common_sw_init(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ if (amdgpu_sriov_vf(adev))
+ xgpu_nv_mailbox_add_irq_id(adev);
+
return 0;
}
@@ -704,6 +738,9 @@ static int soc21_common_hw_fini(void *handle)
/* disable the doorbell aperture */
soc21_enable_doorbell_aperture(adev, false);
+ if (amdgpu_sriov_vf(adev))
+ xgpu_nv_mailbox_put_irq(adev);
+
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
index 5d5d031c9..72fd963 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
@@ -187,9 +187,9 @@ static void umc_v6_7_ecc_info_query_ras_error_count(struct amdgpu_device *adev,
}
}
-static void umc_v6_7_convert_error_address(struct amdgpu_device *adev,
- struct ras_err_data *err_data, uint64_t err_addr,
- uint32_t ch_inst, uint32_t umc_inst)
+void umc_v6_7_convert_error_address(struct amdgpu_device *adev,
+ struct ras_err_data *err_data, uint64_t err_addr,
+ uint32_t ch_inst, uint32_t umc_inst)
{
uint32_t channel_index;
uint64_t soc_pa, retired_page, column;
@@ -553,5 +553,4 @@ struct amdgpu_umc_ras umc_v6_7_ras = {
.query_ras_poison_mode = umc_v6_7_query_ras_poison_mode,
.ecc_info_query_ras_error_count = umc_v6_7_ecc_info_query_ras_error_count,
.ecc_info_query_ras_error_address = umc_v6_7_ecc_info_query_ras_error_address,
- .convert_ras_error_address = umc_v6_7_convert_error_address,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.h b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.h
index fe41ed2..105245d 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.h
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.h
@@ -71,5 +71,7 @@ extern const uint32_t
umc_v6_7_channel_idx_tbl_second[UMC_V6_7_UMC_INSTANCE_NUM][UMC_V6_7_CHANNEL_INSTANCE_NUM];
extern const uint32_t
umc_v6_7_channel_idx_tbl_first[UMC_V6_7_UMC_INSTANCE_NUM][UMC_V6_7_CHANNEL_INSTANCE_NUM];
-
+void umc_v6_7_convert_error_address(struct amdgpu_device *adev,
+ struct ras_err_data *err_data, uint64_t err_addr,
+ uint32_t ch_inst, uint32_t umc_inst);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v8_10.c b/drivers/gpu/drm/amd/amdgpu/umc_v8_10.c
index 91235df..b7da452 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v8_10.c
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v8_10.c
@@ -46,6 +46,16 @@ const struct channelnum_map_colbit umc_v8_10_channelnum_map_colbit_table[] = {
};
const uint32_t
+ umc_v8_10_channel_idx_tbl_ext0[]
+ [UMC_V8_10_UMC_INSTANCE_NUM]
+ [UMC_V8_10_CHANNEL_INSTANCE_NUM] = {
+ {{1, 5}, {7, 3}},
+ {{14, 15}, {13, 12}},
+ {{10, 11}, {9, 8}},
+ {{6, 2}, {0, 4}}
+ };
+
+const uint32_t
umc_v8_10_channel_idx_tbl[]
[UMC_V8_10_UMC_INSTANCE_NUM]
[UMC_V8_10_CHANNEL_INSTANCE_NUM] = {
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v8_10.h b/drivers/gpu/drm/amd/amdgpu/umc_v8_10.h
index 849ede8..25eaf4a 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v8_10.h
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v8_10.h
@@ -66,5 +66,9 @@ extern const uint32_t
[UMC_V8_10_UMC_INSTANCE_NUM]
[UMC_V8_10_CHANNEL_INSTANCE_NUM];
+extern const uint32_t
+ umc_v8_10_channel_idx_tbl_ext0[]
+ [UMC_V8_10_UMC_INSTANCE_NUM]
+ [UMC_V8_10_CHANNEL_INSTANCE_NUM];
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
index 375c440..5fe872f 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
@@ -216,8 +216,8 @@ static int uvd_v6_0_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t handle
uint64_t addr;
int i, r;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4,
+ AMDGPU_IB_POOL_DIRECT, &job);
if (r)
return r;
@@ -280,8 +280,8 @@ static int uvd_v6_0_enc_get_destroy_msg(struct amdgpu_ring *ring,
uint64_t addr;
int i, r;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4,
+ AMDGPU_IB_POOL_DIRECT, &job);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
index e668b3b..e407be6 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
@@ -213,7 +213,7 @@ static int uvd_v7_0_enc_ring_test_ring(struct amdgpu_ring *ring)
*
* Open up a stream for HW test
*/
-static int uvd_v7_0_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t handle,
+static int uvd_v7_0_enc_get_create_msg(struct amdgpu_ring *ring, u32 handle,
struct amdgpu_bo *bo,
struct dma_fence **fence)
{
@@ -224,8 +224,8 @@ static int uvd_v7_0_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t handle
uint64_t addr;
int i, r;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4,
+ AMDGPU_IB_POOL_DIRECT, &job);
if (r)
return r;
@@ -276,7 +276,7 @@ static int uvd_v7_0_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t handle
*
* Close up a stream for HW test or if userspace failed to do so
*/
-static int uvd_v7_0_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle,
+static int uvd_v7_0_enc_get_destroy_msg(struct amdgpu_ring *ring, u32 handle,
struct amdgpu_bo *bo,
struct dma_fence **fence)
{
@@ -287,8 +287,8 @@ static int uvd_v7_0_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handl
uint64_t addr;
int i, r;
- r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4,
- AMDGPU_IB_POOL_DIRECT, &job);
+ r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4,
+ AMDGPU_IB_POOL_DIRECT, &job);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
index 8a7006d..ec87b00 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
@@ -770,6 +770,33 @@ static void vcn_v2_5_enable_clock_gating(struct amdgpu_device *adev)
}
}
+static void vcn_v2_6_enable_ras(struct amdgpu_device *adev, int inst_idx,
+ bool indirect)
+{
+ uint32_t tmp;
+
+ if (adev->ip_versions[UVD_HWIP][0] != IP_VERSION(2, 6, 0))
+ return;
+
+ tmp = VCN_RAS_CNTL__VCPU_VCODEC_REARM_MASK |
+ VCN_RAS_CNTL__VCPU_VCODEC_IH_EN_MASK |
+ VCN_RAS_CNTL__VCPU_VCODEC_PMI_EN_MASK |
+ VCN_RAS_CNTL__VCPU_VCODEC_STALL_EN_MASK;
+ WREG32_SOC15_DPG_MODE(inst_idx,
+ SOC15_DPG_MODE_OFFSET(VCN, 0, mmVCN_RAS_CNTL),
+ tmp, 0, indirect);
+
+ tmp = UVD_VCPU_INT_EN__RASCNTL_VCPU_VCODEC_EN_MASK;
+ WREG32_SOC15_DPG_MODE(inst_idx,
+ SOC15_DPG_MODE_OFFSET(VCN, 0, mmUVD_VCPU_INT_EN),
+ tmp, 0, indirect);
+
+ tmp = UVD_SYS_INT_EN__RASCNTL_VCPU_VCODEC_EN_MASK;
+ WREG32_SOC15_DPG_MODE(inst_idx,
+ SOC15_DPG_MODE_OFFSET(VCN, 0, mmUVD_SYS_INT_EN),
+ tmp, 0, indirect);
+}
+
static int vcn_v2_5_start_dpg_mode(struct amdgpu_device *adev, int inst_idx, bool indirect)
{
volatile struct amdgpu_fw_shared *fw_shared = adev->vcn.inst[inst_idx].fw_shared.cpu_addr;
@@ -849,6 +876,8 @@ static int vcn_v2_5_start_dpg_mode(struct amdgpu_device *adev, int inst_idx, boo
WREG32_SOC15_DPG_MODE(inst_idx, SOC15_DPG_MODE_OFFSET(
VCN, 0, mmUVD_LMI_CTRL2), 0, 0, indirect);
+ vcn_v2_6_enable_ras(adev, inst_idx, indirect);
+
/* unblock VCPU register access */
WREG32_SOC15_DPG_MODE(inst_idx, SOC15_DPG_MODE_OFFSET(
VCN, 0, mmUVD_RB_ARB_CTRL), 0, 0, indirect);
@@ -2002,16 +2031,5 @@ static void vcn_v2_5_set_ras_funcs(struct amdgpu_device *adev)
break;
}
- if (adev->vcn.ras) {
- amdgpu_ras_register_ras_block(adev, &adev->vcn.ras->ras_block);
-
- strcpy(adev->vcn.ras->ras_block.ras_comm.name, "vcn");
- adev->vcn.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__VCN;
- adev->vcn.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__POISON;
- adev->vcn.ras_if = &adev->vcn.ras->ras_block.ras_comm;
-
- /* If don't define special ras_late_init function, use default ras_late_init */
- if (!adev->vcn.ras->ras_block.ras_late_init)
- adev->vcn.ras->ras_block.ras_late_init = amdgpu_ras_block_late_init;
- }
+ amdgpu_vcn_set_ras_funcs(adev);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
index dcc49b0..1e2b222 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
@@ -31,6 +31,7 @@
#include "soc15_hw_ip.h"
#include "vcn_v2_0.h"
#include "mmsch_v4_0.h"
+#include "vcn_v4_0.h"
#include "vcn/vcn_4_0_0_offset.h"
#include "vcn/vcn_4_0_0_sh_mask.h"
@@ -64,6 +65,7 @@ static int vcn_v4_0_set_powergating_state(void *handle,
static int vcn_v4_0_pause_dpg_mode(struct amdgpu_device *adev,
int inst_idx, struct dpg_pause_state *new_state);
static void vcn_v4_0_unified_ring_set_wptr(struct amdgpu_ring *ring);
+static void vcn_v4_0_set_ras_funcs(struct amdgpu_device *adev);
/**
* vcn_v4_0_early_init - set function pointers
@@ -84,6 +86,7 @@ static int vcn_v4_0_early_init(void *handle)
vcn_v4_0_set_unified_ring_funcs(adev);
vcn_v4_0_set_irq_funcs(adev);
+ vcn_v4_0_set_ras_funcs(adev);
return 0;
}
@@ -125,6 +128,12 @@ static int vcn_v4_0_sw_init(void *handle)
if (r)
return r;
+ /* VCN POISON TRAP */
+ r = amdgpu_irq_add_id(adev, amdgpu_ih_clientid_vcns[i],
+ VCN_4_0__SRCID_UVD_POISON, &adev->vcn.inst[i].irq);
+ if (r)
+ return r;
+
ring = &adev->vcn.inst[i].ring_enc[0];
ring->use_doorbell = true;
if (amdgpu_sriov_vf(adev))
@@ -289,6 +298,7 @@ static int vcn_v4_0_hw_fini(void *handle)
}
}
+ amdgpu_irq_put(adev, &adev->vcn.inst[i].irq, 0);
}
return 0;
@@ -852,6 +862,28 @@ static void vcn_v4_0_enable_clock_gating(struct amdgpu_device *adev, int inst)
return;
}
+static void vcn_v4_0_enable_ras(struct amdgpu_device *adev, int inst_idx,
+ bool indirect)
+{
+ uint32_t tmp;
+
+ if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__VCN))
+ return;
+
+ tmp = VCN_RAS_CNTL__VCPU_VCODEC_REARM_MASK |
+ VCN_RAS_CNTL__VCPU_VCODEC_IH_EN_MASK |
+ VCN_RAS_CNTL__VCPU_VCODEC_PMI_EN_MASK |
+ VCN_RAS_CNTL__VCPU_VCODEC_STALL_EN_MASK;
+ WREG32_SOC15_DPG_MODE(inst_idx,
+ SOC15_DPG_MODE_OFFSET(VCN, 0, regVCN_RAS_CNTL),
+ tmp, 0, indirect);
+
+ tmp = UVD_SYS_INT_EN__RASCNTL_VCPU_VCODEC_EN_MASK;
+ WREG32_SOC15_DPG_MODE(inst_idx,
+ SOC15_DPG_MODE_OFFSET(VCN, 0, regUVD_SYS_INT_EN),
+ tmp, 0, indirect);
+}
+
/**
* vcn_v4_0_start_dpg_mode - VCN start with dpg mode
*
@@ -940,6 +972,8 @@ static int vcn_v4_0_start_dpg_mode(struct amdgpu_device *adev, int inst_idx, boo
WREG32_SOC15_DPG_MODE(inst_idx, SOC15_DPG_MODE_OFFSET(
VCN, inst_idx, regUVD_LMI_CTRL2), tmp, 0, indirect);
+ vcn_v4_0_enable_ras(adev, inst_idx, indirect);
+
/* enable master interrupt */
WREG32_SOC15_DPG_MODE(inst_idx, SOC15_DPG_MODE_OFFSET(
VCN, inst_idx, regUVD_MASTINT_EN),
@@ -1932,6 +1966,9 @@ static int vcn_v4_0_process_interrupt(struct amdgpu_device *adev, struct amdgpu_
case VCN_4_0__SRCID__UVD_ENC_GENERAL_PURPOSE:
amdgpu_fence_process(&adev->vcn.inst[ip_instance].ring_enc[0]);
break;
+ case VCN_4_0__SRCID_UVD_POISON:
+ amdgpu_vcn_process_poison_irq(adev, source, entry);
+ break;
default:
DRM_ERROR("Unhandled interrupt: %d %d\n",
entry->src_id, entry->src_data[0]);
@@ -1994,3 +2031,60 @@ const struct amdgpu_ip_block_version vcn_v4_0_ip_block =
.rev = 0,
.funcs = &vcn_v4_0_ip_funcs,
};
+
+static uint32_t vcn_v4_0_query_poison_by_instance(struct amdgpu_device *adev,
+ uint32_t instance, uint32_t sub_block)
+{
+ uint32_t poison_stat = 0, reg_value = 0;
+
+ switch (sub_block) {
+ case AMDGPU_VCN_V4_0_VCPU_VCODEC:
+ reg_value = RREG32_SOC15(VCN, instance, regUVD_RAS_VCPU_VCODEC_STATUS);
+ poison_stat = REG_GET_FIELD(reg_value, UVD_RAS_VCPU_VCODEC_STATUS, POISONED_PF);
+ break;
+ default:
+ break;
+ }
+
+ if (poison_stat)
+ dev_info(adev->dev, "Poison detected in VCN%d, sub_block%d\n",
+ instance, sub_block);
+
+ return poison_stat;
+}
+
+static bool vcn_v4_0_query_ras_poison_status(struct amdgpu_device *adev)
+{
+ uint32_t inst, sub;
+ uint32_t poison_stat = 0;
+
+ for (inst = 0; inst < adev->vcn.num_vcn_inst; inst++)
+ for (sub = 0; sub < AMDGPU_VCN_V4_0_MAX_SUB_BLOCK; sub++)
+ poison_stat +=
+ vcn_v4_0_query_poison_by_instance(adev, inst, sub);
+
+ return !!poison_stat;
+}
+
+const struct amdgpu_ras_block_hw_ops vcn_v4_0_ras_hw_ops = {
+ .query_poison_status = vcn_v4_0_query_ras_poison_status,
+};
+
+static struct amdgpu_vcn_ras vcn_v4_0_ras = {
+ .ras_block = {
+ .hw_ops = &vcn_v4_0_ras_hw_ops,
+ },
+};
+
+static void vcn_v4_0_set_ras_funcs(struct amdgpu_device *adev)
+{
+ switch (adev->ip_versions[VCN_HWIP][0]) {
+ case IP_VERSION(4, 0, 0):
+ adev->vcn.ras = &vcn_v4_0_ras;
+ break;
+ default:
+ break;
+ }
+
+ amdgpu_vcn_set_ras_funcs(adev);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.h b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.h
index 7c5c9d9..7d3d11f 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.h
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.h
@@ -24,6 +24,12 @@
#ifndef __VCN_V4_0_H__
#define __VCN_V4_0_H__
+enum amdgpu_vcn_v4_0_sub_block {
+ AMDGPU_VCN_V4_0_VCPU_VCODEC = 0,
+
+ AMDGPU_VCN_V4_0_MAX_SUB_BLOCK,
+};
+
extern const struct amdgpu_ip_block_version vcn_v4_0_ip_block;
#endif /* __VCN_V4_0_H__ */
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
index 59dfca0..1706081 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
@@ -267,7 +267,7 @@ static void vega20_ih_reroute_ih(struct amdgpu_device *adev)
/* vega20 ih reroute will go through psp this
* function is used for newer asics starting arcturus
*/
- if (adev->asic_type >= CHIP_ARCTURUS) {
+ if (adev->ip_versions[OSSSYS_HWIP][0] >= IP_VERSION(4, 2, 1)) {
/* Reroute to IH ring 1 for VMC */
WREG32_SOC15(OSSSYS, 0, mmIH_CLIENT_CFG_INDEX, 0x12);
tmp = RREG32_SOC15(OSSSYS, 0, mmIH_CLIENT_CFG_DATA);
@@ -308,7 +308,7 @@ static int vega20_ih_irq_init(struct amdgpu_device *adev)
adev->nbio.funcs->ih_control(adev);
- if (adev->asic_type == CHIP_ARCTURUS &&
+ if ((adev->ip_versions[OSSSYS_HWIP][0] == IP_VERSION(4, 2, 1)) &&
adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) {
ih_chicken = RREG32_SOC15(OSSSYS, 0, mmIH_CHICKEN);
if (adev->irq.ih.use_bus_addr) {
@@ -321,7 +321,7 @@ static int vega20_ih_irq_init(struct amdgpu_device *adev)
/* psp firmware won't program IH_CHICKEN for aldebaran
* driver needs to program it properly according to
* MC_SPACE type in IH_RB_CNTL */
- if (adev->asic_type == CHIP_ALDEBARAN) {
+ if (adev->ip_versions[OSSSYS_HWIP][0] == IP_VERSION(4, 4, 0)) {
ih_chicken = RREG32_SOC15(OSSSYS, 0, mmIH_CHICKEN_ALDEBARAN);
if (adev->irq.ih.use_bus_addr) {
ih_chicken = REG_SET_FIELD(ih_chicken, IH_CHICKEN,
diff --git a/drivers/gpu/drm/amd/amdgpu/vi.c b/drivers/gpu/drm/amd/amdgpu/vi.c
index f6ffd7c..12ef782 100644
--- a/drivers/gpu/drm/amd/amdgpu/vi.c
+++ b/drivers/gpu/drm/amd/amdgpu/vi.c
@@ -2111,6 +2111,8 @@ void vi_set_virt_ops(struct amdgpu_device *adev)
int vi_set_ip_blocks(struct amdgpu_device *adev)
{
+ amdgpu_device_set_sriov_virtual_display(adev);
+
switch (adev->asic_type) {
case CHIP_TOPAZ:
/* topaz has no DCE, UVD, VCE */
@@ -2130,7 +2132,7 @@ int vi_set_ip_blocks(struct amdgpu_device *adev)
amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block);
amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block);
amdgpu_device_ip_block_add(adev, &pp_smu_ip_block);
- if (adev->enable_virtual_display || amdgpu_sriov_vf(adev))
+ if (adev->enable_virtual_display)
amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block);
#if defined(CONFIG_DRM_AMD_DC)
else if (amdgpu_device_has_dc_support(adev))
@@ -2150,7 +2152,7 @@ int vi_set_ip_blocks(struct amdgpu_device *adev)
amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block);
amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block);
amdgpu_device_ip_block_add(adev, &pp_smu_ip_block);
- if (adev->enable_virtual_display || amdgpu_sriov_vf(adev))
+ if (adev->enable_virtual_display)
amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block);
#if defined(CONFIG_DRM_AMD_DC)
else if (amdgpu_device_has_dc_support(adev))
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
index 8bfdfd0..3251f47 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
@@ -50,16 +50,6 @@ static inline unsigned int get_and_inc_gpu_processor_id(
return current_id;
}
-/* Static table to describe GPU Cache information */
-struct kfd_gpu_cache_info {
- uint32_t cache_size;
- uint32_t cache_level;
- uint32_t flags;
- /* Indicates how many Compute Units share this cache
- * within a SA. Value = 1 indicates the cache is not shared
- */
- uint32_t num_cu_shared;
-};
static struct kfd_gpu_cache_info kaveri_cache_info[] = {
{
@@ -891,6 +881,54 @@ static struct kfd_gpu_cache_info gc_10_3_6_cache_info[] = {
},
};
+static struct kfd_gpu_cache_info dummy_cache_info[] = {
+ {
+ /* TCP L1 Cache per CU */
+ .cache_size = 16,
+ .cache_level = 1,
+ .flags = (CRAT_CACHE_FLAGS_ENABLED |
+ CRAT_CACHE_FLAGS_DATA_CACHE |
+ CRAT_CACHE_FLAGS_SIMD_CACHE),
+ .num_cu_shared = 1,
+ },
+ {
+ /* Scalar L1 Instruction Cache per SQC */
+ .cache_size = 32,
+ .cache_level = 1,
+ .flags = (CRAT_CACHE_FLAGS_ENABLED |
+ CRAT_CACHE_FLAGS_INST_CACHE |
+ CRAT_CACHE_FLAGS_SIMD_CACHE),
+ .num_cu_shared = 2,
+ },
+ {
+ /* Scalar L1 Data Cache per SQC */
+ .cache_size = 16,
+ .cache_level = 1,
+ .flags = (CRAT_CACHE_FLAGS_ENABLED |
+ CRAT_CACHE_FLAGS_DATA_CACHE |
+ CRAT_CACHE_FLAGS_SIMD_CACHE),
+ .num_cu_shared = 2,
+ },
+ {
+ /* GL1 Data Cache per SA */
+ .cache_size = 128,
+ .cache_level = 1,
+ .flags = (CRAT_CACHE_FLAGS_ENABLED |
+ CRAT_CACHE_FLAGS_DATA_CACHE |
+ CRAT_CACHE_FLAGS_SIMD_CACHE),
+ .num_cu_shared = 6,
+ },
+ {
+ /* L2 Data Cache per GPU (Total Tex Cache) */
+ .cache_size = 2048,
+ .cache_level = 2,
+ .flags = (CRAT_CACHE_FLAGS_ENABLED |
+ CRAT_CACHE_FLAGS_DATA_CACHE |
+ CRAT_CACHE_FLAGS_SIMD_CACHE),
+ .num_cu_shared = 6,
+ },
+};
+
static void kfd_populated_cu_info_cpu(struct kfd_topology_device *dev,
struct crat_subtype_computeunit *cu)
{
@@ -1071,8 +1109,12 @@ static int kfd_parse_subtype_cache(struct crat_subtype_cache *cache,
props->cachelines_per_tag = cache->lines_per_tag;
props->cache_assoc = cache->associativity;
props->cache_latency = cache->cache_latency;
+
memcpy(props->sibling_map, cache->sibling_map,
- sizeof(props->sibling_map));
+ CRAT_SIBLINGMAP_SIZE);
+
+ /* set the sibling_map_size as 32 for CRAT from ACPI */
+ props->sibling_map_size = CRAT_SIBLINGMAP_SIZE;
if (cache->flags & CRAT_CACHE_FLAGS_DATA_CACHE)
props->cache_type |= HSA_CACHE_TYPE_DATA;
@@ -1083,7 +1125,6 @@ static int kfd_parse_subtype_cache(struct crat_subtype_cache *cache,
if (cache->flags & CRAT_CACHE_FLAGS_SIMD_CACHE)
props->cache_type |= HSA_CACHE_TYPE_HSACU;
- dev->cache_count++;
dev->node_props.caches_count++;
list_add_tail(&props->list, &dev->cache_props);
@@ -1291,125 +1332,6 @@ int kfd_parse_crat_table(void *crat_image, struct list_head *device_list,
return ret;
}
-/* Helper function. See kfd_fill_gpu_cache_info for parameter description */
-static int fill_in_l1_pcache(struct crat_subtype_cache *pcache,
- struct kfd_gpu_cache_info *pcache_info,
- struct kfd_cu_info *cu_info,
- int mem_available,
- int cu_bitmask,
- int cache_type, unsigned int cu_processor_id,
- int cu_block)
-{
- unsigned int cu_sibling_map_mask;
- int first_active_cu;
-
- /* First check if enough memory is available */
- if (sizeof(struct crat_subtype_cache) > mem_available)
- return -ENOMEM;
-
- cu_sibling_map_mask = cu_bitmask;
- cu_sibling_map_mask >>= cu_block;
- cu_sibling_map_mask &=
- ((1 << pcache_info[cache_type].num_cu_shared) - 1);
- first_active_cu = ffs(cu_sibling_map_mask);
-
- /* CU could be inactive. In case of shared cache find the first active
- * CU. and incase of non-shared cache check if the CU is inactive. If
- * inactive active skip it
- */
- if (first_active_cu) {
- memset(pcache, 0, sizeof(struct crat_subtype_cache));
- pcache->type = CRAT_SUBTYPE_CACHE_AFFINITY;
- pcache->length = sizeof(struct crat_subtype_cache);
- pcache->flags = pcache_info[cache_type].flags;
- pcache->processor_id_low = cu_processor_id
- + (first_active_cu - 1);
- pcache->cache_level = pcache_info[cache_type].cache_level;
- pcache->cache_size = pcache_info[cache_type].cache_size;
-
- /* Sibling map is w.r.t processor_id_low, so shift out
- * inactive CU
- */
- cu_sibling_map_mask =
- cu_sibling_map_mask >> (first_active_cu - 1);
-
- pcache->sibling_map[0] = (uint8_t)(cu_sibling_map_mask & 0xFF);
- pcache->sibling_map[1] =
- (uint8_t)((cu_sibling_map_mask >> 8) & 0xFF);
- pcache->sibling_map[2] =
- (uint8_t)((cu_sibling_map_mask >> 16) & 0xFF);
- pcache->sibling_map[3] =
- (uint8_t)((cu_sibling_map_mask >> 24) & 0xFF);
- return 0;
- }
- return 1;
-}
-
-/* Helper function. See kfd_fill_gpu_cache_info for parameter description */
-static int fill_in_l2_l3_pcache(struct crat_subtype_cache *pcache,
- struct kfd_gpu_cache_info *pcache_info,
- struct kfd_cu_info *cu_info,
- int mem_available,
- int cache_type, unsigned int cu_processor_id)
-{
- unsigned int cu_sibling_map_mask;
- int first_active_cu;
- int i, j, k;
-
- /* First check if enough memory is available */
- if (sizeof(struct crat_subtype_cache) > mem_available)
- return -ENOMEM;
-
- cu_sibling_map_mask = cu_info->cu_bitmap[0][0];
- cu_sibling_map_mask &=
- ((1 << pcache_info[cache_type].num_cu_shared) - 1);
- first_active_cu = ffs(cu_sibling_map_mask);
-
- /* CU could be inactive. In case of shared cache find the first active
- * CU. and incase of non-shared cache check if the CU is inactive. If
- * inactive active skip it
- */
- if (first_active_cu) {
- memset(pcache, 0, sizeof(struct crat_subtype_cache));
- pcache->type = CRAT_SUBTYPE_CACHE_AFFINITY;
- pcache->length = sizeof(struct crat_subtype_cache);
- pcache->flags = pcache_info[cache_type].flags;
- pcache->processor_id_low = cu_processor_id
- + (first_active_cu - 1);
- pcache->cache_level = pcache_info[cache_type].cache_level;
- pcache->cache_size = pcache_info[cache_type].cache_size;
-
- /* Sibling map is w.r.t processor_id_low, so shift out
- * inactive CU
- */
- cu_sibling_map_mask =
- cu_sibling_map_mask >> (first_active_cu - 1);
- k = 0;
- for (i = 0; i < cu_info->num_shader_engines; i++) {
- for (j = 0; j < cu_info->num_shader_arrays_per_engine;
- j++) {
- pcache->sibling_map[k] =
- (uint8_t)(cu_sibling_map_mask & 0xFF);
- pcache->sibling_map[k+1] =
- (uint8_t)((cu_sibling_map_mask >> 8) & 0xFF);
- pcache->sibling_map[k+2] =
- (uint8_t)((cu_sibling_map_mask >> 16) & 0xFF);
- pcache->sibling_map[k+3] =
- (uint8_t)((cu_sibling_map_mask >> 24) & 0xFF);
- k += 4;
- cu_sibling_map_mask =
- cu_info->cu_bitmap[i % 4][j + i / 4];
- cu_sibling_map_mask &= (
- (1 << pcache_info[cache_type].num_cu_shared)
- - 1);
- }
- }
- return 0;
- }
- return 1;
-}
-
-#define KFD_MAX_CACHE_TYPES 6
static int kfd_fill_gpu_cache_info_from_gfx_config(struct kfd_dev *kdev,
struct kfd_gpu_cache_info *pcache_info)
@@ -1483,228 +1405,134 @@ static int kfd_fill_gpu_cache_info_from_gfx_config(struct kfd_dev *kdev,
return i;
}
-/* kfd_fill_gpu_cache_info - Fill GPU cache info using kfd_gpu_cache_info
- * tables
- *
- * @kdev - [IN] GPU device
- * @gpu_processor_id - [IN] GPU processor ID to which these caches
- * associate
- * @available_size - [IN] Amount of memory available in pcache
- * @cu_info - [IN] Compute Unit info obtained from KGD
- * @pcache - [OUT] memory into which cache data is to be filled in.
- * @size_filled - [OUT] amount of data used up in pcache.
- * @num_of_entries - [OUT] number of caches added
- */
-static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
- int gpu_processor_id,
- int available_size,
- struct kfd_cu_info *cu_info,
- struct crat_subtype_cache *pcache,
- int *size_filled,
- int *num_of_entries)
+int kfd_get_gpu_cache_info(struct kfd_dev *kdev, struct kfd_gpu_cache_info **pcache_info)
{
- struct kfd_gpu_cache_info *pcache_info;
- struct kfd_gpu_cache_info cache_info[KFD_MAX_CACHE_TYPES];
int num_of_cache_types = 0;
- int i, j, k;
- int ct = 0;
- int mem_available = available_size;
- unsigned int cu_processor_id;
- int ret;
- unsigned int num_cu_shared;
switch (kdev->adev->asic_type) {
case CHIP_KAVERI:
- pcache_info = kaveri_cache_info;
+ *pcache_info = kaveri_cache_info;
num_of_cache_types = ARRAY_SIZE(kaveri_cache_info);
break;
case CHIP_HAWAII:
- pcache_info = hawaii_cache_info;
+ *pcache_info = hawaii_cache_info;
num_of_cache_types = ARRAY_SIZE(hawaii_cache_info);
break;
case CHIP_CARRIZO:
- pcache_info = carrizo_cache_info;
+ *pcache_info = carrizo_cache_info;
num_of_cache_types = ARRAY_SIZE(carrizo_cache_info);
break;
case CHIP_TONGA:
- pcache_info = tonga_cache_info;
+ *pcache_info = tonga_cache_info;
num_of_cache_types = ARRAY_SIZE(tonga_cache_info);
break;
case CHIP_FIJI:
- pcache_info = fiji_cache_info;
+ *pcache_info = fiji_cache_info;
num_of_cache_types = ARRAY_SIZE(fiji_cache_info);
break;
case CHIP_POLARIS10:
- pcache_info = polaris10_cache_info;
+ *pcache_info = polaris10_cache_info;
num_of_cache_types = ARRAY_SIZE(polaris10_cache_info);
break;
case CHIP_POLARIS11:
- pcache_info = polaris11_cache_info;
+ *pcache_info = polaris11_cache_info;
num_of_cache_types = ARRAY_SIZE(polaris11_cache_info);
break;
case CHIP_POLARIS12:
- pcache_info = polaris12_cache_info;
+ *pcache_info = polaris12_cache_info;
num_of_cache_types = ARRAY_SIZE(polaris12_cache_info);
break;
case CHIP_VEGAM:
- pcache_info = vegam_cache_info;
+ *pcache_info = vegam_cache_info;
num_of_cache_types = ARRAY_SIZE(vegam_cache_info);
break;
default:
switch (KFD_GC_VERSION(kdev)) {
case IP_VERSION(9, 0, 1):
- pcache_info = vega10_cache_info;
+ *pcache_info = vega10_cache_info;
num_of_cache_types = ARRAY_SIZE(vega10_cache_info);
break;
case IP_VERSION(9, 2, 1):
- pcache_info = vega12_cache_info;
+ *pcache_info = vega12_cache_info;
num_of_cache_types = ARRAY_SIZE(vega12_cache_info);
break;
case IP_VERSION(9, 4, 0):
case IP_VERSION(9, 4, 1):
- pcache_info = vega20_cache_info;
+ *pcache_info = vega20_cache_info;
num_of_cache_types = ARRAY_SIZE(vega20_cache_info);
break;
case IP_VERSION(9, 4, 2):
- pcache_info = aldebaran_cache_info;
+ *pcache_info = aldebaran_cache_info;
num_of_cache_types = ARRAY_SIZE(aldebaran_cache_info);
break;
case IP_VERSION(9, 1, 0):
case IP_VERSION(9, 2, 2):
- pcache_info = raven_cache_info;
+ *pcache_info = raven_cache_info;
num_of_cache_types = ARRAY_SIZE(raven_cache_info);
break;
case IP_VERSION(9, 3, 0):
- pcache_info = renoir_cache_info;
+ *pcache_info = renoir_cache_info;
num_of_cache_types = ARRAY_SIZE(renoir_cache_info);
break;
case IP_VERSION(10, 1, 10):
case IP_VERSION(10, 1, 2):
case IP_VERSION(10, 1, 3):
case IP_VERSION(10, 1, 4):
- pcache_info = navi10_cache_info;
+ *pcache_info = navi10_cache_info;
num_of_cache_types = ARRAY_SIZE(navi10_cache_info);
break;
case IP_VERSION(10, 1, 1):
- pcache_info = navi14_cache_info;
+ *pcache_info = navi14_cache_info;
num_of_cache_types = ARRAY_SIZE(navi14_cache_info);
break;
case IP_VERSION(10, 3, 0):
- pcache_info = sienna_cichlid_cache_info;
+ *pcache_info = sienna_cichlid_cache_info;
num_of_cache_types = ARRAY_SIZE(sienna_cichlid_cache_info);
break;
case IP_VERSION(10, 3, 2):
- pcache_info = navy_flounder_cache_info;
+ *pcache_info = navy_flounder_cache_info;
num_of_cache_types = ARRAY_SIZE(navy_flounder_cache_info);
break;
case IP_VERSION(10, 3, 4):
- pcache_info = dimgrey_cavefish_cache_info;
+ *pcache_info = dimgrey_cavefish_cache_info;
num_of_cache_types = ARRAY_SIZE(dimgrey_cavefish_cache_info);
break;
case IP_VERSION(10, 3, 1):
- pcache_info = vangogh_cache_info;
+ *pcache_info = vangogh_cache_info;
num_of_cache_types = ARRAY_SIZE(vangogh_cache_info);
break;
case IP_VERSION(10, 3, 5):
- pcache_info = beige_goby_cache_info;
+ *pcache_info = beige_goby_cache_info;
num_of_cache_types = ARRAY_SIZE(beige_goby_cache_info);
break;
case IP_VERSION(10, 3, 3):
- pcache_info = yellow_carp_cache_info;
+ *pcache_info = yellow_carp_cache_info;
num_of_cache_types = ARRAY_SIZE(yellow_carp_cache_info);
break;
case IP_VERSION(10, 3, 6):
- pcache_info = gc_10_3_6_cache_info;
+ *pcache_info = gc_10_3_6_cache_info;
num_of_cache_types = ARRAY_SIZE(gc_10_3_6_cache_info);
break;
case IP_VERSION(10, 3, 7):
- pcache_info = gfx1037_cache_info;
+ *pcache_info = gfx1037_cache_info;
num_of_cache_types = ARRAY_SIZE(gfx1037_cache_info);
break;
case IP_VERSION(11, 0, 0):
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
- pcache_info = cache_info;
+ case IP_VERSION(11, 0, 4):
num_of_cache_types =
- kfd_fill_gpu_cache_info_from_gfx_config(kdev, pcache_info);
+ kfd_fill_gpu_cache_info_from_gfx_config(kdev, *pcache_info);
break;
default:
- return -EINVAL;
- }
- }
-
- *size_filled = 0;
- *num_of_entries = 0;
-
- /* For each type of cache listed in the kfd_gpu_cache_info table,
- * go through all available Compute Units.
- * The [i,j,k] loop will
- * if kfd_gpu_cache_info.num_cu_shared = 1
- * will parse through all available CU
- * If (kfd_gpu_cache_info.num_cu_shared != 1)
- * then it will consider only one CU from
- * the shared unit
- */
-
- for (ct = 0; ct < num_of_cache_types; ct++) {
- cu_processor_id = gpu_processor_id;
- if (pcache_info[ct].cache_level == 1) {
- for (i = 0; i < cu_info->num_shader_engines; i++) {
- for (j = 0; j < cu_info->num_shader_arrays_per_engine; j++) {
- for (k = 0; k < cu_info->num_cu_per_sh;
- k += pcache_info[ct].num_cu_shared) {
- ret = fill_in_l1_pcache(pcache,
- pcache_info,
- cu_info,
- mem_available,
- cu_info->cu_bitmap[i % 4][j + i / 4],
- ct,
- cu_processor_id,
- k);
-
- if (ret < 0)
+ *pcache_info = dummy_cache_info;
+ num_of_cache_types = ARRAY_SIZE(dummy_cache_info);
+ pr_warn("dummy cache info is used temporarily and real cache info need update later.\n");
break;
-
- if (!ret) {
- pcache++;
- (*num_of_entries)++;
- mem_available -= sizeof(*pcache);
- (*size_filled) += sizeof(*pcache);
- }
-
- /* Move to next CU block */
- num_cu_shared = ((k + pcache_info[ct].num_cu_shared) <=
- cu_info->num_cu_per_sh) ?
- pcache_info[ct].num_cu_shared :
- (cu_info->num_cu_per_sh - k);
- cu_processor_id += num_cu_shared;
}
- }
- }
- } else {
- ret = fill_in_l2_l3_pcache(pcache,
- pcache_info,
- cu_info,
- mem_available,
- ct,
- cu_processor_id);
-
- if (ret < 0)
- break;
-
- if (!ret) {
- pcache++;
- (*num_of_entries)++;
- mem_available -= sizeof(*pcache);
- (*size_filled) += sizeof(*pcache);
- }
- }
}
-
- pr_debug("Added [%d] GPU cache entries\n", *num_of_entries);
-
- return 0;
+ return num_of_cache_types;
}
static bool kfd_ignore_crat(void)
@@ -2063,8 +1891,8 @@ static void kfd_find_numa_node_in_srat(struct kfd_dev *kdev)
struct acpi_table_header *table_header = NULL;
struct acpi_subtable_header *sub_header = NULL;
unsigned long table_end, subtable_len;
- u32 pci_id = pci_domain_nr(kdev->pdev->bus) << 16 |
- pci_dev_id(kdev->pdev);
+ u32 pci_id = pci_domain_nr(kdev->adev->pdev->bus) << 16 |
+ pci_dev_id(kdev->adev->pdev);
u32 bdf;
acpi_status status;
struct acpi_srat_cpu_affinity *cpu;
@@ -2139,7 +1967,7 @@ static void kfd_find_numa_node_in_srat(struct kfd_dev *kdev)
numa_node = 0;
if (numa_node != NUMA_NO_NODE)
- set_dev_node(&kdev->pdev->dev, numa_node);
+ set_dev_node(&kdev->adev->pdev->dev, numa_node);
}
#endif
@@ -2200,14 +2028,14 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size,
sub_type_hdr->proximity_domain_from = proximity_domain;
#ifdef CONFIG_ACPI_NUMA
- if (kdev->pdev->dev.numa_node == NUMA_NO_NODE)
+ if (kdev->adev->pdev->dev.numa_node == NUMA_NO_NODE)
kfd_find_numa_node_in_srat(kdev);
#endif
#ifdef CONFIG_NUMA
- if (kdev->pdev->dev.numa_node == NUMA_NO_NODE)
+ if (kdev->adev->pdev->dev.numa_node == NUMA_NO_NODE)
sub_type_hdr->proximity_domain_to = 0;
else
- sub_type_hdr->proximity_domain_to = kdev->pdev->dev.numa_node;
+ sub_type_hdr->proximity_domain_to = kdev->adev->pdev->dev.numa_node;
#else
sub_type_hdr->proximity_domain_to = 0;
#endif
@@ -2263,8 +2091,6 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image,
struct kfd_cu_info cu_info;
int avail_size = *size;
uint32_t total_num_of_cu;
- int num_of_cache_entries = 0;
- int cache_mem_filled = 0;
uint32_t nid = 0;
int ret = 0;
@@ -2365,31 +2191,12 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image,
crat_table->length += sizeof(struct crat_subtype_memory);
crat_table->total_entries++;
- /* TODO: Fill in cache information. This information is NOT readily
- * available in KGD
- */
- sub_type_hdr = (typeof(sub_type_hdr))((char *)sub_type_hdr +
- sub_type_hdr->length);
- ret = kfd_fill_gpu_cache_info(kdev, cu->processor_id_low,
- avail_size,
- &cu_info,
- (struct crat_subtype_cache *)sub_type_hdr,
- &cache_mem_filled,
- &num_of_cache_entries);
-
- if (ret < 0)
- return ret;
-
- crat_table->length += cache_mem_filled;
- crat_table->total_entries += num_of_cache_entries;
- avail_size -= cache_mem_filled;
-
/* Fill in Subtype: IO_LINKS
* Only direct links are added here which is Link from GPU to
* its NUMA node. Indirect links are added by userspace.
*/
sub_type_hdr = (typeof(sub_type_hdr))((char *)sub_type_hdr +
- cache_mem_filled);
+ sub_type_hdr->length);
ret = kfd_fill_gpu_direct_io_link_to_cpu(&avail_size, kdev,
(struct crat_subtype_iolink *)sub_type_hdr, proximity_domain);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.h b/drivers/gpu/drm/amd/amdkfd/kfd_crat.h
index 482ba84..8d1e8ba5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.h
@@ -29,11 +29,10 @@
#pragma pack(1)
/*
- * 4CC signature values for the CRAT and CDIT ACPI tables
+ * 4CC signature value for the CRAT ACPI table
*/
#define CRAT_SIGNATURE "CRAT"
-#define CDIT_SIGNATURE "CDIT"
/*
* Component Resource Association Table (CRAT)
@@ -292,31 +291,22 @@ struct crat_subtype_generic {
uint32_t flags;
};
-/*
- * Component Locality Distance Information Table (CDIT)
- */
-#define CDIT_OEMID_LENGTH 6
-#define CDIT_OEMTABLEID_LENGTH 8
-
-struct cdit_header {
- uint32_t signature;
- uint32_t length;
- uint8_t revision;
- uint8_t checksum;
- uint8_t oem_id[CDIT_OEMID_LENGTH];
- uint8_t oem_table_id[CDIT_OEMTABLEID_LENGTH];
- uint32_t oem_revision;
- uint32_t creator_id;
- uint32_t creator_revision;
- uint32_t total_entries;
- uint16_t num_domains;
- uint8_t entry[1];
-};
-
#pragma pack()
struct kfd_dev;
+/* Static table to describe GPU Cache information */
+struct kfd_gpu_cache_info {
+ uint32_t cache_size;
+ uint32_t cache_level;
+ uint32_t flags;
+ /* Indicates how many Compute Units share this cache
+ * within a SA. Value = 1 indicates the cache is not shared
+ */
+ uint32_t num_cu_shared;
+};
+int kfd_get_gpu_cache_info(struct kfd_dev *kdev, struct kfd_gpu_cache_info **pcache_info);
+
int kfd_create_crat_image_acpi(void **crat_image, size_t *size);
void kfd_destroy_crat_image(void *crat_image);
int kfd_parse_crat_table(void *crat_image, struct list_head *device_list,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 65a1d4f..b893634 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -153,6 +153,7 @@ static void kfd_device_info_set_event_interrupt_class(struct kfd_dev *kfd)
case IP_VERSION(11, 0, 1):
case IP_VERSION(11, 0, 2):
case IP_VERSION(11, 0, 3):
+ case IP_VERSION(11, 0, 4):
kfd->device_info.event_interrupt_class = &event_interrupt_class_v11;
break;
default:
@@ -227,7 +228,6 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
{
struct kfd_dev *kfd = NULL;
const struct kfd2kgd_calls *f2g = NULL;
- struct pci_dev *pdev = adev->pdev;
uint32_t gfx_target_version = 0;
switch (adev->asic_type) {
@@ -395,6 +395,7 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
f2g = &gfx_v11_kfd2kgd;
break;
case IP_VERSION(11, 0, 1):
+ case IP_VERSION(11, 0, 4):
gfx_target_version = 110003;
f2g = &gfx_v11_kfd2kgd;
break;
@@ -429,7 +430,6 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
kfd->adev = adev;
kfd_device_info_init(kfd, vf, gfx_target_version);
- kfd->pdev = pdev;
kfd->init_complete = false;
kfd->kfd2kgd = f2g;
atomic_set(&kfd->compute_profile, 0);
@@ -497,7 +497,10 @@ static int kfd_gws_init(struct kfd_dev *kfd)
(KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 1)
&& kfd->mec2_fw_version >= 0x30) ||
(KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 2)
- && kfd->mec2_fw_version >= 0x28))))
+ && kfd->mec2_fw_version >= 0x28) ||
+ (KFD_GC_VERSION(kfd) >= IP_VERSION(10, 3, 0)
+ && KFD_GC_VERSION(kfd) < IP_VERSION(11, 0, 0)
+ && kfd->mec2_fw_version >= 0x6b))))
ret = amdgpu_amdkfd_alloc_gws(kfd->adev,
kfd->adev->gds.gws_size, &kfd->gws);
@@ -511,12 +514,10 @@ static void kfd_smi_init(struct kfd_dev *dev)
}
bool kgd2kfd_device_init(struct kfd_dev *kfd,
- struct drm_device *ddev,
const struct kgd2kfd_shared_resources *gpu_resources)
{
unsigned int size, map_process_packet_size;
- kfd->ddev = ddev;
kfd->mec_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev,
KGD_ENGINE_MEC1);
kfd->mec2_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev,
@@ -541,7 +542,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
kfd->mec_fw_version < kfd->device_info.no_atomic_fw_version)) {
dev_info(kfd_device,
"skipped device %x:%x, PCI rejects atomics %d<%d\n",
- kfd->pdev->vendor, kfd->pdev->device,
+ kfd->adev->pdev->vendor, kfd->adev->pdev->device,
kfd->mec_fw_version,
kfd->device_info.no_atomic_fw_version);
return false;
@@ -650,8 +651,8 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
kfd_smi_init(kfd);
kfd->init_complete = true;
- dev_info(kfd_device, "added device %x:%x\n", kfd->pdev->vendor,
- kfd->pdev->device);
+ dev_info(kfd_device, "added device %x:%x\n", kfd->adev->pdev->vendor,
+ kfd->adev->pdev->device);
pr_debug("Starting kfd with the following scheduling policy %d\n",
kfd->dqm->sched_policy);
@@ -676,7 +677,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
amdgpu_amdkfd_free_gws(kfd->adev, kfd->gws);
dev_err(kfd_device,
"device %x:%x NOT added due to errors\n",
- kfd->pdev->vendor, kfd->pdev->device);
+ kfd->adev->pdev->vendor, kfd->adev->pdev->device);
out:
return kfd->init_complete;
}
@@ -789,7 +790,7 @@ int kgd2kfd_resume_iommu(struct kfd_dev *kfd)
if (err)
dev_err(kfd_device,
"Failed to resume IOMMU for device %x:%x\n",
- kfd->pdev->vendor, kfd->pdev->device);
+ kfd->adev->pdev->vendor, kfd->adev->pdev->device);
return err;
}
@@ -801,7 +802,7 @@ static int kfd_resume(struct kfd_dev *kfd)
if (err)
dev_err(kfd_device,
"Error starting queue manager for device %x:%x\n",
- kfd->pdev->vendor, kfd->pdev->device);
+ kfd->adev->pdev->vendor, kfd->adev->pdev->device);
return err;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
index fbd0afe..ec1bf61 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
@@ -49,7 +49,7 @@ int kfd_iommu_check_device(struct kfd_dev *kfd)
return -ENODEV;
iommu_info.flags = 0;
- err = amd_iommu_device_info(kfd->pdev, &iommu_info);
+ err = amd_iommu_device_info(kfd->adev->pdev, &iommu_info);
if (err)
return err;
@@ -71,7 +71,7 @@ int kfd_iommu_device_init(struct kfd_dev *kfd)
return 0;
iommu_info.flags = 0;
- err = amd_iommu_device_info(kfd->pdev, &iommu_info);
+ err = amd_iommu_device_info(kfd->adev->pdev, &iommu_info);
if (err < 0) {
dev_err(kfd_device,
"error getting iommu info. is the iommu enabled?\n");
@@ -121,7 +121,7 @@ int kfd_iommu_bind_process_to_device(struct kfd_process_device *pdd)
return -EINVAL;
}
- err = amd_iommu_bind_pasid(dev->pdev, p->pasid, p->lead_thread);
+ err = amd_iommu_bind_pasid(dev->adev->pdev, p->pasid, p->lead_thread);
if (!err)
pdd->bound = PDD_BOUND;
@@ -139,7 +139,8 @@ void kfd_iommu_unbind_process(struct kfd_process *p)
for (i = 0; i < p->n_pdds; i++)
if (p->pdds[i]->bound == PDD_BOUND)
- amd_iommu_unbind_pasid(p->pdds[i]->dev->pdev, p->pasid);
+ amd_iommu_unbind_pasid(p->pdds[i]->dev->adev->pdev,
+ p->pasid);
}
/* Callback for process shutdown invoked by the IOMMU driver */
@@ -222,7 +223,7 @@ static int kfd_bind_processes_to_device(struct kfd_dev *kfd)
continue;
}
- err = amd_iommu_bind_pasid(kfd->pdev, p->pasid,
+ err = amd_iommu_bind_pasid(kfd->adev->pdev, p->pasid,
p->lead_thread);
if (err < 0) {
pr_err("Unexpected pasid 0x%x binding failure\n",
@@ -282,9 +283,9 @@ void kfd_iommu_suspend(struct kfd_dev *kfd)
kfd_unbind_processes_from_device(kfd);
- amd_iommu_set_invalidate_ctx_cb(kfd->pdev, NULL);
- amd_iommu_set_invalid_ppr_cb(kfd->pdev, NULL);
- amd_iommu_free_device(kfd->pdev);
+ amd_iommu_set_invalidate_ctx_cb(kfd->adev->pdev, NULL);
+ amd_iommu_set_invalid_ppr_cb(kfd->adev->pdev, NULL);
+ amd_iommu_free_device(kfd->adev->pdev);
}
/** kfd_iommu_resume - Restore IOMMU after resume
@@ -302,20 +303,20 @@ int kfd_iommu_resume(struct kfd_dev *kfd)
pasid_limit = kfd_get_pasid_limit();
- err = amd_iommu_init_device(kfd->pdev, pasid_limit);
+ err = amd_iommu_init_device(kfd->adev->pdev, pasid_limit);
if (err)
return -ENXIO;
- amd_iommu_set_invalidate_ctx_cb(kfd->pdev,
+ amd_iommu_set_invalidate_ctx_cb(kfd->adev->pdev,
iommu_pasid_shutdown_callback);
- amd_iommu_set_invalid_ppr_cb(kfd->pdev,
+ amd_iommu_set_invalid_ppr_cb(kfd->adev->pdev,
iommu_invalid_ppr_cb);
err = kfd_bind_processes_to_device(kfd);
if (err) {
- amd_iommu_set_invalidate_ctx_cb(kfd->pdev, NULL);
- amd_iommu_set_invalid_ppr_cb(kfd->pdev, NULL);
- amd_iommu_free_device(kfd->pdev);
+ amd_iommu_set_invalidate_ctx_cb(kfd->adev->pdev, NULL);
+ amd_iommu_set_invalid_ppr_cb(kfd->adev->pdev, NULL);
+ amd_iommu_free_device(kfd->adev->pdev);
return err;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
index 22b077a..10048ce 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
@@ -28,7 +28,6 @@
#include "amdgpu_sync.h"
#include "amdgpu_object.h"
#include "amdgpu_vm.h"
-#include "amdgpu_mn.h"
#include "amdgpu_res_cursor.h"
#include "kfd_priv.h"
#include "kfd_svm.h"
@@ -65,8 +64,11 @@ svm_migrate_gart_map(struct amdgpu_ring *ring, uint64_t npages,
num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
num_bytes = npages * 8;
- r = amdgpu_job_alloc_with_ib(adev, num_dw * 4 + num_bytes,
- AMDGPU_IB_POOL_DELAYED, &job);
+ r = amdgpu_job_alloc_with_ib(adev, &adev->mman.entity,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ num_dw * 4 + num_bytes,
+ AMDGPU_IB_POOL_DELAYED,
+ &job);
if (r)
return r;
@@ -89,18 +91,10 @@ svm_migrate_gart_map(struct amdgpu_ring *ring, uint64_t npages,
cpu_addr = &job->ibs[0].ptr[num_dw];
amdgpu_gart_map(adev, 0, npages, addr, pte_flags, cpu_addr);
- r = amdgpu_job_submit(job, &adev->mman.entity,
- AMDGPU_FENCE_OWNER_UNDEFINED, &fence);
- if (r)
- goto error_free;
-
+ fence = amdgpu_job_submit(job);
dma_fence_put(fence);
return r;
-
-error_free:
- amdgpu_job_free(job);
- return r;
}
/**
@@ -529,8 +523,8 @@ svm_migrate_ram_to_vram(struct svm_range *prange, uint32_t best_loc,
for (addr = start; addr < end;) {
unsigned long next;
- vma = find_vma(mm, addr);
- if (!vma || addr < vma->vm_start)
+ vma = vma_lookup(mm, addr);
+ if (!vma)
break;
next = min(vma->vm_end, end);
@@ -798,8 +792,8 @@ int svm_migrate_vram_to_ram(struct svm_range *prange, struct mm_struct *mm,
for (addr = start; addr < end;) {
unsigned long next;
- vma = find_vma(mm, addr);
- if (!vma || addr < vma->vm_start) {
+ vma = vma_lookup(mm, addr);
+ if (!vma) {
pr_debug("failed to find vma for prange %p\n", prange);
r = -EFAULT;
break;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers_diq.h b/drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers_diq.h
deleted file mode 100644
index f9cd286..0000000
--- a/drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers_diq.h
+++ /dev/null
@@ -1,291 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 OR MIT */
-/*
- * Copyright 2014-2022 Advanced Micro Devices, Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- */
-
-#ifndef KFD_PM4_HEADERS_DIQ_H_
-#define KFD_PM4_HEADERS_DIQ_H_
-
-/*--------------------_INDIRECT_BUFFER-------------------- */
-
-#ifndef _PM4__INDIRECT_BUFFER_DEFINED
-#define _PM4__INDIRECT_BUFFER_DEFINED
-enum _INDIRECT_BUFFER_cache_policy_enum {
- cache_policy___indirect_buffer__lru = 0,
- cache_policy___indirect_buffer__stream = 1,
- cache_policy___indirect_buffer__bypass = 2
-};
-
-enum {
- IT_INDIRECT_BUFFER_PASID = 0x5C
-};
-
-struct pm4__indirect_buffer_pasid {
- union {
- union PM4_MES_TYPE_3_HEADER header; /* header */
- unsigned int ordinal1;
- };
-
- union {
- struct {
- unsigned int reserved1:2;
- unsigned int ib_base_lo:30;
- } bitfields2;
- unsigned int ordinal2;
- };
-
- union {
- struct {
- unsigned int ib_base_hi:16;
- unsigned int reserved2:16;
- } bitfields3;
- unsigned int ordinal3;
- };
-
- union {
- unsigned int control;
- unsigned int ordinal4;
- };
-
- union {
- struct {
- unsigned int pasid:10;
- unsigned int reserved4:22;
- } bitfields5;
- unsigned int ordinal5;
- };
-
-};
-
-#endif
-
-/*--------------------_RELEASE_MEM-------------------- */
-
-#ifndef _PM4__RELEASE_MEM_DEFINED
-#define _PM4__RELEASE_MEM_DEFINED
-enum _RELEASE_MEM_event_index_enum {
- event_index___release_mem__end_of_pipe = 5,
- event_index___release_mem__shader_done = 6
-};
-
-enum _RELEASE_MEM_cache_policy_enum {
- cache_policy___release_mem__lru = 0,
- cache_policy___release_mem__stream = 1,
- cache_policy___release_mem__bypass = 2
-};
-
-enum _RELEASE_MEM_dst_sel_enum {
- dst_sel___release_mem__memory_controller = 0,
- dst_sel___release_mem__tc_l2 = 1,
- dst_sel___release_mem__queue_write_pointer_register = 2,
- dst_sel___release_mem__queue_write_pointer_poll_mask_bit = 3
-};
-
-enum _RELEASE_MEM_int_sel_enum {
- int_sel___release_mem__none = 0,
- int_sel___release_mem__send_interrupt_only = 1,
- int_sel___release_mem__send_interrupt_after_write_confirm = 2,
- int_sel___release_mem__send_data_after_write_confirm = 3
-};
-
-enum _RELEASE_MEM_data_sel_enum {
- data_sel___release_mem__none = 0,
- data_sel___release_mem__send_32_bit_low = 1,
- data_sel___release_mem__send_64_bit_data = 2,
- data_sel___release_mem__send_gpu_clock_counter = 3,
- data_sel___release_mem__send_cp_perfcounter_hi_lo = 4,
- data_sel___release_mem__store_gds_data_to_memory = 5
-};
-
-struct pm4__release_mem {
- union {
- union PM4_MES_TYPE_3_HEADER header; /*header */
- unsigned int ordinal1;
- };
-
- union {
- struct {
- unsigned int event_type:6;
- unsigned int reserved1:2;
- enum _RELEASE_MEM_event_index_enum event_index:4;
- unsigned int tcl1_vol_action_ena:1;
- unsigned int tc_vol_action_ena:1;
- unsigned int reserved2:1;
- unsigned int tc_wb_action_ena:1;
- unsigned int tcl1_action_ena:1;
- unsigned int tc_action_ena:1;
- unsigned int reserved3:6;
- unsigned int atc:1;
- enum _RELEASE_MEM_cache_policy_enum cache_policy:2;
- unsigned int reserved4:5;
- } bitfields2;
- unsigned int ordinal2;
- };
-
- union {
- struct {
- unsigned int reserved5:16;
- enum _RELEASE_MEM_dst_sel_enum dst_sel:2;
- unsigned int reserved6:6;
- enum _RELEASE_MEM_int_sel_enum int_sel:3;
- unsigned int reserved7:2;
- enum _RELEASE_MEM_data_sel_enum data_sel:3;
- } bitfields3;
- unsigned int ordinal3;
- };
-
- union {
- struct {
- unsigned int reserved8:2;
- unsigned int address_lo_32b:30;
- } bitfields4;
- struct {
- unsigned int reserved9:3;
- unsigned int address_lo_64b:29;
- } bitfields5;
- unsigned int ordinal4;
- };
-
- unsigned int address_hi;
-
- unsigned int data_lo;
-
- unsigned int data_hi;
-
-};
-#endif
-
-
-/*--------------------_SET_CONFIG_REG-------------------- */
-
-#ifndef _PM4__SET_CONFIG_REG_DEFINED
-#define _PM4__SET_CONFIG_REG_DEFINED
-
-struct pm4__set_config_reg {
- union {
- union PM4_MES_TYPE_3_HEADER header; /*header */
- unsigned int ordinal1;
- };
-
- union {
- struct {
- unsigned int reg_offset:16;
- unsigned int reserved1:7;
- unsigned int vmid_shift:5;
- unsigned int insert_vmid:1;
- unsigned int reserved2:3;
- } bitfields2;
- unsigned int ordinal2;
- };
-
- unsigned int reg_data[1]; /*1..N of these fields */
-
-};
-#endif
-
-/*--------------------_WAIT_REG_MEM-------------------- */
-
-#ifndef _PM4__WAIT_REG_MEM_DEFINED
-#define _PM4__WAIT_REG_MEM_DEFINED
-enum _WAIT_REG_MEM_function_enum {
- function___wait_reg_mem__always_pass = 0,
- function___wait_reg_mem__less_than_ref_value = 1,
- function___wait_reg_mem__less_than_equal_to_the_ref_value = 2,
- function___wait_reg_mem__equal_to_the_reference_value = 3,
- function___wait_reg_mem__not_equal_reference_value = 4,
- function___wait_reg_mem__greater_than_or_equal_reference_value = 5,
- function___wait_reg_mem__greater_than_reference_value = 6,
- function___wait_reg_mem__reserved = 7
-};
-
-enum _WAIT_REG_MEM_mem_space_enum {
- mem_space___wait_reg_mem__register_space = 0,
- mem_space___wait_reg_mem__memory_space = 1
-};
-
-enum _WAIT_REG_MEM_operation_enum {
- operation___wait_reg_mem__wait_reg_mem = 0,
- operation___wait_reg_mem__wr_wait_wr_reg = 1
-};
-
-struct pm4__wait_reg_mem {
- union {
- union PM4_MES_TYPE_3_HEADER header; /*header */
- unsigned int ordinal1;
- };
-
- union {
- struct {
- enum _WAIT_REG_MEM_function_enum function:3;
- unsigned int reserved1:1;
- enum _WAIT_REG_MEM_mem_space_enum mem_space:2;
- enum _WAIT_REG_MEM_operation_enum operation:2;
- unsigned int reserved2:24;
- } bitfields2;
- unsigned int ordinal2;
- };
-
- union {
- struct {
- unsigned int reserved3:2;
- unsigned int memory_poll_addr_lo:30;
- } bitfields3;
- struct {
- unsigned int register_poll_addr:16;
- unsigned int reserved4:16;
- } bitfields4;
- struct {
- unsigned int register_write_addr:16;
- unsigned int reserved5:16;
- } bitfields5;
- unsigned int ordinal3;
- };
-
- union {
- struct {
- unsigned int poll_address_hi:16;
- unsigned int reserved6:16;
- } bitfields6;
- struct {
- unsigned int register_write_addr:16;
- unsigned int reserved7:16;
- } bitfields7;
- unsigned int ordinal4;
- };
-
- unsigned int reference;
-
- unsigned int mask;
-
- union {
- struct {
- unsigned int poll_interval:16;
- unsigned int reserved8:16;
- } bitfields8;
- unsigned int ordinal7;
- };
-
-};
-#endif
-
-
-#endif /* KFD_PM4_HEADERS_DIQ_H_ */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index bf610e3..552c3ac8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -254,8 +254,6 @@ struct kfd_dev {
struct amdgpu_device *adev;
struct kfd_device_info device_info;
- struct pci_dev *pdev;
- struct drm_device *ddev;
unsigned int id; /* topology stub index */
@@ -1365,7 +1363,7 @@ void kfd_dec_compute_active(struct kfd_dev *dev);
static inline int kfd_devcgroup_check_permission(struct kfd_dev *kfd)
{
#if defined(CONFIG_CGROUP_DEVICE) || defined(CONFIG_CGROUP_BPF)
- struct drm_device *ddev = kfd->ddev;
+ struct drm_device *ddev = adev_to_drm(kfd->adev);
return devcgroup_check_permission(DEVCG_DEV_CHAR, DRM_MAJOR,
ddev->render->index,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 951b6367..a262571 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1050,8 +1050,8 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
* for auto suspend
*/
if (pdd->runtime_inuse) {
- pm_runtime_mark_last_busy(pdd->dev->ddev->dev);
- pm_runtime_put_autosuspend(pdd->dev->ddev->dev);
+ pm_runtime_mark_last_busy(adev_to_drm(pdd->dev->adev)->dev);
+ pm_runtime_put_autosuspend(adev_to_drm(pdd->dev->adev)->dev);
pdd->runtime_inuse = false;
}
@@ -1633,9 +1633,9 @@ struct kfd_process_device *kfd_bind_process_to_device(struct kfd_dev *dev,
* pdd is destroyed.
*/
if (!pdd->runtime_inuse) {
- err = pm_runtime_get_sync(dev->ddev->dev);
+ err = pm_runtime_get_sync(adev_to_drm(dev->adev)->dev);
if (err < 0) {
- pm_runtime_put_autosuspend(dev->ddev->dev);
+ pm_runtime_put_autosuspend(adev_to_drm(dev->adev)->dev);
return ERR_PTR(err);
}
}
@@ -1655,8 +1655,8 @@ struct kfd_process_device *kfd_bind_process_to_device(struct kfd_dev *dev,
out:
/* balance runpm reference count and exit with error */
if (!pdd->runtime_inuse) {
- pm_runtime_mark_last_busy(dev->ddev->dev);
- pm_runtime_put_autosuspend(dev->ddev->dev);
+ pm_runtime_mark_last_busy(adev_to_drm(dev->adev)->dev);
+ pm_runtime_put_autosuspend(adev_to_drm(dev->adev)->dev);
}
return ERR_PTR(err);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index 64fdf63..814f998 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -26,7 +26,7 @@
#include "amdgpu_sync.h"
#include "amdgpu_object.h"
#include "amdgpu_vm.h"
-#include "amdgpu_mn.h"
+#include "amdgpu_hmm.h"
#include "amdgpu.h"
#include "amdgpu_xgmi.h"
#include "kfd_priv.h"
@@ -259,7 +259,7 @@ void svm_range_free_dma_mappings(struct svm_range *prange)
pr_debug("failed to find device idx %d\n", gpuidx);
continue;
}
- dev = &pdd->dev->pdev->dev;
+ dev = &pdd->dev->adev->pdev->dev;
svm_range_dma_unmap(dev, dma_addr, 0, prange->npages);
kvfree(dma_addr);
prange->dma_addr[gpuidx] = NULL;
@@ -1586,8 +1586,8 @@ static int svm_range_validate_and_map(struct mm_struct *mm,
unsigned long npages;
bool readonly;
- vma = find_vma(mm, addr);
- if (!vma || addr < vma->vm_start) {
+ vma = vma_lookup(mm, addr);
+ if (!vma) {
r = -EFAULT;
goto unreserve_out;
}
@@ -1596,9 +1596,9 @@ static int svm_range_validate_and_map(struct mm_struct *mm,
next = min(vma->vm_end, end);
npages = (next - addr) >> PAGE_SHIFT;
WRITE_ONCE(p->svms.faulting_task, current);
- r = amdgpu_hmm_range_get_pages(&prange->notifier, mm, NULL,
- addr, npages, &hmm_range,
- readonly, true, owner);
+ r = amdgpu_hmm_range_get_pages(&prange->notifier, addr, npages,
+ readonly, owner, NULL,
+ &hmm_range);
WRITE_ONCE(p->svms.faulting_task, NULL);
if (r) {
pr_debug("failed %d to get svm range pages\n", r);
@@ -2542,8 +2542,8 @@ svm_range_get_range_boundaries(struct kfd_process *p, int64_t addr,
struct interval_tree_node *node;
unsigned long start_limit, end_limit;
- vma = find_vma(p->mm, addr << PAGE_SHIFT);
- if (!vma || (addr << PAGE_SHIFT) < vma->vm_start) {
+ vma = vma_lookup(p->mm, addr << PAGE_SHIFT);
+ if (!vma) {
pr_debug("VMA does not exist in address [0x%llx]\n", addr);
return -EFAULT;
}
@@ -2871,8 +2871,8 @@ svm_range_restore_pages(struct amdgpu_device *adev, unsigned int pasid,
/* __do_munmap removed VMA, return success as we are handling stale
* retry fault.
*/
- vma = find_vma(mm, addr << PAGE_SHIFT);
- if (!vma || (addr << PAGE_SHIFT) < vma->vm_start) {
+ vma = vma_lookup(mm, addr << PAGE_SHIFT);
+ if (!vma) {
pr_debug("address 0x%llx VMA is removed\n", addr);
r = 0;
goto out_unlock_range;
@@ -3152,9 +3152,8 @@ svm_range_is_valid(struct kfd_process *p, uint64_t start, uint64_t size)
start <<= PAGE_SHIFT;
end = start + (size << PAGE_SHIFT);
do {
- vma = find_vma(p->mm, start);
- if (!vma || start < vma->vm_start ||
- (vma->vm_flags & device_vma))
+ vma = vma_lookup(p->mm, start);
+ if (!vma || (vma->vm_flags & device_vma))
return -EFAULT;
start = min(end, vma->vm_end);
} while (start < end);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
index 3f0a4a4..bceb1a5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
@@ -115,7 +115,7 @@ struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev)
down_read(&topology_lock);
list_for_each_entry(top_dev, &topology_device_list, list)
- if (top_dev->gpu && top_dev->gpu->pdev == pdev) {
+ if (top_dev->gpu && top_dev->gpu->adev->pdev == pdev) {
device = top_dev->gpu;
break;
}
@@ -364,7 +364,6 @@ static ssize_t kfd_cache_show(struct kobject *kobj, struct attribute *attr,
/* Making sure that the buffer is an empty string */
buffer[0] = 0;
-
cache = container_of(attr, struct kfd_cache_properties, attr);
if (cache->gpu && kfd_devcgroup_check_permission(cache->gpu))
return -EPERM;
@@ -379,12 +378,13 @@ static ssize_t kfd_cache_show(struct kobject *kobj, struct attribute *attr,
sysfs_show_32bit_prop(buffer, offs, "association", cache->cache_assoc);
sysfs_show_32bit_prop(buffer, offs, "latency", cache->cache_latency);
sysfs_show_32bit_prop(buffer, offs, "type", cache->cache_type);
+
offs += snprintf(buffer+offs, PAGE_SIZE-offs, "sibling_map ");
- for (i = 0; i < CRAT_SIBLINGMAP_SIZE; i++)
+ for (i = 0; i < cache->sibling_map_size; i++)
for (j = 0; j < sizeof(cache->sibling_map[0])*8; j++)
/* Check each bit */
offs += snprintf(buffer+offs, PAGE_SIZE-offs, "%d,",
- (cache->sibling_map[i] >> j) & 1);
+ (cache->sibling_map[i] >> j) & 1);
/* Replace the last "," with end of line */
buffer[offs-1] = '\n';
@@ -1169,13 +1169,12 @@ static uint32_t kfd_generate_gpu_id(struct kfd_dev *gpu)
local_mem_size = gpu->local_mem_info.local_mem_size_private +
gpu->local_mem_info.local_mem_size_public;
-
- buf[0] = gpu->pdev->devfn;
- buf[1] = gpu->pdev->subsystem_vendor |
- (gpu->pdev->subsystem_device << 16);
- buf[2] = pci_domain_nr(gpu->pdev->bus);
- buf[3] = gpu->pdev->device;
- buf[4] = gpu->pdev->bus->number;
+ buf[0] = gpu->adev->pdev->devfn;
+ buf[1] = gpu->adev->pdev->subsystem_vendor |
+ (gpu->adev->pdev->subsystem_device << 16);
+ buf[2] = pci_domain_nr(gpu->adev->pdev->bus);
+ buf[3] = gpu->adev->pdev->device;
+ buf[4] = gpu->adev->pdev->bus->number;
buf[5] = lower_32_bits(local_mem_size);
buf[6] = upper_32_bits(local_mem_size);
@@ -1198,7 +1197,6 @@ static struct kfd_topology_device *kfd_assign_gpu(struct kfd_dev *gpu)
struct kfd_iolink_properties *iolink;
struct kfd_iolink_properties *p2plink;
- down_write(&topology_lock);
list_for_each_entry(dev, &topology_device_list, list) {
/* Discrete GPUs need their own topology device list
* entries. Don't assign them to CPU/APU nodes.
@@ -1222,7 +1220,6 @@ static struct kfd_topology_device *kfd_assign_gpu(struct kfd_dev *gpu)
break;
}
}
- up_write(&topology_lock);
return out_dev;
}
@@ -1269,7 +1266,7 @@ static void kfd_set_iolink_no_atomics(struct kfd_topology_device *dev,
if (target_gpu_dev) {
uint32_t cap;
- pcie_capability_read_dword(target_gpu_dev->gpu->pdev,
+ pcie_capability_read_dword(target_gpu_dev->gpu->adev->pdev,
PCI_EXP_DEVCAP2, &cap);
if (!(cap & (PCI_EXP_DEVCAP2_ATOMIC_COMP32 |
@@ -1593,21 +1590,290 @@ static int kfd_dev_create_p2p_links(void)
return ret;
}
+
+/* Helper function. See kfd_fill_gpu_cache_info for parameter description */
+static int fill_in_l1_pcache(struct kfd_cache_properties **props_ext,
+ struct kfd_gpu_cache_info *pcache_info,
+ struct kfd_cu_info *cu_info,
+ int cu_bitmask,
+ int cache_type, unsigned int cu_processor_id,
+ int cu_block)
+{
+ unsigned int cu_sibling_map_mask;
+ int first_active_cu;
+ struct kfd_cache_properties *pcache = NULL;
+
+ cu_sibling_map_mask = cu_bitmask;
+ cu_sibling_map_mask >>= cu_block;
+ cu_sibling_map_mask &= ((1 << pcache_info[cache_type].num_cu_shared) - 1);
+ first_active_cu = ffs(cu_sibling_map_mask);
+
+ /* CU could be inactive. In case of shared cache find the first active
+ * CU. and incase of non-shared cache check if the CU is inactive. If
+ * inactive active skip it
+ */
+ if (first_active_cu) {
+ pcache = kfd_alloc_struct(pcache);
+ if (!pcache)
+ return -ENOMEM;
+
+ memset(pcache, 0, sizeof(struct kfd_cache_properties));
+ pcache->processor_id_low = cu_processor_id + (first_active_cu - 1);
+ pcache->cache_level = pcache_info[cache_type].cache_level;
+ pcache->cache_size = pcache_info[cache_type].cache_size;
+
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_DATA_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_DATA;
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_INST_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_INSTRUCTION;
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_CPU_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_CPU;
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_SIMD_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_HSACU;
+
+ /* Sibling map is w.r.t processor_id_low, so shift out
+ * inactive CU
+ */
+ cu_sibling_map_mask =
+ cu_sibling_map_mask >> (first_active_cu - 1);
+
+ pcache->sibling_map[0] = (uint8_t)(cu_sibling_map_mask & 0xFF);
+ pcache->sibling_map[1] =
+ (uint8_t)((cu_sibling_map_mask >> 8) & 0xFF);
+ pcache->sibling_map[2] =
+ (uint8_t)((cu_sibling_map_mask >> 16) & 0xFF);
+ pcache->sibling_map[3] =
+ (uint8_t)((cu_sibling_map_mask >> 24) & 0xFF);
+
+ pcache->sibling_map_size = 4;
+ *props_ext = pcache;
+
+ return 0;
+ }
+ return 1;
+}
+
+/* Helper function. See kfd_fill_gpu_cache_info for parameter description */
+static int fill_in_l2_l3_pcache(struct kfd_cache_properties **props_ext,
+ struct kfd_gpu_cache_info *pcache_info,
+ struct kfd_cu_info *cu_info,
+ int cache_type, unsigned int cu_processor_id)
+{
+ unsigned int cu_sibling_map_mask;
+ int first_active_cu;
+ int i, j, k;
+ struct kfd_cache_properties *pcache = NULL;
+
+ cu_sibling_map_mask = cu_info->cu_bitmap[0][0];
+ cu_sibling_map_mask &=
+ ((1 << pcache_info[cache_type].num_cu_shared) - 1);
+ first_active_cu = ffs(cu_sibling_map_mask);
+
+ /* CU could be inactive. In case of shared cache find the first active
+ * CU. and incase of non-shared cache check if the CU is inactive. If
+ * inactive active skip it
+ */
+ if (first_active_cu) {
+ pcache = kfd_alloc_struct(pcache);
+ if (!pcache)
+ return -ENOMEM;
+
+ memset(pcache, 0, sizeof(struct kfd_cache_properties));
+ pcache->processor_id_low = cu_processor_id
+ + (first_active_cu - 1);
+ pcache->cache_level = pcache_info[cache_type].cache_level;
+ pcache->cache_size = pcache_info[cache_type].cache_size;
+
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_DATA_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_DATA;
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_INST_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_INSTRUCTION;
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_CPU_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_CPU;
+ if (pcache_info[cache_type].flags & CRAT_CACHE_FLAGS_SIMD_CACHE)
+ pcache->cache_type |= HSA_CACHE_TYPE_HSACU;
+
+ /* Sibling map is w.r.t processor_id_low, so shift out
+ * inactive CU
+ */
+ cu_sibling_map_mask = cu_sibling_map_mask >> (first_active_cu - 1);
+ k = 0;
+
+ for (i = 0; i < cu_info->num_shader_engines; i++) {
+ for (j = 0; j < cu_info->num_shader_arrays_per_engine; j++) {
+ pcache->sibling_map[k] = (uint8_t)(cu_sibling_map_mask & 0xFF);
+ pcache->sibling_map[k+1] = (uint8_t)((cu_sibling_map_mask >> 8) & 0xFF);
+ pcache->sibling_map[k+2] = (uint8_t)((cu_sibling_map_mask >> 16) & 0xFF);
+ pcache->sibling_map[k+3] = (uint8_t)((cu_sibling_map_mask >> 24) & 0xFF);
+ k += 4;
+
+ cu_sibling_map_mask = cu_info->cu_bitmap[i % 4][j + i / 4];
+ cu_sibling_map_mask &= ((1 << pcache_info[cache_type].num_cu_shared) - 1);
+ }
+ }
+ pcache->sibling_map_size = k;
+ *props_ext = pcache;
+ return 0;
+ }
+ return 1;
+}
+
+#define KFD_MAX_CACHE_TYPES 6
+
+/* kfd_fill_cache_non_crat_info - Fill GPU cache info using kfd_gpu_cache_info
+ * tables
+ */
+static void kfd_fill_cache_non_crat_info(struct kfd_topology_device *dev, struct kfd_dev *kdev)
+{
+ struct kfd_gpu_cache_info *pcache_info = NULL;
+ int i, j, k;
+ int ct = 0;
+ unsigned int cu_processor_id;
+ int ret;
+ unsigned int num_cu_shared;
+ struct kfd_cu_info cu_info;
+ struct kfd_cu_info *pcu_info;
+ int gpu_processor_id;
+ struct kfd_cache_properties *props_ext;
+ int num_of_entries = 0;
+ int num_of_cache_types = 0;
+ struct kfd_gpu_cache_info cache_info[KFD_MAX_CACHE_TYPES];
+
+ amdgpu_amdkfd_get_cu_info(kdev->adev, &cu_info);
+ pcu_info = &cu_info;
+
+ gpu_processor_id = dev->node_props.simd_id_base;
+
+ pcache_info = cache_info;
+ num_of_cache_types = kfd_get_gpu_cache_info(kdev, &pcache_info);
+ if (!num_of_cache_types) {
+ pr_warn("no cache info found\n");
+ return;
+ }
+
+ /* For each type of cache listed in the kfd_gpu_cache_info table,
+ * go through all available Compute Units.
+ * The [i,j,k] loop will
+ * if kfd_gpu_cache_info.num_cu_shared = 1
+ * will parse through all available CU
+ * If (kfd_gpu_cache_info.num_cu_shared != 1)
+ * then it will consider only one CU from
+ * the shared unit
+ */
+ for (ct = 0; ct < num_of_cache_types; ct++) {
+ cu_processor_id = gpu_processor_id;
+ if (pcache_info[ct].cache_level == 1) {
+ for (i = 0; i < pcu_info->num_shader_engines; i++) {
+ for (j = 0; j < pcu_info->num_shader_arrays_per_engine; j++) {
+ for (k = 0; k < pcu_info->num_cu_per_sh; k += pcache_info[ct].num_cu_shared) {
+
+ ret = fill_in_l1_pcache(&props_ext, pcache_info, pcu_info,
+ pcu_info->cu_bitmap[i % 4][j + i / 4], ct,
+ cu_processor_id, k);
+
+ if (ret < 0)
+ break;
+
+ if (!ret) {
+ num_of_entries++;
+ list_add_tail(&props_ext->list, &dev->cache_props);
+ }
+
+ /* Move to next CU block */
+ num_cu_shared = ((k + pcache_info[ct].num_cu_shared) <=
+ pcu_info->num_cu_per_sh) ?
+ pcache_info[ct].num_cu_shared :
+ (pcu_info->num_cu_per_sh - k);
+ cu_processor_id += num_cu_shared;
+ }
+ }
+ }
+ } else {
+ ret = fill_in_l2_l3_pcache(&props_ext, pcache_info,
+ pcu_info, ct, cu_processor_id);
+
+ if (ret < 0)
+ break;
+
+ if (!ret) {
+ num_of_entries++;
+ list_add_tail(&props_ext->list, &dev->cache_props);
+ }
+ }
+ }
+ dev->node_props.caches_count += num_of_entries;
+ pr_debug("Added [%d] GPU cache entries\n", num_of_entries);
+}
+
+static int kfd_topology_add_device_locked(struct kfd_dev *gpu, uint32_t gpu_id,
+ struct kfd_topology_device **dev)
+{
+ int proximity_domain = ++topology_crat_proximity_domain;
+ struct list_head temp_topology_device_list;
+ void *crat_image = NULL;
+ size_t image_size = 0;
+ int res;
+
+ res = kfd_create_crat_image_virtual(&crat_image, &image_size,
+ COMPUTE_UNIT_GPU, gpu,
+ proximity_domain);
+ if (res) {
+ pr_err("Error creating VCRAT for GPU (ID: 0x%x)\n",
+ gpu_id);
+ topology_crat_proximity_domain--;
+ goto err;
+ }
+
+ INIT_LIST_HEAD(&temp_topology_device_list);
+
+ res = kfd_parse_crat_table(crat_image,
+ &temp_topology_device_list,
+ proximity_domain);
+ if (res) {
+ pr_err("Error parsing VCRAT for GPU (ID: 0x%x)\n",
+ gpu_id);
+ topology_crat_proximity_domain--;
+ goto err;
+ }
+
+ kfd_topology_update_device_list(&temp_topology_device_list,
+ &topology_device_list);
+
+ *dev = kfd_assign_gpu(gpu);
+ if (WARN_ON(!*dev)) {
+ res = -ENODEV;
+ goto err;
+ }
+
+ /* Fill the cache affinity information here for the GPUs
+ * using VCRAT
+ */
+ kfd_fill_cache_non_crat_info(*dev, gpu);
+
+ /* Update the SYSFS tree, since we added another topology
+ * device
+ */
+ res = kfd_topology_update_sysfs();
+ if (!res)
+ sys_props.generation_count++;
+ else
+ pr_err("Failed to update GPU (ID: 0x%x) to sysfs topology. res=%d\n",
+ gpu_id, res);
+
+err:
+ kfd_destroy_crat_image(crat_image);
+ return res;
+}
+
int kfd_topology_add_device(struct kfd_dev *gpu)
{
uint32_t gpu_id;
struct kfd_topology_device *dev;
struct kfd_cu_info cu_info;
int res = 0;
- struct list_head temp_topology_device_list;
- void *crat_image = NULL;
- size_t image_size = 0;
- int proximity_domain;
int i;
const char *asic_name = amdgpu_asic_name[gpu->adev->asic_type];
- INIT_LIST_HEAD(&temp_topology_device_list);
-
gpu_id = kfd_generate_gpu_id(gpu);
pr_debug("Adding new GPU (ID: 0x%x) to topology\n", gpu_id);
@@ -1617,50 +1883,13 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
* CRAT to create a new topology device. Once created assign the gpu to
* that topology device
*/
+ down_write(&topology_lock);
dev = kfd_assign_gpu(gpu);
- if (!dev) {
- down_write(&topology_lock);
- proximity_domain = ++topology_crat_proximity_domain;
-
- res = kfd_create_crat_image_virtual(&crat_image, &image_size,
- COMPUTE_UNIT_GPU, gpu,
- proximity_domain);
- if (res) {
- pr_err("Error creating VCRAT for GPU (ID: 0x%x)\n",
- gpu_id);
- topology_crat_proximity_domain--;
- return res;
- }
- res = kfd_parse_crat_table(crat_image,
- &temp_topology_device_list,
- proximity_domain);
- if (res) {
- pr_err("Error parsing VCRAT for GPU (ID: 0x%x)\n",
- gpu_id);
- topology_crat_proximity_domain--;
- goto err;
- }
-
- kfd_topology_update_device_list(&temp_topology_device_list,
- &topology_device_list);
-
- /* Update the SYSFS tree, since we added another topology
- * device
- */
- res = kfd_topology_update_sysfs();
- up_write(&topology_lock);
-
- if (!res)
- sys_props.generation_count++;
- else
- pr_err("Failed to update GPU (ID: 0x%x) to sysfs topology. res=%d\n",
- gpu_id, res);
- dev = kfd_assign_gpu(gpu);
- if (WARN_ON(!dev)) {
- res = -ENODEV;
- goto err;
- }
- }
+ if (!dev)
+ res = kfd_topology_add_device_locked(gpu, gpu_id, &dev);
+ up_write(&topology_lock);
+ if (res)
+ return res;
dev->gpu_id = gpu_id;
gpu->id = gpu_id;
@@ -1688,13 +1917,13 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
cu_info.num_shader_arrays_per_engine;
dev->node_props.gfx_target_version = gpu->device_info.gfx_target_version;
- dev->node_props.vendor_id = gpu->pdev->vendor;
- dev->node_props.device_id = gpu->pdev->device;
+ dev->node_props.vendor_id = gpu->adev->pdev->vendor;
+ dev->node_props.device_id = gpu->adev->pdev->device;
dev->node_props.capability |=
((dev->gpu->adev->rev_id << HSA_CAP_ASIC_REVISION_SHIFT) &
HSA_CAP_ASIC_REVISION_MASK);
- dev->node_props.location_id = pci_dev_id(gpu->pdev);
- dev->node_props.domain = pci_domain_nr(gpu->pdev->bus);
+ dev->node_props.location_id = pci_dev_id(gpu->adev->pdev);
+ dev->node_props.domain = pci_domain_nr(gpu->adev->pdev->bus);
dev->node_props.max_engine_clk_fcompute =
amdgpu_amdkfd_get_max_engine_clock_in_mhz(dev->gpu->adev);
dev->node_props.max_engine_clk_ccompute =
@@ -1783,11 +2012,9 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
kfd_debug_print_topology();
- if (!res)
- kfd_notify_gpu_change(gpu_id, 1);
-err:
- kfd_destroy_crat_image(crat_image);
- return res;
+ kfd_notify_gpu_change(gpu_id, 1);
+
+ return 0;
}
/**
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
index 9f6c949..fca30d0 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
@@ -80,6 +80,8 @@ struct kfd_mem_properties {
struct attribute attr;
};
+#define CACHE_SIBLINGMAP_SIZE 64
+
struct kfd_cache_properties {
struct list_head list;
uint32_t processor_id_low;
@@ -90,10 +92,11 @@ struct kfd_cache_properties {
uint32_t cache_assoc;
uint32_t cache_latency;
uint32_t cache_type;
- uint8_t sibling_map[CRAT_SIBLINGMAP_SIZE];
+ uint8_t sibling_map[CACHE_SIBLINGMAP_SIZE];
struct kfd_dev *gpu;
struct kobject *kobj;
struct attribute attr;
+ uint32_t sibling_map_size;
};
struct kfd_iolink_properties {
@@ -128,7 +131,6 @@ struct kfd_topology_device {
uint32_t proximity_domain;
struct kfd_node_properties node_props;
struct list_head mem_props;
- uint32_t cache_count;
struct list_head cache_props;
struct list_head io_link_props;
struct list_head p2p_link_props;
diff --git a/drivers/gpu/drm/amd/display/Kconfig b/drivers/gpu/drm/amd/display/Kconfig
index f4f3d26..2efe93f 100644
--- a/drivers/gpu/drm/amd/display/Kconfig
+++ b/drivers/gpu/drm/amd/display/Kconfig
@@ -7,7 +7,8 @@
default y
depends on BROKEN || !CC_IS_CLANG || X86_64 || SPARC64 || ARM64
select SND_HDA_COMPONENT if SND_HDA_CORE
- select DRM_AMD_DC_DCN if (X86 || PPC_LONG_DOUBLE_128)
+ # !CC_IS_CLANG: https://github.com/ClangBuiltLinux/linux/issues/1752
+ select DRM_AMD_DC_DCN if (X86 || PPC_LONG_DOUBLE_128 || (ARM64 && KERNEL_MODE_NEON && !CC_IS_CLANG))
help
Choose this option if you want to use the new display engine
support for AMDGPU. This adds required support for Vega and
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 512c323..77277d9 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -82,7 +82,6 @@
#include <drm/drm_atomic_uapi.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_blend.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_edid.h>
#include <drm/drm_vblank.h>
@@ -147,14 +146,6 @@ MODULE_FIRMWARE(FIRMWARE_NAVI12_DMCU);
/* Number of bytes in PSP footer for firmware. */
#define PSP_FOOTER_BYTES 0x100
-/*
- * DMUB Async to Sync Mechanism Status
- */
-#define DMUB_ASYNC_TO_SYNC_ACCESS_FAIL 1
-#define DMUB_ASYNC_TO_SYNC_ACCESS_TIMEOUT 2
-#define DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS 3
-#define DMUB_ASYNC_TO_SYNC_ACCESS_INVALID 4
-
/**
* DOC: overview
*
@@ -1105,7 +1096,7 @@ static int dm_dmub_hw_init(struct amdgpu_device *adev)
/* Initialize hardware. */
memset(&hw_params, 0, sizeof(hw_params));
hw_params.fb_base = adev->gmc.fb_start;
- hw_params.fb_offset = adev->gmc.aper_base;
+ hw_params.fb_offset = adev->vm_manager.vram_base_offset;
/* backdoor load firmware and trigger dmub running */
if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP)
@@ -1227,7 +1218,7 @@ static void mmhub_read_system_context(struct amdgpu_device *adev, struct dc_phy_
pa_config->system_aperture.agp_top = (uint64_t)agp_top << 24;
pa_config->system_aperture.fb_base = adev->gmc.fb_start;
- pa_config->system_aperture.fb_offset = adev->gmc.aper_base;
+ pa_config->system_aperture.fb_offset = adev->vm_manager.vram_base_offset;
pa_config->system_aperture.fb_top = adev->gmc.fb_end;
pa_config->gart_config.page_table_start_addr = page_table_start.quad_part << 12;
@@ -1442,9 +1433,9 @@ static int amdgpu_dm_init(struct amdgpu_device *adev)
memset(&init_params, 0, sizeof(init_params));
#endif
+ mutex_init(&adev->dm.dpia_aux_lock);
mutex_init(&adev->dm.dc_lock);
mutex_init(&adev->dm.audio_lock);
- spin_lock_init(&adev->dm.vblank_lock);
if(amdgpu_dm_irq_init(adev)) {
DRM_ERROR("amdgpu: failed to initialize DM IRQ support.\n");
@@ -1807,6 +1798,7 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev)
mutex_destroy(&adev->dm.audio_lock);
mutex_destroy(&adev->dm.dc_lock);
+ mutex_destroy(&adev->dm.dpia_aux_lock);
return;
}
@@ -2858,7 +2850,6 @@ const struct amdgpu_ip_block_version dm_ip_block =
static const struct drm_mode_config_funcs amdgpu_dm_mode_funcs = {
.fb_create = amdgpu_display_user_framebuffer_create,
.get_format_info = amd_get_format_info,
- .output_poll_changed = drm_fb_helper_output_poll_changed,
.atomic_check = amdgpu_dm_atomic_check,
.atomic_commit = drm_atomic_helper_commit,
};
@@ -3864,8 +3855,6 @@ static int amdgpu_dm_mode_config_init(struct amdgpu_device *adev)
/* indicates support for immediate flip */
adev_to_drm(adev)->mode_config.async_page_flip = true;
- adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
-
state = kzalloc(sizeof(*state), GFP_KERNEL);
if (!state)
return -ENOMEM;
@@ -4640,6 +4629,7 @@ static int dm_early_init(void *handle)
adev_to_drm(adev)->dev,
&dev_attr_s3_debug);
#endif
+ adev->dc_enabled = true;
return 0;
}
@@ -4879,6 +4869,35 @@ static int fill_dc_plane_attributes(struct amdgpu_device *adev,
return 0;
}
+static inline void fill_dc_dirty_rect(struct drm_plane *plane,
+ struct rect *dirty_rect, int32_t x,
+ int32_t y, int32_t width, int32_t height,
+ int *i, bool ffu)
+{
+ if (*i > DC_MAX_DIRTY_RECTS)
+ return;
+
+ if (*i == DC_MAX_DIRTY_RECTS)
+ goto out;
+
+ dirty_rect->x = x;
+ dirty_rect->y = y;
+ dirty_rect->width = width;
+ dirty_rect->height = height;
+
+ if (ffu)
+ drm_dbg(plane->dev,
+ "[PLANE:%d] PSR FFU dirty rect size (%d, %d)\n",
+ plane->base.id, width, height);
+ else
+ drm_dbg(plane->dev,
+ "[PLANE:%d] PSR SU dirty rect at (%d, %d) size (%d, %d)",
+ plane->base.id, x, y, width, height);
+
+out:
+ (*i)++;
+}
+
/**
* fill_dc_dirty_rects() - Fill DC dirty regions for PSR selective updates
*
@@ -4899,10 +4918,6 @@ static int fill_dc_plane_attributes(struct amdgpu_device *adev,
* addition, certain use cases - such as cursor and multi-plane overlay (MPO) -
* implicitly provide damage clips without any client support via the plane
* bounds.
- *
- * Today, amdgpu_dm only supports the MPO and cursor usecase.
- *
- * TODO: Also enable for FB_DAMAGE_CLIPS
*/
static void fill_dc_dirty_rects(struct drm_plane *plane,
struct drm_plane_state *old_plane_state,
@@ -4913,12 +4928,11 @@ static void fill_dc_dirty_rects(struct drm_plane *plane,
struct dm_crtc_state *dm_crtc_state = to_dm_crtc_state(crtc_state);
struct rect *dirty_rects = flip_addrs->dirty_rects;
uint32_t num_clips;
+ struct drm_mode_rect *clips;
bool bb_changed;
bool fb_changed;
uint32_t i = 0;
- flip_addrs->dirty_rect_count = 0;
-
/*
* Cursor plane has it's own dirty rect update interface. See
* dcn10_dmub_update_cursor_data and dmub_cmd_update_cursor_info_data
@@ -4926,20 +4940,20 @@ static void fill_dc_dirty_rects(struct drm_plane *plane,
if (plane->type == DRM_PLANE_TYPE_CURSOR)
return;
- /*
- * Today, we only consider MPO use-case for PSR SU. If MPO not
- * requested, and there is a plane update, do FFU.
- */
+ num_clips = drm_plane_get_damage_clips_count(new_plane_state);
+ clips = drm_plane_get_damage_clips(new_plane_state);
+
if (!dm_crtc_state->mpo_requested) {
- dirty_rects[0].x = 0;
- dirty_rects[0].y = 0;
- dirty_rects[0].width = dm_crtc_state->base.mode.crtc_hdisplay;
- dirty_rects[0].height = dm_crtc_state->base.mode.crtc_vdisplay;
- flip_addrs->dirty_rect_count = 1;
- DRM_DEBUG_DRIVER("[PLANE:%d] PSR FFU dirty rect size (%d, %d)\n",
- new_plane_state->plane->base.id,
- dm_crtc_state->base.mode.crtc_hdisplay,
- dm_crtc_state->base.mode.crtc_vdisplay);
+ if (!num_clips || num_clips > DC_MAX_DIRTY_RECTS)
+ goto ffu;
+
+ for (; flip_addrs->dirty_rect_count < num_clips; clips++)
+ fill_dc_dirty_rect(new_plane_state->plane,
+ &dirty_rects[i], clips->x1,
+ clips->y1, clips->x2 - clips->x1,
+ clips->y2 - clips->y1,
+ &flip_addrs->dirty_rect_count,
+ false);
return;
}
@@ -4950,7 +4964,6 @@ static void fill_dc_dirty_rects(struct drm_plane *plane,
* If plane is moved or resized, also add old bounding box to dirty
* rects.
*/
- num_clips = drm_plane_get_damage_clips_count(new_plane_state);
fb_changed = old_plane_state->fb->base.id !=
new_plane_state->fb->base.id;
bb_changed = (old_plane_state->crtc_x != new_plane_state->crtc_x ||
@@ -4958,36 +4971,51 @@ static void fill_dc_dirty_rects(struct drm_plane *plane,
old_plane_state->crtc_w != new_plane_state->crtc_w ||
old_plane_state->crtc_h != new_plane_state->crtc_h);
- DRM_DEBUG_DRIVER("[PLANE:%d] PSR bb_changed:%d fb_changed:%d num_clips:%d\n",
- new_plane_state->plane->base.id,
- bb_changed, fb_changed, num_clips);
+ drm_dbg(plane->dev,
+ "[PLANE:%d] PSR bb_changed:%d fb_changed:%d num_clips:%d\n",
+ new_plane_state->plane->base.id,
+ bb_changed, fb_changed, num_clips);
- if (num_clips || fb_changed || bb_changed) {
- dirty_rects[i].x = new_plane_state->crtc_x;
- dirty_rects[i].y = new_plane_state->crtc_y;
- dirty_rects[i].width = new_plane_state->crtc_w;
- dirty_rects[i].height = new_plane_state->crtc_h;
- DRM_DEBUG_DRIVER("[PLANE:%d] PSR SU dirty rect at (%d, %d) size (%d, %d)\n",
- new_plane_state->plane->base.id,
- dirty_rects[i].x, dirty_rects[i].y,
- dirty_rects[i].width, dirty_rects[i].height);
- i += 1;
- }
-
- /* Add old plane bounding-box if plane is moved or resized */
if (bb_changed) {
- dirty_rects[i].x = old_plane_state->crtc_x;
- dirty_rects[i].y = old_plane_state->crtc_y;
- dirty_rects[i].width = old_plane_state->crtc_w;
- dirty_rects[i].height = old_plane_state->crtc_h;
- DRM_DEBUG_DRIVER("[PLANE:%d] PSR SU dirty rect at (%d, %d) size (%d, %d)\n",
- old_plane_state->plane->base.id,
- dirty_rects[i].x, dirty_rects[i].y,
- dirty_rects[i].width, dirty_rects[i].height);
- i += 1;
+ fill_dc_dirty_rect(new_plane_state->plane, &dirty_rects[i],
+ new_plane_state->crtc_x,
+ new_plane_state->crtc_y,
+ new_plane_state->crtc_w,
+ new_plane_state->crtc_h, &i, false);
+
+ /* Add old plane bounding-box if plane is moved or resized */
+ fill_dc_dirty_rect(new_plane_state->plane, &dirty_rects[i],
+ old_plane_state->crtc_x,
+ old_plane_state->crtc_y,
+ old_plane_state->crtc_w,
+ old_plane_state->crtc_h, &i, false);
}
+ if (num_clips) {
+ for (; i < num_clips; clips++)
+ fill_dc_dirty_rect(new_plane_state->plane,
+ &dirty_rects[i], clips->x1,
+ clips->y1, clips->x2 - clips->x1,
+ clips->y2 - clips->y1, &i, false);
+ } else if (fb_changed && !bb_changed) {
+ fill_dc_dirty_rect(new_plane_state->plane, &dirty_rects[i],
+ new_plane_state->crtc_x,
+ new_plane_state->crtc_y,
+ new_plane_state->crtc_w,
+ new_plane_state->crtc_h, &i, false);
+ }
+
+ if (i > DC_MAX_DIRTY_RECTS)
+ goto ffu;
+
flip_addrs->dirty_rect_count = i;
+ return;
+
+ffu:
+ fill_dc_dirty_rect(new_plane_state->plane, &dirty_rects[0], 0, 0,
+ dm_crtc_state->base.mode.crtc_hdisplay,
+ dm_crtc_state->base.mode.crtc_vdisplay,
+ &flip_addrs->dirty_rect_count, true);
}
static void update_stream_scaling_settings(const struct drm_display_mode *mode,
@@ -5652,16 +5680,14 @@ static void apply_dsc_policy_for_stream(struct amdgpu_dm_connector *aconnector,
{
struct drm_connector *drm_connector = &aconnector->base;
uint32_t link_bandwidth_kbps;
- uint32_t max_dsc_target_bpp_limit_override = 0;
struct dc *dc = sink->ctx->dc;
uint32_t max_supported_bw_in_kbps, timing_bw_in_kbps;
uint32_t dsc_max_supported_bw_in_kbps;
+ uint32_t max_dsc_target_bpp_limit_override =
+ drm_connector->display_info.max_dsc_bpp;
link_bandwidth_kbps = dc_link_bandwidth_kbps(aconnector->dc_link,
dc_link_get_link_cap(aconnector->dc_link));
- if (stream->link && stream->link->local_sink)
- max_dsc_target_bpp_limit_override =
- stream->link->local_sink->edid_caps.panel_patch.max_dsc_target_bpp_limit;
/* Set DSC policy according to dsc_clock_en */
dc_dsc_policy_set_enable_dsc_when_not_needed(
@@ -5734,7 +5760,7 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector,
const struct drm_connector_state *con_state =
dm_state ? &dm_state->base : NULL;
struct dc_stream_state *stream = NULL;
- struct drm_display_mode mode = *drm_mode;
+ struct drm_display_mode mode;
struct drm_display_mode saved_mode;
struct drm_display_mode *freesync_mode = NULL;
bool native_mode_found = false;
@@ -5742,12 +5768,14 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector,
bool scale = dm_state ? (dm_state->scaling != RMX_OFF) : false;
int mode_refresh;
int preferred_refresh = 0;
+ enum color_transfer_func tf = TRANSFER_FUNC_UNKNOWN;
#if defined(CONFIG_DRM_AMD_DC_DCN)
struct dsc_dec_dpcd_caps dsc_caps;
#endif
struct dc_sink *sink = NULL;
+ drm_mode_init(&mode, drm_mode);
memset(&saved_mode, 0, sizeof(saved_mode));
if (aconnector == NULL) {
@@ -5865,7 +5893,9 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector,
if (stream->link->dpcd_caps.dprx_feature.bits.VSC_SDP_COLORIMETRY_SUPPORTED)
stream->use_vsc_sdp_for_colorimetry = true;
}
- mod_build_vsc_infopacket(stream, &stream->vsc_infopacket, stream->output_color_space);
+ if (stream->out_transfer_func->tf == TRANSFER_FUNCTION_GAMMA22)
+ tf = TRANSFER_FUNC_GAMMA_22;
+ mod_build_vsc_infopacket(stream, &stream->vsc_infopacket, stream->output_color_space, tf);
aconnector->psr_skip_count = AMDGPU_DM_PSR_ENTRY_DELAY;
}
@@ -6158,7 +6188,6 @@ static void create_eml_sink(struct amdgpu_dm_connector *aconnector)
aconnector->base.name);
aconnector->base.force = DRM_FORCE_OFF;
- aconnector->base.override_edid = false;
return;
}
@@ -6193,11 +6222,73 @@ static void handle_edid_mgmt(struct amdgpu_dm_connector *aconnector)
link->verified_link_cap.link_rate = LINK_RATE_HIGH2;
}
-
- aconnector->base.override_edid = true;
create_eml_sink(aconnector);
}
+static enum dc_status dm_validate_stream_and_context(struct dc *dc,
+ struct dc_stream_state *stream)
+{
+ enum dc_status dc_result = DC_ERROR_UNEXPECTED;
+ struct dc_plane_state *dc_plane_state = NULL;
+ struct dc_state *dc_state = NULL;
+
+ if (!stream)
+ goto cleanup;
+
+ dc_plane_state = dc_create_plane_state(dc);
+ if (!dc_plane_state)
+ goto cleanup;
+
+ dc_state = dc_create_state(dc);
+ if (!dc_state)
+ goto cleanup;
+
+ /* populate stream to plane */
+ dc_plane_state->src_rect.height = stream->src.height;
+ dc_plane_state->src_rect.width = stream->src.width;
+ dc_plane_state->dst_rect.height = stream->src.height;
+ dc_plane_state->dst_rect.width = stream->src.width;
+ dc_plane_state->clip_rect.height = stream->src.height;
+ dc_plane_state->clip_rect.width = stream->src.width;
+ dc_plane_state->plane_size.surface_pitch = ((stream->src.width + 255) / 256) * 256;
+ dc_plane_state->plane_size.surface_size.height = stream->src.height;
+ dc_plane_state->plane_size.surface_size.width = stream->src.width;
+ dc_plane_state->plane_size.chroma_size.height = stream->src.height;
+ dc_plane_state->plane_size.chroma_size.width = stream->src.width;
+ dc_plane_state->tiling_info.gfx9.swizzle = DC_SW_UNKNOWN;
+ dc_plane_state->format = SURFACE_PIXEL_FORMAT_GRPH_ARGB8888;
+ dc_plane_state->tiling_info.gfx9.swizzle = DC_SW_UNKNOWN;
+ dc_plane_state->rotation = ROTATION_ANGLE_0;
+ dc_plane_state->is_tiling_rotated = false;
+ dc_plane_state->tiling_info.gfx8.array_mode = DC_ARRAY_LINEAR_GENERAL;
+
+ dc_result = dc_validate_stream(dc, stream);
+ if (dc_result == DC_OK)
+ dc_result = dc_validate_plane(dc, dc_plane_state);
+
+ if (dc_result == DC_OK)
+ dc_result = dc_add_stream_to_ctx(dc, dc_state, stream);
+
+ if (dc_result == DC_OK && !dc_add_plane_to_context(
+ dc,
+ stream,
+ dc_plane_state,
+ dc_state))
+ dc_result = DC_FAIL_ATTACH_SURFACES;
+
+ if (dc_result == DC_OK)
+ dc_result = dc_validate_global_state(dc, dc_state, true);
+
+cleanup:
+ if (dc_state)
+ dc_release_state(dc_state);
+
+ if (dc_plane_state)
+ dc_plane_state_release(dc_plane_state);
+
+ return dc_result;
+}
+
struct dc_stream_state *
create_validate_stream_for_sink(struct amdgpu_dm_connector *aconnector,
const struct drm_display_mode *drm_mode,
@@ -6224,6 +6315,9 @@ create_validate_stream_for_sink(struct amdgpu_dm_connector *aconnector,
if (dc_result == DC_OK && stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST)
dc_result = dm_dp_mst_is_port_support_mode(aconnector, stream);
+ if (dc_result == DC_OK)
+ dc_result = dm_validate_stream_and_context(adev->dm.dc, stream);
+
if (dc_result != DC_OK) {
DRM_DEBUG_KMS("Mode %dx%d (clk %d) failed DC validation with error %d (%s)\n",
drm_mode->hdisplay,
@@ -7889,6 +7983,9 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state,
*/
if (acrtc_state->stream->link->psr_settings.psr_version >= DC_PSR_VERSION_SU_1 &&
acrtc_attach->dm_irq_params.allow_psr_entry &&
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+ !amdgpu_dm_crc_window_is_activated(acrtc_state->base.crtc) &&
+#endif
!acrtc_state->stream->link->psr_settings.psr_allow_active)
amdgpu_dm_psr_enable(acrtc_state->stream);
} else {
@@ -8350,8 +8447,8 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
if (amdgpu_dm_crc_window_is_activated(crtc)) {
spin_lock_irqsave(&adev_to_drm(adev)->event_lock, flags);
- acrtc->dm_irq_params.crc_window.update_win = true;
- acrtc->dm_irq_params.crc_window.skip_frame_cnt = 2;
+ acrtc->dm_irq_params.window_param.update_win = true;
+ acrtc->dm_irq_params.window_param.skip_frame_cnt = 2;
spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock);
crc_rd_wrk->crtc = crtc;
spin_unlock_irq(&crc_rd_wrk->crc_rd_work_lock);
@@ -10139,91 +10236,92 @@ uint32_t dm_read_reg_func(const struct dc_context *ctx, uint32_t address,
return value;
}
-static int amdgpu_dm_set_dmub_async_sync_status(bool is_cmd_aux,
- struct dc_context *ctx,
- uint8_t status_type,
- uint32_t *operation_result)
+int amdgpu_dm_process_dmub_aux_transfer_sync(
+ struct dc_context *ctx,
+ unsigned int link_index,
+ struct aux_payload *payload,
+ enum aux_return_code_type *operation_result)
{
struct amdgpu_device *adev = ctx->driver_context;
- int return_status = -1;
struct dmub_notification *p_notify = adev->dm.dmub_notify;
+ int ret = -1;
- if (is_cmd_aux) {
- if (status_type == DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS) {
- return_status = p_notify->aux_reply.length;
- *operation_result = p_notify->result;
- } else if (status_type == DMUB_ASYNC_TO_SYNC_ACCESS_TIMEOUT) {
- *operation_result = AUX_RET_ERROR_TIMEOUT;
- } else if (status_type == DMUB_ASYNC_TO_SYNC_ACCESS_FAIL) {
- *operation_result = AUX_RET_ERROR_ENGINE_ACQUIRE;
- } else if (status_type == DMUB_ASYNC_TO_SYNC_ACCESS_INVALID) {
- *operation_result = AUX_RET_ERROR_INVALID_REPLY;
- } else {
- *operation_result = AUX_RET_ERROR_UNKNOWN;
- }
- } else {
- if (status_type == DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS) {
- return_status = 0;
- *operation_result = p_notify->sc_status;
- } else {
- *operation_result = SET_CONFIG_UNKNOWN_ERROR;
- }
+ mutex_lock(&adev->dm.dpia_aux_lock);
+ if (!dc_process_dmub_aux_transfer_async(ctx->dc, link_index, payload)) {
+ *operation_result = AUX_RET_ERROR_ENGINE_ACQUIRE;
+ goto out;
+ }
+
+ if (!wait_for_completion_timeout(&adev->dm.dmub_aux_transfer_done, 10 * HZ)) {
+ DRM_ERROR("wait_for_completion_timeout timeout!");
+ *operation_result = AUX_RET_ERROR_TIMEOUT;
+ goto out;
}
- return return_status;
+ if (p_notify->result != AUX_RET_SUCCESS) {
+ /*
+ * Transient states before tunneling is enabled could
+ * lead to this error. We can ignore this for now.
+ */
+ if (p_notify->result != AUX_RET_ERROR_PROTOCOL_ERROR) {
+ DRM_WARN("DPIA AUX failed on 0x%x(%d), error %d\n",
+ payload->address, payload->length,
+ p_notify->result);
+ }
+ *operation_result = AUX_RET_ERROR_INVALID_REPLY;
+ goto out;
+ }
+
+
+ payload->reply[0] = adev->dm.dmub_notify->aux_reply.command;
+ if (!payload->write && p_notify->aux_reply.length &&
+ (payload->reply[0] == AUX_TRANSACTION_REPLY_AUX_ACK)) {
+
+ if (payload->length != p_notify->aux_reply.length) {
+ DRM_WARN("invalid read length %d from DPIA AUX 0x%x(%d)!\n",
+ p_notify->aux_reply.length,
+ payload->address, payload->length);
+ *operation_result = AUX_RET_ERROR_INVALID_REPLY;
+ goto out;
+ }
+
+ memcpy(payload->data, p_notify->aux_reply.data,
+ p_notify->aux_reply.length);
+ }
+
+ /* success */
+ ret = p_notify->aux_reply.length;
+ *operation_result = p_notify->result;
+out:
+ mutex_unlock(&adev->dm.dpia_aux_lock);
+ return ret;
}
-int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux, struct dc_context *ctx,
- unsigned int link_index, void *cmd_payload, void *operation_result)
+int amdgpu_dm_process_dmub_set_config_sync(
+ struct dc_context *ctx,
+ unsigned int link_index,
+ struct set_config_cmd_payload *payload,
+ enum set_config_status *operation_result)
{
struct amdgpu_device *adev = ctx->driver_context;
- int ret = 0;
+ bool is_cmd_complete;
+ int ret;
- if (is_cmd_aux) {
- dc_process_dmub_aux_transfer_async(ctx->dc,
- link_index, (struct aux_payload *)cmd_payload);
- } else if (dc_process_dmub_set_config_async(ctx->dc, link_index,
- (struct set_config_cmd_payload *)cmd_payload,
- adev->dm.dmub_notify)) {
- return amdgpu_dm_set_dmub_async_sync_status(is_cmd_aux,
- ctx, DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS,
- (uint32_t *)operation_result);
- }
+ mutex_lock(&adev->dm.dpia_aux_lock);
+ is_cmd_complete = dc_process_dmub_set_config_async(ctx->dc,
+ link_index, payload, adev->dm.dmub_notify);
- ret = wait_for_completion_timeout(&adev->dm.dmub_aux_transfer_done, 10 * HZ);
- if (ret == 0) {
+ if (is_cmd_complete || wait_for_completion_timeout(&adev->dm.dmub_aux_transfer_done, 10 * HZ)) {
+ ret = 0;
+ *operation_result = adev->dm.dmub_notify->sc_status;
+ } else {
DRM_ERROR("wait_for_completion_timeout timeout!");
- return amdgpu_dm_set_dmub_async_sync_status(is_cmd_aux,
- ctx, DMUB_ASYNC_TO_SYNC_ACCESS_TIMEOUT,
- (uint32_t *)operation_result);
+ ret = -1;
+ *operation_result = SET_CONFIG_UNKNOWN_ERROR;
}
- if (is_cmd_aux) {
- if (adev->dm.dmub_notify->result == AUX_RET_SUCCESS) {
- struct aux_payload *payload = (struct aux_payload *)cmd_payload;
-
- payload->reply[0] = adev->dm.dmub_notify->aux_reply.command;
- if (!payload->write && adev->dm.dmub_notify->aux_reply.length &&
- payload->reply[0] == AUX_TRANSACTION_REPLY_AUX_ACK) {
-
- if (payload->length != adev->dm.dmub_notify->aux_reply.length) {
- DRM_WARN("invalid read from DPIA AUX %x(%d) got length %d!\n",
- payload->address, payload->length,
- adev->dm.dmub_notify->aux_reply.length);
- return amdgpu_dm_set_dmub_async_sync_status(is_cmd_aux, ctx,
- DMUB_ASYNC_TO_SYNC_ACCESS_INVALID,
- (uint32_t *)operation_result);
- }
-
- memcpy(payload->data, adev->dm.dmub_notify->aux_reply.data,
- adev->dm.dmub_notify->aux_reply.length);
- }
- }
- }
-
- return amdgpu_dm_set_dmub_async_sync_status(is_cmd_aux,
- ctx, DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS,
- (uint32_t *)operation_result);
+ mutex_unlock(&adev->dm.dpia_aux_lock);
+ return ret;
}
/*
@@ -10235,8 +10333,8 @@ int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux, struct dc_context
*/
bool check_seamless_boot_capability(struct amdgpu_device *adev)
{
- switch (adev->asic_type) {
- case CHIP_VANGOGH:
+ switch (adev->ip_versions[DCE_HWIP][0]) {
+ case IP_VERSION(3, 0, 1):
if (!adev->mman.keep_stolen_vga_memory)
return true;
break;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
index 635c398..df3c25e 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
@@ -59,7 +59,9 @@
#include "signal_types.h"
#include "amdgpu_dm_crc.h"
struct aux_payload;
+struct set_config_cmd_payload;
enum aux_return_code_type;
+enum set_config_status;
/* Forward declarations */
struct amdgpu_device;
@@ -360,13 +362,6 @@ struct amdgpu_display_manager {
struct mutex audio_lock;
/**
- * @vblank_lock:
- *
- * Guards access to deferred vblank work state.
- */
- spinlock_t vblank_lock;
-
- /**
* @audio_component:
*
* Used to notify ELD changes to sound driver.
@@ -549,6 +544,13 @@ struct amdgpu_display_manager {
* occurred on certain intel platform
*/
bool aux_hpd_discon_quirk;
+
+ /**
+ * @dpia_aux_lock:
+ *
+ * Guards access to DPIA AUX
+ */
+ struct mutex dpia_aux_lock;
};
enum dsc_clock_force_state {
@@ -792,9 +794,11 @@ void amdgpu_dm_update_connector_after_detect(
extern const struct drm_encoder_helper_funcs amdgpu_dm_encoder_helper_funcs;
-int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux,
- struct dc_context *ctx, unsigned int link_index,
- void *payload, void *operation_result);
+int amdgpu_dm_process_dmub_aux_transfer_sync(struct dc_context *ctx, unsigned int link_index,
+ struct aux_payload *payload, enum aux_return_code_type *operation_result);
+
+int amdgpu_dm_process_dmub_set_config_sync(struct dc_context *ctx, unsigned int link_index,
+ struct set_config_cmd_payload *payload, enum set_config_status *operation_result);
bool check_seamless_boot_capability(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c
index 8a441a2..66df239 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c
@@ -89,13 +89,13 @@ static void amdgpu_dm_set_crc_window_default(struct drm_crtc *crtc)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- acrtc->dm_irq_params.crc_window.x_start = 0;
- acrtc->dm_irq_params.crc_window.y_start = 0;
- acrtc->dm_irq_params.crc_window.x_end = 0;
- acrtc->dm_irq_params.crc_window.y_end = 0;
- acrtc->dm_irq_params.crc_window.activated = false;
- acrtc->dm_irq_params.crc_window.update_win = false;
- acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ acrtc->dm_irq_params.window_param.x_start = 0;
+ acrtc->dm_irq_params.window_param.y_start = 0;
+ acrtc->dm_irq_params.window_param.x_end = 0;
+ acrtc->dm_irq_params.window_param.y_end = 0;
+ acrtc->dm_irq_params.window_param.activated = false;
+ acrtc->dm_irq_params.window_param.update_win = false;
+ acrtc->dm_irq_params.window_param.skip_frame_cnt = 0;
spin_unlock_irq(&drm_dev->event_lock);
}
@@ -123,6 +123,8 @@ static void amdgpu_dm_crtc_notify_ta_to_read(struct work_struct *work)
phy_id = crc_rd_wrk->phy_inst;
spin_unlock_irq(&crc_rd_wrk->crc_rd_work_lock);
+ mutex_lock(&psp->securedisplay_context.mutex);
+
psp_prep_securedisplay_cmd_buf(psp, &securedisplay_cmd,
TA_SECUREDISPLAY_COMMAND__SEND_ROI_CRC);
securedisplay_cmd->securedisplay_in_message.send_roi_crc.phy_id =
@@ -133,6 +135,24 @@ static void amdgpu_dm_crtc_notify_ta_to_read(struct work_struct *work)
psp_securedisplay_parse_resp_status(psp, securedisplay_cmd->status);
}
}
+
+ mutex_unlock(&psp->securedisplay_context.mutex);
+}
+
+static void
+amdgpu_dm_forward_crc_window(struct work_struct *work)
+{
+ struct crc_fw_work *crc_fw_wrk;
+ struct amdgpu_display_manager *dm;
+
+ crc_fw_wrk = container_of(work, struct crc_fw_work, forward_roi_work);
+ dm = crc_fw_wrk->dm;
+
+ mutex_lock(&dm->dc_lock);
+ dc_stream_forward_crc_window(dm->dc, &crc_fw_wrk->rect, crc_fw_wrk->stream, crc_fw_wrk->is_stop_cmd);
+ mutex_unlock(&dm->dc_lock);
+
+ kfree(crc_fw_wrk);
}
bool amdgpu_dm_crc_window_is_activated(struct drm_crtc *crtc)
@@ -142,7 +162,7 @@ bool amdgpu_dm_crc_window_is_activated(struct drm_crtc *crtc)
bool ret = false;
spin_lock_irq(&drm_dev->event_lock);
- ret = acrtc->dm_irq_params.crc_window.activated;
+ ret = acrtc->dm_irq_params.window_param.activated;
spin_unlock_irq(&drm_dev->event_lock);
return ret;
@@ -187,9 +207,11 @@ int amdgpu_dm_crtc_configure_crc_source(struct drm_crtc *crtc,
if (adev->dm.crc_rd_wrk) {
flush_work(&adev->dm.crc_rd_wrk->notify_ta_work);
spin_lock_irq(&adev->dm.crc_rd_wrk->crc_rd_work_lock);
+
if (adev->dm.crc_rd_wrk->crtc == crtc) {
- dc_stream_stop_dmcu_crc_win_update(stream_state->ctx->dc,
- dm_crtc_state->stream);
+ /* stop ROI update on this crtc */
+ dc_stream_forward_crc_window(stream_state->ctx->dc,
+ NULL, stream_state, true);
adev->dm.crc_rd_wrk->crtc = NULL;
}
spin_unlock_irq(&adev->dm.crc_rd_wrk->crc_rd_work_lock);
@@ -439,14 +461,9 @@ void amdgpu_dm_crtc_handle_crc_window_irq(struct drm_crtc *crtc)
enum amdgpu_dm_pipe_crc_source cur_crc_src;
struct amdgpu_crtc *acrtc = NULL;
struct amdgpu_device *adev = NULL;
- struct crc_rd_work *crc_rd_wrk = NULL;
- struct crc_params *crc_window = NULL, tmp_window;
+ struct crc_rd_work *crc_rd_wrk;
+ struct crc_fw_work *crc_fw_wrk;
unsigned long flags1, flags2;
- struct crtc_position position;
- uint32_t v_blank;
- uint32_t v_back_porch;
- uint32_t crc_window_latch_up_line;
- struct dc_crtc_timing *timing_out;
if (crtc == NULL)
return;
@@ -458,74 +475,54 @@ void amdgpu_dm_crtc_handle_crc_window_irq(struct drm_crtc *crtc)
spin_lock_irqsave(&drm_dev->event_lock, flags1);
stream_state = acrtc->dm_irq_params.stream;
cur_crc_src = acrtc->dm_irq_params.crc_src;
- timing_out = &stream_state->timing;
/* Early return if CRC capture is not enabled. */
if (!amdgpu_dm_is_valid_crc_source(cur_crc_src))
goto cleanup;
- if (dm_is_crc_source_crtc(cur_crc_src)) {
- if (acrtc->dm_irq_params.crc_window.activated) {
- if (acrtc->dm_irq_params.crc_window.update_win) {
- if (acrtc->dm_irq_params.crc_window.skip_frame_cnt) {
- acrtc->dm_irq_params.crc_window.skip_frame_cnt -= 1;
- goto cleanup;
- }
- crc_window = &tmp_window;
+ if (!dm_is_crc_source_crtc(cur_crc_src))
+ goto cleanup;
- tmp_window.windowa_x_start =
- acrtc->dm_irq_params.crc_window.x_start;
- tmp_window.windowa_y_start =
- acrtc->dm_irq_params.crc_window.y_start;
- tmp_window.windowa_x_end =
- acrtc->dm_irq_params.crc_window.x_end;
- tmp_window.windowa_y_end =
- acrtc->dm_irq_params.crc_window.y_end;
- tmp_window.windowb_x_start =
- acrtc->dm_irq_params.crc_window.x_start;
- tmp_window.windowb_y_start =
- acrtc->dm_irq_params.crc_window.y_start;
- tmp_window.windowb_x_end =
- acrtc->dm_irq_params.crc_window.x_end;
- tmp_window.windowb_y_end =
- acrtc->dm_irq_params.crc_window.y_end;
+ if (!acrtc->dm_irq_params.window_param.activated)
+ goto cleanup;
- dc_stream_forward_dmcu_crc_window(stream_state->ctx->dc,
- stream_state, crc_window);
+ if (acrtc->dm_irq_params.window_param.update_win) {
+ if (acrtc->dm_irq_params.window_param.skip_frame_cnt) {
+ acrtc->dm_irq_params.window_param.skip_frame_cnt -= 1;
+ goto cleanup;
+ }
- acrtc->dm_irq_params.crc_window.update_win = false;
+ /* prepare work for dmub to update ROI */
+ crc_fw_wrk = kzalloc(sizeof(*crc_fw_wrk), GFP_ATOMIC);
+ if (!crc_fw_wrk)
+ goto cleanup;
- dc_stream_get_crtc_position(stream_state->ctx->dc, &stream_state, 1,
- &position.vertical_count,
- &position.nominal_vcount);
+ INIT_WORK(&crc_fw_wrk->forward_roi_work, amdgpu_dm_forward_crc_window);
+ crc_fw_wrk->dm = &adev->dm;
+ crc_fw_wrk->stream = stream_state;
+ crc_fw_wrk->rect.x = acrtc->dm_irq_params.window_param.x_start;
+ crc_fw_wrk->rect.y = acrtc->dm_irq_params.window_param.y_start;
+ crc_fw_wrk->rect.width = acrtc->dm_irq_params.window_param.x_end -
+ acrtc->dm_irq_params.window_param.x_start;
+ crc_fw_wrk->rect.height = acrtc->dm_irq_params.window_param.y_end -
+ acrtc->dm_irq_params.window_param.y_start;
+ schedule_work(&crc_fw_wrk->forward_roi_work);
- v_blank = timing_out->v_total - timing_out->v_border_top -
- timing_out->v_addressable - timing_out->v_border_bottom;
+ acrtc->dm_irq_params.window_param.update_win = false;
+ acrtc->dm_irq_params.window_param.skip_frame_cnt = 1;
- v_back_porch = v_blank - timing_out->v_front_porch -
- timing_out->v_sync_width;
+ } else {
+ if (acrtc->dm_irq_params.window_param.skip_frame_cnt) {
+ acrtc->dm_irq_params.window_param.skip_frame_cnt -= 1;
+ goto cleanup;
+ }
- crc_window_latch_up_line = v_back_porch + timing_out->v_sync_width;
-
- /* take 3 lines margin*/
- if ((position.vertical_count + 3) >= crc_window_latch_up_line)
- acrtc->dm_irq_params.crc_window.skip_frame_cnt = 1;
- else
- acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
- } else {
- if (acrtc->dm_irq_params.crc_window.skip_frame_cnt == 0) {
- if (adev->dm.crc_rd_wrk) {
- crc_rd_wrk = adev->dm.crc_rd_wrk;
- spin_lock_irqsave(&crc_rd_wrk->crc_rd_work_lock, flags2);
- crc_rd_wrk->phy_inst =
- stream_state->link->link_enc_hw_inst;
- spin_unlock_irqrestore(&crc_rd_wrk->crc_rd_work_lock, flags2);
- schedule_work(&crc_rd_wrk->notify_ta_work);
- }
- } else {
- acrtc->dm_irq_params.crc_window.skip_frame_cnt -= 1;
- }
- }
+ if (adev->dm.crc_rd_wrk) {
+ crc_rd_wrk = adev->dm.crc_rd_wrk;
+ spin_lock_irqsave(&crc_rd_wrk->crc_rd_work_lock, flags2);
+ crc_rd_wrk->phy_inst = stream_state->link->link_enc_hw_inst;
+ spin_unlock_irqrestore(&crc_rd_wrk->crc_rd_work_lock, flags2);
+ schedule_work(&crc_rd_wrk->notify_ta_work);
}
}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h
index f07850db..71bce60 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h
@@ -40,7 +40,7 @@ enum amdgpu_dm_pipe_crc_source {
};
#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
-struct crc_window_parm {
+struct crc_window_param {
uint16_t x_start;
uint16_t y_start;
uint16_t x_end;
@@ -53,6 +53,7 @@ struct crc_window_parm {
int skip_frame_cnt;
};
+/* read_work for driver to call PSP to read */
struct crc_rd_work {
struct work_struct notify_ta_work;
/* To protect crc_rd_work carried fields*/
@@ -60,6 +61,15 @@ struct crc_rd_work {
struct drm_crtc *crtc;
uint8_t phy_inst;
};
+
+/* forward_work for driver to forward ROI to dmu */
+struct crc_fw_work {
+ struct work_struct forward_roi_work;
+ struct amdgpu_display_manager *dm;
+ struct dc_stream_state *stream;
+ struct rect rect;
+ bool is_stop_cmd;
+};
#endif
static inline bool amdgpu_dm_is_valid_crc_source(enum amdgpu_dm_pipe_crc_source source)
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
index 64dd029..22125da 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
@@ -127,6 +127,9 @@ static void vblank_control_worker(struct work_struct *work)
amdgpu_dm_psr_disable(vblank_work->stream);
} else if (vblank_work->stream->link->psr_settings.psr_feature_enabled &&
!vblank_work->stream->link->psr_settings.psr_allow_active &&
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+ !amdgpu_dm_crc_window_is_activated(&vblank_work->acrtc->base) &&
+#endif
vblank_work->acrtc->dm_irq_params.allow_psr_entry) {
amdgpu_dm_psr_enable(vblank_work->stream);
}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
index ee242d9..461037a 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
@@ -38,6 +38,10 @@
#include "link_hwss.h"
#include "dc/dc_dmub_srv.h"
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+#include "amdgpu_dm_psr.h"
+#endif
+
struct dmub_debugfs_trace_header {
uint32_t entry_count;
uint32_t reserved[3];
@@ -299,6 +303,8 @@ static ssize_t dp_link_settings_write(struct file *f, const char __user *buf,
case LINK_RATE_HIGH2:
case LINK_RATE_HIGH3:
case LINK_RATE_UHBR10:
+ case LINK_RATE_UHBR13_5:
+ case LINK_RATE_UHBR20:
break;
default:
valid_input = false;
@@ -2633,6 +2639,25 @@ static int dp_mst_progress_status_show(struct seq_file *m, void *unused)
return 0;
}
+/*
+ * Reports whether the connected display is a USB4 DPIA tunneled display
+ * Example usage: cat /sys/kernel/debug/dri/0/DP-8/is_dpia_link
+ */
+static int is_dpia_link_show(struct seq_file *m, void *data)
+{
+ struct drm_connector *connector = m->private;
+ struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(connector);
+ struct dc_link *link = aconnector->dc_link;
+
+ if (connector->status != connector_status_connected)
+ return -ENODEV;
+
+ seq_printf(m, "%s\n", (link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA) ? "yes" :
+ (link->ep_type == DISPLAY_ENDPOINT_PHY) ? "no" : "unknown");
+
+ return 0;
+}
+
DEFINE_SHOW_ATTRIBUTE(dp_dsc_fec_support);
DEFINE_SHOW_ATTRIBUTE(dmub_fw_state);
DEFINE_SHOW_ATTRIBUTE(dmub_tracebuffer);
@@ -2644,6 +2669,7 @@ DEFINE_SHOW_ATTRIBUTE(internal_display);
DEFINE_SHOW_ATTRIBUTE(psr_capability);
DEFINE_SHOW_ATTRIBUTE(dp_is_mst_connector);
DEFINE_SHOW_ATTRIBUTE(dp_mst_progress_status);
+DEFINE_SHOW_ATTRIBUTE(is_dpia_link);
static const struct file_operations dp_dsc_clock_en_debugfs_fops = {
.owner = THIS_MODULE,
@@ -2788,7 +2814,8 @@ static const struct {
{"max_bpc", &dp_max_bpc_debugfs_fops},
{"dsc_disable_passthrough", &dp_dsc_disable_passthrough_debugfs_fops},
{"is_mst_connector", &dp_is_mst_connector_fops},
- {"mst_progress_status", &dp_mst_progress_status_fops}
+ {"mst_progress_status", &dp_mst_progress_status_fops},
+ {"is_dpia_link", &is_dpia_link_fops}
};
#ifdef CONFIG_DRM_AMD_DC_HDCP
@@ -3079,8 +3106,8 @@ static int crc_win_x_start_set(void *data, u64 val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- acrtc->dm_irq_params.crc_window.x_start = (uint16_t) val;
- acrtc->dm_irq_params.crc_window.update_win = false;
+ acrtc->dm_irq_params.window_param.x_start = (uint16_t) val;
+ acrtc->dm_irq_params.window_param.update_win = false;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3096,7 +3123,7 @@ static int crc_win_x_start_get(void *data, u64 *val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- *val = acrtc->dm_irq_params.crc_window.x_start;
+ *val = acrtc->dm_irq_params.window_param.x_start;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3116,8 +3143,8 @@ static int crc_win_y_start_set(void *data, u64 val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- acrtc->dm_irq_params.crc_window.y_start = (uint16_t) val;
- acrtc->dm_irq_params.crc_window.update_win = false;
+ acrtc->dm_irq_params.window_param.y_start = (uint16_t) val;
+ acrtc->dm_irq_params.window_param.update_win = false;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3133,7 +3160,7 @@ static int crc_win_y_start_get(void *data, u64 *val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- *val = acrtc->dm_irq_params.crc_window.y_start;
+ *val = acrtc->dm_irq_params.window_param.y_start;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3152,8 +3179,8 @@ static int crc_win_x_end_set(void *data, u64 val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- acrtc->dm_irq_params.crc_window.x_end = (uint16_t) val;
- acrtc->dm_irq_params.crc_window.update_win = false;
+ acrtc->dm_irq_params.window_param.x_end = (uint16_t) val;
+ acrtc->dm_irq_params.window_param.update_win = false;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3169,7 +3196,7 @@ static int crc_win_x_end_get(void *data, u64 *val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- *val = acrtc->dm_irq_params.crc_window.x_end;
+ *val = acrtc->dm_irq_params.window_param.x_end;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3188,8 +3215,8 @@ static int crc_win_y_end_set(void *data, u64 val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- acrtc->dm_irq_params.crc_window.y_end = (uint16_t) val;
- acrtc->dm_irq_params.crc_window.update_win = false;
+ acrtc->dm_irq_params.window_param.y_end = (uint16_t) val;
+ acrtc->dm_irq_params.window_param.update_win = false;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3205,7 +3232,7 @@ static int crc_win_y_end_get(void *data, u64 *val)
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
spin_lock_irq(&drm_dev->event_lock);
- *val = acrtc->dm_irq_params.crc_window.y_end;
+ *val = acrtc->dm_irq_params.window_param.y_end;
spin_unlock_irq(&drm_dev->event_lock);
return 0;
@@ -3228,31 +3255,38 @@ static int crc_win_update_set(void *data, u64 val)
return 0;
if (val) {
+ new_acrtc = to_amdgpu_crtc(new_crtc);
+ mutex_lock(&adev->dm.dc_lock);
+ /* PSR may write to OTG CRC window control register,
+ * so close it before starting secure_display.
+ */
+ amdgpu_dm_psr_disable(new_acrtc->dm_irq_params.stream);
+
spin_lock_irq(&adev_to_drm(adev)->event_lock);
spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock);
if (crc_rd_wrk->crtc) {
old_crtc = crc_rd_wrk->crtc;
old_acrtc = to_amdgpu_crtc(old_crtc);
}
- new_acrtc = to_amdgpu_crtc(new_crtc);
if (old_crtc && old_crtc != new_crtc) {
- old_acrtc->dm_irq_params.crc_window.activated = false;
- old_acrtc->dm_irq_params.crc_window.update_win = false;
- old_acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ old_acrtc->dm_irq_params.window_param.activated = false;
+ old_acrtc->dm_irq_params.window_param.update_win = false;
+ old_acrtc->dm_irq_params.window_param.skip_frame_cnt = 0;
- new_acrtc->dm_irq_params.crc_window.activated = true;
- new_acrtc->dm_irq_params.crc_window.update_win = true;
- new_acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ new_acrtc->dm_irq_params.window_param.activated = true;
+ new_acrtc->dm_irq_params.window_param.update_win = true;
+ new_acrtc->dm_irq_params.window_param.skip_frame_cnt = 0;
crc_rd_wrk->crtc = new_crtc;
} else {
- new_acrtc->dm_irq_params.crc_window.activated = true;
- new_acrtc->dm_irq_params.crc_window.update_win = true;
- new_acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ new_acrtc->dm_irq_params.window_param.activated = true;
+ new_acrtc->dm_irq_params.window_param.update_win = true;
+ new_acrtc->dm_irq_params.window_param.skip_frame_cnt = 0;
crc_rd_wrk->crtc = new_crtc;
}
spin_unlock_irq(&crc_rd_wrk->crc_rd_work_lock);
spin_unlock_irq(&adev_to_drm(adev)->event_lock);
+ mutex_unlock(&adev->dm.dc_lock);
}
return 0;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
index 6202e31..a7fd98f 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
@@ -495,7 +495,7 @@ static void update_config(void *handle, struct cp_psp_stream_config *config)
link->dp.mst_enabled = config->mst_enabled;
link->dp.usb4_enabled = config->usb4_enabled;
display->adjust.disable = MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION;
- link->adjust.auth_delay = 3;
+ link->adjust.auth_delay = 0;
link->adjust.hdcp1.disable = 0;
conn_state = aconnector->base.state;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
index f0b01c8..6994c9a 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
@@ -42,39 +42,6 @@
#include "dm_helpers.h"
#include "ddc_service_types.h"
-struct monitor_patch_info {
- unsigned int manufacturer_id;
- unsigned int product_id;
- void (*patch_func)(struct dc_edid_caps *edid_caps, unsigned int param);
- unsigned int patch_param;
-};
-static void set_max_dsc_bpp_limit(struct dc_edid_caps *edid_caps, unsigned int param);
-
-static const struct monitor_patch_info monitor_patch_table[] = {
-{0x6D1E, 0x5BBF, set_max_dsc_bpp_limit, 15},
-{0x6D1E, 0x5B9A, set_max_dsc_bpp_limit, 15},
-};
-
-static void set_max_dsc_bpp_limit(struct dc_edid_caps *edid_caps, unsigned int param)
-{
- if (edid_caps)
- edid_caps->panel_patch.max_dsc_target_bpp_limit = param;
-}
-
-static int amdgpu_dm_patch_edid_caps(struct dc_edid_caps *edid_caps)
-{
- int i, ret = 0;
-
- for (i = 0; i < ARRAY_SIZE(monitor_patch_table); i++)
- if ((edid_caps->manufacturer_id == monitor_patch_table[i].manufacturer_id)
- && (edid_caps->product_id == monitor_patch_table[i].product_id)) {
- monitor_patch_table[i].patch_func(edid_caps, monitor_patch_table[i].patch_param);
- ret++;
- }
-
- return ret;
-}
-
/* dm_helpers_parse_edid_caps
*
* Parse edid caps
@@ -149,8 +116,6 @@ enum dc_edid_status dm_helpers_parse_edid_caps(
kfree(sads);
kfree(sadb);
- amdgpu_dm_patch_edid_caps(edid_caps);
-
return result;
}
@@ -852,9 +817,8 @@ int dm_helper_dmub_aux_transfer_sync(
struct aux_payload *payload,
enum aux_return_code_type *operation_result)
{
- return amdgpu_dm_process_dmub_aux_transfer_sync(true, ctx,
- link->link_index, (void *)payload,
- (void *)operation_result);
+ return amdgpu_dm_process_dmub_aux_transfer_sync(ctx, link->link_index, payload,
+ operation_result);
}
int dm_helpers_dmub_set_config_sync(struct dc_context *ctx,
@@ -862,9 +826,8 @@ int dm_helpers_dmub_set_config_sync(struct dc_context *ctx,
struct set_config_cmd_payload *payload,
enum set_config_status *operation_result)
{
- return amdgpu_dm_process_dmub_aux_transfer_sync(false, ctx,
- link->link_index, (void *)payload,
- (void *)operation_result);
+ return amdgpu_dm_process_dmub_set_config_sync(ctx, link->link_index, payload,
+ operation_result);
}
void dm_set_dcn_clocks(struct dc_context *ctx, struct dc_clocks *clks)
@@ -1006,3 +969,11 @@ void dm_helpers_enable_periodic_detection(struct dc_context *ctx, bool enable)
{
/* TODO: add periodic detection implementation */
}
+
+void dm_helpers_dp_mst_update_branch_bandwidth(
+ struct dc_context *ctx,
+ struct dc_link *link)
+{
+ // TODO
+}
+
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h
index 79b5f99..5c93032 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h
@@ -39,7 +39,7 @@ struct dm_irq_params {
#ifdef CONFIG_DEBUG_FS
enum amdgpu_dm_pipe_crc_source crc_src;
#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
- struct crc_window_parm crc_window;
+ struct crc_window_param window_param;
#endif
#endif
};
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
index 6483ba2..1edf738 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
@@ -642,15 +642,18 @@ static void set_dsc_configs_from_fairness_vars(struct dsc_mst_fairness_params *p
int count,
int k)
{
+ struct drm_connector *drm_connector;
int i;
for (i = 0; i < count; i++) {
+ drm_connector = ¶ms[i].aconnector->base;
+
memset(¶ms[i].timing->dsc_cfg, 0, sizeof(params[i].timing->dsc_cfg));
if (vars[i + k].dsc_enabled && dc_dsc_compute_config(
params[i].sink->ctx->dc->res_pool->dscs[0],
¶ms[i].sink->dsc_caps.dsc_dec_caps,
params[i].sink->ctx->dc->debug.dsc_min_slice_height_override,
- params[i].sink->edid_caps.panel_patch.max_dsc_target_bpp_limit,
+ drm_connector->display_info.max_dsc_bpp,
0,
params[i].timing,
¶ms[i].timing->dsc_cfg)) {
@@ -692,12 +695,16 @@ static int bpp_x16_from_pbn(struct dsc_mst_fairness_params param, int pbn)
struct dc_dsc_config dsc_config;
u64 kbps;
+ struct drm_connector *drm_connector = ¶m.aconnector->base;
+ uint32_t max_dsc_target_bpp_limit_override =
+ drm_connector->display_info.max_dsc_bpp;
+
kbps = div_u64((u64)pbn * 994 * 8 * 54, 64);
dc_dsc_compute_config(
param.sink->ctx->dc->res_pool->dscs[0],
¶m.sink->dsc_caps.dsc_dec_caps,
param.sink->ctx->dc->debug.dsc_min_slice_height_override,
- param.sink->edid_caps.panel_patch.max_dsc_target_bpp_limit,
+ max_dsc_target_bpp_limit_override,
(int) kbps, param.timing, &dsc_config);
return dsc_config.bits_per_pixel;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
index e6854f7..3c50b3f 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c
@@ -1600,6 +1600,10 @@ int amdgpu_dm_plane_init(struct amdgpu_display_manager *dm,
drm_plane_create_rotation_property(plane, DRM_MODE_ROTATE_0,
supported_rotations);
+ if (dm->adev->ip_versions[DCE_HWIP][0] > IP_VERSION(3, 0, 1) &&
+ plane->type != DRM_PLANE_TYPE_CURSOR)
+ drm_plane_enable_fb_damage_clips(plane);
+
drm_plane_helper_add(plane, &dm_plane_helper_funcs);
#ifdef CONFIG_DRM_AMD_DC_HDR
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h
index d3bc9dc..0f580ea3 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h
@@ -37,6 +37,7 @@
#include <drm/drm_framebuffer.h>
#include <drm/drm_encoder.h>
#include <drm/drm_atomic.h>
+#include "dcn10/dcn10_optc.h"
#include "dc/inc/core_types.h"
@@ -662,6 +663,69 @@ TRACE_EVENT(dcn_fpu,
)
);
+TRACE_EVENT(dcn_optc_lock_unlock_state,
+ TP_PROTO(const struct optc *optc_state, int instance, bool lock, const char *function, const int line),
+ TP_ARGS(optc_state, instance, lock, function, line),
+
+ TP_STRUCT__entry(
+ __field(const char *, function)
+ __field(int, instance)
+ __field(bool, lock)
+ __field(int, line)
+ __field(int, opp_count)
+ __field(int, max_h_total)
+ __field(int, max_v_total)
+ __field(int, min_h_blank)
+ __field(int, min_h_sync_width)
+ __field(int, min_v_sync_width)
+ __field(int, min_v_blank)
+ __field(int, min_v_blank_interlace)
+ __field(int, vstartup_start)
+ __field(int, vupdate_offset)
+ __field(int, vupdate_width)
+ __field(int, vready_offset)
+ ),
+ TP_fast_assign(
+ __entry->function = function;
+ __entry->instance = instance;
+ __entry->lock = lock;
+ __entry->line = line;
+ __entry->opp_count = optc_state->opp_count;
+ __entry->max_h_total = optc_state->max_h_total;
+ __entry->max_v_total = optc_state->max_v_total;
+ __entry->min_h_blank = optc_state->min_h_blank;
+ __entry->min_h_sync_width = optc_state->min_h_sync_width;
+ __entry->min_v_sync_width = optc_state->min_v_sync_width;
+ __entry->min_v_blank = optc_state->min_v_blank;
+ __entry->min_v_blank_interlace = optc_state->min_v_blank_interlace;
+ __entry->vstartup_start = optc_state->vstartup_start;
+ __entry->vupdate_offset = optc_state->vupdate_offset;
+ __entry->vupdate_width = optc_state->vupdate_width;
+ __entry->vready_offset = optc_state->vupdate_offset;
+ ),
+ TP_printk("%s: %s()+%d: optc_instance=%d opp_count=%d max_h_total=%d max_v_total=%d "
+ "min_h_blank=%d min_h_sync_width=%d min_v_sync_width=%d min_v_blank=%d "
+ "min_v_blank_interlace=%d vstartup_start=%d vupdate_offset=%d vupdate_width=%d "
+ "vready_offset=%d",
+ __entry->lock ? "Lock" : "Unlock",
+ __entry->function,
+ __entry->line,
+ __entry->instance,
+ __entry->opp_count,
+ __entry->max_h_total,
+ __entry->max_v_total,
+ __entry->min_h_blank,
+ __entry->min_h_sync_width,
+ __entry->min_v_sync_width,
+ __entry->min_v_blank,
+ __entry->min_v_blank_interlace,
+ __entry->vstartup_start,
+ __entry->vupdate_offset,
+ __entry->vupdate_width,
+ __entry->vready_offset
+ )
+);
+
#endif /* _AMDGPU_DM_TRACE_H_ */
#undef TRACE_INCLUDE_PATH
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/dc_fpu.c b/drivers/gpu/drm/amd/display/amdgpu_dm/dc_fpu.c
index ab0c6d19..1743ca0 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/dc_fpu.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/dc_fpu.c
@@ -31,6 +31,8 @@
#elif defined(CONFIG_PPC64)
#include <asm/switch_to.h>
#include <asm/cputable.h>
+#elif defined(CONFIG_ARM64)
+#include <asm/neon.h>
#endif
/**
@@ -99,6 +101,8 @@ void dc_fpu_begin(const char *function_name, const int line)
preempt_disable();
enable_kernel_fp();
}
+#elif defined(CONFIG_ARM64)
+ kernel_neon_begin();
#endif
}
@@ -136,6 +140,8 @@ void dc_fpu_end(const char *function_name, const int line)
disable_kernel_fp();
preempt_enable();
}
+#elif defined(CONFIG_ARM64)
+ kernel_neon_end();
#endif
}
diff --git a/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c b/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c
index 9b8ea6e..a1a00f4 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c
@@ -138,7 +138,9 @@ static uint8_t get_number_of_objects(struct bios_parser *bp, uint32_t offset)
uint32_t object_table_offset = bp->object_info_tbl_offset + offset;
- table = GET_IMAGE(ATOM_OBJECT_TABLE, object_table_offset);
+ table = ((ATOM_OBJECT_TABLE *) bios_get_image(&bp->base,
+ object_table_offset,
+ struct_size(table, asObjects, 1)));
if (!table)
return 0;
@@ -166,8 +168,9 @@ static struct graphics_object_id bios_parser_get_connector_id(
uint32_t connector_table_offset = bp->object_info_tbl_offset
+ le16_to_cpu(bp->object_info_tbl.v1_1->usConnectorObjectTableOffset);
- ATOM_OBJECT_TABLE *tbl =
- GET_IMAGE(ATOM_OBJECT_TABLE, connector_table_offset);
+ ATOM_OBJECT_TABLE *tbl = ((ATOM_OBJECT_TABLE *) bios_get_image(&bp->base,
+ connector_table_offset,
+ struct_size(tbl, asObjects, 1)));
if (!tbl) {
dm_error("Can't get connector table from atom bios.\n");
@@ -662,8 +665,9 @@ static enum bp_result get_ss_info_v3_1(
if (!DATA_TABLES(ASIC_InternalSS_Info))
return BP_RESULT_UNSUPPORTED;
- ss_table_header_include = GET_IMAGE(ATOM_ASIC_INTERNAL_SS_INFO_V3,
- DATA_TABLES(ASIC_InternalSS_Info));
+ ss_table_header_include = ((ATOM_ASIC_INTERNAL_SS_INFO_V3 *) bios_get_image(&bp->base,
+ DATA_TABLES(ASIC_InternalSS_Info),
+ struct_size(ss_table_header_include, asSpreadSpectrum, 1)));
table_size =
(le16_to_cpu(ss_table_header_include->sHeader.usStructureSize)
- sizeof(ATOM_COMMON_TABLE_HEADER))
@@ -1029,8 +1033,10 @@ static enum bp_result get_ss_info_from_internal_ss_info_tbl_V2_1(
if (!DATA_TABLES(ASIC_InternalSS_Info))
return result;
- header = GET_IMAGE(ATOM_ASIC_INTERNAL_SS_INFO_V2,
- DATA_TABLES(ASIC_InternalSS_Info));
+ header = ((ATOM_ASIC_INTERNAL_SS_INFO_V2 *) bios_get_image(
+ &bp->base,
+ DATA_TABLES(ASIC_InternalSS_Info),
+ struct_size(header, asSpreadSpectrum, 1)));
memset(info, 0, sizeof(struct spread_spectrum_info));
@@ -1709,8 +1715,10 @@ static uint32_t get_ss_entry_number_from_internal_ss_info_tbl_v2_1(
if (!DATA_TABLES(ASIC_InternalSS_Info))
return 0;
- header_include = GET_IMAGE(ATOM_ASIC_INTERNAL_SS_INFO_V2,
- DATA_TABLES(ASIC_InternalSS_Info));
+ header_include = ((ATOM_ASIC_INTERNAL_SS_INFO_V2 *) bios_get_image(
+ &bp->base,
+ DATA_TABLES(ASIC_InternalSS_Info),
+ struct_size(header_include, asSpreadSpectrum, 1)));
size = (le16_to_cpu(header_include->sHeader.usStructureSize)
- sizeof(ATOM_COMMON_TABLE_HEADER))
@@ -1746,8 +1754,9 @@ static uint32_t get_ss_entry_number_from_internal_ss_info_tbl_V3_1(
if (!DATA_TABLES(ASIC_InternalSS_Info))
return number;
- header_include = GET_IMAGE(ATOM_ASIC_INTERNAL_SS_INFO_V3,
- DATA_TABLES(ASIC_InternalSS_Info));
+ header_include = ((ATOM_ASIC_INTERNAL_SS_INFO_V3 *) bios_get_image(&bp->base,
+ DATA_TABLES(ASIC_InternalSS_Info),
+ struct_size(header_include, asSpreadSpectrum, 1)));
size = (le16_to_cpu(header_include->sHeader.usStructureSize) -
sizeof(ATOM_COMMON_TABLE_HEADER)) /
sizeof(ATOM_ASIC_SS_ASSIGNMENT_V3);
@@ -1789,11 +1798,13 @@ static enum bp_result bios_parser_get_gpio_pin_info(
if (!DATA_TABLES(GPIO_Pin_LUT))
return BP_RESULT_BADBIOSTABLE;
- header = GET_IMAGE(ATOM_GPIO_PIN_LUT, DATA_TABLES(GPIO_Pin_LUT));
+ header = ((ATOM_GPIO_PIN_LUT *) bios_get_image(&bp->base,
+ DATA_TABLES(GPIO_Pin_LUT),
+ struct_size(header, asGPIO_Pin, 1)));
if (!header)
return BP_RESULT_BADBIOSTABLE;
- if (sizeof(ATOM_COMMON_TABLE_HEADER) + sizeof(ATOM_GPIO_PIN_LUT)
+ if (sizeof(ATOM_COMMON_TABLE_HEADER) + struct_size(header, asGPIO_Pin, 1)
> le16_to_cpu(header->sHeader.usStructureSize))
return BP_RESULT_BADBIOSTABLE;
@@ -1978,7 +1989,8 @@ static ATOM_OBJECT *get_bios_object(struct bios_parser *bp,
offset += bp->object_info_tbl_offset;
- tbl = GET_IMAGE(ATOM_OBJECT_TABLE, offset);
+ tbl = ((ATOM_OBJECT_TABLE *) bios_get_image(&bp->base, offset,
+ struct_size(tbl, asObjects, 1)));
if (!tbl)
return NULL;
@@ -2600,8 +2612,7 @@ static enum bp_result update_slot_layout_info(
for (;;) {
- record_header = (ATOM_COMMON_RECORD_HEADER *)
- GET_IMAGE(ATOM_COMMON_RECORD_HEADER, record_offset);
+ record_header = GET_IMAGE(ATOM_COMMON_RECORD_HEADER, record_offset);
if (record_header == NULL) {
result = BP_RESULT_BADBIOSTABLE;
break;
@@ -2615,7 +2626,7 @@ static enum bp_result update_slot_layout_info(
if (record_header->ucRecordType ==
ATOM_BRACKET_LAYOUT_RECORD_TYPE &&
- sizeof(ATOM_BRACKET_LAYOUT_RECORD)
+ struct_size(record, asConnInfo, 1)
<= record_header->ucRecordSize) {
record = (ATOM_BRACKET_LAYOUT_RECORD *)
(record_header);
@@ -2709,8 +2720,9 @@ static enum bp_result get_bracket_layout_record(
genericTableOffset = bp->object_info_tbl_offset +
bp->object_info_tbl.v1_3->usMiscObjectTableOffset;
- object_table = (ATOM_OBJECT_TABLE *)
- GET_IMAGE(ATOM_OBJECT_TABLE, genericTableOffset);
+ object_table = ((ATOM_OBJECT_TABLE *) bios_get_image(&bp->base,
+ genericTableOffset,
+ struct_size(object_table, asObjects, 1)));
if (!object_table)
return BP_RESULT_FAILURE;
diff --git a/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c b/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
index e0c8d6f..074e70a 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
@@ -462,6 +462,7 @@ static enum bp_result get_gpio_i2c_info(
uint32_t count = 0;
unsigned int table_index = 0;
bool find_valid = false;
+ struct atom_gpio_pin_assignment *pin;
if (!info)
return BP_RESULT_BADINPUT;
@@ -489,20 +490,17 @@ static enum bp_result get_gpio_i2c_info(
- sizeof(struct atom_common_table_header))
/ sizeof(struct atom_gpio_pin_assignment);
+ pin = (struct atom_gpio_pin_assignment *) header->gpio_pin;
+
for (table_index = 0; table_index < count; table_index++) {
- if (((record->i2c_id & I2C_HW_CAP) == (
- header->gpio_pin[table_index].gpio_id &
- I2C_HW_CAP)) &&
- ((record->i2c_id & I2C_HW_ENGINE_ID_MASK) ==
- (header->gpio_pin[table_index].gpio_id &
- I2C_HW_ENGINE_ID_MASK)) &&
- ((record->i2c_id & I2C_HW_LANE_MUX) ==
- (header->gpio_pin[table_index].gpio_id &
- I2C_HW_LANE_MUX))) {
+ if (((record->i2c_id & I2C_HW_CAP) == (pin->gpio_id & I2C_HW_CAP)) &&
+ ((record->i2c_id & I2C_HW_ENGINE_ID_MASK) == (pin->gpio_id & I2C_HW_ENGINE_ID_MASK)) &&
+ ((record->i2c_id & I2C_HW_LANE_MUX) == (pin->gpio_id & I2C_HW_LANE_MUX))) {
/* still valid */
find_valid = true;
break;
}
+ pin = (struct atom_gpio_pin_assignment *)((uint8_t *)pin + sizeof(struct atom_gpio_pin_assignment));
}
/* If we don't find the entry that we are looking for then
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.h b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.h
index 3e5df27..1ce19d8 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.h
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.h
@@ -26,6 +26,8 @@
#ifndef DAL_DC_RN_CLK_MGR_VBIOS_SMU_H_
#define DAL_DC_RN_CLK_MGR_VBIOS_SMU_H_
+enum dcn_pwr_state;
+
int rn_vbios_smu_get_smu_version(struct clk_mgr_internal *clk_mgr);
int rn_vbios_smu_set_dispclk(struct clk_mgr_internal *clk_mgr, int requested_dispclk_khz);
int rn_vbios_smu_set_dprefclk(struct clk_mgr_internal *clk_mgr);
@@ -33,7 +35,7 @@ int rn_vbios_smu_set_hard_min_dcfclk(struct clk_mgr_internal *clk_mgr, int reque
int rn_vbios_smu_set_min_deep_sleep_dcfclk(struct clk_mgr_internal *clk_mgr, int requested_min_ds_dcfclk_khz);
void rn_vbios_smu_set_phyclk(struct clk_mgr_internal *clk_mgr, int requested_phyclk_khz);
int rn_vbios_smu_set_dppclk(struct clk_mgr_internal *clk_mgr, int requested_dpp_khz);
-void rn_vbios_smu_set_dcn_low_power_state(struct clk_mgr_internal *clk_mgr, int display_count);
+void rn_vbios_smu_set_dcn_low_power_state(struct clk_mgr_internal *clk_mgr, enum dcn_pwr_state);
void rn_vbios_smu_enable_48mhz_tmdp_refclk_pwrdwn(struct clk_mgr_internal *clk_mgr, bool enable);
void rn_vbios_smu_enable_pme_wa(struct clk_mgr_internal *clk_mgr);
int rn_vbios_smu_is_periodic_retraining_disabled(struct clk_mgr_internal *clk_mgr);
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_smu11_driver_if.h b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_smu11_driver_if.h
index 8ea8ee5..61bb1d8 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_smu11_driver_if.h
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_smu11_driver_if.h
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: MIT
// This is a stripped-down version of the smu11_driver_if.h file for the relevant DAL interfaces.
#define SMU11_DRIVER_IF_VERSION 0x40
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c
index c1eaf57..1c0569b 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c
@@ -609,8 +609,10 @@ static void dcn31_clk_mgr_helper_populate_bw_params(struct clk_mgr_internal *clk
}
bw_params->vram_type = bios_info->memory_type;
- bw_params->num_channels = bios_info->ma_channel_number;
+ bw_params->dram_channel_width_bytes = bios_info->memory_type == 0x22 ? 8 : 4;
+ //bw_params->dram_channel_width_bytes = dc->ctx->asic_id.vram_width;
+ bw_params->num_channels = bios_info->ma_channel_number ? bios_info->ma_channel_number : 4;
for (i = 0; i < WM_SET_COUNT; i++) {
bw_params->wm_table.entries[i].wm_inst = i;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c
index 090b2c0..0827c7d 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c
@@ -333,8 +333,8 @@ void dcn31_smu_set_zstate_support(struct clk_mgr_internal *clk_mgr, enum dcn_zst
(support == DCN_ZSTATE_SUPPORT_ALLOW_Z10_ONLY))
support = DCN_ZSTATE_SUPPORT_DISALLOW;
-
- if (support == DCN_ZSTATE_SUPPORT_ALLOW_Z10_ONLY)
+ if (support == DCN_ZSTATE_SUPPORT_ALLOW_Z10_ONLY ||
+ support == DCN_ZSTATE_SUPPORT_ALLOW_Z8_Z10_ONLY)
param = 1;
else
param = 0;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_smu.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_smu.c
index 2db5956..f47cfe6 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_smu.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_smu.c
@@ -346,8 +346,6 @@ void dcn314_smu_set_zstate_support(struct clk_mgr_internal *clk_mgr, enum dcn_zs
if (!clk_mgr->smu_present)
return;
- // Arg[15:0] = 8/9/0 for Z8/Z9/disallow -> existing bits
- // Arg[16] = Disallow Z9 -> new bit
switch (support) {
case DCN_ZSTATE_SUPPORT_ALLOW:
@@ -366,6 +364,16 @@ void dcn314_smu_set_zstate_support(struct clk_mgr_internal *clk_mgr, enum dcn_zs
param = (1 << 10);
break;
+ case DCN_ZSTATE_SUPPORT_ALLOW_Z8_Z10_ONLY:
+ msg_id = VBIOSSMC_MSG_AllowZstatesEntry;
+ param = (1 << 10) | (1 << 8);
+ break;
+
+ case DCN_ZSTATE_SUPPORT_ALLOW_Z8_ONLY:
+ msg_id = VBIOSSMC_MSG_AllowZstatesEntry;
+ param = (1 << 8);
+ break;
+
default: //DCN_ZSTATE_SUPPORT_UNKNOWN
msg_id = VBIOSSMC_MSG_AllowZstatesEntry;
param = 0;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c
index 893991a..07edd977 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c
@@ -458,19 +458,6 @@ static void dcn315_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
dcn315_smu_transfer_dpm_table_smu_2_dram(clk_mgr);
}
-static uint32_t find_max_clk_value(const uint32_t clocks[], uint32_t num_clocks)
-{
- uint32_t max = 0;
- int i;
-
- for (i = 0; i < num_clocks; ++i) {
- if (clocks[i] > max)
- max = clocks[i];
- }
-
- return max;
-}
-
static void dcn315_clk_mgr_helper_populate_bw_params(
struct clk_mgr_internal *clk_mgr,
struct integrated_info *bios_info,
@@ -478,29 +465,21 @@ static void dcn315_clk_mgr_helper_populate_bw_params(
{
int i;
struct clk_bw_params *bw_params = clk_mgr->base.bw_params;
- uint32_t max_pstate = 0, max_fclk = 0, min_pstate = 0;
+ uint32_t max_pstate = clock_table->NumDfPstatesEnabled - 1;
struct clk_limit_table_entry def_max = bw_params->clk_table.entries[bw_params->clk_table.num_entries - 1];
- /* Find highest fclk pstate */
- for (i = 0; i < clock_table->NumDfPstatesEnabled; i++) {
- if (clock_table->DfPstateTable[i].FClk > max_fclk) {
- max_fclk = clock_table->DfPstateTable[i].FClk;
- max_pstate = i;
- }
- }
-
/* For 315 we want to base clock table on dcfclk, need at least one entry regardless of pmfw table */
for (i = 0; i < clock_table->NumDcfClkLevelsEnabled; i++) {
int j;
- uint32_t min_fclk = clock_table->DfPstateTable[0].FClk;
- for (j = 1; j < clock_table->NumDfPstatesEnabled; j++) {
- if (clock_table->DfPstateTable[j].Voltage <= clock_table->SocVoltage[i]
- && clock_table->DfPstateTable[j].FClk < min_fclk) {
- min_fclk = clock_table->DfPstateTable[j].FClk;
- min_pstate = j;
- }
+ /* DF table is sorted with clocks decreasing */
+ for (j = clock_table->NumDfPstatesEnabled - 2; j >= 0; j--) {
+ if (clock_table->DfPstateTable[j].Voltage <= clock_table->SocVoltage[i])
+ max_pstate = j;
}
+ /* Max DCFCLK should match up with max pstate */
+ if (i == clock_table->NumDcfClkLevelsEnabled - 1)
+ max_pstate = 0;
/* First search defaults for the clocks we don't read using closest lower or equal default dcfclk */
for (j = bw_params->clk_table.num_entries - 1; j > 0; j--)
@@ -511,9 +490,9 @@ static void dcn315_clk_mgr_helper_populate_bw_params(
bw_params->clk_table.entries[i].dtbclk_mhz = bw_params->clk_table.entries[j].dtbclk_mhz;
/* Now update clocks we do read */
- bw_params->clk_table.entries[i].fclk_mhz = min_fclk;
- bw_params->clk_table.entries[i].memclk_mhz = clock_table->DfPstateTable[min_pstate].MemClk;
- bw_params->clk_table.entries[i].voltage = clock_table->DfPstateTable[min_pstate].Voltage;
+ bw_params->clk_table.entries[i].fclk_mhz = clock_table->DfPstateTable[max_pstate].FClk;
+ bw_params->clk_table.entries[i].memclk_mhz = clock_table->DfPstateTable[max_pstate].MemClk;
+ bw_params->clk_table.entries[i].voltage = clock_table->SocVoltage[i];
bw_params->clk_table.entries[i].dcfclk_mhz = clock_table->DcfClocks[i];
bw_params->clk_table.entries[i].socclk_mhz = clock_table->SocClocks[i];
bw_params->clk_table.entries[i].dispclk_mhz = clock_table->DispClocks[i];
@@ -521,25 +500,16 @@ static void dcn315_clk_mgr_helper_populate_bw_params(
bw_params->clk_table.entries[i].wck_ratio = 1;
}
- /* Make sure to include at least one entry and highest pstate */
- if (max_pstate != min_pstate || i == 0) {
- bw_params->clk_table.entries[i].fclk_mhz = max_fclk;
- bw_params->clk_table.entries[i].memclk_mhz = clock_table->DfPstateTable[max_pstate].MemClk;
- bw_params->clk_table.entries[i].voltage = clock_table->DfPstateTable[max_pstate].Voltage;
- bw_params->clk_table.entries[i].dcfclk_mhz = find_max_clk_value(clock_table->DcfClocks, NUM_DCFCLK_DPM_LEVELS);
+ /* Make sure to include at least one entry */
+ if (i == 0) {
+ bw_params->clk_table.entries[i].fclk_mhz = clock_table->DfPstateTable[0].FClk;
+ bw_params->clk_table.entries[i].memclk_mhz = clock_table->DfPstateTable[0].MemClk;
+ bw_params->clk_table.entries[i].voltage = clock_table->DfPstateTable[0].Voltage;
+ bw_params->clk_table.entries[i].dcfclk_mhz = clock_table->DcfClocks[0];
bw_params->clk_table.entries[i].wck_ratio = 1;
i++;
}
- bw_params->clk_table.num_entries = i--;
-
- /* Make sure all highest clocks are included*/
- bw_params->clk_table.entries[i].socclk_mhz = find_max_clk_value(clock_table->SocClocks, NUM_SOCCLK_DPM_LEVELS);
- bw_params->clk_table.entries[i].dispclk_mhz = find_max_clk_value(clock_table->DispClocks, NUM_DISPCLK_DPM_LEVELS);
- bw_params->clk_table.entries[i].dppclk_mhz = find_max_clk_value(clock_table->DppClocks, NUM_DPPCLK_DPM_LEVELS);
- ASSERT(clock_table->DcfClocks[i] == find_max_clk_value(clock_table->DcfClocks, NUM_DCFCLK_DPM_LEVELS));
- bw_params->clk_table.entries[i].phyclk_mhz = def_max.phyclk_mhz;
- bw_params->clk_table.entries[i].phyclk_d18_mhz = def_max.phyclk_d18_mhz;
- bw_params->clk_table.entries[i].dtbclk_mhz = def_max.dtbclk_mhz;
+ bw_params->clk_table.num_entries = i;
/* Set any 0 clocks to max default setting. Not an issue for
* power since we aren't doing switching in such case anyway
@@ -565,6 +535,11 @@ static void dcn315_clk_mgr_helper_populate_bw_params(
if (!bw_params->clk_table.entries[i].dtbclk_mhz)
bw_params->clk_table.entries[i].dtbclk_mhz = def_max.dtbclk_mhz;
}
+
+ /* Make sure all highest default clocks are included*/
+ ASSERT(bw_params->clk_table.entries[i-1].phyclk_mhz == def_max.phyclk_mhz);
+ ASSERT(bw_params->clk_table.entries[i-1].phyclk_d18_mhz == def_max.phyclk_d18_mhz);
+ ASSERT(bw_params->clk_table.entries[i-1].dtbclk_mhz == def_max.dtbclk_mhz);
ASSERT(bw_params->clk_table.entries[i-1].dcfclk_mhz);
bw_params->vram_type = bios_info->memory_type;
bw_params->num_channels = bios_info->ma_channel_number;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c
index 187f5b27..3edc81e 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c
@@ -553,6 +553,7 @@ static void dcn316_clk_mgr_helper_populate_bw_params(
bw_params->vram_type = bios_info->memory_type;
bw_params->num_channels = bios_info->ma_channel_number;
+ bw_params->dram_channel_width_bytes = bios_info->memory_type == 0x22 ? 8 : 4;
for (i = 0; i < WM_SET_COUNT; i++) {
bw_params->wm_table.entries[i].wm_inst = i;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c
index 6f77d8e..200fcec 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c
@@ -233,41 +233,6 @@ void dcn32_init_clocks(struct clk_mgr *clk_mgr_base)
DC_FP_END();
}
-static void dcn32_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr,
- struct dc_state *context,
- int ref_dtbclk_khz)
-{
- struct dccg *dccg = clk_mgr->dccg;
- uint32_t tg_mask = 0;
- int i;
-
- for (i = 0; i < clk_mgr->base.ctx->dc->res_pool->pipe_count; i++) {
- struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[i];
- struct dtbclk_dto_params dto_params = {0};
-
- /* use mask to program DTO once per tg */
- if (pipe_ctx->stream_res.tg &&
- !(tg_mask & (1 << pipe_ctx->stream_res.tg->inst))) {
- tg_mask |= (1 << pipe_ctx->stream_res.tg->inst);
-
- dto_params.otg_inst = pipe_ctx->stream_res.tg->inst;
- dto_params.ref_dtbclk_khz = ref_dtbclk_khz;
-
- if (is_dp_128b_132b_signal(pipe_ctx)) {
- dto_params.pixclk_khz = pipe_ctx->stream->phy_pix_clk;
-
- if (pipe_ctx->stream_res.audio != NULL)
- dto_params.req_audio_dtbclk_khz = 24000;
- }
- if (dc_is_hdmi_signal(pipe_ctx->stream->signal))
- dto_params.is_hdmi = true;
-
- dccg->funcs->set_dtbclk_dto(clk_mgr->dccg, &dto_params);
- //dccg->funcs->set_audio_dtbclk_dto(clk_mgr->dccg, &dto_params);
- }
- }
-}
-
/* Since DPPCLK request to PMFW needs to be exact (due to DPP DTO programming),
* update DPPCLK to be the exact frequency that will be set after the DPPCLK
* divider is updated. This will prevent rounding issues that could cause DPP
@@ -438,7 +403,7 @@ static void dcn32_update_clocks(struct clk_mgr *clk_mgr_base,
}
if (!new_clocks->dtbclk_en) {
- new_clocks->ref_dtbclk_khz = 0;
+ new_clocks->ref_dtbclk_khz = clk_mgr_base->bw_params->clk_table.entries[0].dtbclk_mhz * 1000;
}
/* clock limits are received with MHz precision, divide by 1000 to prevent setting clocks at every call */
@@ -447,8 +412,6 @@ static void dcn32_update_clocks(struct clk_mgr *clk_mgr_base,
/* DCCG requires KHz precision for DTBCLK */
clk_mgr_base->clks.ref_dtbclk_khz =
dcn32_smu_set_hard_min_by_freq(clk_mgr, PPCLK_DTBCLK, khz_to_mhz_ceil(new_clocks->ref_dtbclk_khz));
-
- dcn32_update_clocks_update_dtb_dto(clk_mgr, context, clk_mgr_base->clks.ref_dtbclk_khz);
}
if (dc->config.forced_clocks == false || (force_reset && safe_to_lower)) {
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_smu13_driver_if.h b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_smu13_driver_if.h
index d30fbbdd..d3d5a8c 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_smu13_driver_if.h
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_smu13_driver_if.h
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: MIT
// This is a stripped-down version of the smu13_driver_if.h file for the relevant DAL interfaces.
#define SMU13_DRIVER_IF_VERSION 0x18
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c
index 997ab03..0cb8d1f 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc.c
@@ -135,9 +135,7 @@ static const char DC_BUILD_ID[] = "production-build";
* one or two (in the pipe-split case).
*/
-/*******************************************************************************
- * Private functions
- ******************************************************************************/
+/* Private functions */
static inline void elevate_update_type(enum surface_update_type *original, enum surface_update_type new)
{
@@ -401,9 +399,6 @@ bool dc_stream_adjust_vmin_vmax(struct dc *dc,
{
int i;
- if (memcmp(adjust, &stream->adjust, sizeof(struct dc_crtc_timing_adjust)) == 0)
- return true;
-
stream->adjust.v_total_max = adjust->v_total_max;
stream->adjust.v_total_mid = adjust->v_total_mid;
stream->adjust.v_total_mid_frame_num = adjust->v_total_mid_frame_num;
@@ -424,18 +419,14 @@ bool dc_stream_adjust_vmin_vmax(struct dc *dc,
}
/**
- *****************************************************************************
- * Function: dc_stream_get_last_vrr_vtotal
+ * dc_stream_get_last_used_drr_vtotal - dc_stream_get_last_vrr_vtotal
*
- * @brief
- * Looks up the pipe context of dc_stream_state and gets the
- * last VTOTAL used by DRR (Dynamic Refresh Rate)
+ * @dc: [in] dc reference
+ * @stream: [in] Initial dc stream state
+ * @adjust: [in] Updated parameters for vertical_total_min and
*
- * @param [in] dc: dc reference
- * @param [in] stream: Initial dc stream state
- * @param [in] adjust: Updated parameters for vertical_total_min and
- * vertical_total_max
- *****************************************************************************
+ * Looks up the pipe context of dc_stream_state and gets the last VTOTAL used
+ * by DRR (Dynamic Refresh Rate)
*/
bool dc_stream_get_last_used_drr_vtotal(struct dc *dc,
struct dc_stream_state *stream,
@@ -491,86 +482,79 @@ bool dc_stream_get_crtc_position(struct dc *dc,
}
#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
-bool dc_stream_forward_dmcu_crc_window(struct dc *dc, struct dc_stream_state *stream,
- struct crc_params *crc_window)
+static inline void
+dc_stream_forward_dmub_crc_window(struct dc_dmub_srv *dmub_srv,
+ struct rect *rect, struct otg_phy_mux *mux_mapping, bool is_stop)
{
- int i;
- struct dmcu *dmcu = dc->res_pool->dmcu;
- struct pipe_ctx *pipe;
- struct crc_region tmp_win, *crc_win;
- struct otg_phy_mux mapping_tmp, *mux_mapping;
+ union dmub_rb_cmd cmd = {0};
- /*crc window can't be null*/
- if (!crc_window)
- return false;
+ cmd.secure_display.roi_info.phy_id = mux_mapping->phy_output_num;
+ cmd.secure_display.roi_info.otg_id = mux_mapping->otg_output_num;
- if ((dmcu != NULL && dmcu->funcs->is_dmcu_initialized(dmcu))) {
- crc_win = &tmp_win;
- mux_mapping = &mapping_tmp;
- /*set crc window*/
- tmp_win.x_start = crc_window->windowa_x_start;
- tmp_win.y_start = crc_window->windowa_y_start;
- tmp_win.x_end = crc_window->windowa_x_end;
- tmp_win.y_end = crc_window->windowa_y_end;
-
- for (i = 0; i < MAX_PIPES; i++) {
- pipe = &dc->current_state->res_ctx.pipe_ctx[i];
- if (pipe->stream == stream && !pipe->top_pipe && !pipe->prev_odm_pipe)
- break;
- }
-
- /* Stream not found */
- if (i == MAX_PIPES)
- return false;
-
-
- /*set mux routing info*/
- mapping_tmp.phy_output_num = stream->link->link_enc_hw_inst;
- mapping_tmp.otg_output_num = pipe->stream_res.tg->inst;
-
- dmcu->funcs->forward_crc_window(dmcu, crc_win, mux_mapping);
+ if (is_stop) {
+ cmd.secure_display.header.type = DMUB_CMD__SECURE_DISPLAY;
+ cmd.secure_display.header.sub_type = DMUB_CMD__SECURE_DISPLAY_CRC_STOP_UPDATE;
} else {
- DC_LOG_DC("dmcu is not initialized");
- return false;
+ cmd.secure_display.header.type = DMUB_CMD__SECURE_DISPLAY;
+ cmd.secure_display.header.sub_type = DMUB_CMD__SECURE_DISPLAY_CRC_WIN_NOTIFY;
+ cmd.secure_display.roi_info.x_start = rect->x;
+ cmd.secure_display.roi_info.y_start = rect->y;
+ cmd.secure_display.roi_info.x_end = rect->x + rect->width;
+ cmd.secure_display.roi_info.y_end = rect->y + rect->height;
}
- return true;
+ dc_dmub_srv_cmd_queue(dmub_srv, &cmd);
+ dc_dmub_srv_cmd_execute(dmub_srv);
}
-bool dc_stream_stop_dmcu_crc_win_update(struct dc *dc, struct dc_stream_state *stream)
+static inline void
+dc_stream_forward_dmcu_crc_window(struct dmcu *dmcu,
+ struct rect *rect, struct otg_phy_mux *mux_mapping, bool is_stop)
{
- int i;
- struct dmcu *dmcu = dc->res_pool->dmcu;
- struct pipe_ctx *pipe;
- struct otg_phy_mux mapping_tmp, *mux_mapping;
-
- if ((dmcu != NULL && dmcu->funcs->is_dmcu_initialized(dmcu))) {
- mux_mapping = &mapping_tmp;
-
- for (i = 0; i < MAX_PIPES; i++) {
- pipe = &dc->current_state->res_ctx.pipe_ctx[i];
- if (pipe->stream == stream && !pipe->top_pipe && !pipe->prev_odm_pipe)
- break;
- }
-
- /* Stream not found */
- if (i == MAX_PIPES)
- return false;
-
-
- /*set mux routing info*/
- mapping_tmp.phy_output_num = stream->link->link_enc_hw_inst;
- mapping_tmp.otg_output_num = pipe->stream_res.tg->inst;
-
+ if (is_stop)
dmcu->funcs->stop_crc_win_update(dmcu, mux_mapping);
- } else {
- DC_LOG_DC("dmcu is not initialized");
- return false;
+ else
+ dmcu->funcs->forward_crc_window(dmcu, rect, mux_mapping);
+}
+
+bool
+dc_stream_forward_crc_window(struct dc *dc,
+ struct rect *rect, struct dc_stream_state *stream, bool is_stop)
+{
+ struct dmcu *dmcu;
+ struct dc_dmub_srv *dmub_srv;
+ struct otg_phy_mux mux_mapping;
+ struct pipe_ctx *pipe;
+ int i;
+
+ for (i = 0; i < MAX_PIPES; i++) {
+ pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+ if (pipe->stream == stream && !pipe->top_pipe && !pipe->prev_odm_pipe)
+ break;
}
+ /* Stream not found */
+ if (i == MAX_PIPES)
+ return false;
+
+ mux_mapping.phy_output_num = stream->link->link_enc_hw_inst;
+ mux_mapping.otg_output_num = pipe->stream_res.tg->inst;
+
+ dmcu = dc->res_pool->dmcu;
+ dmub_srv = dc->ctx->dmub_srv;
+
+ /* forward to dmub */
+ if (dmub_srv)
+ dc_stream_forward_dmub_crc_window(dmub_srv, rect, &mux_mapping, is_stop);
+ /* forward to dmcu */
+ else if (dmcu && dmcu->funcs->is_dmcu_initialized(dmcu))
+ dc_stream_forward_dmcu_crc_window(dmcu, rect, &mux_mapping, is_stop);
+ else
+ return false;
+
return true;
}
-#endif
+#endif /* CONFIG_DRM_AMD_SECURE_DISPLAY */
/**
* dc_stream_configure_crc() - Configure CRC capture for the given stream.
@@ -1070,6 +1054,8 @@ static void disable_dangling_plane(struct dc *dc, struct dc_state *context)
int i, j;
struct dc_state *dangling_context = dc_create_state(dc);
struct dc_state *current_ctx;
+ struct pipe_ctx *pipe;
+ struct timing_generator *tg;
if (dangling_context == NULL)
return;
@@ -1112,6 +1098,18 @@ static void disable_dangling_plane(struct dc *dc, struct dc_state *context)
}
if (should_disable && old_stream) {
+ pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+ tg = pipe->stream_res.tg;
+ /* When disabling plane for a phantom pipe, we must turn on the
+ * phantom OTG so the disable programming gets the double buffer
+ * update. Otherwise the pipe will be left in a partially disabled
+ * state that can result in underflow or hang when enabling it
+ * again for different use.
+ */
+ if (old_stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ if (tg->funcs->enable_crtc)
+ tg->funcs->enable_crtc(tg);
+ }
dc_rem_all_planes_for_stream(dc, old_stream, dangling_context);
disable_all_writeback_pipes_for_stream(dc, old_stream, dangling_context);
@@ -1127,6 +1125,15 @@ static void disable_dangling_plane(struct dc *dc, struct dc_state *context)
dc->hwss.interdependent_update_lock(dc, dc->current_state, false);
dc->hwss.post_unlock_program_front_end(dc, dangling_context);
}
+ /* We need to put the phantom OTG back into it's default (disabled) state or we
+ * can get corruption when transition from one SubVP config to a different one.
+ * The OTG is set to disable on falling edge of VUPDATE so the plane disable
+ * will still get it's double buffer update.
+ */
+ if (old_stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ if (tg->funcs->disable_phantom_crtc)
+ tg->funcs->disable_phantom_crtc(tg);
+ }
}
}
@@ -1219,9 +1226,7 @@ static void wait_for_no_pipes_pending(struct dc *dc, struct dc_state *context)
PERF_TRACE();
}
-/*******************************************************************************
- * Public functions
- ******************************************************************************/
+/* Public functions */
struct dc *dc_create(const struct dc_init_data *init_params)
{
@@ -1488,17 +1493,19 @@ static void program_timing_sync(
}
}
-static bool context_changed(
- struct dc *dc,
- struct dc_state *context)
+static bool streams_changed(struct dc *dc,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count)
{
uint8_t i;
- if (context->stream_count != dc->current_state->stream_count)
+ if (stream_count != dc->current_state->stream_count)
return true;
for (i = 0; i < dc->current_state->stream_count; i++) {
- if (dc->current_state->streams[i] != context->streams[i])
+ if (dc->current_state->streams[i] != streams[i])
+ return true;
+ if (!streams[i]->link->link_state_valid)
return true;
}
@@ -1549,6 +1556,9 @@ bool dc_validate_boot_timing(const struct dc *dc,
if (tg_inst >= dc->res_pool->timing_generator_count)
return false;
+ if (tg_inst != link->link_enc->preferred_engine)
+ return false;
+
tg = dc->res_pool->timing_generators[tg_inst];
if (!tg->funcs->get_hw_timing)
@@ -1722,8 +1732,13 @@ void dc_z10_save_init(struct dc *dc)
dc->hwss.z10_save_init(dc);
}
-/*
- * Applies given context to HW and copy it into current context.
+/**
+ * dc_commit_state_no_check - Apply context to the hardware
+ *
+ * @dc: DC object with the current status to be updated
+ * @context: New state that will become the current status at the end of this function
+ *
+ * Applies given context to the hardware and copy it into current context.
* It's up to the user to release the src context afterwards.
*/
static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *context)
@@ -1760,6 +1775,12 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c
context->stream_count == 0)
dc->hwss.prepare_bandwidth(dc, context);
+ /* When SubVP is active, all HW programming must be done while
+ * SubVP lock is acquired
+ */
+ if (dc->hwss.subvp_pipe_control_lock)
+ dc->hwss.subvp_pipe_control_lock(dc, context, true, true, NULL, subvp_prev_use);
+
if (dc->debug.enable_double_buffered_dsc_pg_support)
dc->hwss.update_dsc_pg(dc, context, false);
@@ -1787,9 +1808,6 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c
dc->hwss.wait_for_mpcc_disconnect(dc, dc->res_pool, pipe);
}
- if (dc->hwss.subvp_pipe_control_lock)
- dc->hwss.subvp_pipe_control_lock(dc, context, true, true, NULL, subvp_prev_use);
-
result = dc->hwss.apply_ctx_to_hw(dc, context);
if (result != DC_OK) {
@@ -1888,12 +1906,108 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c
return result;
}
+/**
+ * dc_commit_streams - Commit current stream state
+ *
+ * @dc: DC object with the commit state to be configured in the hardware
+ * @streams: Array with a list of stream state
+ * @stream_count: Total of streams
+ *
+ * Function responsible for commit streams change to the hardware.
+ *
+ * Return:
+ * Return DC_OK if everything work as expected, otherwise, return a dc_status
+ * code.
+ */
+enum dc_status dc_commit_streams(struct dc *dc,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count)
+{
+ int i, j;
+ struct dc_state *context;
+ enum dc_status res = DC_OK;
+ struct dc_validation_set set[MAX_STREAMS] = {0};
+
+ if (dc->ctx->dce_environment == DCE_ENV_VIRTUAL_HW)
+ return res;
+
+ if (!streams_changed(dc, streams, stream_count))
+ return res;
+
+ DC_LOG_DC("%s: %d streams\n", __func__, stream_count);
+
+ for (i = 0; i < stream_count; i++) {
+ struct dc_stream_state *stream = streams[i];
+ struct dc_stream_status *status = dc_stream_get_status(stream);
+
+ dc_stream_log(dc, stream);
+
+ set[i].stream = stream;
+
+ if (status) {
+ set[i].plane_count = status->plane_count;
+ for (j = 0; j < status->plane_count; j++)
+ set[i].plane_states[j] = status->plane_states[j];
+ }
+ }
+
+ context = dc_create_state(dc);
+ if (!context)
+ goto context_alloc_fail;
+
+ dc_resource_state_copy_construct_current(dc, context);
+
+ res = dc_validate_with_context(dc, set, stream_count, context, false);
+ if (res != DC_OK) {
+ BREAK_TO_DEBUGGER();
+ goto fail;
+ }
+
+ res = dc_commit_state_no_check(dc, context);
+
+ for (i = 0; i < stream_count; i++) {
+ for (j = 0; j < context->stream_count; j++) {
+ if (streams[i]->stream_id == context->streams[j]->stream_id)
+ streams[i]->out.otg_offset = context->stream_status[j].primary_otg_inst;
+
+ if (dc_is_embedded_signal(streams[i]->signal)) {
+ struct dc_stream_status *status = dc_stream_get_status_from_state(context, streams[i]);
+
+ if (dc->hwss.is_abm_supported)
+ status->is_abm_supported = dc->hwss.is_abm_supported(dc, context, streams[i]);
+ else
+ status->is_abm_supported = true;
+ }
+ }
+ }
+
+fail:
+ dc_release_state(context);
+
+context_alloc_fail:
+
+ DC_LOG_DC("%s Finished.\n", __func__);
+
+ return res;
+}
+
+/* TODO: When the transition to the new commit sequence is done, remove this
+ * function in favor of dc_commit_streams. */
bool dc_commit_state(struct dc *dc, struct dc_state *context)
{
enum dc_status result = DC_ERROR_UNEXPECTED;
int i;
- if (!context_changed(dc, context))
+ /* TODO: Since change commit sequence can have a huge impact,
+ * we decided to only enable it for DCN3x. However, as soon as
+ * we get more confident about this change we'll need to enable
+ * the new sequence for all ASICs. */
+ if (dc->ctx->dce_version >= DCN_VERSION_3_2) {
+ result = dc_commit_streams(dc, context->streams, context->stream_count);
+ return result == DC_OK;
+ }
+
+ if (!streams_changed(dc, context->streams, context->stream_count))
return DC_OK;
DC_LOG_DC("%s: %d streams\n",
@@ -2950,7 +3064,7 @@ static bool update_planes_and_stream_state(struct dc *dc,
* Ensures that we have enough pipes for newly added MPO planes
*/
if (dc->res_pool->funcs->remove_phantom_pipes)
- dc->res_pool->funcs->remove_phantom_pipes(dc, context);
+ dc->res_pool->funcs->remove_phantom_pipes(dc, context, false);
/*remove old surfaces from context */
if (!dc_rem_all_planes_for_stream(dc, stream, context)) {
@@ -2987,6 +3101,19 @@ static bool update_planes_and_stream_state(struct dc *dc,
if (update_type == UPDATE_TYPE_FULL) {
if (!dc->res_pool->funcs->validate_bandwidth(dc, context, false)) {
+ /* For phantom pipes we remove and create a new set of phantom pipes
+ * for each full update (because we don't know if we'll need phantom
+ * pipes until after the first round of validation). However, if validation
+ * fails we need to keep the existing phantom pipes (because we don't update
+ * the dc->current_state).
+ *
+ * The phantom stream/plane refcount is decremented for validation because
+ * we assume it'll be removed (the free comes when the dc_state is freed),
+ * but if validation fails we have to increment back the refcount so it's
+ * consistent.
+ */
+ if (dc->res_pool->funcs->retain_phantom_pipes)
+ dc->res_pool->funcs->retain_phantom_pipes(dc, dc->current_state);
BREAK_TO_DEBUGGER();
goto fail;
}
@@ -3297,22 +3424,6 @@ static void commit_planes_for_stream(struct dc *dc,
dc->hwss.pipe_control_lock(dc, top_pipe_to_program, true);
}
- if (update_type != UPDATE_TYPE_FAST) {
- for (i = 0; i < dc->res_pool->pipe_count; i++) {
- struct pipe_ctx *new_pipe = &context->res_ctx.pipe_ctx[i];
-
- if ((new_pipe->stream && new_pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) ||
- subvp_prev_use) {
- // If old context or new context has phantom pipes, apply
- // the phantom timings now. We can't change the phantom
- // pipe configuration safely without driver acquiring
- // the DMCUB lock first.
- dc->hwss.apply_ctx_to_hw(dc, context);
- break;
- }
- }
- }
-
dc_dmub_update_dirty_rect(dc, surface_count, stream, srf_updates, context);
if (update_type != UPDATE_TYPE_FAST) {
@@ -3370,6 +3481,24 @@ static void commit_planes_for_stream(struct dc *dc,
return;
}
+ if (update_type != UPDATE_TYPE_FAST) {
+ for (j = 0; j < dc->res_pool->pipe_count; j++) {
+ struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[j];
+
+ if (dc->debug.visual_confirm == VISUAL_CONFIRM_SUBVP &&
+ pipe_ctx->stream && pipe_ctx->plane_state) {
+ /* Only update visual confirm for SUBVP here.
+ * The bar appears on all pipes, so we need to update the bar on all displays,
+ * so the information doesn't get stale.
+ */
+ struct mpcc_blnd_cfg blnd_cfg = { 0 };
+
+ dc->hwss.update_visual_confirm_color(dc, pipe_ctx, &blnd_cfg.black_color,
+ pipe_ctx->plane_res.hubp->inst);
+ }
+ }
+ }
+
if (!IS_DIAG_DC(dc->ctx->dce_environment)) {
for (i = 0; i < surface_count; i++) {
struct dc_plane_state *plane_state = srf_updates[i].surface;
@@ -3487,7 +3616,6 @@ static void commit_planes_for_stream(struct dc *dc,
dc->hwss.update_plane_addr(dc, pipe_ctx);
}
}
-
}
if (should_lock_all_pipes && dc->hwss.interdependent_update_lock) {
@@ -3524,6 +3652,44 @@ static void commit_planes_for_stream(struct dc *dc,
top_pipe_to_program->stream_res.tg);
}
+ /* For phantom pipe OTG enable, it has to be done after any previous pipe
+ * that was in use has already been programmed at gotten its double buffer
+ * update for "disable".
+ */
+ if (update_type != UPDATE_TYPE_FAST) {
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ struct pipe_ctx *old_pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+
+ /* If an active, non-phantom pipe is being transitioned into a phantom
+ * pipe, wait for the double buffer update to complete first before we do
+ * ANY phantom pipe programming.
+ */
+ if (pipe->stream && pipe->stream->mall_stream_config.type == SUBVP_PHANTOM &&
+ old_pipe->stream && old_pipe->stream->mall_stream_config.type != SUBVP_PHANTOM) {
+ old_pipe->stream_res.tg->funcs->wait_for_state(
+ old_pipe->stream_res.tg,
+ CRTC_STATE_VBLANK);
+ old_pipe->stream_res.tg->funcs->wait_for_state(
+ old_pipe->stream_res.tg,
+ CRTC_STATE_VACTIVE);
+ }
+ }
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *new_pipe = &context->res_ctx.pipe_ctx[i];
+
+ if ((new_pipe->stream && new_pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) ||
+ subvp_prev_use) {
+ // If old context or new context has phantom pipes, apply
+ // the phantom timings now. We can't change the phantom
+ // pipe configuration safely without driver acquiring
+ // the DMCUB lock first.
+ dc->hwss.apply_ctx_to_hw(dc, context);
+ break;
+ }
+ }
+ }
+
if (update_type != UPDATE_TYPE_FAST)
dc->hwss.post_unlock_program_front_end(dc, context);
if (update_type != UPDATE_TYPE_FAST)
@@ -3563,10 +3729,24 @@ static void commit_planes_for_stream(struct dc *dc,
}
}
-/* Determines if the incoming context requires a applying transition state with unnecessary
- * pipe splitting and ODM disabled, due to hardware limitations. In a case where
- * the OPP associated with an MPCC might change due to plane additions, this function
+/**
+ * could_mpcc_tree_change_for_active_pipes - Check if an OPP associated with MPCC might change
+ *
+ * @dc: Used to get the current state status
+ * @stream: Target stream, which we want to remove the attached planes
+ * @surface_count: Number of surface update
+ * @is_plane_addition: [in] Fill out with true if it is a plane addition case
+ *
+ * DCN32x and newer support a feature named Dynamic ODM which can conflict with
+ * the MPO if used simultaneously in some specific configurations (e.g.,
+ * 4k@144). This function checks if the incoming context requires applying a
+ * transition state with unnecessary pipe splitting and ODM disabled to
+ * circumvent our hardware limitations to prevent this edge case. If the OPP
+ * associated with an MPCC might change due to plane additions, this function
* returns true.
+ *
+ * Return:
+ * Return true if OPP and MPCC might change, otherwise, return false.
*/
static bool could_mpcc_tree_change_for_active_pipes(struct dc *dc,
struct dc_stream_state *stream,
@@ -3576,6 +3756,7 @@ static bool could_mpcc_tree_change_for_active_pipes(struct dc *dc,
struct dc_stream_status *cur_stream_status = stream_get_status(dc->current_state, stream);
bool force_minimal_pipe_splitting = false;
+ bool subvp_active = false;
uint32_t i;
*is_plane_addition = false;
@@ -3608,39 +3789,55 @@ static bool could_mpcc_tree_change_for_active_pipes(struct dc *dc,
}
}
- /* For SubVP pipe split case when adding MPO video
- * we need to add a minimal transition. In this case
- * there will be 2 streams (1 main stream, 1 phantom
- * stream).
- */
- if (cur_stream_status &&
- dc->current_state->stream_count == 2 &&
- stream->mall_stream_config.type == SUBVP_MAIN) {
- bool is_pipe_split = false;
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &dc->current_state->res_ctx.pipe_ctx[i];
- for (i = 0; i < dc->res_pool->pipe_count; i++) {
- if (dc->current_state->res_ctx.pipe_ctx[i].stream == stream &&
- (dc->current_state->res_ctx.pipe_ctx[i].bottom_pipe ||
- dc->current_state->res_ctx.pipe_ctx[i].next_odm_pipe)) {
- is_pipe_split = true;
- break;
- }
+ if (pipe->stream && pipe->stream->mall_stream_config.type != SUBVP_NONE) {
+ subvp_active = true;
+ break;
}
+ }
+ /* For SubVP when adding or removing planes we need to add a minimal transition
+ * (even when disabling all planes). Whenever disabling a phantom pipe, we
+ * must use the minimal transition path to disable the pipe correctly.
+ *
+ * We want to use the minimal transition whenever subvp is active, not only if
+ * a plane is being added / removed from a subvp stream (MPO plane can be added
+ * to a DRR pipe of SubVP + DRR config, in which case we still want to run through
+ * a min transition to disable subvp.
+ */
+ if (cur_stream_status && subvp_active) {
/* determine if minimal transition is required due to SubVP*/
- if (surface_count > 0 && is_pipe_split) {
- if (cur_stream_status->plane_count > surface_count) {
- force_minimal_pipe_splitting = true;
- } else if (cur_stream_status->plane_count < surface_count) {
- force_minimal_pipe_splitting = true;
- *is_plane_addition = true;
- }
+ if (cur_stream_status->plane_count > surface_count) {
+ force_minimal_pipe_splitting = true;
+ } else if (cur_stream_status->plane_count < surface_count) {
+ force_minimal_pipe_splitting = true;
+ *is_plane_addition = true;
}
}
return force_minimal_pipe_splitting;
}
+/**
+ * commit_minimal_transition_state - Create a transition pipe split state
+ *
+ * @dc: Used to get the current state status
+ * @transition_base_context: New transition state
+ *
+ * In some specific configurations, such as pipe split on multi-display with
+ * MPO and/or Dynamic ODM, removing a plane may cause unsupported pipe
+ * programming when moving to new planes. To mitigate those types of problems,
+ * this function adds a transition state that minimizes pipe usage before
+ * programming the new configuration. When adding a new plane, the current
+ * state requires the least pipes, so it is applied without splitting. When
+ * removing a plane, the new state requires the least pipes, so it is applied
+ * without splitting.
+ *
+ * Return:
+ * Return false if something is wrong in the transition state.
+ */
static bool commit_minimal_transition_state(struct dc *dc,
struct dc_state *transition_base_context)
{
@@ -3650,9 +3847,48 @@ static bool commit_minimal_transition_state(struct dc *dc,
bool temp_subvp_policy;
enum dc_status ret = DC_ERROR_UNEXPECTED;
unsigned int i, j;
+ unsigned int pipe_in_use = 0;
+ bool subvp_in_use = false;
if (!transition_context)
return false;
+ /* Setup:
+ * Store the current ODM and MPC config in some temp variables to be
+ * restored after we commit the transition state.
+ */
+
+ /* check current pipes in use*/
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &transition_base_context->res_ctx.pipe_ctx[i];
+
+ if (pipe->plane_state)
+ pipe_in_use++;
+ }
+
+ /* If SubVP is enabled and we are adding or removing planes from any main subvp
+ * pipe, we must use the minimal transition.
+ */
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+
+ if (pipe->stream && pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ subvp_in_use = true;
+ break;
+ }
+ }
+
+ /* When the OS add a new surface if we have been used all of pipes with odm combine
+ * and mpc split feature, it need use commit_minimal_transition_state to transition safely.
+ * After OS exit MPO, it will back to use odm and mpc split with all of pipes, we need
+ * call it again. Otherwise return true to skip.
+ *
+ * Reduce the scenarios to use dc_commit_state_no_check in the stage of flip. Especially
+ * enter/exit MPO when DCN still have enough resources.
+ */
+ if (pipe_in_use != dc->res_pool->pipe_count && !subvp_in_use) {
+ dc_release_state(transition_context);
+ return true;
+ }
if (!dc->config.is_vmin_only_asic) {
tmp_mpc_policy = dc->debug.pipe_split_policy;
@@ -3667,7 +3903,7 @@ static bool commit_minimal_transition_state(struct dc *dc,
dc_resource_state_copy_construct(transition_base_context, transition_context);
- //commit minimal state
+ /* commit minimal state */
if (dc->res_pool->funcs->validate_bandwidth(dc, transition_context, false)) {
for (i = 0; i < transition_context->stream_count; i++) {
struct dc_stream_status *stream_status = &transition_context->stream_status[i];
@@ -3685,10 +3921,12 @@ static bool commit_minimal_transition_state(struct dc *dc,
ret = dc_commit_state_no_check(dc, transition_context);
}
- /*always release as dc_commit_state_no_check retains in good case*/
+ /* always release as dc_commit_state_no_check retains in good case */
dc_release_state(transition_context);
- /*restore previous pipe split and odm policy*/
+ /* TearDown:
+ * Restore original configuration for ODM and MPO.
+ */
if (!dc->config.is_vmin_only_asic)
dc->debug.pipe_split_policy = tmp_mpc_policy;
@@ -3696,12 +3934,12 @@ static bool commit_minimal_transition_state(struct dc *dc,
dc->debug.force_disable_subvp = temp_subvp_policy;
if (ret != DC_OK) {
- /*this should never happen*/
+ /* this should never happen */
BREAK_TO_DEBUGGER();
return false;
}
- /*force full surface update*/
+ /* force full surface update */
for (i = 0; i < dc->current_state->stream_count; i++) {
for (j = 0; j < dc->current_state->stream_status[i].plane_count; j++) {
dc->current_state->stream_status[i].plane_states[j]->update_flags.raw = 0xFFFFFFFF;
@@ -3719,6 +3957,7 @@ bool dc_update_planes_and_stream(struct dc *dc,
struct dc_state *context;
enum surface_update_type update_type;
int i;
+ struct mall_temp_config mall_temp_config;
/* In cases where MPO and split or ODM are used transitions can
* cause underflow. Apply stream configuration with minimal pipe
@@ -3750,11 +3989,29 @@ bool dc_update_planes_and_stream(struct dc *dc,
/* on plane removal, minimal state is the new one */
if (force_minimal_pipe_splitting && !is_plane_addition) {
+ /* Since all phantom pipes are removed in full validation,
+ * we have to save and restore the subvp/mall config when
+ * we do a minimal transition since the flags marking the
+ * pipe as subvp/phantom will be cleared (dc copy constructor
+ * creates a shallow copy).
+ */
+ if (dc->res_pool->funcs->save_mall_state)
+ dc->res_pool->funcs->save_mall_state(dc, context, &mall_temp_config);
if (!commit_minimal_transition_state(dc, context)) {
dc_release_state(context);
return false;
}
+ if (dc->res_pool->funcs->restore_mall_state)
+ dc->res_pool->funcs->restore_mall_state(dc, context, &mall_temp_config);
+ /* If we do a minimal transition with plane removal and the context
+ * has subvp we also have to retain back the phantom stream / planes
+ * since the refcount is decremented as part of the min transition
+ * (we commit a state with no subvp, so the phantom streams / planes
+ * had to be removed).
+ */
+ if (dc->res_pool->funcs->retain_phantom_pipes)
+ dc->res_pool->funcs->retain_phantom_pipes(dc, context);
update_type = UPDATE_TYPE_FULL;
}
@@ -3806,6 +4063,18 @@ void dc_commit_updates_for_stream(struct dc *dc,
struct dc_context *dc_ctx = dc->ctx;
int i, j;
+ /* TODO: Since change commit sequence can have a huge impact,
+ * we decided to only enable it for DCN3x. However, as soon as
+ * we get more confident about this change we'll need to enable
+ * the new sequence for all ASICs.
+ */
+ if (dc->ctx->dce_version >= DCN_VERSION_3_2) {
+ dc_update_planes_and_stream(dc, srf_updates,
+ surface_count, stream,
+ stream_update);
+ return;
+ }
+
stream_status = dc_stream_get_status(stream);
context = dc->current_state;
@@ -4387,21 +4656,17 @@ void dc_mclk_switch_using_fw_based_vblank_stretch_shut_down(struct dc *dc)
dc->current_state->bw_ctx.bw.dcn.clk.fw_based_mclk_switching_shut_down = true;
}
-/*
- *****************************************************************************
- * Function: dc_is_dmub_outbox_supported -
+/**
+ * dc_is_dmub_outbox_supported - Check if DMUB firmware support outbox notification
*
- * @brief
- * Checks whether DMUB FW supports outbox notifications, if supported
- * DM should register outbox interrupt prior to actually enabling interrupts
- * via dc_enable_dmub_outbox
+ * @dc: [in] dc structure
*
- * @param
- * [in] dc: dc structure
+ * Checks whether DMUB FW supports outbox notifications, if supported DM
+ * should register outbox interrupt prior to actually enabling interrupts
+ * via dc_enable_dmub_outbox
*
- * @return
- * True if DMUB FW supports outbox notifications, False otherwise
- *****************************************************************************
+ * Return:
+ * True if DMUB FW supports outbox notifications, False otherwise
*/
bool dc_is_dmub_outbox_supported(struct dc *dc)
{
@@ -4419,21 +4684,17 @@ bool dc_is_dmub_outbox_supported(struct dc *dc)
return dc->debug.enable_dmub_aux_for_legacy_ddc;
}
-/*
- *****************************************************************************
- * Function: dc_enable_dmub_notifications
+/**
+ * dc_enable_dmub_notifications - Check if dmub fw supports outbox
*
- * @brief
- * Calls dc_is_dmub_outbox_supported to check if dmub fw supports outbox
- * notifications. All DMs shall switch to dc_is_dmub_outbox_supported.
- * This API shall be removed after switching.
+ * @dc: [in] dc structure
*
- * @param
- * [in] dc: dc structure
+ * Calls dc_is_dmub_outbox_supported to check if dmub fw supports outbox
+ * notifications. All DMs shall switch to dc_is_dmub_outbox_supported. This
+ * API shall be removed after switching.
*
- * @return
- * True if DMUB FW supports outbox notifications, False otherwise
- *****************************************************************************
+ * Return:
+ * True if DMUB FW supports outbox notifications, False otherwise
*/
bool dc_enable_dmub_notifications(struct dc *dc)
{
@@ -4441,18 +4702,11 @@ bool dc_enable_dmub_notifications(struct dc *dc)
}
/**
- *****************************************************************************
- * Function: dc_enable_dmub_outbox
+ * dc_enable_dmub_outbox - Enables DMUB unsolicited notification
*
- * @brief
- * Enables DMUB unsolicited notifications to x86 via outbox
+ * dc: [in] dc structure
*
- * @param
- * [in] dc: dc structure
- *
- * @return
- * None
- *****************************************************************************
+ * Enables DMUB unsolicited notifications to x86 via outbox.
*/
void dc_enable_dmub_outbox(struct dc *dc)
{
@@ -4553,21 +4807,17 @@ uint8_t get_link_index_from_dpia_port_index(const struct dc *dc,
}
/**
- *****************************************************************************
- * Function: dc_process_dmub_set_config_async
+ * dc_process_dmub_set_config_async - Submits set_config command
*
- * @brief
- * Submits set_config command to dmub via inbox message
+ * @dc: [in] dc structure
+ * @link_index: [in] link_index: link index
+ * @payload: [in] aux payload
+ * @notify: [out] set_config immediate reply
*
- * @param
- * [in] dc: dc structure
- * [in] link_index: link index
- * [in] payload: aux payload
- * [out] notify: set_config immediate reply
+ * Submits set_config command to dmub via inbox message.
*
- * @return
- * True if successful, False if failure
- *****************************************************************************
+ * Return:
+ * True if successful, False if failure
*/
bool dc_process_dmub_set_config_async(struct dc *dc,
uint32_t link_index,
@@ -4603,21 +4853,17 @@ bool dc_process_dmub_set_config_async(struct dc *dc,
}
/**
- *****************************************************************************
- * Function: dc_process_dmub_set_mst_slots
+ * dc_process_dmub_set_mst_slots - Submits MST solt allocation
*
- * @brief
- * Submits mst slot allocation command to dmub via inbox message
+ * @dc: [in] dc structure
+ * @link_index: [in] link index
+ * @mst_alloc_slots: [in] mst slots to be allotted
+ * @mst_slots_in_use: [out] mst slots in use returned in failure case
*
- * @param
- * [in] dc: dc structure
- * [in] link_index: link index
- * [in] mst_alloc_slots: mst slots to be allotted
- * [out] mst_slots_in_use: mst slots in use returned in failure case
+ * Submits mst slot allocation command to dmub via inbox message
*
- * @return
- * DC_OK if successful, DC_ERROR if failure
- *****************************************************************************
+ * Return:
+ * DC_OK if successful, DC_ERROR if failure
*/
enum dc_status dc_process_dmub_set_mst_slots(const struct dc *dc,
uint32_t link_index,
@@ -4657,19 +4903,12 @@ enum dc_status dc_process_dmub_set_mst_slots(const struct dc *dc,
}
/**
- *****************************************************************************
- * Function: dc_process_dmub_dpia_hpd_int_enable
+ * dc_process_dmub_dpia_hpd_int_enable - Submits DPIA DPD interruption
*
- * @brief
- * Submits dpia hpd int enable command to dmub via inbox message
+ * @dc [in]: dc structure
+ * @hpd_int_enable [in]: 1 for hpd int enable, 0 to disable
*
- * @param
- * [in] dc: dc structure
- * [in] hpd_int_enable: 1 for hpd int enable, 0 to disable
- *
- * @return
- * None
- *****************************************************************************
+ * Submits dpia hpd int enable command to dmub via inbox message
*/
void dc_process_dmub_dpia_hpd_int_enable(const struct dc *dc,
uint32_t hpd_int_enable)
@@ -4698,16 +4937,13 @@ void dc_disable_accelerated_mode(struct dc *dc)
/**
- *****************************************************************************
- * dc_notify_vsync_int_state() - notifies vsync enable/disable state
+ * dc_notify_vsync_int_state - notifies vsync enable/disable state
* @dc: dc structure
- * @stream: stream where vsync int state changed
- * @enable: whether vsync is enabled or disabled
+ * @stream: stream where vsync int state changed
+ * @enable: whether vsync is enabled or disabled
*
- * Called when vsync is enabled/disabled
- * Will notify DMUB to start/stop ABM interrupts after steady state is reached
- *
- *****************************************************************************
+ * Called when vsync is enabled/disabled Will notify DMUB to start/stop ABM
+ * interrupts after steady state is reached.
*/
void dc_notify_vsync_int_state(struct dc *dc, struct dc_stream_state *stream, bool enable)
{
@@ -4749,14 +4985,18 @@ void dc_notify_vsync_int_state(struct dc *dc, struct dc_stream_state *stream, bo
if (pipe->stream_res.abm && pipe->stream_res.abm->funcs->set_abm_pause)
pipe->stream_res.abm->funcs->set_abm_pause(pipe->stream_res.abm, !enable, i, pipe->stream_res.tg->inst);
}
-/*
- * dc_extended_blank_supported: Decide whether extended blank is supported
+
+/**
+ * dc_extended_blank_supported 0 Decide whether extended blank is supported
*
- * Extended blank is a freesync optimization feature to be enabled in the future.
- * During the extra vblank period gained from freesync, we have the ability to enter z9/z10.
+ * @dc: [in] Current DC state
*
- * @param [in] dc: Current DC state
- * @return: Indicate whether extended blank is supported (true or false)
+ * Extended blank is a freesync optimization feature to be enabled in the
+ * future. During the extra vblank period gained from freesync, we have the
+ * ability to enter z9/z10.
+ *
+ * Return:
+ * Indicate whether extended blank is supported (true or false)
*/
bool dc_extended_blank_supported(struct dc *dc)
{
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
index 7c2e3b8..471078f 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
@@ -366,6 +366,7 @@ void get_hdr_visual_confirm_color(
struct tg_color *color)
{
uint32_t color_value = MAX_TG_COLOR_VALUE;
+ bool is_sdr = false;
/* Determine the overscan color based on the top-most (desktop) plane's context */
struct pipe_ctx *top_pipe_ctx = pipe_ctx;
@@ -382,7 +383,8 @@ void get_hdr_visual_confirm_color(
/* FreeSync 2 ARGB2101010 - set border color to pink */
color->color_r_cr = color_value;
color->color_b_cb = color_value;
- }
+ } else
+ is_sdr = true;
break;
case PIXEL_FORMAT_FP16:
if (top_pipe_ctx->stream->out_transfer_func->tf == TRANSFER_FUNCTION_PQ) {
@@ -391,14 +393,19 @@ void get_hdr_visual_confirm_color(
} else if (top_pipe_ctx->stream->out_transfer_func->tf == TRANSFER_FUNCTION_GAMMA22) {
/* FreeSync 2 HDR - set border color to green */
color->color_g_y = color_value;
- }
+ } else
+ is_sdr = true;
break;
default:
+ is_sdr = true;
+ break;
+ }
+
+ if (is_sdr) {
/* SDR - set border color to Gray */
color->color_r_cr = color_value/2;
color->color_b_cb = color_value/2;
color->color_g_y = color_value/2;
- break;
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link.c b/drivers/gpu/drm/amd/display/dc/core/dc_link.c
index d7b1ace6..342e906 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link.c
@@ -3378,7 +3378,7 @@ bool dc_link_setup_psr(struct dc_link *link,
case FAMILY_YELLOW_CARP:
case AMDGPU_FAMILY_GC_10_3_6:
case AMDGPU_FAMILY_GC_11_0_1:
- if (dc->debug.disable_z10)
+ if (dc->debug.disable_z10 || dc->debug.psr_skip_crtc_disable)
psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true;
break;
default:
@@ -4229,6 +4229,7 @@ static void fpga_dp_hpo_enable_link_and_stream(struct dc_state *state, struct pi
link_hwss->ext.set_throttled_vcp_size(pipe_ctx, avg_time_slots_per_mtp);
dc->hwss.unblank_stream(pipe_ctx, &stream->link->cur_link_settings);
+ dc->hwss.enable_audio_stream(pipe_ctx);
}
void core_link_enable_stream(
@@ -4308,10 +4309,7 @@ void core_link_enable_stream(
/* Still enable stream features & audio on seamless boot for DP external displays */
if (pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT) {
enable_stream_features(pipe_ctx);
- if (pipe_ctx->stream_res.audio != NULL) {
- pipe_ctx->stream_res.stream_enc->funcs->dp_audio_enable(pipe_ctx->stream_res.stream_enc);
- dc->hwss.enable_audio_stream(pipe_ctx);
- }
+ dc->hwss.enable_audio_stream(pipe_ctx);
}
#if defined(CONFIG_DRM_AMD_DC_HDCP)
@@ -4665,6 +4663,10 @@ void dc_link_set_preferred_training_settings(struct dc *dc,
link->preferred_link_setting.link_rate = LINK_RATE_UNKNOWN;
}
+ if (link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT &&
+ link->type == dc_connection_mst_branch)
+ dm_helpers_dp_mst_update_branch_bandwidth(dc->ctx, link);
+
/* Retrain now, or wait until next stream update to apply */
if (skip_immediate_retrain == false)
dc_link_set_preferred_link_settings(dc, &link->preferred_link_setting, link);
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
index 6512313..ce8d6a5 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
@@ -82,6 +82,7 @@ struct dp_hdmi_dongle_signature_data {
#define HDMI_SCDC_STATUS_FLAGS 0x40
#define HDMI_SCDC_ERR_DETECT 0x50
#define HDMI_SCDC_TEST_CONFIG 0xC0
+#define HDMI_SCDC_DEVICE_ID 0xD3
union hdmi_scdc_update_read_data {
uint8_t byte[2];
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
index 1254d38..dedd124 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
@@ -1912,7 +1912,7 @@ enum dc_status dpcd_configure_lttpr_mode(struct dc_link *link, struct link_train
return status;
}
-static void dpcd_exit_training_mode(struct dc_link *link)
+static void dpcd_exit_training_mode(struct dc_link *link, enum dp_link_encoding encoding)
{
uint8_t sink_status = 0;
uint8_t i;
@@ -1920,12 +1920,14 @@ static void dpcd_exit_training_mode(struct dc_link *link)
/* clear training pattern set */
dpcd_set_training_pattern(link, DP_TRAINING_PATTERN_VIDEOIDLE);
- /* poll for intra-hop disable */
- for (i = 0; i < 10; i++) {
- if ((core_link_read_dpcd(link, DP_SINK_STATUS, &sink_status, 1) == DC_OK) &&
- (sink_status & DP_INTRA_HOP_AUX_REPLY_INDICATION) == 0)
- break;
- udelay(1000);
+ if (encoding == DP_128b_132b_ENCODING) {
+ /* poll for intra-hop disable */
+ for (i = 0; i < 10; i++) {
+ if ((core_link_read_dpcd(link, DP_SINK_STATUS, &sink_status, 1) == DC_OK) &&
+ (sink_status & DP_INTRA_HOP_AUX_REPLY_INDICATION) == 0)
+ break;
+ udelay(1000);
+ }
}
}
@@ -2649,7 +2651,7 @@ enum link_training_result dc_link_dp_perform_link_training(
<_settings);
/* reset previous training states */
- dpcd_exit_training_mode(link);
+ dpcd_exit_training_mode(link, encoding);
/* configure link prior to entering training mode */
dpcd_configure_lttpr_mode(link, <_settings);
@@ -2670,7 +2672,7 @@ enum link_training_result dc_link_dp_perform_link_training(
ASSERT(0);
/* exit training mode */
- dpcd_exit_training_mode(link);
+ dpcd_exit_training_mode(link, encoding);
/* switch to video idle */
if ((status == LINK_TRAINING_SUCCESS) || !skip_video_pattern)
@@ -2771,8 +2773,11 @@ bool perform_link_training_with_retries(
/* Update verified link settings to current one
* Because DPIA LT might fallback to lower link setting.
*/
- link->verified_link_cap.link_rate = link->cur_link_settings.link_rate;
- link->verified_link_cap.lane_count = link->cur_link_settings.lane_count;
+ if (stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) {
+ link->verified_link_cap.link_rate = link->cur_link_settings.link_rate;
+ link->verified_link_cap.lane_count = link->cur_link_settings.lane_count;
+ dm_helpers_dp_mst_update_branch_bandwidth(link->ctx, link);
+ }
}
} else {
status = dc_link_dp_perform_link_training(link,
@@ -3020,7 +3025,7 @@ static enum dc_link_rate get_lttpr_max_link_rate(struct dc_link *link)
static enum dc_link_rate get_cable_max_link_rate(struct dc_link *link)
{
- enum dc_link_rate cable_max_link_rate = LINK_RATE_HIGH3;
+ enum dc_link_rate cable_max_link_rate = LINK_RATE_UNKNOWN;
if (link->dpcd_caps.cable_id.bits.UHBR10_20_CAPABILITY & DP_UHBR20)
cable_max_link_rate = LINK_RATE_UHBR20;
@@ -3083,15 +3088,29 @@ struct dc_link_settings dp_get_max_link_cap(struct dc_link *link)
max_link_cap.link_spread =
link->reported_link_cap.link_spread;
- /* Lower link settings based on cable attributes */
+ /* Lower link settings based on cable attributes
+ * Cable ID is a DP2 feature to identify max certified link rate that
+ * a cable can carry. The cable identification method requires both
+ * cable and display hardware support. Since the specs comes late, it is
+ * anticipated that the first round of DP2 cables and displays may not
+ * be fully compatible to reliably return cable ID data. Therefore the
+ * decision of our cable id policy is that if the cable can return non
+ * zero cable id data, we will take cable's link rate capability into
+ * account. However if we get zero data, the cable link rate capability
+ * is considered inconclusive. In this case, we will not take cable's
+ * capability into account to avoid of over limiting hardware capability
+ * from users. The max overall link rate capability is still determined
+ * after actual dp pre-training. Cable id is considered as an auxiliary
+ * method of determining max link bandwidth capability.
+ */
cable_max_link_rate = get_cable_max_link_rate(link);
if (!link->dc->debug.ignore_cable_id &&
+ cable_max_link_rate != LINK_RATE_UNKNOWN &&
cable_max_link_rate < max_link_cap.link_rate)
max_link_cap.link_rate = cable_max_link_rate;
- /*
- * account for lttpr repeaters cap
+ /* account for lttpr repeaters cap
* notes: repeaters do not snoop in the DPRX Capabilities addresses (3.6.3).
*/
if (dp_is_lttpr_present(link)) {
@@ -4540,9 +4559,19 @@ void dc_link_dp_handle_link_loss(struct dc_link *link)
for (i = 0; i < MAX_PIPES; i++) {
pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i];
- if (pipe_ctx && pipe_ctx->stream && !pipe_ctx->stream->dpms_off &&
- pipe_ctx->stream->link == link && !pipe_ctx->prev_odm_pipe)
+ if (pipe_ctx && pipe_ctx->stream && !pipe_ctx->stream->dpms_off
+ && pipe_ctx->stream->link == link && !pipe_ctx->prev_odm_pipe) {
+ // Always use max settings here for DP 1.4a LL Compliance CTS
+ if (link->is_automated) {
+ pipe_ctx->link_config.dp_link_settings.lane_count =
+ link->verified_link_cap.lane_count;
+ pipe_ctx->link_config.dp_link_settings.link_rate =
+ link->verified_link_cap.link_rate;
+ pipe_ctx->link_config.dp_link_settings.link_spread =
+ link->verified_link_cap.link_spread;
+ }
core_link_enable_stream(link->dc->current_state, pipe_ctx);
+ }
}
}
@@ -4583,6 +4612,8 @@ bool dc_link_handle_hpd_rx_irq(struct dc_link *link, union hpd_irq_data *out_hpd
}
if (hpd_irq_dpcd_data.bytes.device_service_irq.bits.AUTOMATED_TEST) {
+ // Workaround for DP 1.4a LL Compliance CTS as USB4 has to share encoders unlike DP and USBC
+ link->is_automated = true;
device_service_clear.bits.AUTOMATED_TEST = 1;
core_link_write_dpcd(
link,
@@ -5031,7 +5062,7 @@ static bool dpcd_read_sink_ext_caps(struct dc_link *link)
return true;
}
-bool dp_retrieve_lttpr_cap(struct dc_link *link)
+enum dc_status dp_retrieve_lttpr_cap(struct dc_link *link)
{
uint8_t lttpr_dpcd_data[8];
enum dc_status status = DC_ERROR_UNEXPECTED;
@@ -5099,7 +5130,7 @@ bool dp_retrieve_lttpr_cap(struct dc_link *link)
CONN_DATA_DETECT(link, lttpr_dpcd_data, sizeof(lttpr_dpcd_data), "LTTPR Caps: ");
DC_LOG_DC("is_lttpr_present = %d\n", is_lttpr_present);
- return is_lttpr_present;
+ return status;
}
bool dp_is_lttpr_present(struct dc_link *link)
@@ -5227,122 +5258,11 @@ static void retrieve_cable_id(struct dc_link *link)
&link->dpcd_caps.cable_id, &usbc_cable_id);
}
-/* DPRX may take some time to respond to AUX messages after HPD asserted.
- * If AUX read unsuccessful, try to wake unresponsive DPRX by toggling DPCD SET_POWER (0x600).
- */
-static enum dc_status wa_try_to_wake_dprx(struct dc_link *link, uint64_t timeout_ms)
+static enum dc_status wake_up_aux_channel(struct dc_link *link)
{
enum dc_status status = DC_ERROR_UNEXPECTED;
- uint8_t dpcd_data = 0;
- uint64_t start_ts = 0;
- uint64_t current_ts = 0;
- uint64_t time_taken_ms = 0;
- enum dc_connection_type type = dc_connection_none;
- bool lttpr_present;
- bool vbios_lttpr_interop = link->dc->caps.vbios_lttpr_aware;
-
- lttpr_present = dp_is_lttpr_present(link) ||
- (!vbios_lttpr_interop || !link->dc->caps.extended_aux_timeout_support);
- DC_LOG_DC("lttpr_present = %d.\n", lttpr_present ? 1 : 0);
-
- /* Issue an AUX read to test DPRX responsiveness. If LTTPR is supported the first read is expected to
- * be to determine LTTPR capabilities. Otherwise trying to read power state should be an innocuous AUX read.
- */
- if (lttpr_present)
- status = core_link_read_dpcd(
- link,
- DP_LT_TUNABLE_PHY_REPEATER_FIELD_DATA_STRUCTURE_REV,
- &dpcd_data,
- sizeof(dpcd_data));
- else
- status = core_link_read_dpcd(
- link,
- DP_SET_POWER,
- &dpcd_data,
- sizeof(dpcd_data));
-
- if (status != DC_OK) {
- DC_LOG_WARNING("%s: Read DPCD LTTPR_CAP failed - try to toggle DPCD SET_POWER for %lld ms.",
- __func__,
- timeout_ms);
- start_ts = dm_get_timestamp(link->ctx);
-
- do {
- if (!dc_link_detect_sink(link, &type) || type == dc_connection_none)
- break;
-
- dpcd_data = DP_SET_POWER_D3;
- status = core_link_write_dpcd(
- link,
- DP_SET_POWER,
- &dpcd_data,
- sizeof(dpcd_data));
-
- dpcd_data = DP_SET_POWER_D0;
- status = core_link_write_dpcd(
- link,
- DP_SET_POWER,
- &dpcd_data,
- sizeof(dpcd_data));
-
- current_ts = dm_get_timestamp(link->ctx);
- time_taken_ms = div_u64(dm_get_elapse_time_in_ns(link->ctx, current_ts, start_ts), 1000000);
- } while (status != DC_OK && time_taken_ms < timeout_ms);
-
- DC_LOG_WARNING("%s: DPCD SET_POWER %s after %lld ms%s",
- __func__,
- (status == DC_OK) ? "succeeded" : "failed",
- time_taken_ms,
- (type == dc_connection_none) ? ". Unplugged." : ".");
- }
-
- return status;
-}
-
-static bool retrieve_link_cap(struct dc_link *link)
-{
- /* DP_ADAPTER_CAP - DP_DPCD_REV + 1 == 16 and also DP_DSC_BITS_PER_PIXEL_INC - DP_DSC_SUPPORT + 1 == 16,
- * which means size 16 will be good for both of those DPCD register block reads
- */
- uint8_t dpcd_data[16];
- /*Only need to read 1 byte starting from DP_DPRX_FEATURE_ENUMERATION_LIST.
- */
- uint8_t dpcd_dprx_data = '\0';
- uint8_t dpcd_power_state = '\0';
-
- struct dp_device_vendor_id sink_id;
- union down_stream_port_count down_strm_port_count;
- union edp_configuration_cap edp_config_cap;
- union dp_downstream_port_present ds_port = { 0 };
- enum dc_status status = DC_ERROR_UNEXPECTED;
- uint32_t read_dpcd_retry_cnt = 3;
uint32_t aux_channel_retry_cnt = 0;
- int i;
- struct dp_sink_hw_fw_revision dp_hw_fw_revision;
- const uint32_t post_oui_delay = 30; // 30ms
- bool is_lttpr_present = false;
-
- memset(dpcd_data, '\0', sizeof(dpcd_data));
- memset(&down_strm_port_count,
- '\0', sizeof(union down_stream_port_count));
- memset(&edp_config_cap, '\0',
- sizeof(union edp_configuration_cap));
-
- /* if extended timeout is supported in hardware,
- * default to LTTPR timeout (3.2ms) first as a W/A for DP link layer
- * CTS 4.2.1.1 regression introduced by CTS specs requirement update.
- */
- dc_link_aux_try_to_configure_timeout(link->ddc,
- LINK_AUX_DEFAULT_LTTPR_TIMEOUT_PERIOD);
-
- /* Try to ensure AUX channel active before proceeding. */
- if (link->dc->debug.aux_wake_wa.bits.enable_wa) {
- uint64_t timeout_ms = link->dc->debug.aux_wake_wa.bits.timeout_ms;
-
- if (link->dc->debug.aux_wake_wa.bits.use_default_timeout)
- timeout_ms = LINK_AUX_WAKE_TIMEOUT_MS;
- status = wa_try_to_wake_dprx(link, timeout_ms);
- }
+ uint8_t dpcd_power_state = '\0';
while (status != DC_OK && aux_channel_retry_cnt < 10) {
status = core_link_read_dpcd(link, DP_SET_POWER,
@@ -5359,7 +5279,6 @@ static bool retrieve_link_cap(struct dc_link *link)
}
}
- /* If aux channel is not active, return false and trigger another detect*/
if (status != DC_OK) {
dpcd_power_state = DP_SET_POWER_D0;
status = core_link_write_dpcd(
@@ -5374,12 +5293,56 @@ static bool retrieve_link_cap(struct dc_link *link)
DP_SET_POWER,
&dpcd_power_state,
sizeof(dpcd_power_state));
- return false;
+ return DC_ERROR_UNEXPECTED;
}
- is_lttpr_present = dp_retrieve_lttpr_cap(link);
+ return DC_OK;
+}
- if (is_lttpr_present)
+static bool retrieve_link_cap(struct dc_link *link)
+{
+ /* DP_ADAPTER_CAP - DP_DPCD_REV + 1 == 16 and also DP_DSC_BITS_PER_PIXEL_INC - DP_DSC_SUPPORT + 1 == 16,
+ * which means size 16 will be good for both of those DPCD register block reads
+ */
+ uint8_t dpcd_data[16];
+ /*Only need to read 1 byte starting from DP_DPRX_FEATURE_ENUMERATION_LIST.
+ */
+ uint8_t dpcd_dprx_data = '\0';
+
+ struct dp_device_vendor_id sink_id;
+ union down_stream_port_count down_strm_port_count;
+ union edp_configuration_cap edp_config_cap;
+ union dp_downstream_port_present ds_port = { 0 };
+ enum dc_status status = DC_ERROR_UNEXPECTED;
+ uint32_t read_dpcd_retry_cnt = 3;
+ int i;
+ struct dp_sink_hw_fw_revision dp_hw_fw_revision;
+ const uint32_t post_oui_delay = 30; // 30ms
+
+ memset(dpcd_data, '\0', sizeof(dpcd_data));
+ memset(&down_strm_port_count,
+ '\0', sizeof(union down_stream_port_count));
+ memset(&edp_config_cap, '\0',
+ sizeof(union edp_configuration_cap));
+
+ /* if extended timeout is supported in hardware,
+ * default to LTTPR timeout (3.2ms) first as a W/A for DP link layer
+ * CTS 4.2.1.1 regression introduced by CTS specs requirement update.
+ */
+ dc_link_aux_try_to_configure_timeout(link->ddc,
+ LINK_AUX_DEFAULT_LTTPR_TIMEOUT_PERIOD);
+
+ status = dp_retrieve_lttpr_cap(link);
+
+ if (status != DC_OK) {
+ status = wake_up_aux_channel(link);
+ if (status == DC_OK)
+ dp_retrieve_lttpr_cap(link);
+ else
+ return false;
+ }
+
+ if (dp_is_lttpr_present(link))
configure_lttpr_mode_transparent(link);
/* Read DP tunneling information. */
@@ -5406,7 +5369,7 @@ static bool retrieve_link_cap(struct dc_link *link)
return false;
}
- if (!is_lttpr_present)
+ if (!dp_is_lttpr_present(link))
dc_link_aux_try_to_configure_timeout(link->ddc, LINK_AUX_DEFAULT_TIMEOUT_PERIOD);
{
@@ -7294,6 +7257,7 @@ void dp_retrain_link_dp_test(struct dc_link *link,
struct pipe_ctx *pipes =
&link->dc->current_state->res_ctx.pipe_ctx[0];
unsigned int i;
+ bool do_fallback = false;
for (i = 0; i < MAX_PIPES; i++) {
@@ -7326,32 +7290,23 @@ void dp_retrain_link_dp_test(struct dc_link *link,
memset(&link->cur_link_settings, 0,
sizeof(link->cur_link_settings));
+ if (link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA)
+ do_fallback = true;
+
perform_link_training_with_retries(
link_setting,
skip_video_pattern,
LINK_TRAINING_ATTEMPTS,
&pipes[i],
SIGNAL_TYPE_DISPLAY_PORT,
- false);
+ do_fallback);
link->dc->hwss.enable_stream(&pipes[i]);
link->dc->hwss.unblank_stream(&pipes[i],
link_setting);
- if (pipes[i].stream_res.audio) {
- /* notify audio driver for
- * audio modes of monitor */
- pipes[i].stream_res.audio->funcs->az_enable(
- pipes[i].stream_res.audio);
-
- /* un-mute audio */
- /* TODO: audio should be per stream rather than
- * per link */
- pipes[i].stream_res.stream_enc->funcs->
- audio_mute_control(
- pipes[i].stream_res.stream_enc, false);
- }
+ link->dc->hwss.enable_audio_stream(&pipes[i]);
}
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c
index 74e36b3..d130d58 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_dpia.c
@@ -791,10 +791,14 @@ static enum link_training_result dpia_training_eq_transparent(
}
if (dp_is_ch_eq_done(lane_count, dpcd_lane_status) &&
- dp_is_symbol_locked(link->cur_link_settings.lane_count, dpcd_lane_status) &&
- dp_is_interlane_aligned(dpcd_lane_status_updated)) {
- result = LINK_TRAINING_SUCCESS;
- break;
+ dp_is_symbol_locked(link->cur_link_settings.lane_count, dpcd_lane_status)) {
+ /* Take into consideration corner case for DP 1.4a LL Compliance CTS as USB4
+ * has to share encoders unlike DP and USBC
+ */
+ if (dp_is_interlane_aligned(dpcd_lane_status_updated) || (link->is_automated && retries_eq)) {
+ result = LINK_TRAINING_SUCCESS;
+ break;
+ }
}
/* Update VS/PE. */
@@ -1008,7 +1012,8 @@ enum link_training_result dc_link_dpia_perform_link_training(
*/
if (result == LINK_TRAINING_SUCCESS) {
msleep(5);
- result = dp_check_link_loss_status(link, <_settings);
+ if (!link->is_automated)
+ result = dp_check_link_loss_status(link, <_settings);
} else if (result == LINK_TRAINING_ABORT) {
dpia_training_abort(link, <_settings, repeater_id);
} else {
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
index fd8db48..da1646855 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
@@ -1768,6 +1768,17 @@ bool dc_remove_plane_from_context(
return true;
}
+/**
+ * dc_rem_all_planes_for_stream - Remove planes attached to the target stream.
+ *
+ * @dc: Current dc state.
+ * @stream: Target stream, which we want to remove the attached plans.
+ * @context: New context.
+ *
+ * Return:
+ * Return true if DC was able to remove all planes from the target
+ * stream, otherwise, return false.
+ */
bool dc_rem_all_planes_for_stream(
const struct dc *dc,
struct dc_stream_state *stream,
@@ -2562,9 +2573,12 @@ enum dc_status resource_map_pool_resources(
/**
* dc_resource_state_copy_construct_current() - Creates a new dc_state from existing state
- * Is a shallow copy. Increments refcounts on existing streams and planes.
+ *
* @dc: copy out of dc->current_state
* @dst_ctx: copy into this
+ *
+ * This function makes a shallow copy of the current DC state and increments
+ * refcounts on existing streams and planes.
*/
void dc_resource_state_copy_construct_current(
const struct dc *dc,
@@ -2593,15 +2607,241 @@ bool dc_resource_is_dsc_encoding_supported(const struct dc *dc)
return dc->res_pool->res_cap->num_dsc > 0;
}
+static bool planes_changed_for_existing_stream(struct dc_state *context,
+ struct dc_stream_state *stream,
+ const struct dc_validation_set set[],
+ int set_count)
+{
+ int i, j;
+ struct dc_stream_status *stream_status = NULL;
+
+ for (i = 0; i < context->stream_count; i++) {
+ if (context->streams[i] == stream) {
+ stream_status = &context->stream_status[i];
+ break;
+ }
+ }
+
+ if (!stream_status)
+ ASSERT(0);
+
+ for (i = 0; i < set_count; i++)
+ if (set[i].stream == stream)
+ break;
+
+ if (i == set_count)
+ ASSERT(0);
+
+ if (set[i].plane_count != stream_status->plane_count)
+ return true;
+
+ for (j = 0; j < set[i].plane_count; j++)
+ if (set[i].plane_states[j] != stream_status->plane_states[j])
+ return true;
+
+ return false;
+}
/**
- * dc_validate_global_state() - Determine if HW can support a given state
- * Checks HW resource availability and bandwidth requirement.
+ * dc_validate_with_context - Validate and update the potential new stream in the context object
+ *
+ * @dc: Used to get the current state status
+ * @set: An array of dc_validation_set with all the current streams reference
+ * @set_count: Total of streams
+ * @context: New context
+ * @fast_validate: Enable or disable fast validation
+ *
+ * This function updates the potential new stream in the context object. It
+ * creates multiple lists for the add, remove, and unchanged streams. In
+ * particular, if the unchanged streams have a plane that changed, it is
+ * necessary to remove all planes from the unchanged streams. In summary, this
+ * function is responsible for validating the new context.
+ *
+ * Return:
+ * In case of success, return DC_OK (1), otherwise, return a DC error.
+ */
+enum dc_status dc_validate_with_context(struct dc *dc,
+ const struct dc_validation_set set[],
+ int set_count,
+ struct dc_state *context,
+ bool fast_validate)
+{
+ struct dc_stream_state *unchanged_streams[MAX_PIPES] = { 0 };
+ struct dc_stream_state *del_streams[MAX_PIPES] = { 0 };
+ struct dc_stream_state *add_streams[MAX_PIPES] = { 0 };
+ int old_stream_count = context->stream_count;
+ enum dc_status res = DC_ERROR_UNEXPECTED;
+ int unchanged_streams_count = 0;
+ int del_streams_count = 0;
+ int add_streams_count = 0;
+ bool found = false;
+ int i, j, k;
+
+ DC_LOGGER_INIT(dc->ctx->logger);
+
+ /* First build a list of streams to be remove from current context */
+ for (i = 0; i < old_stream_count; i++) {
+ struct dc_stream_state *stream = context->streams[i];
+
+ for (j = 0; j < set_count; j++) {
+ if (stream == set[j].stream) {
+ found = true;
+ break;
+ }
+ }
+
+ if (!found)
+ del_streams[del_streams_count++] = stream;
+
+ found = false;
+ }
+
+ /* Second, build a list of new streams */
+ for (i = 0; i < set_count; i++) {
+ struct dc_stream_state *stream = set[i].stream;
+
+ for (j = 0; j < old_stream_count; j++) {
+ if (stream == context->streams[j]) {
+ found = true;
+ break;
+ }
+ }
+
+ if (!found)
+ add_streams[add_streams_count++] = stream;
+
+ found = false;
+ }
+
+ /* Build a list of unchanged streams which is necessary for handling
+ * planes change such as added, removed, and updated.
+ */
+ for (i = 0; i < set_count; i++) {
+ /* Check if stream is part of the delete list */
+ for (j = 0; j < del_streams_count; j++) {
+ if (set[i].stream == del_streams[j]) {
+ found = true;
+ break;
+ }
+ }
+
+ if (!found) {
+ /* Check if stream is part of the add list */
+ for (j = 0; j < add_streams_count; j++) {
+ if (set[i].stream == add_streams[j]) {
+ found = true;
+ break;
+ }
+ }
+ }
+
+ if (!found)
+ unchanged_streams[unchanged_streams_count++] = set[i].stream;
+
+ found = false;
+ }
+
+ /* Remove all planes for unchanged streams if planes changed */
+ for (i = 0; i < unchanged_streams_count; i++) {
+ if (planes_changed_for_existing_stream(context,
+ unchanged_streams[i],
+ set,
+ set_count)) {
+ if (!dc_rem_all_planes_for_stream(dc,
+ unchanged_streams[i],
+ context)) {
+ res = DC_FAIL_DETACH_SURFACES;
+ goto fail;
+ }
+ }
+ }
+
+ /* Remove all planes for removed streams and then remove the streams */
+ for (i = 0; i < del_streams_count; i++) {
+ /* Need to cpy the dwb data from the old stream in order to efc to work */
+ if (del_streams[i]->num_wb_info > 0) {
+ for (j = 0; j < add_streams_count; j++) {
+ if (del_streams[i]->sink == add_streams[j]->sink) {
+ add_streams[j]->num_wb_info = del_streams[i]->num_wb_info;
+ for (k = 0; k < del_streams[i]->num_wb_info; k++)
+ add_streams[j]->writeback_info[k] = del_streams[i]->writeback_info[k];
+ }
+ }
+ }
+
+ if (!dc_rem_all_planes_for_stream(dc, del_streams[i], context)) {
+ res = DC_FAIL_DETACH_SURFACES;
+ goto fail;
+ }
+
+ res = dc_remove_stream_from_ctx(dc, context, del_streams[i]);
+ if (res != DC_OK)
+ goto fail;
+ }
+
+ /* Swap seamless boot stream to pipe 0 (if needed) to ensure pipe_ctx
+ * matches. This may change in the future if seamless_boot_stream can be
+ * multiple.
+ */
+ for (i = 0; i < add_streams_count; i++) {
+ mark_seamless_boot_stream(dc, add_streams[i]);
+ if (add_streams[i]->apply_seamless_boot_optimization && i != 0) {
+ struct dc_stream_state *temp = add_streams[0];
+
+ add_streams[0] = add_streams[i];
+ add_streams[i] = temp;
+ break;
+ }
+ }
+
+ /* Add new streams and then add all planes for the new stream */
+ for (i = 0; i < add_streams_count; i++) {
+ calculate_phy_pix_clks(add_streams[i]);
+ res = dc_add_stream_to_ctx(dc, context, add_streams[i]);
+ if (res != DC_OK)
+ goto fail;
+
+ if (!add_all_planes_for_stream(dc, add_streams[i], set, set_count, context)) {
+ res = DC_FAIL_ATTACH_SURFACES;
+ goto fail;
+ }
+ }
+
+ /* Add all planes for unchanged streams if planes changed */
+ for (i = 0; i < unchanged_streams_count; i++) {
+ if (planes_changed_for_existing_stream(context,
+ unchanged_streams[i],
+ set,
+ set_count)) {
+ if (!add_all_planes_for_stream(dc, unchanged_streams[i], set, set_count, context)) {
+ res = DC_FAIL_ATTACH_SURFACES;
+ goto fail;
+ }
+ }
+ }
+
+ res = dc_validate_global_state(dc, context, fast_validate);
+
+fail:
+ if (res != DC_OK)
+ DC_LOG_WARNING("%s:resource validation failed, dc_status:%d\n",
+ __func__,
+ res);
+
+ return res;
+}
+
+/**
+ * dc_validate_global_state() - Determine if hardware can support a given state
+ *
* @dc: dc struct for this driver
* @new_ctx: state to be validated
* @fast_validate: set to true if only yes/no to support matters
*
- * Return: DC_OK if the result can be programmed. Otherwise, an error code.
+ * Checks hardware resource availability and bandwidth requirement.
+ *
+ * Return:
+ * DC_OK if the result can be programmed. Otherwise, an error code.
*/
enum dc_status dc_validate_global_state(
struct dc *dc,
@@ -2789,6 +3029,12 @@ static void set_avi_info_frame(
hdmi_info.bits.C0_C1 = COLORIMETRY_EXTENDED;
}
+ if (pixel_encoding && color_space == COLOR_SPACE_2020_YCBCR &&
+ stream->out_transfer_func->tf == TRANSFER_FUNCTION_GAMMA22) {
+ hdmi_info.bits.EC0_EC2 = 0;
+ hdmi_info.bits.C0_C1 = COLORIMETRY_ITU709;
+ }
+
/* TODO: un-hardcode aspect ratio */
aspect = stream->timing.aspect_ratio;
@@ -3734,4 +3980,4 @@ bool dc_resource_acquire_secondary_pipe_for_mpc_odm(
}
return true;
-}
\ No newline at end of file
+}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_stream.c b/drivers/gpu/drm/amd/display/dc/core/dc_stream.c
index 38d71b5c..20e534f 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_stream.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_stream.c
@@ -332,9 +332,21 @@ bool dc_stream_set_cursor_attributes(
dc = stream->ctx->dc;
- if (dc->debug.allow_sw_cursor_fallback && attributes->height * attributes->width * 4 > 16384)
- if (stream->mall_stream_config.type == SUBVP_MAIN)
+ /* SubVP is not compatible with HW cursor larger than 64 x 64 x 4.
+ * Therefore, if cursor is greater than 64 x 64 x 4, fallback to SW cursor in the following case:
+ * 1. For single display cases, if resolution is >= 5K and refresh rate < 120hz
+ * 2. For multi display cases, if resolution is >= 4K and refresh rate < 120hz
+ *
+ * [< 120hz is a requirement for SubVP configs]
+ */
+ if (dc->debug.allow_sw_cursor_fallback && attributes->height * attributes->width * 4 > 16384) {
+ if (dc->current_state->stream_count == 1 && stream->timing.v_addressable >= 2880 &&
+ ((stream->timing.pix_clk_100hz * 100) / stream->timing.v_total / stream->timing.h_total) < 120)
return false;
+ else if (dc->current_state->stream_count > 1 && stream->timing.v_addressable >= 2160 &&
+ ((stream->timing.pix_clk_100hz * 100) / stream->timing.v_total / stream->timing.h_total) < 120)
+ return false;
+ }
stream->cursor_attributes = *attributes;
diff --git a/drivers/gpu/drm/amd/display/dc/dc.h b/drivers/gpu/drm/amd/display/dc/dc.h
index 0598465..85ebeaa 100644
--- a/drivers/gpu/drm/amd/display/dc/dc.h
+++ b/drivers/gpu/drm/amd/display/dc/dc.h
@@ -47,7 +47,7 @@ struct aux_payload;
struct set_config_cmd_payload;
struct dmub_notification;
-#define DC_VER "3.2.207"
+#define DC_VER "3.2.215"
#define MAX_SURFACES 3
#define MAX_PLANES 6
@@ -56,9 +56,7 @@ struct dmub_notification;
#define MIN_VIEWPORT_SIZE 12
#define MAX_NUM_EDP 2
-/*******************************************************************************
- * Display Core Interfaces
- ******************************************************************************/
+/* Display Core Interfaces */
struct dc_versions {
const char *dc_ver;
struct dmcu_version dmcu_version;
@@ -263,11 +261,13 @@ struct dc_caps {
uint32_t cache_line_size;
uint32_t cache_num_ways;
uint16_t subvp_fw_processing_delay_us;
+ uint8_t subvp_drr_max_vblank_margin_us;
uint16_t subvp_prefetch_end_to_mall_start_us;
uint8_t subvp_swath_height_margin_lines; // subvp start line must be aligned to 2 x swath height
uint16_t subvp_pstate_allow_width_us;
uint16_t subvp_vertical_int_margin_us;
bool seamless_odm;
+ uint8_t subvp_drr_vblank_start_margin_us;
};
struct dc_bug_wa {
@@ -395,6 +395,7 @@ struct dc_config {
bool disable_dmcu;
bool enable_4to1MPC;
bool enable_windowed_mpo_odm;
+ bool forceHBR2CP2520; // Used for switching between test patterns TPS4 and CP2520
uint32_t allow_edp_hotplug_detection;
bool clamp_min_dcfclk;
uint64_t vblank_alignment_dto_params;
@@ -408,6 +409,7 @@ struct dc_config {
bool use_default_clock_table;
bool force_bios_enable_lttpr;
uint8_t force_bios_fixed_vs;
+ int sdpif_request_limit_words_per_umc;
};
@@ -457,15 +459,15 @@ enum pipe_split_policy {
MPC_SPLIT_DYNAMIC = 0,
/**
- * @MPC_SPLIT_DYNAMIC: Avoid pipe split, which means that DC will not
+ * @MPC_SPLIT_AVOID: Avoid pipe split, which means that DC will not
* try any sort of split optimization.
*/
MPC_SPLIT_AVOID = 1,
/**
- * @MPC_SPLIT_DYNAMIC: With this option, DC will only try to optimize
- * the pipe utilization when using a single display; if the user
- * connects to a second display, DC will avoid pipe split.
+ * @MPC_SPLIT_AVOID_MULT_DISP: With this option, DC will only try to
+ * optimize the pipe utilization when using a single display; if the
+ * user connects to a second display, DC will avoid pipe split.
*/
MPC_SPLIT_AVOID_MULT_DISP = 2,
};
@@ -491,12 +493,17 @@ enum dcn_pwr_state {
enum dcn_zstate_support_state {
DCN_ZSTATE_SUPPORT_UNKNOWN,
DCN_ZSTATE_SUPPORT_ALLOW,
+ DCN_ZSTATE_SUPPORT_ALLOW_Z8_ONLY,
+ DCN_ZSTATE_SUPPORT_ALLOW_Z8_Z10_ONLY,
DCN_ZSTATE_SUPPORT_ALLOW_Z10_ONLY,
DCN_ZSTATE_SUPPORT_DISALLOW,
};
-/*
- * For any clocks that may differ per pipe
- * only the max is stored in this structure
+
+/**
+ * struct dc_clocks - DC pipe clocks
+ *
+ * For any clocks that may differ per pipe only the max is stored in this
+ * structure
*/
struct dc_clocks {
int dispclk_khz;
@@ -523,6 +530,16 @@ struct dc_clocks {
bool prev_p_state_change_support;
bool fclk_prev_p_state_change_support;
int num_ways;
+
+ /*
+ * @fw_based_mclk_switching
+ *
+ * DC has a mechanism that leverage the variable refresh rate to switch
+ * memory clock in cases that we have a large latency to achieve the
+ * memory clock change and a short vblank window. DC has some
+ * requirements to enable this feature, and this field describes if the
+ * system support or not such a feature.
+ */
bool fw_based_mclk_switching;
bool fw_based_mclk_switching_shut_down;
int prev_num_ways;
@@ -764,7 +781,6 @@ struct dc_debug_options {
bool disable_mem_low_power;
bool pstate_enabled;
bool disable_dmcu;
- bool disable_psr;
bool force_abm_enable;
bool disable_stereo_support;
bool vsr_support;
@@ -828,6 +844,7 @@ struct dc_debug_options {
int crb_alloc_policy_min_disp_count;
bool disable_z10;
bool enable_z9_disable_interface;
+ bool psr_skip_crtc_disable;
union dpia_debug_options dpia_debug;
bool disable_fixed_vs_aux_timeout_wa;
bool force_disable_subvp;
@@ -836,6 +853,7 @@ struct dc_debug_options {
unsigned int force_subvp_num_ways;
unsigned int force_mall_ss_num_ways;
bool alloc_extra_way_for_cursor;
+ uint32_t subvp_extra_lines;
bool force_usr_allow;
/* uses value at boot and disables switch */
bool disable_dtb_ref_clk_switch;
@@ -990,9 +1008,7 @@ void dc_init_callbacks(struct dc *dc,
void dc_deinit_callbacks(struct dc *dc);
void dc_destroy(struct dc **dc);
-/*******************************************************************************
- * Surface Interfaces
- ******************************************************************************/
+/* Surface Interfaces */
enum {
TRANSFER_FUNC_POINTS = 1025
@@ -1271,12 +1287,23 @@ void dc_post_update_surfaces_to_stream(
#include "dc_stream.h"
-/*
- * Structure to store surface/stream associations for validation
+/**
+ * struct dc_validation_set - Struct to store surface/stream associations for validation
*/
struct dc_validation_set {
+ /**
+ * @stream: Stream state properties
+ */
struct dc_stream_state *stream;
+
+ /**
+ * @plane_state: Surface state
+ */
struct dc_plane_state *plane_states[MAX_SURFACES];
+
+ /**
+ * @plane_count: Total of active planes
+ */
uint8_t plane_count;
};
@@ -1288,6 +1315,12 @@ enum dc_status dc_validate_plane(struct dc *dc, const struct dc_plane_state *pla
void get_clock_requirements_for_state(struct dc_state *state, struct AsicStateEx *info);
+enum dc_status dc_validate_with_context(struct dc *dc,
+ const struct dc_validation_set set[],
+ int set_count,
+ struct dc_state *context,
+ bool fast_validate);
+
bool dc_set_generic_gpio_for_stereo(bool enable,
struct gpio_service *gpio_service);
@@ -1323,15 +1356,12 @@ void dc_resource_state_destruct(struct dc_state *context);
bool dc_resource_is_dsc_encoding_supported(const struct dc *dc);
-/*
- * TODO update to make it about validation sets
- * Set up streams and links associated to drive sinks
- * The streams parameter is an absolute set of all active streams.
- *
- * After this call:
- * Phy, Encoder, Timing Generator are programmed and enabled.
- * New streams are enabled with blank stream; no memory read.
- */
+enum dc_status dc_commit_streams(struct dc *dc,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count);
+
+/* TODO: When the transition to the new commit sequence is done, remove this
+ * function in favor of dc_commit_streams. */
bool dc_commit_state(struct dc *dc, struct dc_state *context);
struct dc_state *dc_create_state(struct dc *dc);
@@ -1339,9 +1369,7 @@ struct dc_state *dc_copy_state(struct dc_state *src_ctx);
void dc_retain_state(struct dc_state *context);
void dc_release_state(struct dc_state *context);
-/*******************************************************************************
- * Link Interfaces
- ******************************************************************************/
+/* Link Interfaces */
struct dpcd_caps {
union dpcd_rev dpcd_rev;
@@ -1443,9 +1471,7 @@ struct hdcp_caps {
uint32_t dc_get_opp_for_plane(struct dc *dc, struct dc_plane_state *plane);
-/*******************************************************************************
- * Sink Interfaces - A sink corresponds to a display output device
- ******************************************************************************/
+/* Sink Interfaces - A sink corresponds to a display output device */
struct dc_container_id {
// 128bit GUID in binary form
@@ -1528,9 +1554,7 @@ struct dc_cursor {
};
-/*******************************************************************************
- * Interrupt interfaces
- ******************************************************************************/
+/* Interrupt interfaces */
enum dc_irq_source dc_interrupt_to_irq_source(
struct dc *dc,
uint32_t src_id,
@@ -1542,9 +1566,7 @@ enum dc_irq_source dc_get_hpd_irq_source_at_index(
void dc_notify_vsync_int_state(struct dc *dc, struct dc_stream_state *stream, bool enable);
-/*******************************************************************************
- * Power Interfaces
- ******************************************************************************/
+/* Power Interfaces */
void dc_set_power_state(
struct dc *dc,
@@ -1617,14 +1639,10 @@ enum dc_status dc_process_dmub_set_mst_slots(const struct dc *dc,
void dc_process_dmub_dpia_hpd_int_enable(const struct dc *dc,
uint32_t hpd_int_enable);
-/*******************************************************************************
- * DSC Interfaces
- ******************************************************************************/
+/* DSC Interfaces */
#include "dc_dsc.h"
-/*******************************************************************************
- * Disable acc mode Interfaces
- ******************************************************************************/
+/* Disable acc mode Interfaces */
void dc_disable_accelerated_mode(struct dc *dc);
#endif /* DC_INTERFACE_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
index 0541e87..6ccf477 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
+++ b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
@@ -423,25 +423,20 @@ void dc_dmub_srv_get_visual_confirm_color_cmd(struct dc *dc, struct pipe_ctx *pi
#ifdef CONFIG_DRM_AMD_DC_DCN
/**
- * ***********************************************************************************************
- * populate_subvp_cmd_drr_info: Helper to populate DRR pipe info for the DMCUB subvp command
+ * populate_subvp_cmd_drr_info - Helper to populate DRR pipe info for the DMCUB subvp command
*
- * Populate the DMCUB SubVP command with DRR pipe info. All the information required for calculating
- * the SubVP + DRR microschedule is populated here.
+ * @dc: [in] current dc state
+ * @subvp_pipe: [in] pipe_ctx for the SubVP pipe
+ * @vblank_pipe: [in] pipe_ctx for the DRR pipe
+ * @pipe_data: [in] Pipe data which stores the VBLANK/DRR info
+ *
+ * Populate the DMCUB SubVP command with DRR pipe info. All the information
+ * required for calculating the SubVP + DRR microschedule is populated here.
*
* High level algorithm:
* 1. Get timing for SubVP pipe, phantom pipe, and DRR pipe
* 2. Calculate the min and max vtotal which supports SubVP + DRR microschedule
* 3. Populate the drr_info with the min and max supported vtotal values
- *
- * @param [in] dc: current dc state
- * @param [in] subvp_pipe: pipe_ctx for the SubVP pipe
- * @param [in] vblank_pipe: pipe_ctx for the DRR pipe
- * @param [in] pipe_data: Pipe data which stores the VBLANK/DRR info
- *
- * @return: void
- *
- * ***********************************************************************************************
*/
static void populate_subvp_cmd_drr_info(struct dc *dc,
struct pipe_ctx *subvp_pipe,
@@ -482,33 +477,38 @@ static void populate_subvp_cmd_drr_info(struct dc *dc,
(((uint64_t)main_timing->pix_clk_100hz * 100)));
drr_active_us = div64_u64(((uint64_t)drr_timing->v_addressable * drr_timing->h_total * 1000000),
(((uint64_t)drr_timing->pix_clk_100hz * 100)));
- max_drr_vblank_us = div64_u64((subvp_active_us - prefetch_us - drr_active_us), 2) + drr_active_us;
- max_drr_mallregion_us = subvp_active_us - prefetch_us - mall_region_us;
+ max_drr_vblank_us = div64_u64((subvp_active_us - prefetch_us -
+ dc->caps.subvp_fw_processing_delay_us - drr_active_us), 2) + drr_active_us;
+ max_drr_mallregion_us = subvp_active_us - prefetch_us - mall_region_us - dc->caps.subvp_fw_processing_delay_us;
max_drr_supported_us = max_drr_vblank_us > max_drr_mallregion_us ? max_drr_vblank_us : max_drr_mallregion_us;
max_vtotal_supported = div64_u64(((uint64_t)drr_timing->pix_clk_100hz * 100 * max_drr_supported_us),
(((uint64_t)drr_timing->h_total * 1000000)));
+ /* When calculating the max vtotal supported for SubVP + DRR cases, add
+ * margin due to possible rounding errors (being off by 1 line in the
+ * FW calculation can incorrectly push the P-State switch to wait 1 frame
+ * longer).
+ */
+ max_vtotal_supported = max_vtotal_supported - dc->caps.subvp_drr_max_vblank_margin_us;
+
pipe_data->pipe_config.vblank_data.drr_info.min_vtotal_supported = min_vtotal_supported;
pipe_data->pipe_config.vblank_data.drr_info.max_vtotal_supported = max_vtotal_supported;
+ pipe_data->pipe_config.vblank_data.drr_info.drr_vblank_start_margin = dc->caps.subvp_drr_vblank_start_margin_us;
}
/**
- * ***********************************************************************************************
- * populate_subvp_cmd_vblank_pipe_info: Helper to populate VBLANK pipe info for the DMUB subvp command
+ * populate_subvp_cmd_vblank_pipe_info - Helper to populate VBLANK pipe info for the DMUB subvp command
*
- * Populate the DMCUB SubVP command with VBLANK pipe info. All the information required to calculate
- * the microschedule for SubVP + VBLANK case is stored in the pipe_data (subvp_data and vblank_data).
- * Also check if the VBLANK pipe is a DRR display -- if it is make a call to populate drr_info.
+ * @dc: [in] current dc state
+ * @context: [in] new dc state
+ * @cmd: [in] DMUB cmd to be populated with SubVP info
+ * @vblank_pipe: [in] pipe_ctx for the VBLANK pipe
+ * @cmd_pipe_index: [in] index for the pipe array in DMCUB SubVP cmd
*
- * @param [in] dc: current dc state
- * @param [in] context: new dc state
- * @param [in] cmd: DMUB cmd to be populated with SubVP info
- * @param [in] vblank_pipe: pipe_ctx for the VBLANK pipe
- * @param [in] cmd_pipe_index: index for the pipe array in DMCUB SubVP cmd
- *
- * @return: void
- *
- * ***********************************************************************************************
+ * Populate the DMCUB SubVP command with VBLANK pipe info. All the information
+ * required to calculate the microschedule for SubVP + VBLANK case is stored in
+ * the pipe_data (subvp_data and vblank_data). Also check if the VBLANK pipe
+ * is a DRR display -- if it is make a call to populate drr_info.
*/
static void populate_subvp_cmd_vblank_pipe_info(struct dc *dc,
struct dc_state *context,
@@ -551,22 +551,18 @@ static void populate_subvp_cmd_vblank_pipe_info(struct dc *dc,
}
/**
- * ***********************************************************************************************
- * update_subvp_prefetch_end_to_mall_start: Helper for SubVP + SubVP case
+ * update_subvp_prefetch_end_to_mall_start - Helper for SubVP + SubVP case
*
- * For SubVP + SubVP, we use a single vertical interrupt to start the microschedule for both
- * SubVP pipes. In order for this to work correctly, the MALL REGION of both SubVP pipes must
- * start at the same time. This function lengthens the prefetch end to mall start delay of the
- * SubVP pipe that has the shorter prefetch so that both MALL REGION's will start at the same time.
+ * @dc: [in] current dc state
+ * @context: [in] new dc state
+ * @cmd: [in] DMUB cmd to be populated with SubVP info
+ * @subvp_pipes: [in] Array of SubVP pipes (should always be length 2)
*
- * @param [in] dc: current dc state
- * @param [in] context: new dc state
- * @param [in] cmd: DMUB cmd to be populated with SubVP info
- * @param [in] subvp_pipes: Array of SubVP pipes (should always be length 2)
- *
- * @return: void
- *
- * ***********************************************************************************************
+ * For SubVP + SubVP, we use a single vertical interrupt to start the
+ * microschedule for both SubVP pipes. In order for this to work correctly, the
+ * MALL REGION of both SubVP pipes must start at the same time. This function
+ * lengthens the prefetch end to mall start delay of the SubVP pipe that has
+ * the shorter prefetch so that both MALL REGION's will start at the same time.
*/
static void update_subvp_prefetch_end_to_mall_start(struct dc *dc,
struct dc_state *context,
@@ -608,22 +604,17 @@ static void update_subvp_prefetch_end_to_mall_start(struct dc *dc,
}
/**
- * ***************************************************************************************
- * setup_subvp_dmub_command: Helper to populate the SubVP pipe info for the DMUB subvp command
+ * populate_subvp_cmd_pipe_info - Helper to populate the SubVP pipe info for the DMUB subvp command
*
- * Populate the DMCUB SubVP command with SubVP pipe info. All the information required to
- * calculate the microschedule for the SubVP pipe is stored in the pipe_data of the DMCUB
- * SubVP command.
+ * @dc: [in] current dc state
+ * @context: [in] new dc state
+ * @cmd: [in] DMUB cmd to be populated with SubVP info
+ * @subvp_pipe: [in] pipe_ctx for the SubVP pipe
+ * @cmd_pipe_index: [in] index for the pipe array in DMCUB SubVP cmd
*
- * @param [in] dc: current dc state
- * @param [in] context: new dc state
- * @param [in] cmd: DMUB cmd to be populated with SubVP info
- * @param [in] subvp_pipe: pipe_ctx for the SubVP pipe
- * @param [in] cmd_pipe_index: index for the pipe array in DMCUB SubVP cmd
- *
- * @return: void
- *
- * ***************************************************************************************
+ * Populate the DMCUB SubVP command with SubVP pipe info. All the information
+ * required to calculate the microschedule for the SubVP pipe is stored in the
+ * pipe_data of the DMCUB SubVP command.
*/
static void populate_subvp_cmd_pipe_info(struct dc *dc,
struct dc_state *context,
@@ -703,19 +694,14 @@ static void populate_subvp_cmd_pipe_info(struct dc *dc,
}
/**
- * ***************************************************************************************
- * dc_dmub_setup_subvp_dmub_command: Populate the DMCUB SubVP command
+ * dc_dmub_setup_subvp_dmub_command - Populate the DMCUB SubVP command
*
- * This function loops through each pipe and populates the DMUB
- * SubVP CMD info based on the pipe (e.g. SubVP, VBLANK).
+ * @dc: [in] current dc state
+ * @context: [in] new dc state
+ * @cmd: [in] DMUB cmd to be populated with SubVP info
*
- * @param [in] dc: current dc state
- * @param [in] context: new dc state
- * @param [in] cmd: DMUB cmd to be populated with SubVP info
- *
- * @return: void
- *
- * ***************************************************************************************
+ * This function loops through each pipe and populates the DMUB SubVP CMD info
+ * based on the pipe (e.g. SubVP, VBLANK).
*/
void dc_dmub_setup_subvp_dmub_command(struct dc *dc,
struct dc_state *context,
@@ -882,11 +868,59 @@ void dc_dmub_srv_log_diagnostic_data(struct dc_dmub_srv *dc_dmub_srv)
diag_data.is_cw6_enabled);
}
+static bool dc_can_pipe_disable_cursor(struct pipe_ctx *pipe_ctx)
+{
+ struct pipe_ctx *test_pipe, *split_pipe;
+ const struct scaler_data *scl_data = &pipe_ctx->plane_res.scl_data;
+ struct rect r1 = scl_data->recout, r2, r2_half;
+ int r1_r = r1.x + r1.width, r1_b = r1.y + r1.height, r2_r, r2_b;
+ int cur_layer = pipe_ctx->plane_state->layer_index;
+
+ /**
+ * Disable the cursor if there's another pipe above this with a
+ * plane that contains this pipe's viewport to prevent double cursor
+ * and incorrect scaling artifacts.
+ */
+ for (test_pipe = pipe_ctx->top_pipe; test_pipe;
+ test_pipe = test_pipe->top_pipe) {
+ // Skip invisible layer and pipe-split plane on same layer
+ if (!test_pipe->plane_state->visible || test_pipe->plane_state->layer_index == cur_layer)
+ continue;
+
+ r2 = test_pipe->plane_res.scl_data.recout;
+ r2_r = r2.x + r2.width;
+ r2_b = r2.y + r2.height;
+ split_pipe = test_pipe;
+
+ /**
+ * There is another half plane on same layer because of
+ * pipe-split, merge together per same height.
+ */
+ for (split_pipe = pipe_ctx->top_pipe; split_pipe;
+ split_pipe = split_pipe->top_pipe)
+ if (split_pipe->plane_state->layer_index == test_pipe->plane_state->layer_index) {
+ r2_half = split_pipe->plane_res.scl_data.recout;
+ r2.x = (r2_half.x < r2.x) ? r2_half.x : r2.x;
+ r2.width = r2.width + r2_half.width;
+ r2_r = r2.x + r2.width;
+ break;
+ }
+
+ if (r1.x >= r2.x && r1.y >= r2.y && r1_r <= r2_r && r1_b <= r2_b)
+ return true;
+ }
+
+ return false;
+}
+
static bool dc_dmub_should_update_cursor_data(struct pipe_ctx *pipe_ctx)
{
if (pipe_ctx->plane_state != NULL) {
if (pipe_ctx->plane_state->address.type == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE)
return false;
+
+ if (dc_can_pipe_disable_cursor(pipe_ctx))
+ return false;
}
if ((pipe_ctx->stream->link->psr_settings.psr_version == DC_PSR_VERSION_SU_1 ||
@@ -962,19 +996,14 @@ static void dc_build_cursor_attribute_update_payload1(
}
/**
- * ***************************************************************************************
- * dc_send_update_cursor_info_to_dmu: Populate the DMCUB Cursor update info command
+ * dc_send_update_cursor_info_to_dmu - Populate the DMCUB Cursor update info command
*
- * This function would store the cursor related information and pass it into dmub
+ * @pCtx: [in] pipe context
+ * @pipe_idx: [in] pipe index
*
- * @param [in] pCtx: pipe context
- * @param [in] pipe_idx: pipe index
- *
- * @return: void
- *
- * ***************************************************************************************
+ * This function would store the cursor related information and pass it into
+ * dmub
*/
-
void dc_send_update_cursor_info_to_dmu(
struct pipe_ctx *pCtx, uint8_t pipe_idx)
{
diff --git a/drivers/gpu/drm/amd/display/dc/dc_link.h b/drivers/gpu/drm/amd/display/dc/dc_link.h
index caf0c7a..2e18bcf 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_link.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_link.h
@@ -117,7 +117,7 @@ struct psr_settings {
* Add a struct dc_panel_config under dc_link
*/
struct dc_panel_config {
- // extra panel power sequence parameters
+ /* extra panel power sequence parameters */
struct pps {
unsigned int extra_t3_ms;
unsigned int extra_t7_ms;
@@ -127,13 +127,21 @@ struct dc_panel_config {
unsigned int extra_t12_ms;
unsigned int extra_post_OUI_ms;
} pps;
- // ABM
+ /* PSR */
+ struct psr {
+ bool disable_psr;
+ bool disallow_psrsu;
+ bool rc_disable;
+ bool rc_allow_static_screen;
+ bool rc_allow_fullscreen_VPB;
+ } psr;
+ /* ABM */
struct varib {
unsigned int varibright_feature_enable;
unsigned int def_varibright_level;
unsigned int abm_config_setting;
} varib;
- // edp DSC
+ /* edp DSC */
struct dsc {
bool disable_dsc_edp;
unsigned int force_dsc_edp_policy;
@@ -143,6 +151,20 @@ struct dc_panel_config {
bool optimize_edp_link_rate; /* eDP ILR */
} ilr;
};
+
+/*
+ * USB4 DPIA BW ALLOCATION STRUCTS
+ */
+struct dc_dpia_bw_alloc {
+ int sink_verified_bw; // The Verified BW that sink can allocated and use that has been verified already
+ int sink_allocated_bw; // The Actual Allocated BW that sink currently allocated
+ int padding_bw; // The Padding "Un-used" BW allocated by CM for padding reasons
+ int sink_max_bw; // The Max BW that sink can require/support
+ int estimated_bw; // The estimated available BW for this DPIA
+ int bw_granularity; // BW Granularity
+ bool bw_alloc_enabled; // The BW Alloc Mode Support is turned ON for all 3: DP-Tx & Dpia & CM
+};
+
/*
* A link contains one or more sinks and their connected status.
* The currently active signal type (HDMI, DP-SST, DP-MST) is also reported.
@@ -158,6 +180,14 @@ struct dc_link {
enum dc_irq_source irq_source_hpd_rx;/* aka DP Short Pulse */
bool is_hpd_filter_disabled;
bool dp_ss_off;
+
+ /**
+ * @link_state_valid:
+ *
+ * If there is no link and local sink, this variable should be set to
+ * false. Otherwise, it should be set to true; usually, the function
+ * core_link_enable_stream sets this field to true.
+ */
bool link_state_valid;
bool aux_access_disabled;
bool sync_lt_in_progress;
@@ -168,6 +198,7 @@ struct dc_link {
bool is_dig_mapping_flexible;
bool hpd_status; /* HPD status of link without physical HPD pin. */
bool is_hpd_pending; /* Indicates a new received hpd */
+ bool is_automated; /* Indicates automated testing */
bool edp_sink_present;
diff --git a/drivers/gpu/drm/amd/display/dc/dc_stream.h b/drivers/gpu/drm/amd/display/dc/dc_stream.h
index 9e6025c..dfd3df1 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_stream.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_stream.h
@@ -41,6 +41,10 @@ struct timing_sync_info {
struct dc_stream_status {
int primary_otg_inst;
int stream_enc_inst;
+
+ /**
+ * @plane_count: Total of planes attached to a single stream
+ */
int plane_count;
int audio_inst;
struct timing_sync_info timing_sync_info;
@@ -140,7 +144,7 @@ struct test_pattern {
unsigned int cust_pattern_size;
};
-#define SUBVP_DRR_MARGIN_US 500 // 500us for DRR margin (SubVP + DRR)
+#define SUBVP_DRR_MARGIN_US 600 // 600us for DRR margin (SubVP + DRR)
enum mall_stream_type {
SUBVP_NONE, // subvp not in use
@@ -156,6 +160,17 @@ struct mall_stream_config {
struct dc_stream_state *paired_stream; // master / slave stream
};
+/* Temp struct used to save and restore MALL config
+ * during validation.
+ *
+ * TODO: Move MALL config into dc_state instead of stream struct
+ * to avoid needing to save/restore.
+ */
+struct mall_temp_config {
+ struct mall_stream_config mall_stream_config[MAX_PIPES];
+ bool is_phantom_plane[MAX_PIPES];
+};
+
struct dc_stream_state {
// sink is deprecated, new code should not reference
// this pointer
@@ -197,7 +212,18 @@ struct dc_stream_state {
bool use_vsc_sdp_for_colorimetry;
bool ignore_msa_timing_param;
+ /**
+ * @allow_freesync:
+ *
+ * It say if Freesync is enabled or not.
+ */
bool allow_freesync;
+
+ /**
+ * @vrr_active_variable:
+ *
+ * It describes if VRR is in use.
+ */
bool vrr_active_variable;
bool freesync_on_desktop;
@@ -517,10 +543,10 @@ bool dc_stream_get_crtc_position(struct dc *dc,
unsigned int *nom_v_pos);
#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
-bool dc_stream_forward_dmcu_crc_window(struct dc *dc, struct dc_stream_state *stream,
- struct crc_params *crc_window);
-bool dc_stream_stop_dmcu_crc_win_update(struct dc *dc,
- struct dc_stream_state *stream);
+bool dc_stream_forward_crc_window(struct dc *dc,
+ struct rect *rect,
+ struct dc_stream_state *stream,
+ bool is_stop);
#endif
bool dc_stream_configure_crc(struct dc *dc,
diff --git a/drivers/gpu/drm/amd/display/dc/dc_trace.h b/drivers/gpu/drm/amd/display/dc/dc_trace.h
index c711797..bbec308 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_trace.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_trace.h
@@ -40,3 +40,5 @@
#define TRACE_DCN_FPU(begin, function, line, ref_count) \
trace_dcn_fpu(begin, function, line, ref_count)
+#define TRACE_OPTC_LOCK_UNLOCK_STATE(optc, inst, lock) \
+ trace_dcn_optc_lock_unlock_state(optc, inst, lock, __func__, __LINE__)
diff --git a/drivers/gpu/drm/amd/display/dc/dc_types.h b/drivers/gpu/drm/amd/display/dc/dc_types.h
index ad90414..dc78e24 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_types.h
@@ -993,4 +993,11 @@ struct display_endpoint_id {
enum display_endpoint_type ep_type;
};
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+struct otg_phy_mux {
+ uint8_t phy_output_num;
+ uint8_t otg_output_num;
+};
+#endif
+
#endif /* DC_TYPES_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c b/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c
index fbb19e2..d3cc5ec 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c
@@ -927,19 +927,20 @@ static bool dcn10_recv_edid_cea_ack(struct dmcu *dmcu, int *offset)
#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
static void dcn10_forward_crc_window(struct dmcu *dmcu,
- struct crc_region *crc_win,
+ struct rect *rect,
struct otg_phy_mux *mux_mapping)
{
struct dce_dmcu *dmcu_dce = TO_DCE_DMCU(dmcu);
unsigned int dmcu_max_retry_on_wait_reg_ready = 801;
unsigned int dmcu_wait_reg_ready_interval = 100;
unsigned int crc_start = 0, crc_end = 0, otg_phy_mux = 0;
+ int x_start, y_start, x_end, y_end;
/* If microcontroller is not running, do nothing */
if (dmcu->dmcu_state != DMCU_RUNNING)
return;
- if (!crc_win)
+ if (!rect)
return;
/* waitDMCUReadyForCmd */
@@ -947,9 +948,14 @@ static void dcn10_forward_crc_window(struct dmcu *dmcu,
dmcu_wait_reg_ready_interval,
dmcu_max_retry_on_wait_reg_ready);
+ x_start = rect->x;
+ y_start = rect->y;
+ x_end = x_start + rect->width;
+ y_end = y_start + rect->height;
+
/* build up nitification data */
- crc_start = (((unsigned int) crc_win->x_start) << 16) | crc_win->y_start;
- crc_end = (((unsigned int) crc_win->x_end) << 16) | crc_win->y_end;
+ crc_start = (((unsigned int) x_start) << 16) | y_start;
+ crc_end = (((unsigned int) x_end) << 16) | y_end;
otg_phy_mux =
(((unsigned int) mux_mapping->otg_output_num) << 16) | mux_mapping->phy_output_num;
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
index bec5e9f..2d3201b 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
@@ -399,7 +399,11 @@ static bool dmub_psr_copy_settings(struct dmub_psr *dmub,
link->psr_settings.force_ffu_mode = 0;
copy_settings_data->force_ffu_mode = link->psr_settings.force_ffu_mode;
- if (link->fec_state == dc_link_fec_enabled &&
+ if (((link->dpcd_caps.fec_cap.bits.FEC_CAPABLE &&
+ !link->dc->debug.disable_fec) &&
+ (link->dpcd_caps.dsc_caps.dsc_basic_caps.fields.dsc_support.DSC_SUPPORT &&
+ !link->panel_config.dsc.disable_dsc_edp &&
+ link->dc->caps.edp_dsc_support)) &&
link->dpcd_caps.sink_dev_id == DP_DEVICE_ID_38EC11 &&
(!memcmp(link->dpcd_caps.sink_dev_id_str, DP_SINK_DEVICE_STR_ID_1,
sizeof(DP_SINK_DEVICE_STR_ID_1)) ||
@@ -409,6 +413,11 @@ static bool dmub_psr_copy_settings(struct dmub_psr *dmub,
else
copy_settings_data->debug.bitfields.force_wakeup_by_tps3 = 0;
+ //WA for PSR1 on specific TCON, require frame delay for frame re-lock
+ copy_settings_data->relock_delay_frame_cnt = 0;
+ if (link->dpcd_caps.sink_dev_id == DP_BRANCH_DEVICE_ID_001CF8)
+ copy_settings_data->relock_delay_frame_cnt = 2;
+
dc_dmub_srv_cmd_queue(dc->dmub_srv, &cmd);
dc_dmub_srv_cmd_execute(dc->dmub_srv);
dc_dmub_srv_wait_idle(dc->dmub_srv);
diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
index d260eaa..913a1fe 100644
--- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
@@ -688,16 +688,6 @@ void dce110_enable_stream(struct pipe_ctx *pipe_ctx)
early_control = lane_count;
tg->funcs->set_early_control(tg, early_control);
-
- /* enable audio only within mode set */
- if (pipe_ctx->stream_res.audio != NULL) {
- if (dc_is_dp_signal(pipe_ctx->stream->signal))
- pipe_ctx->stream_res.stream_enc->funcs->dp_audio_enable(pipe_ctx->stream_res.stream_enc);
- }
-
-
-
-
}
static enum bp_result link_transmitter_control(
@@ -1081,12 +1071,14 @@ void dce110_enable_audio_stream(struct pipe_ctx *pipe_ctx)
struct dc *dc;
struct clk_mgr *clk_mgr;
unsigned int i, num_audio = 1;
+ const struct link_hwss *link_hwss;
if (!pipe_ctx->stream)
return;
dc = pipe_ctx->stream->ctx->dc;
clk_mgr = dc->clk_mgr;
+ link_hwss = get_link_hwss(pipe_ctx->stream->link, &pipe_ctx->link_res);
if (pipe_ctx->stream_res.audio && pipe_ctx->stream_res.audio->enabled == true)
return;
@@ -1103,56 +1095,35 @@ void dce110_enable_audio_stream(struct pipe_ctx *pipe_ctx)
if (num_audio >= 1 && clk_mgr->funcs->enable_pme_wa)
/*this is the first audio. apply the PME w/a in order to wake AZ from D3*/
clk_mgr->funcs->enable_pme_wa(clk_mgr);
- /* un-mute audio */
- /* TODO: audio should be per stream rather than per link */
- if (is_dp_128b_132b_signal(pipe_ctx))
- pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->audio_mute_control(
- pipe_ctx->stream_res.hpo_dp_stream_enc, false);
- else
- pipe_ctx->stream_res.stream_enc->funcs->audio_mute_control(
- pipe_ctx->stream_res.stream_enc, false);
+
+ link_hwss->enable_audio_packet(pipe_ctx);
+
if (pipe_ctx->stream_res.audio)
pipe_ctx->stream_res.audio->enabled = true;
}
-
- if (dc_is_dp_signal(pipe_ctx->stream->signal))
- dp_source_sequence_trace(pipe_ctx->stream->link, DPCD_SOURCE_SEQ_AFTER_ENABLE_AUDIO_STREAM);
}
void dce110_disable_audio_stream(struct pipe_ctx *pipe_ctx)
{
struct dc *dc;
struct clk_mgr *clk_mgr;
+ const struct link_hwss *link_hwss;
if (!pipe_ctx || !pipe_ctx->stream)
return;
dc = pipe_ctx->stream->ctx->dc;
clk_mgr = dc->clk_mgr;
+ link_hwss = get_link_hwss(pipe_ctx->stream->link, &pipe_ctx->link_res);
if (pipe_ctx->stream_res.audio && pipe_ctx->stream_res.audio->enabled == false)
return;
- if (is_dp_128b_132b_signal(pipe_ctx))
- pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->audio_mute_control(
- pipe_ctx->stream_res.hpo_dp_stream_enc, true);
- else
- pipe_ctx->stream_res.stream_enc->funcs->audio_mute_control(
- pipe_ctx->stream_res.stream_enc, true);
+ link_hwss->disable_audio_packet(pipe_ctx);
+
if (pipe_ctx->stream_res.audio) {
pipe_ctx->stream_res.audio->enabled = false;
- if (dc_is_dp_signal(pipe_ctx->stream->signal))
- if (is_dp_128b_132b_signal(pipe_ctx))
- pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_audio_disable(
- pipe_ctx->stream_res.hpo_dp_stream_enc);
- else
- pipe_ctx->stream_res.stream_enc->funcs->dp_audio_disable(
- pipe_ctx->stream_res.stream_enc);
- else
- pipe_ctx->stream_res.stream_enc->funcs->hdmi_audio_disable(
- pipe_ctx->stream_res.stream_enc);
-
if (clk_mgr->funcs->enable_pme_wa)
/*this is the first audio. apply the PME w/a in order to wake AZ from D3*/
clk_mgr->funcs->enable_pme_wa(clk_mgr);
@@ -1163,9 +1134,6 @@ void dce110_disable_audio_stream(struct pipe_ctx *pipe_ctx)
* stream->stream_engine_id);
*/
}
-
- if (dc_is_dp_signal(pipe_ctx->stream->signal))
- dp_source_sequence_trace(pipe_ctx->stream->link, DPCD_SOURCE_SEQ_AFTER_DISABLE_AUDIO_STREAM);
}
void dce110_disable_stream(struct pipe_ctx *pipe_ctx)
@@ -1487,6 +1455,9 @@ static enum dc_status apply_single_controller_ctx_to_hw(
unsigned int event_triggers = 0;
struct pipe_ctx *odm_pipe = pipe_ctx->next_odm_pipe;
struct dce_hwseq *hws = dc->hwseq;
+ const struct link_hwss *link_hwss = get_link_hwss(
+ link, &pipe_ctx->link_res);
+
if (hws->funcs.disable_stream_gating) {
hws->funcs.disable_stream_gating(dc, pipe_ctx);
@@ -1497,23 +1468,8 @@ static enum dc_status apply_single_controller_ctx_to_hw(
build_audio_output(context, pipe_ctx, &audio_output);
- if (dc_is_dp_signal(pipe_ctx->stream->signal))
- if (is_dp_128b_132b_signal(pipe_ctx))
- pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_audio_setup(
- pipe_ctx->stream_res.hpo_dp_stream_enc,
- pipe_ctx->stream_res.audio->inst,
- &pipe_ctx->stream->audio_info);
- else
- pipe_ctx->stream_res.stream_enc->funcs->dp_audio_setup(
- pipe_ctx->stream_res.stream_enc,
- pipe_ctx->stream_res.audio->inst,
- &pipe_ctx->stream->audio_info);
- else
- pipe_ctx->stream_res.stream_enc->funcs->hdmi_audio_setup(
- pipe_ctx->stream_res.stream_enc,
- pipe_ctx->stream_res.audio->inst,
- &pipe_ctx->stream->audio_info,
- &audio_output.crtc_info);
+ link_hwss->setup_audio_output(pipe_ctx, &audio_output,
+ pipe_ctx->stream_res.audio->inst);
pipe_ctx->stream_res.audio->funcs->az_configure(
pipe_ctx->stream_res.audio,
@@ -1605,8 +1561,13 @@ static enum dc_status apply_single_controller_ctx_to_hw(
pipe_ctx->plane_res.scl_data.lb_params.alpha_en = pipe_ctx->bottom_pipe != NULL;
- pipe_ctx->stream->link->psr_settings.psr_feature_enabled = false;
-
+ /* Phantom and main stream share the same link (because the stream
+ * is constructed with the same sink). Make sure not to override
+ * and link programming on the main.
+ */
+ if (pipe_ctx->stream->mall_stream_config.type != SUBVP_PHANTOM) {
+ pipe_ctx->stream->link->psr_settings.psr_feature_enabled = false;
+ }
return DC_OK;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dce60/dce60_resource.c b/drivers/gpu/drm/amd/display/dc/dce60/dce60_resource.c
index fc6aa09..8db9f75 100644
--- a/drivers/gpu/drm/amd/display/dc/dce60/dce60_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dce60/dce60_resource.c
@@ -1128,6 +1128,7 @@ struct resource_pool *dce60_create_resource_pool(
if (dce60_construct(num_virtual_links, dc, pool))
return &pool->base;
+ kfree(pool);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1325,6 +1326,7 @@ struct resource_pool *dce61_create_resource_pool(
if (dce61_construct(num_virtual_links, dc, pool))
return &pool->base;
+ kfree(pool);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1518,6 +1520,7 @@ struct resource_pool *dce64_create_resource_pool(
if (dce64_construct(num_virtual_links, dc, pool))
return &pool->base;
+ kfree(pool);
BREAK_TO_DEBUGGER();
return NULL;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c b/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c
index b280259..5825e6f 100644
--- a/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c
@@ -1137,6 +1137,7 @@ struct resource_pool *dce80_create_resource_pool(
if (dce80_construct(num_virtual_links, dc, pool))
return &pool->base;
+ kfree(pool);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1336,6 +1337,7 @@ struct resource_pool *dce81_create_resource_pool(
if (dce81_construct(num_virtual_links, dc, pool))
return &pool->base;
+ kfree(pool);
BREAK_TO_DEBUGGER();
return NULL;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h
index e48fd044..ba1c062 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h
@@ -171,6 +171,7 @@ struct dcn_hubbub_registers {
uint32_t DCHUBBUB_ARB_FCLK_PSTATE_CHANGE_WATERMARK_B;
uint32_t DCHUBBUB_ARB_FCLK_PSTATE_CHANGE_WATERMARK_C;
uint32_t DCHUBBUB_ARB_FCLK_PSTATE_CHANGE_WATERMARK_D;
+ uint32_t SDPIF_REQUEST_RATE_LIMIT;
};
#define HUBBUB_REG_FIELD_LIST_DCN32(type) \
@@ -360,7 +361,8 @@ struct dcn_hubbub_registers {
type DCHUBBUB_ARB_ALLOW_SR_ENTER_WATERMARK_Z8_C;\
type DCHUBBUB_ARB_ALLOW_SR_EXIT_WATERMARK_Z8_C;\
type DCHUBBUB_ARB_ALLOW_SR_ENTER_WATERMARK_Z8_D;\
- type DCHUBBUB_ARB_ALLOW_SR_EXIT_WATERMARK_Z8_D
+ type DCHUBBUB_ARB_ALLOW_SR_EXIT_WATERMARK_Z8_D;\
+ type SDPIF_REQUEST_RATE_LIMIT
struct dcn_hubbub_shift {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
index 11e4c4e..c8ec118 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
@@ -97,10 +97,12 @@ void dcn10_lock_all_pipes(struct dc *dc,
bool lock)
{
struct pipe_ctx *pipe_ctx;
+ struct pipe_ctx *old_pipe_ctx;
struct timing_generator *tg;
int i;
for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ old_pipe_ctx = &dc->current_state->res_ctx.pipe_ctx[i];
pipe_ctx = &context->res_ctx.pipe_ctx[i];
tg = pipe_ctx->stream_res.tg;
@@ -110,7 +112,7 @@ void dcn10_lock_all_pipes(struct dc *dc,
*/
if (pipe_ctx->top_pipe ||
!pipe_ctx->stream ||
- !pipe_ctx->plane_state ||
+ (!pipe_ctx->plane_state && !old_pipe_ctx->plane_state) ||
!tg->funcs->is_tg_enabled(tg))
continue;
@@ -867,6 +869,32 @@ static void false_optc_underflow_wa(
tg->funcs->clear_optc_underflow(tg);
}
+static int calculate_vready_offset_for_group(struct pipe_ctx *pipe)
+{
+ struct pipe_ctx *other_pipe;
+ int vready_offset = pipe->pipe_dlg_param.vready_offset;
+
+ /* Always use the largest vready_offset of all connected pipes */
+ for (other_pipe = pipe->bottom_pipe; other_pipe != NULL; other_pipe = other_pipe->bottom_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+ for (other_pipe = pipe->top_pipe; other_pipe != NULL; other_pipe = other_pipe->top_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+ for (other_pipe = pipe->next_odm_pipe; other_pipe != NULL; other_pipe = other_pipe->next_odm_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+ for (other_pipe = pipe->prev_odm_pipe; other_pipe != NULL; other_pipe = other_pipe->prev_odm_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+
+ return vready_offset;
+}
+
enum dc_status dcn10_enable_stream_timing(
struct pipe_ctx *pipe_ctx,
struct dc_state *context,
@@ -910,7 +938,7 @@ enum dc_status dcn10_enable_stream_timing(
pipe_ctx->stream_res.tg->funcs->program_timing(
pipe_ctx->stream_res.tg,
&stream->timing,
- pipe_ctx->pipe_dlg_param.vready_offset,
+ calculate_vready_offset_for_group(pipe_ctx),
pipe_ctx->pipe_dlg_param.vstartup_start,
pipe_ctx->pipe_dlg_param.vupdate_offset,
pipe_ctx->pipe_dlg_param.vupdate_width,
@@ -2188,6 +2216,12 @@ void dcn10_enable_vblanks_synchronization(
opp = grouped_pipes[i]->stream_res.opp;
tg = grouped_pipes[i]->stream_res.tg;
tg->funcs->get_otg_active_size(tg, &width, &height);
+
+ if (!tg->funcs->is_tg_enabled(tg)) {
+ DC_SYNC_INFO("Skipping timing sync on disabled OTG\n");
+ return;
+ }
+
if (opp->funcs->opp_program_dpg_dimensions)
opp->funcs->opp_program_dpg_dimensions(opp, width, 2*(height) + 1);
}
@@ -2250,6 +2284,12 @@ void dcn10_enable_timing_synchronization(
opp = grouped_pipes[i]->stream_res.opp;
tg = grouped_pipes[i]->stream_res.tg;
tg->funcs->get_otg_active_size(tg, &width, &height);
+
+ if (!tg->funcs->is_tg_enabled(tg)) {
+ DC_SYNC_INFO("Skipping timing sync on disabled OTG\n");
+ return;
+ }
+
if (opp->funcs->opp_program_dpg_dimensions)
opp->funcs->opp_program_dpg_dimensions(opp, width, 2*(height) + 1);
}
@@ -2900,7 +2940,7 @@ void dcn10_program_pipe(
pipe_ctx->stream_res.tg->funcs->program_global_sync(
pipe_ctx->stream_res.tg,
- pipe_ctx->pipe_dlg_param.vready_offset,
+ calculate_vready_offset_for_group(pipe_ctx),
pipe_ctx->pipe_dlg_param.vstartup_start,
pipe_ctx->pipe_dlg_param.vupdate_offset,
pipe_ctx->pipe_dlg_param.vupdate_width);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
index 33d7802..c9e53dc 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
@@ -27,6 +27,7 @@
#include "reg_helper.h"
#include "dcn10_optc.h"
#include "dc.h"
+#include "dc_trace.h"
#define REG(reg)\
optc1->tg_regs->reg
@@ -657,6 +658,8 @@ void optc1_lock(struct timing_generator *optc)
REG_WAIT(OTG_MASTER_UPDATE_LOCK,
UPDATE_LOCK_STATUS, 1,
1, 10);
+
+ TRACE_OPTC_LOCK_UNLOCK_STATE(optc1, optc->inst, true);
}
void optc1_unlock(struct timing_generator *optc)
@@ -665,6 +668,8 @@ void optc1_unlock(struct timing_generator *optc)
REG_SET(OTG_MASTER_UPDATE_LOCK, 0,
OTG_MASTER_UPDATE_LOCK, 0);
+
+ TRACE_OPTC_LOCK_UNLOCK_STATE(optc1, optc->inst, false);
}
void optc1_get_position(struct timing_generator *optc,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
index 56d30ba..6bfac80 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
@@ -1295,47 +1295,6 @@ static uint32_t read_pipe_fuses(struct dc_context *ctx)
return value;
}
-/*
- * Some architectures don't support soft-float (e.g. aarch64), on those
- * this function has to be called with hardfloat enabled, make sure not
- * to inline it so whatever fp stuff is done stays inside
- */
-static noinline void dcn10_resource_construct_fp(
- struct dc *dc)
-{
- if (dc->ctx->dce_version == DCN_VERSION_1_01) {
- struct dcn_soc_bounding_box *dcn_soc = dc->dcn_soc;
- struct dcn_ip_params *dcn_ip = dc->dcn_ip;
- struct display_mode_lib *dml = &dc->dml;
-
- dml->ip.max_num_dpp = 3;
- /* TODO how to handle 23.84? */
- dcn_soc->dram_clock_change_latency = 23;
- dcn_ip->max_num_dpp = 3;
- }
- if (ASICREV_IS_RV1_F0(dc->ctx->asic_id.hw_internal_rev)) {
- dc->dcn_soc->urgent_latency = 3;
- dc->debug.disable_dmcu = true;
- dc->dcn_soc->fabric_and_dram_bandwidth_vmax0p9 = 41.60f;
- }
-
-
- dc->dcn_soc->number_of_channels = dc->ctx->asic_id.vram_width / ddr4_dram_width;
- ASSERT(dc->dcn_soc->number_of_channels < 3);
- if (dc->dcn_soc->number_of_channels == 0)/*old sbios bug*/
- dc->dcn_soc->number_of_channels = 2;
-
- if (dc->dcn_soc->number_of_channels == 1) {
- dc->dcn_soc->fabric_and_dram_bandwidth_vmax0p9 = 19.2f;
- dc->dcn_soc->fabric_and_dram_bandwidth_vnom0p8 = 17.066f;
- dc->dcn_soc->fabric_and_dram_bandwidth_vmid0p72 = 14.933f;
- dc->dcn_soc->fabric_and_dram_bandwidth_vmin0p65 = 12.8f;
- if (ASICREV_IS_RV1_F0(dc->ctx->asic_id.hw_internal_rev)) {
- dc->dcn_soc->fabric_and_dram_bandwidth_vmax0p9 = 20.80f;
- }
- }
-}
-
static bool verify_clock_values(struct dm_pp_clock_levels_with_voltage *clks)
{
int i;
@@ -1510,8 +1469,9 @@ static bool dcn10_resource_construct(
memcpy(dc->dcn_ip, &dcn10_ip_defaults, sizeof(dcn10_ip_defaults));
memcpy(dc->dcn_soc, &dcn10_soc_defaults, sizeof(dcn10_soc_defaults));
- /* Other architectures we build for build this with soft-float */
+ DC_FP_START();
dcn10_resource_construct_fp(dc);
+ DC_FP_END();
if (!dc->config.is_vmin_only_asic)
if (ASICREV_IS_RAVEN2(dc->ctx->asic_id.hw_internal_rev))
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb.h b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb.h
index a85ed22..a9dd9ae 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dwb.h
@@ -27,204 +27,177 @@
#define TO_DCN20_DWBC(dwbc_base) \
container_of(dwbc_base, struct dcn20_dwbc, base)
-/* DCN */
-#define BASE_INNER(seg) \
- DCE_BASE__INST0_SEG ## seg
-
-#define BASE(seg) \
- BASE_INNER(seg)
-
-#define SR(reg_name)\
- .reg_name = BASE(mm ## reg_name ## _BASE_IDX) + \
- mm ## reg_name
-
-#define SRI(reg_name, block, id)\
- .reg_name = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
- mm ## block ## id ## _ ## reg_name
-
-#define SRI2(reg_name, block, id)\
- .reg_name = BASE(mm ## reg_name ## _BASE_IDX) + \
- mm ## reg_name
-
-#define SRII(reg_name, block, id)\
- .reg_name[id] = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
- mm ## block ## id ## _ ## reg_name
-
-#define SF(reg_name, field_name, post_fix)\
- .field_name = reg_name ## __ ## field_name ## post_fix
-
-
#define DWBC_COMMON_REG_LIST_DCN2_0(inst) \
- SRI2(WB_ENABLE, CNV, inst),\
- SRI2(WB_EC_CONFIG, CNV, inst),\
- SRI2(CNV_MODE, CNV, inst),\
- SRI2(CNV_WINDOW_START, CNV, inst),\
- SRI2(CNV_WINDOW_SIZE, CNV, inst),\
- SRI2(CNV_UPDATE, CNV, inst),\
- SRI2(CNV_SOURCE_SIZE, CNV, inst),\
- SRI2(CNV_TEST_CNTL, CNV, inst),\
- SRI2(CNV_TEST_CRC_RED, CNV, inst),\
- SRI2(CNV_TEST_CRC_GREEN, CNV, inst),\
- SRI2(CNV_TEST_CRC_BLUE, CNV, inst),\
- SRI2(WBSCL_COEF_RAM_SELECT, WBSCL, inst),\
- SRI2(WBSCL_COEF_RAM_TAP_DATA, WBSCL, inst),\
- SRI2(WBSCL_MODE, WBSCL, inst),\
- SRI2(WBSCL_TAP_CONTROL, WBSCL, inst),\
- SRI2(WBSCL_DEST_SIZE, WBSCL, inst),\
- SRI2(WBSCL_HORZ_FILTER_SCALE_RATIO, WBSCL, inst),\
- SRI2(WBSCL_HORZ_FILTER_INIT_Y_RGB, WBSCL, inst),\
- SRI2(WBSCL_HORZ_FILTER_INIT_CBCR, WBSCL, inst),\
- SRI2(WBSCL_VERT_FILTER_SCALE_RATIO, WBSCL, inst),\
- SRI2(WBSCL_VERT_FILTER_INIT_Y_RGB, WBSCL, inst),\
- SRI2(WBSCL_VERT_FILTER_INIT_CBCR, WBSCL, inst),\
- SRI2(WBSCL_ROUND_OFFSET, WBSCL, inst),\
- SRI2(WBSCL_OVERFLOW_STATUS, WBSCL, inst),\
- SRI2(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL, inst),\
- SRI2(WBSCL_TEST_CNTL, WBSCL, inst),\
- SRI2(WBSCL_TEST_CRC_RED, WBSCL, inst),\
- SRI2(WBSCL_TEST_CRC_GREEN, WBSCL, inst),\
- SRI2(WBSCL_TEST_CRC_BLUE, WBSCL, inst),\
- SRI2(WBSCL_BACKPRESSURE_CNT_EN, WBSCL, inst),\
- SRI2(WB_MCIF_BACKPRESSURE_CNT, WBSCL, inst),\
- SRI2(WBSCL_CLAMP_Y_RGB, WBSCL, inst),\
- SRI2(WBSCL_CLAMP_CBCR, WBSCL, inst),\
- SRI2(WBSCL_OUTSIDE_PIX_STRATEGY, WBSCL, inst),\
- SRI2(WBSCL_OUTSIDE_PIX_STRATEGY_CBCR, WBSCL, inst),\
- SRI2(WBSCL_DEBUG, WBSCL, inst),\
- SRI2(WBSCL_TEST_DEBUG_INDEX, WBSCL, inst),\
- SRI2(WBSCL_TEST_DEBUG_DATA, WBSCL, inst),\
- SRI2(WB_DEBUG_CTRL, CNV, inst),\
- SRI2(WB_DBG_MODE, CNV, inst),\
- SRI2(WB_HW_DEBUG, CNV, inst),\
- SRI2(CNV_TEST_DEBUG_INDEX, CNV, inst),\
- SRI2(CNV_TEST_DEBUG_DATA, CNV, inst),\
- SRI2(WB_SOFT_RESET, CNV, inst),\
- SRI2(WB_WARM_UP_MODE_CTL1, CNV, inst),\
- SRI2(WB_WARM_UP_MODE_CTL2, CNV, inst)
+ SRI2_DWB(WB_ENABLE, CNV, inst),\
+ SRI2_DWB(WB_EC_CONFIG, CNV, inst),\
+ SRI2_DWB(CNV_MODE, CNV, inst),\
+ SRI2_DWB(CNV_WINDOW_START, CNV, inst),\
+ SRI2_DWB(CNV_WINDOW_SIZE, CNV, inst),\
+ SRI2_DWB(CNV_UPDATE, CNV, inst),\
+ SRI2_DWB(CNV_SOURCE_SIZE, CNV, inst),\
+ SRI2_DWB(CNV_TEST_CNTL, CNV, inst),\
+ SRI2_DWB(CNV_TEST_CRC_RED, CNV, inst),\
+ SRI2_DWB(CNV_TEST_CRC_GREEN, CNV, inst),\
+ SRI2_DWB(CNV_TEST_CRC_BLUE, CNV, inst),\
+ SRI2_DWB(WBSCL_COEF_RAM_SELECT, WBSCL, inst),\
+ SRI2_DWB(WBSCL_COEF_RAM_TAP_DATA, WBSCL, inst),\
+ SRI2_DWB(WBSCL_MODE, WBSCL, inst),\
+ SRI2_DWB(WBSCL_TAP_CONTROL, WBSCL, inst),\
+ SRI2_DWB(WBSCL_DEST_SIZE, WBSCL, inst),\
+ SRI2_DWB(WBSCL_HORZ_FILTER_SCALE_RATIO, WBSCL, inst),\
+ SRI2_DWB(WBSCL_HORZ_FILTER_INIT_Y_RGB, WBSCL, inst),\
+ SRI2_DWB(WBSCL_HORZ_FILTER_INIT_CBCR, WBSCL, inst),\
+ SRI2_DWB(WBSCL_VERT_FILTER_SCALE_RATIO, WBSCL, inst),\
+ SRI2_DWB(WBSCL_VERT_FILTER_INIT_Y_RGB, WBSCL, inst),\
+ SRI2_DWB(WBSCL_VERT_FILTER_INIT_CBCR, WBSCL, inst),\
+ SRI2_DWB(WBSCL_ROUND_OFFSET, WBSCL, inst),\
+ SRI2_DWB(WBSCL_OVERFLOW_STATUS, WBSCL, inst),\
+ SRI2_DWB(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL, inst),\
+ SRI2_DWB(WBSCL_TEST_CNTL, WBSCL, inst),\
+ SRI2_DWB(WBSCL_TEST_CRC_RED, WBSCL, inst),\
+ SRI2_DWB(WBSCL_TEST_CRC_GREEN, WBSCL, inst),\
+ SRI2_DWB(WBSCL_TEST_CRC_BLUE, WBSCL, inst),\
+ SRI2_DWB(WBSCL_BACKPRESSURE_CNT_EN, WBSCL, inst),\
+ SRI2_DWB(WB_MCIF_BACKPRESSURE_CNT, WBSCL, inst),\
+ SRI2_DWB(WBSCL_CLAMP_Y_RGB, WBSCL, inst),\
+ SRI2_DWB(WBSCL_CLAMP_CBCR, WBSCL, inst),\
+ SRI2_DWB(WBSCL_OUTSIDE_PIX_STRATEGY, WBSCL, inst),\
+ SRI2_DWB(WBSCL_OUTSIDE_PIX_STRATEGY_CBCR, WBSCL, inst),\
+ SRI2_DWB(WBSCL_DEBUG, WBSCL, inst),\
+ SRI2_DWB(WBSCL_TEST_DEBUG_INDEX, WBSCL, inst),\
+ SRI2_DWB(WBSCL_TEST_DEBUG_DATA, WBSCL, inst),\
+ SRI2_DWB(WB_DEBUG_CTRL, CNV, inst),\
+ SRI2_DWB(WB_DBG_MODE, CNV, inst),\
+ SRI2_DWB(WB_HW_DEBUG, CNV, inst),\
+ SRI2_DWB(CNV_TEST_DEBUG_INDEX, CNV, inst),\
+ SRI2_DWB(CNV_TEST_DEBUG_DATA, CNV, inst),\
+ SRI2_DWB(WB_SOFT_RESET, CNV, inst),\
+ SRI2_DWB(WB_WARM_UP_MODE_CTL1, CNV, inst),\
+ SRI2_DWB(WB_WARM_UP_MODE_CTL2, CNV, inst)
#define DWBC_COMMON_MASK_SH_LIST_DCN2_0(mask_sh) \
- SF(WB_ENABLE, WB_ENABLE, mask_sh),\
- SF(WB_EC_CONFIG, DISPCLK_R_WB_GATE_DIS, mask_sh),\
- SF(WB_EC_CONFIG, DISPCLK_G_WB_GATE_DIS, mask_sh),\
- SF(WB_EC_CONFIG, DISPCLK_G_WBSCL_GATE_DIS, mask_sh),\
- SF(WB_EC_CONFIG, WB_TEST_CLK_SEL, mask_sh),\
- SF(WB_EC_CONFIG, WB_LB_LS_DIS, mask_sh),\
- SF(WB_EC_CONFIG, WB_LB_SD_DIS, mask_sh),\
- SF(WB_EC_CONFIG, WB_LUT_LS_DIS, mask_sh),\
- SF(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_MODE_SEL, mask_sh),\
- SF(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_DIS, mask_sh),\
- SF(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_FORCE, mask_sh),\
- SF(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_STATE, mask_sh),\
- SF(WB_EC_CONFIG, WB_RAM_PW_SAVE_MODE, mask_sh),\
- SF(WB_EC_CONFIG, WBSCL_LUT_MEM_PWR_STATE, mask_sh),\
- SF(CNV_MODE, CNV_OUT_BPC, mask_sh),\
- SF(CNV_MODE, CNV_FRAME_CAPTURE_RATE, mask_sh),\
- SF(CNV_MODE, CNV_WINDOW_CROP_EN, mask_sh),\
- SF(CNV_MODE, CNV_STEREO_TYPE, mask_sh),\
- SF(CNV_MODE, CNV_INTERLACED_MODE, mask_sh),\
- SF(CNV_MODE, CNV_EYE_SELECTION, mask_sh),\
- SF(CNV_MODE, CNV_STEREO_POLARITY, mask_sh),\
- SF(CNV_MODE, CNV_INTERLACED_FIELD_ORDER, mask_sh),\
- SF(CNV_MODE, CNV_STEREO_SPLIT, mask_sh),\
- SF(CNV_MODE, CNV_NEW_CONTENT, mask_sh),\
- SF(CNV_MODE, CNV_FRAME_CAPTURE_EN_CURRENT, mask_sh),\
- SF(CNV_MODE, CNV_FRAME_CAPTURE_EN, mask_sh),\
- SF(CNV_WINDOW_START, CNV_WINDOW_START_X, mask_sh),\
- SF(CNV_WINDOW_START, CNV_WINDOW_START_Y, mask_sh),\
- SF(CNV_WINDOW_SIZE, CNV_WINDOW_WIDTH, mask_sh),\
- SF(CNV_WINDOW_SIZE, CNV_WINDOW_HEIGHT, mask_sh),\
- SF(CNV_UPDATE, CNV_UPDATE_PENDING, mask_sh),\
- SF(CNV_UPDATE, CNV_UPDATE_TAKEN, mask_sh),\
- SF(CNV_UPDATE, CNV_UPDATE_LOCK, mask_sh),\
- SF(CNV_SOURCE_SIZE, CNV_SOURCE_WIDTH, mask_sh),\
- SF(CNV_SOURCE_SIZE, CNV_SOURCE_HEIGHT, mask_sh),\
- SF(CNV_TEST_CNTL, CNV_TEST_CRC_EN, mask_sh),\
- SF(CNV_TEST_CNTL, CNV_TEST_CRC_CONT_EN, mask_sh),\
- SF(CNV_TEST_CRC_RED, CNV_TEST_CRC_RED_MASK, mask_sh),\
- SF(CNV_TEST_CRC_RED, CNV_TEST_CRC_SIG_RED, mask_sh),\
- SF(CNV_TEST_CRC_GREEN, CNV_TEST_CRC_GREEN_MASK, mask_sh),\
- SF(CNV_TEST_CRC_GREEN, CNV_TEST_CRC_SIG_GREEN, mask_sh),\
- SF(CNV_TEST_CRC_BLUE, CNV_TEST_CRC_BLUE_MASK, mask_sh),\
- SF(CNV_TEST_CRC_BLUE, CNV_TEST_CRC_SIG_BLUE, mask_sh),\
- SF(WB_DEBUG_CTRL, WB_DEBUG_EN, mask_sh),\
- SF(WB_DEBUG_CTRL, WB_DEBUG_SEL, mask_sh),\
- SF(WB_DBG_MODE, WB_DBG_MODE_EN, mask_sh),\
- SF(WB_DBG_MODE, WB_DBG_DIN_FMT, mask_sh),\
- SF(WB_DBG_MODE, WB_DBG_36MODE, mask_sh),\
- SF(WB_DBG_MODE, WB_DBG_CMAP, mask_sh),\
- SF(WB_DBG_MODE, WB_DBG_PXLRATE_ERROR, mask_sh),\
- SF(WB_DBG_MODE, WB_DBG_SOURCE_WIDTH, mask_sh),\
- SF(WB_HW_DEBUG, WB_HW_DEBUG, mask_sh),\
- SF(WB_SOFT_RESET, WB_SOFT_RESET, mask_sh),\
- SF(CNV_TEST_DEBUG_INDEX, CNV_TEST_DEBUG_INDEX, mask_sh),\
- SF(CNV_TEST_DEBUG_INDEX, CNV_TEST_DEBUG_WRITE_EN, mask_sh),\
- SF(CNV_TEST_DEBUG_DATA, CNV_TEST_DEBUG_DATA, mask_sh),\
- SF(WBSCL_COEF_RAM_SELECT, WBSCL_COEF_RAM_TAP_PAIR_IDX, mask_sh),\
- SF(WBSCL_COEF_RAM_SELECT, WBSCL_COEF_RAM_PHASE, mask_sh),\
- SF(WBSCL_COEF_RAM_SELECT, WBSCL_COEF_RAM_FILTER_TYPE, mask_sh),\
- SF(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_EVEN_TAP_COEF, mask_sh),\
- SF(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_EVEN_TAP_COEF_EN, mask_sh),\
- SF(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_ODD_TAP_COEF, mask_sh),\
- SF(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_ODD_TAP_COEF_EN, mask_sh),\
- SF(WBSCL_MODE, WBSCL_MODE, mask_sh),\
- SF(WBSCL_MODE, WBSCL_OUT_BIT_DEPTH, mask_sh),\
- SF(WBSCL_TAP_CONTROL, WBSCL_V_NUM_OF_TAPS_Y_RGB, mask_sh),\
- SF(WBSCL_TAP_CONTROL, WBSCL_V_NUM_OF_TAPS_CBCR, mask_sh),\
- SF(WBSCL_TAP_CONTROL, WBSCL_H_NUM_OF_TAPS_Y_RGB, mask_sh),\
- SF(WBSCL_TAP_CONTROL, WBSCL_H_NUM_OF_TAPS_CBCR, mask_sh),\
- SF(WBSCL_DEST_SIZE, WBSCL_DEST_HEIGHT, mask_sh),\
- SF(WBSCL_DEST_SIZE, WBSCL_DEST_WIDTH, mask_sh),\
- SF(WBSCL_HORZ_FILTER_SCALE_RATIO, WBSCL_H_SCALE_RATIO, mask_sh),\
- SF(WBSCL_HORZ_FILTER_INIT_Y_RGB, WBSCL_H_INIT_FRAC_Y_RGB, mask_sh),\
- SF(WBSCL_HORZ_FILTER_INIT_Y_RGB, WBSCL_H_INIT_INT_Y_RGB, mask_sh),\
- SF(WBSCL_HORZ_FILTER_INIT_CBCR, WBSCL_H_INIT_FRAC_CBCR, mask_sh),\
- SF(WBSCL_HORZ_FILTER_INIT_CBCR, WBSCL_H_INIT_INT_CBCR, mask_sh),\
- SF(WBSCL_VERT_FILTER_SCALE_RATIO, WBSCL_V_SCALE_RATIO, mask_sh),\
- SF(WBSCL_VERT_FILTER_INIT_Y_RGB, WBSCL_V_INIT_FRAC_Y_RGB, mask_sh),\
- SF(WBSCL_VERT_FILTER_INIT_Y_RGB, WBSCL_V_INIT_INT_Y_RGB, mask_sh),\
- SF(WBSCL_VERT_FILTER_INIT_CBCR, WBSCL_V_INIT_FRAC_CBCR, mask_sh),\
- SF(WBSCL_VERT_FILTER_INIT_CBCR, WBSCL_V_INIT_INT_CBCR, mask_sh),\
- SF(WBSCL_ROUND_OFFSET, WBSCL_ROUND_OFFSET_Y_RGB, mask_sh),\
- SF(WBSCL_ROUND_OFFSET, WBSCL_ROUND_OFFSET_CBCR, mask_sh),\
- SF(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_FLAG, mask_sh),\
- SF(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_ACK, mask_sh),\
- SF(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_MASK, mask_sh),\
- SF(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_INT_STATUS, mask_sh),\
- SF(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_INT_TYPE, mask_sh),\
- SF(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_FLAG, mask_sh),\
- SF(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_ACK, mask_sh),\
- SF(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_MASK, mask_sh),\
- SF(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_INT_STATUS, mask_sh),\
- SF(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_INT_TYPE, mask_sh),\
- SF(WBSCL_TEST_CNTL, WBSCL_TEST_CRC_EN, mask_sh),\
- SF(WBSCL_TEST_CNTL, WBSCL_TEST_CRC_CONT_EN, mask_sh),\
- SF(WBSCL_TEST_CRC_RED, WBSCL_TEST_CRC_RED_MASK, mask_sh),\
- SF(WBSCL_TEST_CRC_RED, WBSCL_TEST_CRC_SIG_RED, mask_sh),\
- SF(WBSCL_TEST_CRC_GREEN, WBSCL_TEST_CRC_GREEN_MASK, mask_sh),\
- SF(WBSCL_TEST_CRC_GREEN, WBSCL_TEST_CRC_SIG_GREEN, mask_sh),\
- SF(WBSCL_TEST_CRC_BLUE, WBSCL_TEST_CRC_BLUE_MASK, mask_sh),\
- SF(WBSCL_TEST_CRC_BLUE, WBSCL_TEST_CRC_SIG_BLUE, mask_sh),\
- SF(WBSCL_BACKPRESSURE_CNT_EN, WBSCL_BACKPRESSURE_CNT_EN, mask_sh),\
- SF(WB_MCIF_BACKPRESSURE_CNT, WB_MCIF_Y_MAX_BACKPRESSURE, mask_sh),\
- SF(WB_MCIF_BACKPRESSURE_CNT, WB_MCIF_C_MAX_BACKPRESSURE, mask_sh),\
- SF(WBSCL_CLAMP_Y_RGB, WBSCL_CLAMP_UPPER_Y_RGB, mask_sh),\
- SF(WBSCL_CLAMP_Y_RGB, WBSCL_CLAMP_LOWER_Y_RGB, mask_sh),\
- SF(WBSCL_CLAMP_CBCR, WBSCL_CLAMP_UPPER_CBCR, mask_sh),\
- SF(WBSCL_CLAMP_CBCR, WBSCL_CLAMP_LOWER_CBCR, mask_sh),\
- SF(WBSCL_OUTSIDE_PIX_STRATEGY, WBSCL_OUTSIDE_PIX_STRATEGY, mask_sh),\
- SF(WBSCL_OUTSIDE_PIX_STRATEGY, WBSCL_BLACK_COLOR_G_Y, mask_sh),\
- SF(WBSCL_OUTSIDE_PIX_STRATEGY_CBCR, WBSCL_BLACK_COLOR_B_CB, mask_sh),\
- SF(WBSCL_OUTSIDE_PIX_STRATEGY_CBCR, WBSCL_BLACK_COLOR_R_CR, mask_sh),\
- SF(WBSCL_DEBUG, WBSCL_DEBUG, mask_sh),\
- SF(WBSCL_TEST_DEBUG_INDEX, WBSCL_TEST_DEBUG_INDEX, mask_sh),\
- SF(WBSCL_TEST_DEBUG_INDEX, WBSCL_TEST_DEBUG_WRITE_EN, mask_sh),\
- SF(WBSCL_TEST_DEBUG_DATA, WBSCL_TEST_DEBUG_DATA, mask_sh),\
- SF(WB_WARM_UP_MODE_CTL1, WIDTH_WARMUP, mask_sh),\
- SF(WB_WARM_UP_MODE_CTL1, HEIGHT_WARMUP, mask_sh),\
- SF(WB_WARM_UP_MODE_CTL1, GMC_WARM_UP_ENABLE, mask_sh),\
- SF(WB_WARM_UP_MODE_CTL2, DATA_VALUE_WARMUP, mask_sh),\
- SF(WB_WARM_UP_MODE_CTL2, MODE_WARMUP, mask_sh),\
- SF(WB_WARM_UP_MODE_CTL2, DATA_DEPTH_WARMUP, mask_sh)
+ SF_DWB(WB_ENABLE, WB_ENABLE, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, DISPCLK_R_WB_GATE_DIS, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, DISPCLK_G_WB_GATE_DIS, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, DISPCLK_G_WBSCL_GATE_DIS, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WB_TEST_CLK_SEL, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WB_LB_LS_DIS, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WB_LB_SD_DIS, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WB_LUT_LS_DIS, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_MODE_SEL, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_DIS, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_FORCE, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WBSCL_LB_MEM_PWR_STATE, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WB_RAM_PW_SAVE_MODE, mask_sh),\
+ SF_DWB(WB_EC_CONFIG, WBSCL_LUT_MEM_PWR_STATE, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_OUT_BPC, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_FRAME_CAPTURE_RATE, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_WINDOW_CROP_EN, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_STEREO_TYPE, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_INTERLACED_MODE, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_EYE_SELECTION, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_STEREO_POLARITY, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_INTERLACED_FIELD_ORDER, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_STEREO_SPLIT, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_NEW_CONTENT, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_FRAME_CAPTURE_EN_CURRENT, mask_sh),\
+ SF_DWB(CNV_MODE, CNV_FRAME_CAPTURE_EN, mask_sh),\
+ SF_DWB(CNV_WINDOW_START, CNV_WINDOW_START_X, mask_sh),\
+ SF_DWB(CNV_WINDOW_START, CNV_WINDOW_START_Y, mask_sh),\
+ SF_DWB(CNV_WINDOW_SIZE, CNV_WINDOW_WIDTH, mask_sh),\
+ SF_DWB(CNV_WINDOW_SIZE, CNV_WINDOW_HEIGHT, mask_sh),\
+ SF_DWB(CNV_UPDATE, CNV_UPDATE_PENDING, mask_sh),\
+ SF_DWB(CNV_UPDATE, CNV_UPDATE_TAKEN, mask_sh),\
+ SF_DWB(CNV_UPDATE, CNV_UPDATE_LOCK, mask_sh),\
+ SF_DWB(CNV_SOURCE_SIZE, CNV_SOURCE_WIDTH, mask_sh),\
+ SF_DWB(CNV_SOURCE_SIZE, CNV_SOURCE_HEIGHT, mask_sh),\
+ SF_DWB(CNV_TEST_CNTL, CNV_TEST_CRC_EN, mask_sh),\
+ SF_DWB(CNV_TEST_CNTL, CNV_TEST_CRC_CONT_EN, mask_sh),\
+ SF_DWB(CNV_TEST_CRC_RED, CNV_TEST_CRC_RED_MASK, mask_sh),\
+ SF_DWB(CNV_TEST_CRC_RED, CNV_TEST_CRC_SIG_RED, mask_sh),\
+ SF_DWB(CNV_TEST_CRC_GREEN, CNV_TEST_CRC_GREEN_MASK, mask_sh),\
+ SF_DWB(CNV_TEST_CRC_GREEN, CNV_TEST_CRC_SIG_GREEN, mask_sh),\
+ SF_DWB(CNV_TEST_CRC_BLUE, CNV_TEST_CRC_BLUE_MASK, mask_sh),\
+ SF_DWB(CNV_TEST_CRC_BLUE, CNV_TEST_CRC_SIG_BLUE, mask_sh),\
+ SF_DWB(WB_DEBUG_CTRL, WB_DEBUG_EN, mask_sh),\
+ SF_DWB(WB_DEBUG_CTRL, WB_DEBUG_SEL, mask_sh),\
+ SF_DWB(WB_DBG_MODE, WB_DBG_MODE_EN, mask_sh),\
+ SF_DWB(WB_DBG_MODE, WB_DBG_DIN_FMT, mask_sh),\
+ SF_DWB(WB_DBG_MODE, WB_DBG_36MODE, mask_sh),\
+ SF_DWB(WB_DBG_MODE, WB_DBG_CMAP, mask_sh),\
+ SF_DWB(WB_DBG_MODE, WB_DBG_PXLRATE_ERROR, mask_sh),\
+ SF_DWB(WB_DBG_MODE, WB_DBG_SOURCE_WIDTH, mask_sh),\
+ SF_DWB(WB_HW_DEBUG, WB_HW_DEBUG, mask_sh),\
+ SF_DWB(WB_SOFT_RESET, WB_SOFT_RESET, mask_sh),\
+ SF_DWB(CNV_TEST_DEBUG_INDEX, CNV_TEST_DEBUG_INDEX, mask_sh),\
+ SF_DWB(CNV_TEST_DEBUG_INDEX, CNV_TEST_DEBUG_WRITE_EN, mask_sh),\
+ SF_DWB(CNV_TEST_DEBUG_DATA, CNV_TEST_DEBUG_DATA, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_SELECT, WBSCL_COEF_RAM_TAP_PAIR_IDX, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_SELECT, WBSCL_COEF_RAM_PHASE, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_SELECT, WBSCL_COEF_RAM_FILTER_TYPE, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_EVEN_TAP_COEF, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_EVEN_TAP_COEF_EN, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_ODD_TAP_COEF, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_TAP_DATA, WBSCL_COEF_RAM_ODD_TAP_COEF_EN, mask_sh),\
+ SF_DWB(WBSCL_MODE, WBSCL_MODE, mask_sh),\
+ SF_DWB(WBSCL_MODE, WBSCL_OUT_BIT_DEPTH, mask_sh),\
+ SF_DWB(WBSCL_TAP_CONTROL, WBSCL_V_NUM_OF_TAPS_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_TAP_CONTROL, WBSCL_V_NUM_OF_TAPS_CBCR, mask_sh),\
+ SF_DWB(WBSCL_TAP_CONTROL, WBSCL_H_NUM_OF_TAPS_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_TAP_CONTROL, WBSCL_H_NUM_OF_TAPS_CBCR, mask_sh),\
+ SF_DWB(WBSCL_DEST_SIZE, WBSCL_DEST_HEIGHT, mask_sh),\
+ SF_DWB(WBSCL_DEST_SIZE, WBSCL_DEST_WIDTH, mask_sh),\
+ SF_DWB(WBSCL_HORZ_FILTER_SCALE_RATIO, WBSCL_H_SCALE_RATIO, mask_sh),\
+ SF_DWB(WBSCL_HORZ_FILTER_INIT_Y_RGB, WBSCL_H_INIT_FRAC_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_HORZ_FILTER_INIT_Y_RGB, WBSCL_H_INIT_INT_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_HORZ_FILTER_INIT_CBCR, WBSCL_H_INIT_FRAC_CBCR, mask_sh),\
+ SF_DWB(WBSCL_HORZ_FILTER_INIT_CBCR, WBSCL_H_INIT_INT_CBCR, mask_sh),\
+ SF_DWB(WBSCL_VERT_FILTER_SCALE_RATIO, WBSCL_V_SCALE_RATIO, mask_sh),\
+ SF_DWB(WBSCL_VERT_FILTER_INIT_Y_RGB, WBSCL_V_INIT_FRAC_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_VERT_FILTER_INIT_Y_RGB, WBSCL_V_INIT_INT_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_VERT_FILTER_INIT_CBCR, WBSCL_V_INIT_FRAC_CBCR, mask_sh),\
+ SF_DWB(WBSCL_VERT_FILTER_INIT_CBCR, WBSCL_V_INIT_INT_CBCR, mask_sh),\
+ SF_DWB(WBSCL_ROUND_OFFSET, WBSCL_ROUND_OFFSET_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_ROUND_OFFSET, WBSCL_ROUND_OFFSET_CBCR, mask_sh),\
+ SF_DWB(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_FLAG, mask_sh),\
+ SF_DWB(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_ACK, mask_sh),\
+ SF_DWB(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_MASK, mask_sh),\
+ SF_DWB(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_INT_STATUS, mask_sh),\
+ SF_DWB(WBSCL_OVERFLOW_STATUS, WBSCL_DATA_OVERFLOW_INT_TYPE, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_FLAG, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_ACK, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_MASK, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_INT_STATUS, mask_sh),\
+ SF_DWB(WBSCL_COEF_RAM_CONFLICT_STATUS, WBSCL_HOST_CONFLICT_INT_TYPE, mask_sh),\
+ SF_DWB(WBSCL_TEST_CNTL, WBSCL_TEST_CRC_EN, mask_sh),\
+ SF_DWB(WBSCL_TEST_CNTL, WBSCL_TEST_CRC_CONT_EN, mask_sh),\
+ SF_DWB(WBSCL_TEST_CRC_RED, WBSCL_TEST_CRC_RED_MASK, mask_sh),\
+ SF_DWB(WBSCL_TEST_CRC_RED, WBSCL_TEST_CRC_SIG_RED, mask_sh),\
+ SF_DWB(WBSCL_TEST_CRC_GREEN, WBSCL_TEST_CRC_GREEN_MASK, mask_sh),\
+ SF_DWB(WBSCL_TEST_CRC_GREEN, WBSCL_TEST_CRC_SIG_GREEN, mask_sh),\
+ SF_DWB(WBSCL_TEST_CRC_BLUE, WBSCL_TEST_CRC_BLUE_MASK, mask_sh),\
+ SF_DWB(WBSCL_TEST_CRC_BLUE, WBSCL_TEST_CRC_SIG_BLUE, mask_sh),\
+ SF_DWB(WBSCL_BACKPRESSURE_CNT_EN, WBSCL_BACKPRESSURE_CNT_EN, mask_sh),\
+ SF_DWB(WB_MCIF_BACKPRESSURE_CNT, WB_MCIF_Y_MAX_BACKPRESSURE, mask_sh),\
+ SF_DWB(WB_MCIF_BACKPRESSURE_CNT, WB_MCIF_C_MAX_BACKPRESSURE, mask_sh),\
+ SF_DWB(WBSCL_CLAMP_Y_RGB, WBSCL_CLAMP_UPPER_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_CLAMP_Y_RGB, WBSCL_CLAMP_LOWER_Y_RGB, mask_sh),\
+ SF_DWB(WBSCL_CLAMP_CBCR, WBSCL_CLAMP_UPPER_CBCR, mask_sh),\
+ SF_DWB(WBSCL_CLAMP_CBCR, WBSCL_CLAMP_LOWER_CBCR, mask_sh),\
+ SF_DWB(WBSCL_OUTSIDE_PIX_STRATEGY, WBSCL_OUTSIDE_PIX_STRATEGY, mask_sh),\
+ SF_DWB(WBSCL_OUTSIDE_PIX_STRATEGY, WBSCL_BLACK_COLOR_G_Y, mask_sh),\
+ SF_DWB(WBSCL_OUTSIDE_PIX_STRATEGY_CBCR, WBSCL_BLACK_COLOR_B_CB, mask_sh),\
+ SF_DWB(WBSCL_OUTSIDE_PIX_STRATEGY_CBCR, WBSCL_BLACK_COLOR_R_CR, mask_sh),\
+ SF_DWB(WBSCL_DEBUG, WBSCL_DEBUG, mask_sh),\
+ SF_DWB(WBSCL_TEST_DEBUG_INDEX, WBSCL_TEST_DEBUG_INDEX, mask_sh),\
+ SF_DWB(WBSCL_TEST_DEBUG_INDEX, WBSCL_TEST_DEBUG_WRITE_EN, mask_sh),\
+ SF_DWB(WBSCL_TEST_DEBUG_DATA, WBSCL_TEST_DEBUG_DATA, mask_sh),\
+ SF_DWB(WB_WARM_UP_MODE_CTL1, WIDTH_WARMUP, mask_sh),\
+ SF_DWB(WB_WARM_UP_MODE_CTL1, HEIGHT_WARMUP, mask_sh),\
+ SF_DWB(WB_WARM_UP_MODE_CTL1, GMC_WARM_UP_ENABLE, mask_sh),\
+ SF_DWB(WB_WARM_UP_MODE_CTL2, DATA_VALUE_WARMUP, mask_sh),\
+ SF_DWB(WB_WARM_UP_MODE_CTL2, MODE_WARMUP, mask_sh),\
+ SF_DWB(WB_WARM_UP_MODE_CTL2, DATA_DEPTH_WARMUP, mask_sh)
#define DWBC_REG_FIELD_LIST_DCN2_0(type) \
type WB_ENABLE;\
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
index a7e0001..6291a24 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
@@ -1079,6 +1079,29 @@ void dcn20_blank_pixel_data(
0);
}
+ if (!blank && dc->debug.enable_single_display_2to1_odm_policy) {
+ /* when exiting dynamic ODM need to reinit DPG state for unused pipes */
+ struct pipe_ctx *old_odm_pipe = dc->current_state->res_ctx.pipe_ctx[pipe_ctx->pipe_idx].next_odm_pipe;
+
+ odm_pipe = pipe_ctx->next_odm_pipe;
+
+ while (old_odm_pipe) {
+ if (!odm_pipe || old_odm_pipe->pipe_idx != odm_pipe->pipe_idx)
+ dc->hwss.set_disp_pattern_generator(dc,
+ old_odm_pipe,
+ CONTROLLER_DP_TEST_PATTERN_VIDEOMODE,
+ CONTROLLER_DP_COLOR_SPACE_UDEFINED,
+ COLOR_DEPTH_888,
+ NULL,
+ 0,
+ 0,
+ 0);
+ old_odm_pipe = old_odm_pipe->next_odm_pipe;
+ if (odm_pipe)
+ odm_pipe = odm_pipe->next_odm_pipe;
+ }
+ }
+
if (!blank)
if (stream_res->abm) {
dc->hwss.set_pipe(pipe_ctx);
@@ -1287,6 +1310,19 @@ static void dcn20_detect_pipe_changes(struct pipe_ctx *old_pipe, struct pipe_ctx
{
new_pipe->update_flags.raw = 0;
+ /* If non-phantom pipe is being transitioned to a phantom pipe,
+ * set disable and return immediately. This is because the pipe
+ * that was previously in use must be fully disabled before we
+ * can "enable" it as a phantom pipe (since the OTG will certainly
+ * be different). The post_unlock sequence will set the correct
+ * update flags to enable the phantom pipe.
+ */
+ if (old_pipe->plane_state && !old_pipe->plane_state->is_phantom &&
+ new_pipe->plane_state && new_pipe->plane_state->is_phantom) {
+ new_pipe->update_flags.bits.disable = 1;
+ return;
+ }
+
/* Exit on unchanged, unused pipe */
if (!old_pipe->plane_state && !new_pipe->plane_state)
return;
@@ -1616,6 +1652,31 @@ static void dcn20_update_dchubp_dpp(
hubp->funcs->phantom_hubp_post_enable(hubp);
}
+static int calculate_vready_offset_for_group(struct pipe_ctx *pipe)
+{
+ struct pipe_ctx *other_pipe;
+ int vready_offset = pipe->pipe_dlg_param.vready_offset;
+
+ /* Always use the largest vready_offset of all connected pipes */
+ for (other_pipe = pipe->bottom_pipe; other_pipe != NULL; other_pipe = other_pipe->bottom_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+ for (other_pipe = pipe->top_pipe; other_pipe != NULL; other_pipe = other_pipe->top_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+ for (other_pipe = pipe->next_odm_pipe; other_pipe != NULL; other_pipe = other_pipe->next_odm_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+ for (other_pipe = pipe->prev_odm_pipe; other_pipe != NULL; other_pipe = other_pipe->prev_odm_pipe) {
+ if (other_pipe->pipe_dlg_param.vready_offset > vready_offset)
+ vready_offset = other_pipe->pipe_dlg_param.vready_offset;
+ }
+
+ return vready_offset;
+}
static void dcn20_program_pipe(
struct dc *dc,
@@ -1634,16 +1695,14 @@ static void dcn20_program_pipe(
&& !pipe_ctx->prev_odm_pipe) {
pipe_ctx->stream_res.tg->funcs->program_global_sync(
pipe_ctx->stream_res.tg,
- pipe_ctx->pipe_dlg_param.vready_offset,
+ calculate_vready_offset_for_group(pipe_ctx),
pipe_ctx->pipe_dlg_param.vstartup_start,
pipe_ctx->pipe_dlg_param.vupdate_offset,
pipe_ctx->pipe_dlg_param.vupdate_width);
if (pipe_ctx->stream->mall_stream_config.type != SUBVP_PHANTOM) {
- pipe_ctx->stream_res.tg->funcs->wait_for_state(
- pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK);
- pipe_ctx->stream_res.tg->funcs->wait_for_state(
- pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE);
+ pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK);
+ pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE);
}
pipe_ctx->stream_res.tg->funcs->set_vtg_params(
@@ -1682,7 +1741,10 @@ static void dcn20_program_pipe(
* only do gamma programming for powering on, internal memcmp to avoid
* updating on slave planes
*/
- if (pipe_ctx->update_flags.bits.enable || pipe_ctx->stream->update_flags.bits.out_tf)
+ if (pipe_ctx->update_flags.bits.enable ||
+ pipe_ctx->update_flags.bits.plane_changed ||
+ pipe_ctx->stream->update_flags.bits.out_tf ||
+ pipe_ctx->plane_state->update_flags.bits.output_tf_change)
hws->funcs.set_output_transfer_func(dc, pipe_ctx, pipe_ctx->stream);
/* If the pipe has been enabled or has a different opp, we
@@ -1813,6 +1875,17 @@ void dcn20_program_front_end_for_ctx(
context->stream_status[0].plane_count > 1) {
pipe->plane_res.hubp->funcs->hubp_wait_pipe_read_start(pipe->plane_res.hubp);
}
+
+ /* when dynamic ODM is active, pipes must be reconfigured when all planes are
+ * disabled, as some transitions will leave software and hardware state
+ * mismatched.
+ */
+ if (dc->debug.enable_single_display_2to1_odm_policy &&
+ pipe->stream &&
+ pipe->update_flags.bits.disable &&
+ !pipe->prev_odm_pipe &&
+ hws->funcs.update_odm)
+ hws->funcs.update_odm(dc, context, pipe);
}
}
@@ -1852,26 +1925,6 @@ void dcn20_post_unlock_program_front_end(
for (i = 0; i < dc->res_pool->pipe_count; i++) {
struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
- struct pipe_ctx *old_pipe = &dc->current_state->res_ctx.pipe_ctx[i];
-
- /* If an active, non-phantom pipe is being transitioned into a phantom
- * pipe, wait for the double buffer update to complete first before we do
- * phantom pipe programming (HUBP_VTG_SEL updates right away so that can
- * cause issues).
- */
- if (pipe->stream && pipe->stream->mall_stream_config.type == SUBVP_PHANTOM &&
- old_pipe->stream && old_pipe->stream->mall_stream_config.type != SUBVP_PHANTOM) {
- old_pipe->stream_res.tg->funcs->wait_for_state(
- old_pipe->stream_res.tg,
- CRTC_STATE_VBLANK);
- old_pipe->stream_res.tg->funcs->wait_for_state(
- old_pipe->stream_res.tg,
- CRTC_STATE_VACTIVE);
- }
- }
-
- for (i = 0; i < dc->res_pool->pipe_count; i++) {
- struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
if (pipe->plane_state && !pipe->top_pipe) {
/* Program phantom pipe here to prevent a frame of underflow in the MPO transition
@@ -1881,6 +1934,11 @@ void dcn20_post_unlock_program_front_end(
*/
while (pipe) {
if (pipe->stream && pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ /* When turning on the phantom pipe we want to run through the
+ * entire enable sequence, so apply all the "enable" flags.
+ */
+ if (dc->hwss.apply_update_flags_for_phantom)
+ dc->hwss.apply_update_flags_for_phantom(pipe);
if (dc->hwss.update_phantom_vp_position)
dc->hwss.update_phantom_vp_position(dc, context, pipe);
dcn20_program_pipe(dc, pipe, context);
@@ -1953,10 +2011,13 @@ void dcn20_prepare_bandwidth(
/* decrease compbuf size */
if (hubbub->funcs->program_compbuf_size) {
- if (context->bw_ctx.dml.ip.min_comp_buffer_size_kbytes)
+ if (context->bw_ctx.dml.ip.min_comp_buffer_size_kbytes) {
compbuf_size_kb = context->bw_ctx.dml.ip.min_comp_buffer_size_kbytes;
- else
+ dc->wm_optimized_required |= (compbuf_size_kb != dc->current_state->bw_ctx.dml.ip.min_comp_buffer_size_kbytes);
+ } else {
compbuf_size_kb = context->bw_ctx.bw.dcn.compbuf_size_kb;
+ dc->wm_optimized_required |= (compbuf_size_kb != dc->current_state->bw_ctx.bw.dcn.compbuf_size_kb);
+ }
hubbub->funcs->program_compbuf_size(hubbub, compbuf_size_kb, false);
}
@@ -2037,7 +2098,7 @@ bool dcn20_update_bandwidth(
pipe_ctx->stream_res.tg->funcs->program_global_sync(
pipe_ctx->stream_res.tg,
- pipe_ctx->pipe_dlg_param.vready_offset,
+ calculate_vready_offset_for_group(pipe_ctx),
pipe_ctx->pipe_dlg_param.vstartup_start,
pipe_ctx->pipe_dlg_param.vupdate_offset,
pipe_ctx->pipe_dlg_param.vupdate_width);
@@ -2601,14 +2662,6 @@ void dcn20_enable_stream(struct pipe_ctx *pipe_ctx)
if (dc->hwseq->funcs.set_pixels_per_cycle)
dc->hwseq->funcs.set_pixels_per_cycle(pipe_ctx);
-
- /* enable audio only within mode set */
- if (pipe_ctx->stream_res.audio != NULL) {
- if (is_dp_128b_132b_signal(pipe_ctx))
- pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_audio_enable(pipe_ctx->stream_res.hpo_dp_stream_enc);
- else if (dc_is_dp_signal(pipe_ctx->stream->signal))
- pipe_ctx->stream_res.stream_enc->funcs->dp_audio_enable(pipe_ctx->stream_res.stream_enc);
- }
}
void dcn20_program_dmdata_engine(struct pipe_ctx *pipe_ctx)
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mmhubbub.h b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mmhubbub.h
index 7bcee58..5ab32aa 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mmhubbub.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_mmhubbub.h
@@ -29,13 +29,6 @@
#define TO_DCN20_MMHUBBUB(mcif_wb_base) \
container_of(mcif_wb_base, struct dcn20_mmhubbub, base)
-/* DCN */
-#define BASE_INNER(seg) \
- DCE_BASE__INST0_SEG ## seg
-
-#define BASE(seg) \
- BASE_INNER(seg)
-
#define MCIF_WB_COMMON_REG_LIST_DCN2_0(inst) \
SRI(MCIF_WB_BUFMGR_SW_CONTROL, MCIF_WB, inst),\
SRI(MCIF_WB_BUFMGR_CUR_LINE_R, MCIF_WB, inst),\
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
index 8224b9b..8a0dd0d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
@@ -124,8 +124,6 @@ enum dcn20_clk_src_array_id {
* macros to expend register list macro defined in HW object header file */
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -138,6 +136,15 @@ enum dcn20_clk_src_array_id {
.reg_name = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
+#define SRI2_DWB(reg_name, block, id)\
+ .reg_name = BASE(mm ## reg_name ## _BASE_IDX) + \
+ mm ## reg_name
+#define SF_DWB(reg_name, field_name, post_fix)\
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define SRIR(var_name, reg_name, block, id)\
.var_name = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
@@ -1454,6 +1461,22 @@ enum dc_status dcn20_remove_stream_from_ctx(struct dc *dc, struct dc_state *new_
return result;
}
+/**
+ * dcn20_split_stream_for_odm - Check if stream can be splited for ODM
+ *
+ * @dc: DC object with resource pool info required for pipe split
+ * @res_ctx: Persistent state of resources
+ * @prev_odm_pipe: Reference to the previous ODM pipe
+ * @next_odm_pipe: Reference to the next ODM pipe
+ *
+ * This function takes a logically active pipe and a logically free pipe and
+ * halves all the scaling parameters that need to be halved while populating
+ * the free pipe with the required resources and configuring the next/previous
+ * ODM pipe pointers.
+ *
+ * Return:
+ * Return true if split stream for ODM is possible, otherwise, return false.
+ */
bool dcn20_split_stream_for_odm(
const struct dc *dc,
struct resource_context *res_ctx,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_vmid.h b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_vmid.h
index f1ef46e..e7a1b7f 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_vmid.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_vmid.h
@@ -28,12 +28,6 @@
#include "vmid.h"
-#define BASE_INNER(seg) \
- DCE_BASE__INST0_SEG ## seg
-
-#define BASE(seg) \
- BASE_INNER(seg)
-
#define DCN20_VMID_REG_LIST(id)\
SRI(CNTL, DCN_VM_CONTEXT, id),\
SRI(PAGE_TABLE_BASE_ADDR_HI32, DCN_VM_CONTEXT, id),\
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
index 8870814..fbcf0afe 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
@@ -94,8 +94,6 @@
* macros to expend register list macro defined in HW object header file */
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) DMU_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -671,12 +669,15 @@ static const struct dc_debug_options debug_defaults_diags = {
.disable_pplib_wm_range = true,
.disable_stutter = true,
.disable_48mhz_pwrdwn = true,
- .disable_psr = true,
.enable_tri_buf = true,
.use_max_lb = true
};
static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
.ilr = {
.optimize_edp_link_rate = true,
},
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb.h b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb.h
index 1010930..fc00ec0 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb.h
@@ -27,21 +27,6 @@
#define TO_DCN30_DWBC(dwbc_base) \
container_of(dwbc_base, struct dcn30_dwbc, base)
-/* DCN */
-#define BASE_INNER(seg) \
- DCE_BASE__INST0_SEG ## seg
-
-#define BASE(seg) \
- BASE_INNER(seg)
-
-#define SF_DWB(reg_name, block, id, field_name, post_fix)\
- .field_name = block ## id ## _ ## reg_name ## __ ## field_name ## post_fix
-
- /* set field name */
-#define SF_DWB2(reg_name, block, id, field_name, post_fix)\
- .field_name = reg_name ## __ ## field_name ## post_fix
-
-
#define DWBC_COMMON_REG_LIST_DCN30(inst) \
SR(DWB_ENABLE_CLK_CTRL),\
SR(DWB_MEM_PWR_CTRL),\
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.h b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.h
index 7446e54..376620a 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mmhubbub.h
@@ -31,13 +31,6 @@
#define TO_DCN30_MMHUBBUB(mcif_wb_base) \
container_of(mcif_wb_base, struct dcn30_mmhubbub, base)
-/* DCN */
-#define BASE_INNER(seg) \
- DCE_BASE__INST0_SEG ## seg
-
-#define BASE(seg) \
- BASE_INNER(seg)
-
#define MCIF_WB_COMMON_REG_LIST_DCN3_0(inst) \
SRI(MCIF_WB_BUFMGR_SW_CONTROL, MCIF_WB, inst),\
SRI(MCIF_WB_BUFMGR_STATUS, MCIF_WB, inst),\
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c
index 892d3c4..867d601 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_optc.c
@@ -30,6 +30,7 @@
#include "dc_dmub_srv.h"
#include "dml/dcn30/dcn30_fpu.h"
+#include "dc_trace.h"
#define REG(reg)\
optc1->tg_regs->reg
@@ -58,6 +59,8 @@ void optc3_triplebuffer_lock(struct timing_generator *optc)
REG_WAIT(OTG_MASTER_UPDATE_LOCK,
UPDATE_LOCK_STATUS, 1,
1, 10);
+
+ TRACE_OPTC_LOCK_UNLOCK_STATE(optc1, optc->inst, true);
}
void optc3_lock_doublebuffer_enable(struct timing_generator *optc)
@@ -93,6 +96,8 @@ void optc3_lock_doublebuffer_enable(struct timing_generator *optc)
MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, 0,
MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, 100,
OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, 1);
+
+ TRACE_OPTC_LOCK_UNLOCK_STATE(optc1, optc->inst, true);
}
void optc3_lock_doublebuffer_disable(struct timing_generator *optc)
@@ -108,6 +113,8 @@ void optc3_lock_doublebuffer_disable(struct timing_generator *optc)
REG_UPDATE(OTG_GLOBAL_CONTROL2, GLOBAL_UPDATE_LOCK_EN, 0);
REG_UPDATE(OTG_GLOBAL_CONTROL0, MASTER_UPDATE_LOCK_DB_EN, 0);
+
+ TRACE_OPTC_LOCK_UNLOCK_STATE(optc1, optc->inst, true);
}
void optc3_lock(struct timing_generator *optc)
@@ -122,6 +129,8 @@ void optc3_lock(struct timing_generator *optc)
REG_WAIT(OTG_MASTER_UPDATE_LOCK,
UPDATE_LOCK_STATUS, 1,
1, 10);
+
+ TRACE_OPTC_LOCK_UNLOCK_STATE(optc1, optc->inst, true);
}
void optc3_set_out_mux(struct timing_generator *optc, enum otg_out_mux_dest dest)
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
index 020f512..c18c52a 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
@@ -108,8 +108,6 @@ enum dcn30_clk_src_array_id {
*/
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -142,6 +140,9 @@ enum dcn30_clk_src_array_id {
.reg_name[id] = BASE(mm ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define DCCG_SRII(reg_name, block, id)\
.block ## _ ## reg_name[id] = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
@@ -723,7 +724,6 @@ static const struct dc_debug_options debug_defaults_drv = {
.underflow_assert_delay_us = 0xFFFFFFFF,
.dwb_fi_phase = -1, // -1 = disable,
.dmub_command_table = true,
- .disable_psr = false,
.use_max_lb = true,
.exit_idle_opt_for_cursor_updates = true
};
@@ -742,11 +742,17 @@ static const struct dc_debug_options debug_defaults_diags = {
.scl_reset_length10 = true,
.dwb_fi_phase = -1, // -1 = disable
.dmub_command_table = true,
- .disable_psr = true,
.enable_tri_buf = true,
.use_max_lb = true
};
+static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
+};
+
static void dcn30_dpp_destroy(struct dpp **dpp)
{
kfree(TO_DCN20_DPP(*dpp));
@@ -1323,6 +1329,7 @@ static struct clock_source *dcn30_clock_source_create(
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -2212,6 +2219,11 @@ void dcn30_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_params
}
}
+static void dcn30_get_panel_config_defaults(struct dc_panel_config *panel_config)
+{
+ *panel_config = panel_config_defaults;
+}
+
static const struct resource_funcs dcn30_res_pool_funcs = {
.destroy = dcn30_destroy_resource_pool,
.link_enc_create = dcn30_link_encoder_create,
@@ -2231,6 +2243,7 @@ static const struct resource_funcs dcn30_res_pool_funcs = {
.release_post_bldn_3dlut = dcn30_release_post_bldn_3dlut,
.update_bw_bounding_box = dcn30_update_bw_bounding_box,
.patch_unknown_plane_state = dcn20_patch_unknown_plane_state,
+ .get_panel_config_defaults = dcn30_get_panel_config_defaults,
};
#define CTX ctx
diff --git a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
index f04595b..8cf1035 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
@@ -107,8 +107,6 @@ enum dcn301_clk_src_array_id {
*/
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -146,6 +144,9 @@ enum dcn301_clk_src_array_id {
.reg_name[id] = BASE(mm ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define DCCG_SRII(reg_name, block, id)\
.block ## _ ## reg_name[id] = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
@@ -1288,6 +1289,7 @@ static struct clock_source *dcn301_clock_source_create(
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1491,6 +1493,8 @@ static bool dcn301_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ dc->caps.dp_hdmi21_pcon_support = true;
+
/* read VBIOS LTTPR caps */
if (ctx->dc_bios->funcs->get_lttpr_caps) {
enum bp_result bp_query_result;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
index b925b6d..47cffd0 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
@@ -112,10 +112,16 @@ static const struct dc_debug_options debug_defaults_diags = {
.dwb_fi_phase = -1, // -1 = disable
.dmub_command_table = true,
.enable_tri_buf = true,
- .disable_psr = true,
.use_max_lb = true
};
+static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
+};
+
enum dcn302_clk_src_array_id {
DCN302_CLK_SRC_PLL0,
DCN302_CLK_SRC_PLL1,
@@ -177,7 +183,6 @@ static const struct dc_plane_cap plane_cap = {
mm ## reg_name
/* DCN */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -210,6 +215,9 @@ static const struct dc_plane_cap plane_cap = {
.reg_name[id] = BASE(mm ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define SRII_MPC_RMU(reg_name, block, id)\
.RMU##_##reg_name[id] = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
@@ -458,6 +466,7 @@ static struct clock_source *dcn302_clock_source_create(struct dc_context *ctx, s
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1132,6 +1141,11 @@ void dcn302_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_param
DC_FP_END();
}
+static void dcn302_get_panel_config_defaults(struct dc_panel_config *panel_config)
+{
+ *panel_config = panel_config_defaults;
+}
+
static struct resource_funcs dcn302_res_pool_funcs = {
.destroy = dcn302_destroy_resource_pool,
.link_enc_create = dcn302_link_encoder_create,
@@ -1151,6 +1165,7 @@ static struct resource_funcs dcn302_res_pool_funcs = {
.release_post_bldn_3dlut = dcn30_release_post_bldn_3dlut,
.update_bw_bounding_box = dcn302_update_bw_bounding_box,
.patch_unknown_plane_state = dcn20_patch_unknown_plane_state,
+ .get_panel_config_defaults = dcn302_get_panel_config_defaults,
};
static struct dc_cap_funcs cap_funcs = {
@@ -1266,6 +1281,8 @@ static bool dcn302_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ dc->caps.dp_hdmi21_pcon_support = true;
+
/* read VBIOS LTTPR caps */
if (ctx->dc_bios->funcs->get_lttpr_caps) {
enum bp_result bp_query_result;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c
index 527d5c9..c14d358 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c
@@ -96,7 +96,13 @@ static const struct dc_debug_options debug_defaults_diags = {
.dwb_fi_phase = -1, // -1 = disable
.dmub_command_table = true,
.enable_tri_buf = true,
- .disable_psr = true,
+};
+
+static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
};
enum dcn303_clk_src_array_id {
@@ -156,7 +162,6 @@ static const struct dc_plane_cap plane_cap = {
mm ## reg_name
/* DCN */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -189,6 +194,9 @@ static const struct dc_plane_cap plane_cap = {
.reg_name[id] = BASE(mm ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define SRII_MPC_RMU(reg_name, block, id)\
.RMU##_##reg_name[id] = BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
@@ -425,6 +433,7 @@ static struct clock_source *dcn303_clock_source_create(struct dc_context *ctx, s
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1055,6 +1064,10 @@ static void dcn303_destroy_resource_pool(struct resource_pool **pool)
*pool = NULL;
}
+static void dcn303_get_panel_config_defaults(struct dc_panel_config *panel_config)
+{
+ *panel_config = panel_config_defaults;
+}
void dcn303_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_params)
{
@@ -1082,6 +1095,7 @@ static struct resource_funcs dcn303_res_pool_funcs = {
.release_post_bldn_3dlut = dcn30_release_post_bldn_3dlut,
.update_bw_bounding_box = dcn303_update_bw_bounding_box,
.patch_unknown_plane_state = dcn20_patch_unknown_plane_state,
+ .get_panel_config_defaults = dcn303_get_panel_config_defaults,
};
static struct dc_cap_funcs cap_funcs = {
@@ -1198,6 +1212,8 @@ static bool dcn303_resource_construct(
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
dc->caps.color.mpc.ocsc = 1;
+ dc->caps.dp_hdmi21_pcon_support = true;
+
/* read VBIOS LTTPR caps */
if (ctx->dc_bios->funcs->get_lttpr_caps) {
enum bp_result bp_query_result;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.c
index de5e18c..24e9ff6 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.c
@@ -134,23 +134,10 @@ static void apg31_se_audio_setup(
/* Disable forced mem power off */
REG_UPDATE(APG_MEM_PWR, APG_MEM_PWR_FORCE, 0);
-
- apg31_enable(apg);
-}
-
-static void apg31_audio_mute_control(
- struct apg *apg,
- bool mute)
-{
- if (mute)
- apg31_disable(apg);
- else
- apg31_enable(apg);
}
static struct apg_funcs dcn31_apg_funcs = {
.se_audio_setup = apg31_se_audio_setup,
- .audio_mute_control = apg31_audio_mute_control,
.enable_apg = apg31_enable,
.disable_apg = apg31_disable,
};
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.h b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.h
index 24f568e..1b81f67 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_apg.h
@@ -84,10 +84,6 @@ struct apg_funcs {
unsigned int az_inst,
struct audio_info *audio_info);
- void (*audio_mute_control)(
- struct apg *apg,
- bool mute);
-
void (*enable_apg)(
struct apg *apg);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c
index 814f401..16639bd 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c
@@ -600,14 +600,6 @@ static void dcn31_hpo_dp_stream_enc_map_stream_to_link(
}
}
-static void dcn31_hpo_dp_stream_enc_mute_control(
- struct hpo_dp_stream_encoder *enc,
- bool mute)
-{
- ASSERT(enc->apg);
- enc->apg->funcs->audio_mute_control(enc->apg, mute);
-}
-
static void dcn31_hpo_dp_stream_enc_audio_setup(
struct hpo_dp_stream_encoder *enc,
unsigned int az_inst,
@@ -726,7 +718,6 @@ static const struct hpo_dp_stream_encoder_funcs dcn30_str_enc_funcs = {
.stop_dp_info_packets = dcn31_hpo_dp_stream_enc_stop_dp_info_packets,
.dp_set_dsc_pps_info_packet = dcn31_hpo_dp_stream_enc_set_dsc_pps_info_packet,
.map_stream_to_link = dcn31_hpo_dp_stream_enc_map_stream_to_link,
- .audio_mute_control = dcn31_hpo_dp_stream_enc_mute_control,
.dp_audio_setup = dcn31_hpo_dp_stream_enc_audio_setup,
.dp_audio_enable = dcn31_hpo_dp_stream_enc_audio_enable,
.dp_audio_disable = dcn31_hpo_dp_stream_enc_audio_disable,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c
index bdf1015..4226a051 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c
@@ -89,7 +89,8 @@ static void enable_memory_low_power(struct dc *dc)
REG_UPDATE(MMHUBBUB_MEM_PWR_CNTL, VGA_MEM_PWR_FORCE, 1);
}
- if (dc->debug.enable_mem_low_power.bits.mpc)
+ if (dc->debug.enable_mem_low_power.bits.mpc &&
+ dc->res_pool->mpc->funcs->set_mpc_mem_lp_mode)
dc->res_pool->mpc->funcs->set_mpc_mem_lp_mode(dc->res_pool->mpc);
@@ -141,7 +142,8 @@ void dcn31_init_hw(struct dc *dc)
if (!dcb->funcs->is_accelerated_mode(dcb)) {
hws->funcs.bios_golden_init(dc);
- hws->funcs.disable_vga(dc->hwseq);
+ if (hws->funcs.disable_vga)
+ hws->funcs.disable_vga(dc->hwseq);
}
// Initialize the dccg
if (res_pool->dccg->funcs->dccg_init)
@@ -621,3 +623,43 @@ void dcn31_setup_hpo_hw_control(const struct dce_hwseq *hws, bool enable)
if (hws->ctx->dc->debug.hpo_optimization)
REG_UPDATE(HPO_TOP_HW_CONTROL, HPO_IO_EN, !!enable);
}
+void dcn31_set_drr(struct pipe_ctx **pipe_ctx,
+ int num_pipes, struct dc_crtc_timing_adjust adjust)
+{
+ int i = 0;
+ struct drr_params params = {0};
+ unsigned int event_triggers = 0x2;/*Bit[1]: OTG_TRIG_A*/
+ unsigned int num_frames = 2;
+ params.vertical_total_max = adjust.v_total_max;
+ params.vertical_total_min = adjust.v_total_min;
+ params.vertical_total_mid = adjust.v_total_mid;
+ params.vertical_total_mid_frame_num = adjust.v_total_mid_frame_num;
+ for (i = 0; i < num_pipes; i++) {
+ if ((pipe_ctx[i]->stream_res.tg != NULL) && pipe_ctx[i]->stream_res.tg->funcs) {
+ if (pipe_ctx[i]->stream_res.tg->funcs->set_drr)
+ pipe_ctx[i]->stream_res.tg->funcs->set_drr(
+ pipe_ctx[i]->stream_res.tg, ¶ms);
+ if (adjust.v_total_max != 0 && adjust.v_total_min != 0)
+ if (pipe_ctx[i]->stream_res.tg->funcs->set_static_screen_control)
+ pipe_ctx[i]->stream_res.tg->funcs->set_static_screen_control(
+ pipe_ctx[i]->stream_res.tg,
+ event_triggers, num_frames);
+ }
+ }
+}
+void dcn31_set_static_screen_control(struct pipe_ctx **pipe_ctx,
+ int num_pipes, const struct dc_static_screen_params *params)
+{
+ unsigned int i;
+ unsigned int triggers = 0;
+ if (params->triggers.surface_update)
+ triggers |= 0x600;/*bit 9 and bit10 : 110 0000 0000*/
+ if (params->triggers.cursor_update)
+ triggers |= 0x10;/*bit4*/
+ if (params->triggers.force_trigger)
+ triggers |= 0x1;
+ for (i = 0; i < num_pipes; i++)
+ pipe_ctx[i]->stream_res.tg->funcs->
+ set_static_screen_control(pipe_ctx[i]->stream_res.tg,
+ triggers, params->num_frames);
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.h b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.h
index edfc01d..e7e03a8 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.h
@@ -56,4 +56,8 @@ bool dcn31_is_abm_supported(struct dc *dc,
void dcn31_init_pipes(struct dc *dc, struct dc_state *context);
void dcn31_setup_hpo_hw_control(const struct dce_hwseq *hws, bool enable);
+void dcn31_set_static_screen_control(struct pipe_ctx **pipe_ctx,
+ int num_pipes, const struct dc_static_screen_params *params);
+void dcn31_set_drr(struct pipe_ctx **pipe_ctx,
+ int num_pipes, struct dc_crtc_timing_adjust adjust);
#endif /* __DC_HWSS_DCN31_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c
index 3a32810..7c2da70 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_init.c
@@ -64,9 +64,9 @@ static const struct hw_sequencer_funcs dcn31_funcs = {
.prepare_bandwidth = dcn20_prepare_bandwidth,
.optimize_bandwidth = dcn20_optimize_bandwidth,
.update_bandwidth = dcn20_update_bandwidth,
- .set_drr = dcn10_set_drr,
+ .set_drr = dcn31_set_drr,
.get_position = dcn10_get_position,
- .set_static_screen_control = dcn10_set_static_screen_control,
+ .set_static_screen_control = dcn31_set_static_screen_control,
.setup_stereo = dcn10_setup_stereo,
.set_avmute = dcn30_set_avmute,
.log_hw_state = dcn10_log_hw_state,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c
index 63a677c..fe449f7 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c
@@ -40,6 +40,7 @@
#define FN(reg_name, field_name) \
optc1->tg_shift->field_name, optc1->tg_mask->field_name
+#define STATIC_SCREEN_EVENT_MASK_DRR_DOUBLE_BUFFER_UPDATE_EN 0x2000 /*bit 13*/
static void optc31_set_odm_combine(struct timing_generator *optc, int *opp_id, int opp_cnt,
struct dc_crtc_timing *timing)
{
@@ -231,6 +232,32 @@ void optc3_init_odm(struct timing_generator *optc)
OPTC_MEM_SEL, 0);
optc1->opp_count = 1;
}
+void optc31_set_static_screen_control(
+ struct timing_generator *optc,
+ uint32_t event_triggers,
+ uint32_t num_frames)
+{
+ struct optc *optc1 = DCN10TG_FROM_TG(optc);
+ uint32_t framecount;
+ uint32_t events;
+
+ if (num_frames > 0xFF)
+ num_frames = 0xFF;
+ REG_GET_2(OTG_STATIC_SCREEN_CONTROL,
+ OTG_STATIC_SCREEN_EVENT_MASK, &events,
+ OTG_STATIC_SCREEN_FRAME_COUNT, &framecount);
+
+ if (events == event_triggers && num_frames == framecount)
+ return;
+ if ((event_triggers & STATIC_SCREEN_EVENT_MASK_DRR_DOUBLE_BUFFER_UPDATE_EN)
+ != 0)
+ event_triggers = event_triggers &
+ ~STATIC_SCREEN_EVENT_MASK_DRR_DOUBLE_BUFFER_UPDATE_EN;
+
+ REG_UPDATE_2(OTG_STATIC_SCREEN_CONTROL,
+ OTG_STATIC_SCREEN_EVENT_MASK, event_triggers,
+ OTG_STATIC_SCREEN_FRAME_COUNT, num_frames);
+}
static struct timing_generator_funcs dcn31_tg_funcs = {
.validate_timing = optc1_validate_timing,
@@ -266,7 +293,7 @@ static struct timing_generator_funcs dcn31_tg_funcs = {
.set_drr = optc31_set_drr,
.get_last_used_drr_vtotal = optc2_get_last_used_drr_vtotal,
.set_vtotal_min_max = optc1_set_vtotal_min_max,
- .set_static_screen_control = optc1_set_static_screen_control,
+ .set_static_screen_control = optc31_set_static_screen_control,
.program_stereo = optc1_program_stereo,
.is_stereo_left_eye = optc1_is_stereo_left_eye,
.tg_init = optc3_tg_init,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.h b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.h
index 30b81a4..5fc6c63 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.h
@@ -263,5 +263,8 @@ bool optc31_immediate_disable_crtc(struct timing_generator *optc);
void optc31_set_drr(struct timing_generator *optc, const struct drr_params *params);
void optc3_init_odm(struct timing_generator *optc);
-
+void optc31_set_static_screen_control(
+ struct timing_generator *optc,
+ uint32_t event_triggers,
+ uint32_t num_frames);
#endif /* __DC_OPTC_DCN31_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c
index fddc21a..3ca517d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c
@@ -119,8 +119,6 @@ enum dcn31_clk_src_array_id {
*/
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -153,6 +151,9 @@ enum dcn31_clk_src_array_id {
.reg_name[id] = BASE(reg ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define DCCG_SRII(reg_name, block, id)\
.block ## _ ## reg_name[id] = BASE(reg ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## reg_name
@@ -911,6 +912,10 @@ static const struct dc_debug_options debug_defaults_diags = {
};
static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
.ilr = {
.optimize_edp_link_rate = true,
},
@@ -1625,6 +1630,7 @@ static struct clock_source *dcn31_clock_source_create(
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1634,6 +1640,31 @@ static bool is_dual_plane(enum surface_pixel_format format)
return format >= SURFACE_PIXEL_FORMAT_VIDEO_BEGIN || format == SURFACE_PIXEL_FORMAT_GRPH_RGBE_ALPHA;
}
+int dcn31x_populate_dml_pipes_from_context(struct dc *dc,
+ struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ bool fast_validate)
+{
+ uint32_t pipe_cnt;
+ int i;
+
+ dc_assert_fp_enabled();
+
+ pipe_cnt = dcn20_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
+
+ for (i = 0; i < pipe_cnt; i++) {
+ pipes[i].pipe.src.gpuvm = 1;
+ if (dc->debug.dml_hostvm_override == DML_HOSTVM_NO_OVERRIDE) {
+ //pipes[pipe_cnt].pipe.src.hostvm = dc->res_pool->hubbub->riommu_active;
+ pipes[i].pipe.src.hostvm = dc->vm_pa_config.is_hvm_enabled;
+ } else if (dc->debug.dml_hostvm_override == DML_HOSTVM_OVERRIDE_FALSE)
+ pipes[i].pipe.src.hostvm = false;
+ else if (dc->debug.dml_hostvm_override == DML_HOSTVM_OVERRIDE_TRUE)
+ pipes[i].pipe.src.hostvm = true;
+ }
+ return pipe_cnt;
+}
+
int dcn31_populate_dml_pipes_from_context(
struct dc *dc, struct dc_state *context,
display_e2e_pipe_params_st *pipes,
@@ -1645,7 +1676,7 @@ int dcn31_populate_dml_pipes_from_context(
bool upscaled = false;
DC_FP_START();
- dcn20_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
+ dcn31x_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
DC_FP_END();
for (i = 0, pipe_cnt = 0; i < dc->res_pool->pipe_count; i++) {
@@ -1675,12 +1706,6 @@ int dcn31_populate_dml_pipes_from_context(
dcn31_zero_pipe_dcc_fraction(pipes, pipe_cnt);
DC_FP_END();
- if (dc->debug.dml_hostvm_override == DML_HOSTVM_NO_OVERRIDE)
- pipes[pipe_cnt].pipe.src.hostvm = dc->res_pool->hubbub->riommu_active;
- else if (dc->debug.dml_hostvm_override == DML_HOSTVM_OVERRIDE_FALSE)
- pipes[pipe_cnt].pipe.src.hostvm = false;
- else if (dc->debug.dml_hostvm_override == DML_HOSTVM_OVERRIDE_TRUE)
- pipes[pipe_cnt].pipe.src.hostvm = true;
if (pipes[pipe_cnt].dout.dsc_enable) {
switch (timing->display_color_depth) {
@@ -1898,6 +1923,8 @@ static bool dcn31_resource_construct(
dc->caps.max_slave_rgb_planes = 2;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
+ if (dc->config.forceHBR2CP2520)
+ dc->caps.force_dp_tps4_for_cp2520 = false;
dc->caps.dp_hpo = true;
dc->caps.dp_hdmi21_pcon_support = true;
dc->caps.edp_dsc_support = true;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dccg.c b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dccg.c
index 389a893..0b769ee 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dccg.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dccg.c
@@ -104,7 +104,7 @@ static void dccg314_set_pixel_rate_div(
}
dccg314_get_pixel_rate_div(dccg, otg_inst, &cur_k1, &cur_k2);
- if (k1 == PIXEL_RATE_DIV_NA || k2 == PIXEL_RATE_DIV_NA || (k1 == cur_k1 && k2 == cur_k2))
+ if (k1 == cur_k1 && k2 == cur_k2)
return;
switch (otg_inst) {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_init.c b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_init.c
index 5b6c2d9..31feb4b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_init.c
@@ -66,9 +66,9 @@ static const struct hw_sequencer_funcs dcn314_funcs = {
.prepare_bandwidth = dcn20_prepare_bandwidth,
.optimize_bandwidth = dcn20_optimize_bandwidth,
.update_bandwidth = dcn20_update_bandwidth,
- .set_drr = dcn10_set_drr,
+ .set_drr = dcn31_set_drr,
.get_position = dcn10_get_position,
- .set_static_screen_control = dcn10_set_static_screen_control,
+ .set_static_screen_control = dcn31_set_static_screen_control,
.setup_stereo = dcn10_setup_stereo,
.set_avmute = dcn30_set_avmute,
.log_hw_state = dcn10_log_hw_state,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_optc.c b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_optc.c
index 7dd36e4..41edbd6 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_optc.c
@@ -228,7 +228,7 @@ static struct timing_generator_funcs dcn314_tg_funcs = {
.set_drr = optc31_set_drr,
.get_last_used_drr_vtotal = optc2_get_last_used_drr_vtotal,
.set_vtotal_min_max = optc1_set_vtotal_min_max,
- .set_static_screen_control = optc1_set_static_screen_control,
+ .set_static_screen_control = optc31_set_static_screen_control,
.program_stereo = optc1_program_stereo,
.is_stereo_left_eye = optc1_is_stereo_left_eye,
.tg_init = optc3_tg_init,
@@ -241,7 +241,6 @@ static struct timing_generator_funcs dcn314_tg_funcs = {
.set_dsc_config = optc3_set_dsc_config,
.get_dsc_status = optc2_get_dsc_status,
.set_dwb_source = NULL,
- .set_odm_bypass = optc3_set_odm_bypass,
.set_odm_combine = optc314_set_odm_combine,
.get_optc_source = optc2_get_optc_source,
.set_out_mux = optc3_set_out_mux,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c
index 9066c51..f9ea1e8 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c
@@ -184,6 +184,9 @@ enum dcn31_clk_src_array_id {
.reg_name[id] = BASE(reg ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define DCCG_SRII(reg_name, block, id)\
.block ## _ ## reg_name[id] = BASE(reg ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## reg_name
@@ -883,6 +886,7 @@ static const struct dc_plane_cap plane_cap = {
static const struct dc_debug_options debug_defaults_drv = {
.disable_z10 = false,
.enable_z9_disable_interface = true,
+ .psr_skip_crtc_disable = true,
.disable_dmcu = true,
.force_abm_enable = false,
.timing_trace = false,
@@ -937,6 +941,10 @@ static const struct dc_debug_options debug_defaults_diags = {
};
static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
.ilr = {
.optimize_edp_link_rate = true,
},
@@ -1766,6 +1774,8 @@ static bool dcn314_resource_construct(
dc->caps.max_slave_rgb_planes = 2;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
+ if (dc->config.forceHBR2CP2520)
+ dc->caps.force_dp_tps4_for_cp2520 = false;
dc->caps.dp_hpo = true;
dc->caps.dp_hdmi21_pcon_support = true;
dc->caps.edp_dsc_support = true;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn315/dcn315_resource.c b/drivers/gpu/drm/amd/display/dc/dcn315/dcn315_resource.c
index 58746c4..7887078 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn315/dcn315_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn315/dcn315_resource.c
@@ -151,8 +151,6 @@ enum dcn31_clk_src_array_id {
*/
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -185,6 +183,9 @@ enum dcn31_clk_src_array_id {
.reg_name[id] = BASE(reg ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define DCCG_SRII(reg_name, block, id)\
.block ## _ ## reg_name[id] = BASE(reg ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## reg_name
@@ -907,6 +908,10 @@ static const struct dc_debug_options debug_defaults_diags = {
};
static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
.ilr = {
.optimize_edp_link_rate = true,
},
@@ -1623,6 +1628,7 @@ static struct clock_source *dcn31_clock_source_create(
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1643,7 +1649,7 @@ static int dcn315_populate_dml_pipes_from_context(
const int max_usable_det = context->bw_ctx.dml.ip.config_return_buffer_size_in_kbytes - DCN3_15_MIN_COMPBUF_SIZE_KB;
DC_FP_START();
- dcn20_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
+ dcn31x_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
DC_FP_END();
for (i = 0, pipe_cnt = 0; i < dc->res_pool->pipe_count; i++) {
@@ -1662,7 +1668,6 @@ static int dcn315_populate_dml_pipes_from_context(
pipes[pipe_cnt].pipe.src.immediate_flip = true;
pipes[pipe_cnt].pipe.src.unbounded_req_mode = false;
- pipes[pipe_cnt].pipe.src.gpuvm = true;
pipes[pipe_cnt].pipe.dest.vfront_porch = timing->v_front_porch;
pipes[pipe_cnt].pipe.src.dcc_rate = 3;
pipes[pipe_cnt].dout.dsc_input_bpc = 0;
@@ -1703,7 +1708,9 @@ static int dcn315_populate_dml_pipes_from_context(
dc->config.enable_4to1MPC = true;
context->bw_ctx.dml.ip.det_buffer_size_kbytes =
(max_usable_det / DCN3_15_CRB_SEGMENT_SIZE_KB / 4) * DCN3_15_CRB_SEGMENT_SIZE_KB;
- } else if (!is_dual_plane(pipe->plane_state->format) && pipe->plane_state->src_rect.width <= 5120) {
+ } else if (!is_dual_plane(pipe->plane_state->format)
+ && pipe->plane_state->src_rect.width <= 5120
+ && pipe->stream->timing.pix_clk_100hz < dcn_get_max_non_odm_pix_rate_100hz(&dc->dml.soc)) {
/* Limit to 5k max to avoid forced pipe split when there is not enough detile for swath */
context->bw_ctx.dml.ip.det_buffer_size_kbytes = 192;
pipes[0].pipe.src.unbounded_req_mode = true;
@@ -1779,6 +1786,8 @@ static bool dcn315_resource_construct(
dc->caps.max_slave_rgb_planes = 2;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
+ if (dc->config.forceHBR2CP2520)
+ dc->caps.force_dp_tps4_for_cp2520 = false;
dc->caps.dp_hpo = true;
dc->caps.dp_hdmi21_pcon_support = true;
dc->caps.edp_dsc_support = true;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn316/dcn316_resource.c b/drivers/gpu/drm/amd/display/dc/dcn316/dcn316_resource.c
index 6b40a11..b4d5076 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn316/dcn316_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn316/dcn316_resource.c
@@ -142,8 +142,6 @@ enum dcn31_clk_src_array_id {
*/
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
#define BASE(seg) BASE_INNER(seg)
@@ -176,6 +174,9 @@ enum dcn31_clk_src_array_id {
.reg_name[id] = BASE(reg ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define DCCG_SRII(reg_name, block, id)\
.block ## _ ## reg_name[id] = BASE(reg ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## reg_name
@@ -906,6 +907,10 @@ static const struct dc_debug_options debug_defaults_diags = {
};
static const struct dc_panel_config panel_config_defaults = {
+ .psr = {
+ .disable_psr = false,
+ .disallow_psrsu = false,
+ },
.ilr = {
.optimize_edp_link_rate = true,
},
@@ -1646,7 +1651,7 @@ static int dcn316_populate_dml_pipes_from_context(
const int max_usable_det = context->bw_ctx.dml.ip.config_return_buffer_size_in_kbytes - DCN3_16_MIN_COMPBUF_SIZE_KB;
DC_FP_START();
- dcn20_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
+ dcn31x_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
DC_FP_END();
for (i = 0, pipe_cnt = 0; i < dc->res_pool->pipe_count; i++) {
@@ -1665,7 +1670,6 @@ static int dcn316_populate_dml_pipes_from_context(
pipes[pipe_cnt].pipe.src.immediate_flip = true;
pipes[pipe_cnt].pipe.src.unbounded_req_mode = false;
- pipes[pipe_cnt].pipe.src.gpuvm = true;
pipes[pipe_cnt].pipe.dest.vfront_porch = timing->v_front_porch;
pipes[pipe_cnt].pipe.src.dcc_rate = 3;
pipes[pipe_cnt].dout.dsc_input_bpc = 0;
@@ -1781,6 +1785,8 @@ static bool dcn316_resource_construct(
dc->caps.max_slave_rgb_planes = 2;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
+ if (dc->config.forceHBR2CP2520)
+ dc->caps.force_dp_tps4_for_cp2520 = false;
dc->caps.dp_hpo = true;
dc->caps.dp_hdmi21_pcon_support = true;
dc->caps.edp_dsc_support = true;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dccg.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dccg.c
index df4f251..e4472c6 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dccg.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dccg.c
@@ -225,11 +225,7 @@ static void dccg32_set_dtbclk_dto(
} else {
REG_UPDATE_2(OTG_PIXEL_RATE_CNTL[params->otg_inst],
DTBCLK_DTO_ENABLE[params->otg_inst], 0,
- PIPE_DTO_SRC_SEL[params->otg_inst], 1);
- if (params->is_hdmi)
- REG_UPDATE(OTG_PIXEL_RATE_CNTL[params->otg_inst],
- PIPE_DTO_SRC_SEL[params->otg_inst], 0);
-
+ PIPE_DTO_SRC_SEL[params->otg_inst], params->is_hdmi ? 0 : 1);
REG_WRITE(DTBCLK_DTO_MODULO[params->otg_inst], 0);
REG_WRITE(DTBCLK_DTO_PHASE[params->otg_inst], 0);
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.c
index 9fbb723..5947c2c 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.c
@@ -41,6 +41,10 @@
#define FN(reg_name, field_name) \
hubbub2->shifts->field_name, hubbub2->masks->field_name
+/**
+ * @DCN32_CRB_SEGMENT_SIZE_KB: Maximum Configurable Return Buffer size for
+ * DCN32
+ */
#define DCN32_CRB_SEGMENT_SIZE_KB 64
static void dcn32_init_crb(struct hubbub *hubbub)
@@ -68,6 +72,23 @@ static void dcn32_init_crb(struct hubbub *hubbub)
REG_UPDATE(DCHUBBUB_DEBUG_CTRL_0, DET_DEPTH, 0x47F);
}
+void hubbub32_set_request_limit(struct hubbub *hubbub, int memory_channel_count, int words_per_channel)
+{
+ struct dcn20_hubbub *hubbub2 = TO_DCN20_HUBBUB(hubbub);
+
+ uint32_t request_limit = 3 * memory_channel_count * words_per_channel / 4;
+
+ ASSERT((request_limit & (~0xFFF)) == 0); //field is only 24 bits long
+ ASSERT(request_limit > 0); //field is only 24 bits long
+
+ if (request_limit > 0xFFF)
+ request_limit = 0xFFF;
+
+ if (request_limit > 0)
+ REG_UPDATE(SDPIF_REQUEST_RATE_LIMIT, SDPIF_REQUEST_RATE_LIMIT, request_limit);
+}
+
+
void dcn32_program_det_size(struct hubbub *hubbub, int hubp_inst, unsigned int det_buffer_size_in_kbyte)
{
struct dcn20_hubbub *hubbub2 = TO_DCN20_HUBBUB(hubbub);
@@ -945,6 +966,7 @@ static const struct hubbub_funcs hubbub32_funcs = {
.init_crb = dcn32_init_crb,
.hubbub_read_state = hubbub2_read_state,
.force_usr_retraining_allow = hubbub32_force_usr_retraining_allow,
+ .set_request_limit = hubbub32_set_request_limit
};
void hubbub32_construct(struct dcn20_hubbub *hubbub2,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.h b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.h
index cda94e0..786f9ce 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hubbub.h
@@ -82,7 +82,8 @@
SR(DCN_VM_FAULT_ADDR_MSB),\
SR(DCN_VM_FAULT_ADDR_LSB),\
SR(DCN_VM_FAULT_CNTL),\
- SR(DCN_VM_FAULT_STATUS)
+ SR(DCN_VM_FAULT_STATUS),\
+ SR(SDPIF_REQUEST_RATE_LIMIT)
#define HUBBUB_MASK_SH_LIST_DCN32(mask_sh)\
HUBBUB_SF(DCHUBBUB_GLOBAL_TIMER_CNTL, DCHUBBUB_GLOBAL_TIMER_ENABLE, mask_sh), \
@@ -159,7 +160,8 @@
HUBBUB_SF(DCN_VM_FAULT_STATUS, DCN_VM_ERROR_VMID, mask_sh), \
HUBBUB_SF(DCN_VM_FAULT_STATUS, DCN_VM_ERROR_TABLE_LEVEL, mask_sh), \
HUBBUB_SF(DCN_VM_FAULT_STATUS, DCN_VM_ERROR_PIPE, mask_sh), \
- HUBBUB_SF(DCN_VM_FAULT_STATUS, DCN_VM_ERROR_INTERRUPT_STATUS, mask_sh)
+ HUBBUB_SF(DCN_VM_FAULT_STATUS, DCN_VM_ERROR_INTERRUPT_STATUS, mask_sh),\
+ HUBBUB_SF(SDPIF_REQUEST_RATE_LIMIT, SDPIF_REQUEST_RATE_LIMIT, mask_sh)
bool hubbub32_program_urgent_watermarks(
struct hubbub *hubbub,
@@ -200,4 +202,6 @@ void hubbub32_construct(struct dcn20_hubbub *hubbub2,
int pixel_chunk_size_kb,
int config_return_buffer_size_kb);
+void hubbub32_set_request_limit(struct hubbub *hubbub, int umc_count, int words_per_umc);
+
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c
index d0b46a3..b8767be 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c
@@ -227,8 +227,13 @@ static uint32_t dcn32_calculate_cab_allocation(struct dc *dc, struct dc_state *c
for (i = 0; i < dc->res_pool->pipe_count; i++) {
struct pipe_ctx *pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+ /* If PSR is supported on an eDP panel that's connected, but that panel is
+ * not in PSR at the time of trying to enter MALL SS, we have to include it
+ * in the static screen CAB calculation
+ */
if (!pipe->stream || !pipe->plane_state ||
- pipe->stream->link->psr_settings.psr_version != DC_PSR_VERSION_UNSUPPORTED ||
+ (pipe->stream->link->psr_settings.psr_version != DC_PSR_VERSION_UNSUPPORTED &&
+ pipe->stream->link->psr_settings.psr_allow_active) ||
pipe->stream->mall_stream_config.type == SUBVP_PHANTOM)
continue;
@@ -257,11 +262,11 @@ static uint32_t dcn32_calculate_cab_allocation(struct dc *dc, struct dc_state *c
num_mblks = ((mall_alloc_width_blk_aligned + mblk_width - 1) / mblk_width) *
((mall_alloc_height_blk_aligned + mblk_height - 1) / mblk_height);
- /* For DCC:
- * meta_num_mblk = CEILING(full_mblk_width_ub_l*full_mblk_height_ub_l*Bpe/256/mblk_bytes, 1)
+ /*For DCC:
+ * meta_num_mblk = CEILING(meta_pitch*full_vp_height*Bpe/256/mblk_bytes, 1)
*/
if (pipe->plane_state->dcc.enable)
- num_mblks += (mall_alloc_width_blk_aligned * mall_alloc_width_blk_aligned * bytes_per_pixel +
+ num_mblks += (pipe->plane_state->dcc.meta_pitch * pipe->plane_res.scl_data.viewport.height * bytes_per_pixel +
(256 * DCN3_2_MALL_MBLK_SIZE_BYTES) - 1) / (256 * DCN3_2_MALL_MBLK_SIZE_BYTES);
bytes_in_mall = num_mblks * DCN3_2_MALL_MBLK_SIZE_BYTES;
@@ -311,8 +316,8 @@ static uint32_t dcn32_calculate_cab_allocation(struct dc *dc, struct dc_state *c
cache_lines_used += (((cursor_size + DCN3_2_MALL_MBLK_SIZE_BYTES - 1) /
DCN3_2_MALL_MBLK_SIZE_BYTES) * DCN3_2_MALL_MBLK_SIZE_BYTES) /
dc->caps.cache_line_size + 2;
+ break;
}
- break;
}
}
@@ -698,11 +703,7 @@ void dcn32_subvp_update_force_pstate(struct dc *dc, struct dc_state *context)
for (i = 0; i < dc->res_pool->pipe_count; i++) {
struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
- // For SubVP + DRR, also force disallow on the DRR pipe
- // (We will force allow in the DMUB sequence -- some DRR timings by default won't allow P-State so we have
- // to force once the vblank is stretched).
- if (pipe->stream && pipe->plane_state && (pipe->stream->mall_stream_config.type == SUBVP_MAIN ||
- (pipe->stream->mall_stream_config.type == SUBVP_NONE && pipe->stream->ignore_msa_timing_param))) {
+ if (pipe->stream && pipe->plane_state && (pipe->stream->mall_stream_config.type == SUBVP_MAIN)) {
struct hubp *hubp = pipe->plane_res.hubp;
if (hubp && hubp->funcs->hubp_update_force_pstate_disallow)
@@ -780,6 +781,10 @@ void dcn32_program_mall_pipe_config(struct dc *dc, struct dc_state *context)
if (hws && hws->funcs.update_mall_sel)
hws->funcs.update_mall_sel(dc, context);
+ //update subvp force pstate
+ if (hws && hws->funcs.subvp_update_force_pstate)
+ dc->hwseq->funcs.subvp_update_force_pstate(dc, context);
+
// Program FORCE_ONE_ROW_FOR_FRAME and CURSOR_REQ_MODE for main subvp pipes
for (i = 0; i < dc->res_pool->pipe_count; i++) {
struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
@@ -980,15 +985,14 @@ void dcn32_init_hw(struct dc *dc)
if (dc->res_pool->hubbub->funcs->init_crb)
dc->res_pool->hubbub->funcs->init_crb(dc->res_pool->hubbub);
+ if (dc->res_pool->hubbub->funcs->set_request_limit && dc->config.sdpif_request_limit_words_per_umc > 0)
+ dc->res_pool->hubbub->funcs->set_request_limit(dc->res_pool->hubbub, dc->ctx->dc_bios->vram_info.num_chans, dc->config.sdpif_request_limit_words_per_umc);
+
// Get DMCUB capabilities
if (dc->ctx->dmub_srv) {
dc_dmub_srv_query_caps_cmd(dc->ctx->dmub_srv->dmub);
dc->caps.dmub_caps.psr = dc->ctx->dmub_srv->dmub->feature_caps.psr;
}
-
- /* Enable support for ODM and windowed MPO if policy flag is set */
- if (dc->debug.enable_single_display_2to1_odm_policy)
- dc->config.enable_windowed_mpo_odm = true;
}
static int calc_mpc_flow_ctrl_cnt(const struct dc_stream_state *stream,
@@ -1180,7 +1184,7 @@ unsigned int dcn32_calculate_dccg_k1_k2_values(struct pipe_ctx *pipe_ctx, unsign
*k2_div = PIXEL_RATE_DIV_BY_2;
else
*k2_div = PIXEL_RATE_DIV_BY_4;
- } else if (dc_is_dp_signal(pipe_ctx->stream->signal)) {
+ } else if (dc_is_dp_signal(pipe_ctx->stream->signal) || dc_is_virtual_signal(pipe_ctx->stream->signal)) {
if (two_pix_per_container) {
*k1_div = PIXEL_RATE_DIV_BY_1;
*k2_div = PIXEL_RATE_DIV_BY_2;
@@ -1365,6 +1369,33 @@ void dcn32_update_phantom_vp_position(struct dc *dc,
}
}
+/* Treat the phantom pipe as if it needs to be fully enabled.
+ * If the pipe was previously in use but not phantom, it would
+ * have been disabled earlier in the sequence so we need to run
+ * the full enable sequence.
+ */
+void dcn32_apply_update_flags_for_phantom(struct pipe_ctx *phantom_pipe)
+{
+ phantom_pipe->update_flags.raw = 0;
+ if (phantom_pipe->stream && phantom_pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ if (phantom_pipe->stream && phantom_pipe->plane_state) {
+ phantom_pipe->update_flags.bits.enable = 1;
+ phantom_pipe->update_flags.bits.mpcc = 1;
+ phantom_pipe->update_flags.bits.dppclk = 1;
+ phantom_pipe->update_flags.bits.hubp_interdependent = 1;
+ phantom_pipe->update_flags.bits.hubp_rq_dlg_ttu = 1;
+ phantom_pipe->update_flags.bits.gamut_remap = 1;
+ phantom_pipe->update_flags.bits.scaler = 1;
+ phantom_pipe->update_flags.bits.viewport = 1;
+ phantom_pipe->update_flags.bits.det_size = 1;
+ if (!phantom_pipe->top_pipe && !phantom_pipe->prev_odm_pipe) {
+ phantom_pipe->update_flags.bits.odm = 1;
+ phantom_pipe->update_flags.bits.global_sync = 1;
+ }
+ }
+ }
+}
+
bool dcn32_dsc_pg_status(
struct dce_hwseq *hws,
unsigned int dsc_inst)
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.h b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.h
index ac3657a..7de36529 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.h
@@ -92,6 +92,8 @@ void dcn32_update_phantom_vp_position(struct dc *dc,
struct dc_state *context,
struct pipe_ctx *phantom_pipe);
+void dcn32_apply_update_flags_for_phantom(struct pipe_ctx *phantom_pipe);
+
bool dcn32_dsc_pg_status(
struct dce_hwseq *hws,
unsigned int dsc_inst);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_init.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_init.c
index 45a949b..dc46494 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_init.c
@@ -110,6 +110,7 @@ static const struct hw_sequencer_funcs dcn32_funcs = {
.update_visual_confirm_color = dcn20_update_visual_confirm_color,
.update_phantom_vp_position = dcn32_update_phantom_vp_position,
.update_dsc_pg = dcn32_update_dsc_pg,
+ .apply_update_flags_for_phantom = dcn32_apply_update_flags_for_phantom,
};
static const struct hwseq_private_funcs dcn32_private_funcs = {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mmhubbub.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mmhubbub.c
index 41b0baf..c3b089b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mmhubbub.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mmhubbub.c
@@ -211,7 +211,7 @@ static void mmhubbub32_config_mcif_arb(struct mcif_wb *mcif_wb,
REG_UPDATE(MCIF_WB_ARBITRATION_CONTROL, MCIF_WB_CLIENT_ARBITRATION_SLICE, params->arbitration_slice);
}
-const struct mcif_wb_funcs dcn32_mmhubbub_funcs = {
+static const struct mcif_wb_funcs dcn32_mmhubbub_funcs = {
.warmup_mcif = mmhubbub32_warmup_mcif,
.enable_mcif = mmhubbub2_enable_mcif,
.disable_mcif = mmhubbub2_disable_mcif,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c
index 4edd065..206a5dd 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c
@@ -982,7 +982,7 @@ static bool mpc32_program_3dlut(
return true;
}
-const struct mpc_funcs dcn32_mpc_funcs = {
+static const struct mpc_funcs dcn32_mpc_funcs = {
.read_mpcc_state = mpc1_read_mpcc_state,
.insert_plane = mpc1_insert_plane,
.remove_mpcc = mpc1_remove_mpcc,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_optc.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_optc.c
index 2b33eeb..2ee7989 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_optc.c
@@ -167,6 +167,13 @@ static void optc32_phantom_crtc_post_enable(struct timing_generator *optc)
REG_WAIT(OTG_CLOCK_CONTROL, OTG_BUSY, 0, 1, 100000);
}
+static void optc32_disable_phantom_otg(struct timing_generator *optc)
+{
+ struct optc *optc1 = DCN10TG_FROM_TG(optc);
+
+ REG_UPDATE(OTG_CONTROL, OTG_MASTER_EN, 0);
+}
+
static void optc32_set_odm_bypass(struct timing_generator *optc,
const struct dc_crtc_timing *dc_crtc_timing)
{
@@ -260,6 +267,7 @@ static struct timing_generator_funcs dcn32_tg_funcs = {
.enable_crtc = optc32_enable_crtc,
.disable_crtc = optc32_disable_crtc,
.phantom_crtc_post_enable = optc32_phantom_crtc_post_enable,
+ .disable_phantom_crtc = optc32_disable_phantom_otg,
/* used by enable_timing_synchronization. Not need for FPGA */
.is_counter_moving = optc1_is_counter_moving,
.get_position = optc1_get_position,
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c
index d1598e3..e4dbc83 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c
@@ -106,8 +106,6 @@ enum dcn32_clk_src_array_id {
*/
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) ctx->dcn_reg_offsets[seg]
#define BASE(seg) BASE_INNER(seg)
@@ -167,6 +165,9 @@ enum dcn32_clk_src_array_id {
REG_STRUCT.reg_name[id] = BASE(reg ## block ## id ## _ ## temp_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## temp_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define DCCG_SRII(reg_name, block, id)\
REG_STRUCT.block ## _ ## reg_name[id] = BASE(reg ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## reg_name
@@ -722,7 +723,7 @@ static const struct dc_debug_options debug_defaults_drv = {
/* Must match enable_single_display_2to1_odm_policy to support dynamic ODM transitions*/
.enable_double_buffered_dsc_pg_support = true,
.enable_dp_dig_pixel_rate_div_policy = 1,
- .allow_sw_cursor_fallback = false,
+ .allow_sw_cursor_fallback = false, // Linux can't do SW cursor "fallback"
.alloc_extra_way_for_cursor = true,
.min_prefetch_in_strobe_ns = 60000, // 60us
};
@@ -830,6 +831,7 @@ static struct clock_source *dcn32_clock_source_create(
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1679,7 +1681,7 @@ static void dcn32_enable_phantom_plane(struct dc *dc,
/* Shadow pipe has small viewport. */
phantom_plane->clip_rect.y = 0;
- phantom_plane->clip_rect.height = phantom_stream->timing.v_addressable;
+ phantom_plane->clip_rect.height = phantom_stream->src.height;
phantom_plane->is_phantom = true;
@@ -1719,8 +1721,29 @@ static struct dc_stream_state *dcn32_enable_phantom_stream(struct dc *dc,
return phantom_stream;
}
+void dcn32_retain_phantom_pipes(struct dc *dc, struct dc_state *context)
+{
+ int i;
+ struct dc_plane_state *phantom_plane = NULL;
+ struct dc_stream_state *phantom_stream = NULL;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+
+ if (!pipe->top_pipe && !pipe->prev_odm_pipe &&
+ pipe->plane_state && pipe->stream &&
+ pipe->stream->mall_stream_config.type == SUBVP_PHANTOM) {
+ phantom_plane = pipe->plane_state;
+ phantom_stream = pipe->stream;
+
+ dc_plane_state_retain(phantom_plane);
+ dc_stream_retain(phantom_stream);
+ }
+ }
+}
+
// return true if removed piped from ctx, false otherwise
-bool dcn32_remove_phantom_pipes(struct dc *dc, struct dc_state *context)
+bool dcn32_remove_phantom_pipes(struct dc *dc, struct dc_state *context, bool fast_update)
{
int i;
bool removed_pipe = false;
@@ -1747,14 +1770,23 @@ bool dcn32_remove_phantom_pipes(struct dc *dc, struct dc_state *context)
removed_pipe = true;
}
- // Clear all phantom stream info
- if (pipe->stream) {
- pipe->stream->mall_stream_config.type = SUBVP_NONE;
- pipe->stream->mall_stream_config.paired_stream = NULL;
- }
+ /* For non-full updates, a shallow copy of the current state
+ * is created. In this case we don't want to erase the current
+ * state (there can be 2 HIRQL threads, one in flip, and one in
+ * checkMPO) that can cause a race condition.
+ *
+ * This is just a workaround, needs a proper fix.
+ */
+ if (!fast_update) {
+ // Clear all phantom stream info
+ if (pipe->stream) {
+ pipe->stream->mall_stream_config.type = SUBVP_NONE;
+ pipe->stream->mall_stream_config.paired_stream = NULL;
+ }
- if (pipe->plane_state) {
- pipe->plane_state->is_phantom = false;
+ if (pipe->plane_state) {
+ pipe->plane_state->is_phantom = false;
+ }
}
}
return removed_pipe;
@@ -1901,7 +1933,7 @@ int dcn32_populate_dml_pipes_from_context(
pipes[pipe_cnt].pipe.dest.odm_combine_policy = dm_odm_combine_policy_dal;
if (context->stream_count == 1 &&
- context->stream_status[0].plane_count <= 1 &&
+ context->stream_status[0].plane_count == 1 &&
!dc_is_hdmi_signal(res_ctx->pipe_ctx[i].stream->signal) &&
is_h_timing_divisible_by_2(res_ctx->pipe_ctx[i].stream) &&
pipe->stream->timing.pix_clk_100hz * 100 > DCN3_2_VMIN_DISPCLK_HZ &&
@@ -1919,30 +1951,36 @@ int dcn32_populate_dml_pipes_from_context(
timing = &pipe->stream->timing;
pipes[pipe_cnt].pipe.src.gpuvm = true;
- pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_luma = 0;
- pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_chroma = 0;
+ DC_FP_START();
+ dcn32_zero_pipe_dcc_fraction(pipes, pipe_cnt);
+ DC_FP_END();
pipes[pipe_cnt].pipe.dest.vfront_porch = timing->v_front_porch;
pipes[pipe_cnt].pipe.src.gpuvm_min_page_size_kbytes = 256; // according to spreadsheet
pipes[pipe_cnt].pipe.src.unbounded_req_mode = false;
pipes[pipe_cnt].pipe.scale_ratio_depth.lb_depth = dm_lb_19;
- switch (pipe->stream->mall_stream_config.type) {
- case SUBVP_MAIN:
- pipes[pipe_cnt].pipe.src.use_mall_for_pstate_change = dm_use_mall_pstate_change_sub_viewport;
- subvp_in_use = true;
- break;
- case SUBVP_PHANTOM:
- pipes[pipe_cnt].pipe.src.use_mall_for_pstate_change = dm_use_mall_pstate_change_phantom_pipe;
- pipes[pipe_cnt].pipe.src.use_mall_for_static_screen = dm_use_mall_static_screen_disable;
- // Disallow unbounded req for SubVP according to DCHUB programming guide
- pipes[pipe_cnt].pipe.src.unbounded_req_mode = false;
- break;
- case SUBVP_NONE:
- pipes[pipe_cnt].pipe.src.use_mall_for_pstate_change = dm_use_mall_pstate_change_disable;
- pipes[pipe_cnt].pipe.src.use_mall_for_static_screen = dm_use_mall_static_screen_disable;
- break;
- default:
- break;
+ /* Only populate DML input with subvp info for full updates.
+ * This is just a workaround -- needs a proper fix.
+ */
+ if (!fast_validate) {
+ switch (pipe->stream->mall_stream_config.type) {
+ case SUBVP_MAIN:
+ pipes[pipe_cnt].pipe.src.use_mall_for_pstate_change = dm_use_mall_pstate_change_sub_viewport;
+ subvp_in_use = true;
+ break;
+ case SUBVP_PHANTOM:
+ pipes[pipe_cnt].pipe.src.use_mall_for_pstate_change = dm_use_mall_pstate_change_phantom_pipe;
+ pipes[pipe_cnt].pipe.src.use_mall_for_static_screen = dm_use_mall_static_screen_disable;
+ // Disallow unbounded req for SubVP according to DCHUB programming guide
+ pipes[pipe_cnt].pipe.src.unbounded_req_mode = false;
+ break;
+ case SUBVP_NONE:
+ pipes[pipe_cnt].pipe.src.use_mall_for_pstate_change = dm_use_mall_pstate_change_disable;
+ pipes[pipe_cnt].pipe.src.use_mall_for_static_screen = dm_use_mall_static_screen_disable;
+ break;
+ default:
+ break;
+ }
}
pipes[pipe_cnt].dout.dsc_input_bpc = 0;
@@ -2030,6 +2068,9 @@ static struct resource_funcs dcn32_res_pool_funcs = {
.update_soc_for_wm_a = dcn30_update_soc_for_wm_a,
.add_phantom_pipes = dcn32_add_phantom_pipes,
.remove_phantom_pipes = dcn32_remove_phantom_pipes,
+ .retain_phantom_pipes = dcn32_retain_phantom_pipes,
+ .save_mall_state = dcn32_save_mall_state,
+ .restore_mall_state = dcn32_restore_mall_state,
};
@@ -2116,16 +2157,20 @@ static bool dcn32_resource_construct(
dc->caps.cache_num_ways = 16;
dc->caps.max_cab_allocation_bytes = 67108864; // 64MB = 1024 * 1024 * 64
dc->caps.subvp_fw_processing_delay_us = 15;
+ dc->caps.subvp_drr_max_vblank_margin_us = 40;
dc->caps.subvp_prefetch_end_to_mall_start_us = 15;
dc->caps.subvp_swath_height_margin_lines = 16;
dc->caps.subvp_pstate_allow_width_us = 20;
dc->caps.subvp_vertical_int_margin_us = 30;
+ dc->caps.subvp_drr_vblank_start_margin_us = 100; // 100us margin
dc->caps.max_slave_planes = 2;
dc->caps.max_slave_yuv_planes = 2;
dc->caps.max_slave_rgb_planes = 2;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
+ if (dc->config.forceHBR2CP2520)
+ dc->caps.force_dp_tps4_for_cp2520 = false;
dc->caps.dp_hpo = true;
dc->caps.dp_hdmi21_pcon_support = true;
dc->caps.edp_dsc_support = true;
@@ -2409,6 +2454,9 @@ static bool dcn32_resource_construct(
pool->base.oem_device = NULL;
}
+ if (ASICREV_IS_GC_11_0_3(dc->ctx->asic_id.hw_internal_rev) && (dc->config.sdpif_request_limit_words_per_umc == 0))
+ dc->config.sdpif_request_limit_words_per_umc = 16;
+
DC_FP_END();
return true;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h
index f76120e..13fbc57 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h
@@ -45,17 +45,6 @@
extern struct _vcs_dpi_ip_params_st dcn3_2_ip;
extern struct _vcs_dpi_soc_bounding_box_st dcn3_2_soc;
-/* Temp struct used to save and restore MALL config
- * during validation.
- *
- * TODO: Move MALL config into dc_state instead of stream struct
- * to avoid needing to save/restore.
- */
-struct mall_temp_config {
- struct mall_stream_config mall_stream_config[MAX_PIPES];
- bool is_phantom_plane[MAX_PIPES];
-};
-
struct dcn32_resource_pool {
struct resource_pool base;
};
@@ -81,6 +70,9 @@ bool dcn32_release_post_bldn_3dlut(
struct dc_transfer_func **shaper);
bool dcn32_remove_phantom_pipes(struct dc *dc,
+ struct dc_state *context, bool fast_update);
+
+void dcn32_retain_phantom_pipes(struct dc *dc,
struct dc_state *context);
void dcn32_add_phantom_pipes(struct dc *dc,
@@ -1244,7 +1236,8 @@ void dcn32_restore_mall_state(struct dc *dc,
SR(DCHUBBUB_ARB_FCLK_PSTATE_CHANGE_WATERMARK_C), \
SR(DCHUBBUB_ARB_FCLK_PSTATE_CHANGE_WATERMARK_D), \
SR(DCN_VM_FAULT_ADDR_MSB), SR(DCN_VM_FAULT_ADDR_LSB), \
- SR(DCN_VM_FAULT_CNTL), SR(DCN_VM_FAULT_STATUS) \
+ SR(DCN_VM_FAULT_CNTL), SR(DCN_VM_FAULT_STATUS), \
+ SR(SDPIF_REQUEST_RATE_LIMIT) \
)
/* DCCG */
diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c
index fa37788..783935c 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c
@@ -97,14 +97,14 @@ uint32_t dcn32_helper_calculate_num_ways_for_subvp(struct dc *dc, struct dc_stat
* FLOOR(vp_x_start, blk_width)
*/
full_vp_width_blk_aligned = ((pipe->plane_res.scl_data.viewport.x +
- pipe->plane_res.scl_data.viewport.width + mblk_width - 1) / mblk_width * mblk_width) +
+ pipe->plane_res.scl_data.viewport.width + mblk_width - 1) / mblk_width * mblk_width) -
(pipe->plane_res.scl_data.viewport.x / mblk_width * mblk_width);
/* full_vp_height_blk_aligned = FLOOR(vp_y_start + full_vp_height + blk_height - 1, blk_height) -
* FLOOR(vp_y_start, blk_height)
*/
full_vp_height_blk_aligned = ((pipe->plane_res.scl_data.viewport.y +
- full_vp_height + mblk_height - 1) / mblk_height * mblk_height) +
+ full_vp_height + mblk_height - 1) / mblk_height * mblk_height) -
(pipe->plane_res.scl_data.viewport.y / mblk_height * mblk_height);
/* mall_alloc_width_blk_aligned_l/c = full_vp_width_blk_aligned_l/c */
@@ -121,14 +121,19 @@ uint32_t dcn32_helper_calculate_num_ways_for_subvp(struct dc *dc, struct dc_stat
*/
num_mblks = ((mall_alloc_width_blk_aligned + mblk_width - 1) / mblk_width) *
((mall_alloc_height_blk_aligned + mblk_height - 1) / mblk_height);
+
+ /*For DCC:
+ * meta_num_mblk = CEILING(meta_pitch*full_vp_height*Bpe/256/mblk_bytes, 1)
+ */
+ if (pipe->plane_state->dcc.enable)
+ num_mblks += (pipe->plane_state->dcc.meta_pitch * pipe->plane_res.scl_data.viewport.height * bytes_per_pixel +
+ (256 * DCN3_2_MALL_MBLK_SIZE_BYTES) - 1) / (256 * DCN3_2_MALL_MBLK_SIZE_BYTES);
+
bytes_in_mall = num_mblks * DCN3_2_MALL_MBLK_SIZE_BYTES;
// cache lines used is total bytes / cache_line size. Add +2 for worst case alignment
// (MALL is 64-byte aligned)
cache_lines_per_plane = bytes_in_mall / dc->caps.cache_line_size + 2;
- /* For DCC divide by 256 */
- if (pipe->plane_state->dcc.enable)
- cache_lines_per_plane = cache_lines_per_plane + (cache_lines_per_plane / 256) + 1;
cache_lines_used += cache_lines_per_plane;
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c b/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c
index 6292ac51..d1f36df 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c
@@ -109,8 +109,6 @@ enum dcn321_clk_src_array_id {
*/
/* DCN */
-/* TODO awful hack. fixup dcn20_dwb.h */
-#undef BASE_INNER
#define BASE_INNER(seg) ctx->dcn_reg_offsets[seg]
#define BASE(seg) BASE_INNER(seg)
@@ -174,6 +172,9 @@ enum dcn321_clk_src_array_id {
REG_STRUCT.block ## _ ## reg_name[id] = BASE(reg ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
reg ## block ## id ## _ ## reg_name
+#define SF_DWB2(reg_name, block, id, field_name, post_fix) \
+ .field_name = reg_name ## __ ## field_name ## post_fix
+
#define VUPDATE_SRII(reg_name, block, id)\
REG_STRUCT.reg_name[id] = BASE(reg ## reg_name ## _ ## block ## id ## _BASE_IDX) + \
reg ## reg_name ## _ ## block ## id
@@ -720,7 +721,7 @@ static const struct dc_debug_options debug_defaults_drv = {
/*must match enable_single_display_2to1_odm_policy to support dynamic ODM transitions*/
.enable_double_buffered_dsc_pg_support = true,
.enable_dp_dig_pixel_rate_div_policy = 1,
- .allow_sw_cursor_fallback = false,
+ .allow_sw_cursor_fallback = false, // Linux can't do SW cursor "fallback"
.alloc_extra_way_for_cursor = true,
.min_prefetch_in_strobe_ns = 60000, // 60us
};
@@ -742,7 +743,7 @@ static const struct dc_debug_options debug_defaults_diags = {
.dmub_command_table = true,
.enable_tri_buf = true,
.use_max_lb = true,
- .force_disable_subvp = true
+ .force_disable_subvp = true,
};
@@ -829,6 +830,7 @@ static struct clock_source *dcn321_clock_source_create(
return &clk_src->base;
}
+ kfree(clk_src);
BREAK_TO_DEBUGGER();
return NULL;
}
@@ -1619,6 +1621,9 @@ static struct resource_funcs dcn321_res_pool_funcs = {
.update_soc_for_wm_a = dcn30_update_soc_for_wm_a,
.add_phantom_pipes = dcn32_add_phantom_pipes,
.remove_phantom_pipes = dcn32_remove_phantom_pipes,
+ .retain_phantom_pipes = dcn32_retain_phantom_pipes,
+ .save_mall_state = dcn32_save_mall_state,
+ .restore_mall_state = dcn32_restore_mall_state,
};
@@ -1704,10 +1709,12 @@ static bool dcn321_resource_construct(
dc->caps.cache_num_ways = 16;
dc->caps.max_cab_allocation_bytes = 33554432; // 32MB = 1024 * 1024 * 32
dc->caps.subvp_fw_processing_delay_us = 15;
+ dc->caps.subvp_drr_max_vblank_margin_us = 40;
dc->caps.subvp_prefetch_end_to_mall_start_us = 15;
dc->caps.subvp_swath_height_margin_lines = 16;
dc->caps.subvp_pstate_allow_width_us = 20;
dc->caps.subvp_vertical_int_margin_us = 30;
+ dc->caps.subvp_drr_vblank_start_margin_us = 100; // 100us margin
dc->caps.max_slave_planes = 1;
dc->caps.max_slave_yuv_planes = 1;
dc->caps.max_slave_rgb_planes = 1;
diff --git a/drivers/gpu/drm/amd/display/dc/dm_helpers.h b/drivers/gpu/drm/amd/display/dc/dm_helpers.h
index e3e5c39..af1c50e 100644
--- a/drivers/gpu/drm/amd/display/dc/dm_helpers.h
+++ b/drivers/gpu/drm/amd/display/dc/dm_helpers.h
@@ -116,6 +116,11 @@ bool dm_helpers_dp_mst_start_top_mgr(
bool dm_helpers_dp_mst_stop_top_mgr(
struct dc_context *ctx,
struct dc_link *link);
+
+void dm_helpers_dp_mst_update_branch_bandwidth(
+ struct dc_context *ctx,
+ struct dc_link *link);
+
/**
* OS specific aux read callback.
*/
diff --git a/drivers/gpu/drm/amd/display/dc/dml/Makefile b/drivers/gpu/drm/amd/display/dc/dml/Makefile
index ca7d240..0ecea87 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/Makefile
+++ b/drivers/gpu/drm/amd/display/dc/dml/Makefile
@@ -33,6 +33,10 @@
dml_ccflags := -mhard-float -maltivec
endif
+ifdef CONFIG_ARM64
+dml_rcflags := -mgeneral-regs-only
+endif
+
ifdef CONFIG_CC_IS_GCC
ifneq ($(call gcc-min-version, 70100),y)
IS_OLD_GCC = 1
@@ -55,8 +59,6 @@
endif
CFLAGS_$(AMDDALPATH)/dc/dml/display_mode_lib.o := $(dml_ccflags)
-
-ifdef CONFIG_DRM_AMD_DC_DCN
CFLAGS_$(AMDDALPATH)/dc/dml/display_mode_vba.o := $(dml_ccflags)
CFLAGS_$(AMDDALPATH)/dc/dml/dcn10/dcn10_fpu.o := $(dml_ccflags)
CFLAGS_$(AMDDALPATH)/dc/dml/dcn20/dcn20_fpu.o := $(dml_ccflags)
@@ -88,7 +90,6 @@
CFLAGS_$(AMDDALPATH)/dc/dml/calcs/dcn_calc_auto.o := $(dml_ccflags)
CFLAGS_$(AMDDALPATH)/dc/dml/calcs/dcn_calc_math.o := $(dml_ccflags) -Wno-tautological-compare
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/display_mode_vba.o := $(dml_rcflags)
-CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn2x/dcn2x.o := $(dml_rcflags)
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn20/display_mode_vba_20.o := $(dml_rcflags)
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn20/display_rq_dlg_calc_20.o := $(dml_rcflags)
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn20/display_mode_vba_20v2.o := $(dml_rcflags)
@@ -105,7 +106,18 @@
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn301/dcn301_fpu.o := $(dml_rcflags)
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/display_mode_lib.o := $(dml_rcflags)
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dsc/rc_calc_fpu.o := $(dml_rcflags)
-endif
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn10/dcn10_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn20/dcn20_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn314/display_mode_vba_314.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn314/display_rq_dlg_calc_314.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn314/dcn314_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn30/dcn30_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn32/dcn32_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn321/dcn321_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn31/dcn31_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn302/dcn302_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn303/dcn303_fpu.o := $(dml_rcflags)
+CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/calcs/dcn_calc_math.o := $(dml_rcflags)
CFLAGS_$(AMDDALPATH)/dc/dml/dml1_display_rq_dlg_calc.o := $(dml_ccflags)
CFLAGS_$(AMDDALPATH)/dc/dml/display_rq_dlg_helpers.o := $(dml_ccflags)
CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dml1_display_rq_dlg_calc.o := $(dml_rcflags)
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dc_features.h b/drivers/gpu/drm/amd/display/dc/dml/dc_features.h
index 74e8673..2cbdd75 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dc_features.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/dc_features.h
@@ -29,6 +29,13 @@
#define DC__PRESENT 1
#define DC__PRESENT__1 1
#define DC__NUM_DPP 4
+
+/**
+ * @DC__VOLTAGE_STATES:
+ *
+ * Define the maximum amount of states supported by the ASIC. Every ASIC has a
+ * specific number of states; this macro defines the maximum number of states.
+ */
#define DC__VOLTAGE_STATES 20
#define DC__NUM_DPP__4 1
#define DC__NUM_DPP__0_PRESENT 1
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.c
index 99644d8..c5e8419 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.c
@@ -27,6 +27,8 @@
#include "dcn10/dcn10_resource.h"
#include "dcn10_fpu.h"
+#include "resource.h"
+#include "amdgpu_dm/dc_fpu.h"
/**
* DOC: DCN10 FPU manipulation Overview
@@ -121,3 +123,37 @@ struct _vcs_dpi_soc_bounding_box_st dcn1_0_soc = {
.writeback_dram_clock_change_latency_us = 23.0,
.return_bus_width_bytes = 64,
};
+
+void dcn10_resource_construct_fp(struct dc *dc)
+{
+ dc_assert_fp_enabled();
+ if (dc->ctx->dce_version == DCN_VERSION_1_01) {
+ struct dcn_soc_bounding_box *dcn_soc = dc->dcn_soc;
+ struct dcn_ip_params *dcn_ip = dc->dcn_ip;
+ struct display_mode_lib *dml = &dc->dml;
+
+ dml->ip.max_num_dpp = 3;
+ /* TODO how to handle 23.84? */
+ dcn_soc->dram_clock_change_latency = 23;
+ dcn_ip->max_num_dpp = 3;
+ }
+ if (ASICREV_IS_RV1_F0(dc->ctx->asic_id.hw_internal_rev)) {
+ dc->dcn_soc->urgent_latency = 3;
+ dc->debug.disable_dmcu = true;
+ dc->dcn_soc->fabric_and_dram_bandwidth_vmax0p9 = 41.60f;
+ }
+
+ dc->dcn_soc->number_of_channels = dc->ctx->asic_id.vram_width / ddr4_dram_width;
+ ASSERT(dc->dcn_soc->number_of_channels < 3);
+ if (dc->dcn_soc->number_of_channels == 0)/*old sbios bug*/
+ dc->dcn_soc->number_of_channels = 2;
+
+ if (dc->dcn_soc->number_of_channels == 1) {
+ dc->dcn_soc->fabric_and_dram_bandwidth_vmax0p9 = 19.2f;
+ dc->dcn_soc->fabric_and_dram_bandwidth_vnom0p8 = 17.066f;
+ dc->dcn_soc->fabric_and_dram_bandwidth_vmid0p72 = 14.933f;
+ dc->dcn_soc->fabric_and_dram_bandwidth_vmin0p65 = 12.8f;
+ if (ASICREV_IS_RV1_F0(dc->ctx->asic_id.hw_internal_rev))
+ dc->dcn_soc->fabric_and_dram_bandwidth_vmax0p9 = 20.80f;
+ }
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.h b/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.h
index e74ed4b..63219ec 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn10/dcn10_fpu.h
@@ -27,4 +27,6 @@
#ifndef __DCN10_FPU_H__
#define __DCN10_FPU_H__
+void dcn10_resource_construct_fp(struct dc *dc);
+
#endif /* __DCN20_FPU_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c
index 45db40c..c26da3b 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c
@@ -565,7 +565,7 @@ struct _vcs_dpi_soc_bounding_box_st dcn2_1_soc = {
.dppclk_mhz = 847.06,
.phyclk_mhz = 810.0,
.socclk_mhz = 953.0,
- .dscclk_mhz = 489.0,
+ .dscclk_mhz = 300.0,
.dram_speed_mts = 2400.0,
},
{
@@ -576,7 +576,7 @@ struct _vcs_dpi_soc_bounding_box_st dcn2_1_soc = {
.dppclk_mhz = 960.00,
.phyclk_mhz = 810.0,
.socclk_mhz = 278.0,
- .dscclk_mhz = 287.67,
+ .dscclk_mhz = 342.86,
.dram_speed_mts = 2666.0,
},
{
@@ -587,7 +587,7 @@ struct _vcs_dpi_soc_bounding_box_st dcn2_1_soc = {
.dppclk_mhz = 1028.57,
.phyclk_mhz = 810.0,
.socclk_mhz = 715.0,
- .dscclk_mhz = 318.334,
+ .dscclk_mhz = 369.23,
.dram_speed_mts = 3200.0,
},
{
@@ -949,6 +949,7 @@ static enum dcn_zstate_support_state decide_zstate_support(struct dc *dc, struc
int plane_count;
int i;
unsigned int optimized_min_dst_y_next_start_us;
+ bool allow_z8 = context->bw_ctx.dml.vba.StutterPeriod > 1000.0;
plane_count = 0;
optimized_min_dst_y_next_start_us = 0;
@@ -963,6 +964,8 @@ static enum dcn_zstate_support_state decide_zstate_support(struct dc *dc, struc
* 2. single eDP, on link 0, 1 plane and stutter period > 5ms
* Z10 only cases:
* 1. single eDP, on link 0, 1 plane and stutter period >= 5ms
+ * Z8 cases:
+ * 1. stutter period sufficient
* Zstate not allowed cases:
* 1. Everything else
*/
@@ -989,12 +992,15 @@ static enum dcn_zstate_support_state decide_zstate_support(struct dc *dc, struc
if (context->bw_ctx.dml.vba.StutterPeriod > 5000.0 || optimized_min_dst_y_next_start_us > 5000)
return DCN_ZSTATE_SUPPORT_ALLOW;
- else if (link->psr_settings.psr_version == DC_PSR_VERSION_1 && !dc->debug.disable_psr)
- return DCN_ZSTATE_SUPPORT_ALLOW_Z10_ONLY;
+ else if (link->psr_settings.psr_version == DC_PSR_VERSION_1 && !link->panel_config.psr.disable_psr)
+ return allow_z8 ? DCN_ZSTATE_SUPPORT_ALLOW_Z8_Z10_ONLY : DCN_ZSTATE_SUPPORT_ALLOW_Z10_ONLY;
else
- return DCN_ZSTATE_SUPPORT_DISALLOW;
- } else
+ return allow_z8 ? DCN_ZSTATE_SUPPORT_ALLOW_Z8_ONLY : DCN_ZSTATE_SUPPORT_DISALLOW;
+ } else if (allow_z8) {
+ return DCN_ZSTATE_SUPPORT_ALLOW_Z8_ONLY;
+ } else {
return DCN_ZSTATE_SUPPORT_DISALLOW;
+ }
}
void dcn20_calculate_dlg_params(
@@ -1296,6 +1302,8 @@ int dcn20_populate_dml_pipes_from_context(
case SIGNAL_TYPE_DISPLAY_PORT_MST:
case SIGNAL_TYPE_DISPLAY_PORT:
pipes[pipe_cnt].dout.output_type = dm_dp;
+ if (is_dp_128b_132b_signal(&res_ctx->pipe_ctx[i]))
+ pipes[pipe_cnt].dout.output_type = dm_dp2p0;
break;
case SIGNAL_TYPE_EDP:
pipes[pipe_cnt].dout.output_type = dm_edp;
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn30/dcn30_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn30/dcn30_fpu.c
index e1e92da..d4c0f9c 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn30/dcn30_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn30/dcn30_fpu.c
@@ -520,9 +520,7 @@ void dcn30_fpu_calculate_wm_and_dlg(
pipe_idx++;
}
- DC_FP_START();
dcn20_calculate_dlg_params(dc, context, pipes, pipe_cnt, vlevel);
- DC_FP_END();
if (!pstate_en)
/* Restore full p-state latency */
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c b/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c
index 479e2c1..379729b 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c
@@ -4851,7 +4851,7 @@ void dml30_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
v->SwathHeightYThisState[k],
v->SwathHeightCThisState[k],
v->HTotal[k] / v->PixelClock[k],
- v->UrgentLatency,
+ v->UrgLatency[i],
v->CursorBufferSize,
v->CursorWidth[k][0],
v->CursorBPP[k][0],
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.c
index 7dd0845..b37d143 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.c
@@ -483,7 +483,7 @@ void dcn31_calculate_wm_and_dlg_fp(
int pipe_cnt,
int vlevel)
{
- int i, pipe_idx, active_dpp_count = 0;
+ int i, pipe_idx, active_hubp_count = 0;
double dcfclk = context->bw_ctx.dml.vba.DCFCLKState[vlevel][context->bw_ctx.dml.vba.maxMpcComb];
dc_assert_fp_enabled();
@@ -529,7 +529,7 @@ void dcn31_calculate_wm_and_dlg_fp(
continue;
if (context->res_ctx.pipe_ctx[i].plane_state)
- active_dpp_count++;
+ active_hubp_count++;
pipes[pipe_idx].clks_cfg.dispclk_mhz = get_dispclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt);
pipes[pipe_idx].clks_cfg.dppclk_mhz = get_dppclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
@@ -547,9 +547,22 @@ void dcn31_calculate_wm_and_dlg_fp(
}
dcn20_calculate_dlg_params(dc, context, pipes, pipe_cnt, vlevel);
- /* For 31x apu pstate change is only supported if possible in vactive or if there are no active dpps */
+ /* For 31x apu pstate change is only supported if possible in vactive*/
context->bw_ctx.bw.dcn.clk.p_state_change_support =
- context->bw_ctx.dml.vba.DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] == dm_dram_clock_change_vactive || !active_dpp_count;
+ context->bw_ctx.dml.vba.DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] == dm_dram_clock_change_vactive;
+ /* If DCN isn't making memory requests we can allow pstate change and lower clocks */
+ if (!active_hubp_count) {
+ context->bw_ctx.bw.dcn.clk.socclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dramclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.fclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.p_state_change_support = true;
+ for (i = 0; i < dc->res_pool->pipe_count; i++)
+ if (context->res_ctx.pipe_ctx[i].stream)
+ context->res_ctx.pipe_ctx[i].plane_res.bw.dppclk_khz = 0;
+ }
}
void dcn31_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_params)
@@ -797,3 +810,8 @@ void dcn316_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_param
else
dml_init_instance(&dc->dml, &dcn3_16_soc, &dcn3_16_ip, DML_PROJECT_DCN31_FPGA);
}
+
+int dcn_get_max_non_odm_pix_rate_100hz(struct _vcs_dpi_soc_bounding_box_st *soc)
+{
+ return soc->clock_limits[0].dispclk_mhz * 10000.0 / (1.0 + soc->dcn_downspread_percent / 100.0);
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.h b/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.h
index fd58b25..687d352 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn31/dcn31_fpu.h
@@ -46,5 +46,10 @@ void dcn31_calculate_wm_and_dlg_fp(
void dcn31_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_params);
void dcn315_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_params);
void dcn316_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_params);
+int dcn_get_max_non_odm_pix_rate_100hz(struct _vcs_dpi_soc_bounding_box_st *soc);
+int dcn31x_populate_dml_pipes_from_context(struct dc *dc,
+ struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ bool fast_validate);
#endif /* __DCN31_FPU_H__*/
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c b/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c
index b612edb..ec351c84 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c
@@ -1052,17 +1052,16 @@ static bool CalculatePrefetchSchedule(
else
bytes_pp = myPipe->BytePerPixelY + myPipe->BytePerPixelC;
/*rev 99*/
- prefetch_bw_pr = dml_min(1, bytes_pp * myPipe->PixelClock / (double) myPipe->DPPPerPlane);
+ prefetch_bw_pr = bytes_pp * myPipe->PixelClock / (double) myPipe->DPPPerPlane;
+ prefetch_bw_pr = dml_min(1, myPipe->VRatio) * prefetch_bw_pr;
max_Tsw = dml_max(PrefetchSourceLinesY, PrefetchSourceLinesC) * LineTime;
prefetch_sw_bytes = PrefetchSourceLinesY * swath_width_luma_ub * myPipe->BytePerPixelY + PrefetchSourceLinesC * swath_width_chroma_ub * myPipe->BytePerPixelC;
- prefetch_bw_oto = dml_max(bytes_pp * myPipe->PixelClock / myPipe->DPPPerPlane, prefetch_sw_bytes / (dml_max(PrefetchSourceLinesY, PrefetchSourceLinesC) * LineTime));
prefetch_bw_oto = dml_max(prefetch_bw_pr, prefetch_sw_bytes / max_Tsw);
min_Lsw = dml_max(1, dml_max(PrefetchSourceLinesY, PrefetchSourceLinesC) / max_vratio_pre);
Lsw_oto = dml_ceil(4 * dml_max(prefetch_sw_bytes / prefetch_bw_oto / LineTime, min_Lsw), 1) / 4;
Tsw_oto = Lsw_oto * LineTime;
- prefetch_bw_oto = (PrefetchSourceLinesY * swath_width_luma_ub * myPipe->BytePerPixelY + PrefetchSourceLinesC * swath_width_chroma_ub * myPipe->BytePerPixelC) / Tsw_oto;
#ifdef __DML_VBA_DEBUG__
dml_print("DML: HTotal: %d\n", myPipe->HTotal);
@@ -5083,7 +5082,7 @@ void dml31_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
v->SwathHeightYThisState[k],
v->SwathHeightCThisState[k],
v->HTotal[k] / v->PixelClock[k],
- v->UrgentLatency,
+ v->UrgLatency[i],
v->CursorBufferSize,
v->CursorWidth[k][0],
v->CursorBPP[k][0],
@@ -5361,6 +5360,58 @@ void dml31_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
v->ModeSupport[i][j] = true;
} else {
v->ModeSupport[i][j] = false;
+#ifdef __DML_VBA_DEBUG__
+ if (v->ScaleRatioAndTapsSupport == false)
+ dml_print("DML SUPPORT: ScaleRatioAndTapsSupport failed");
+ if (v->SourceFormatPixelAndScanSupport == false)
+ dml_print("DML SUPPORT: SourceFormatPixelAndScanSupport failed");
+ if (v->ViewportSizeSupport[i][j] == false)
+ dml_print("DML SUPPORT: ViewportSizeSupport failed");
+ if (v->LinkCapacitySupport[i] == false)
+ dml_print("DML SUPPORT: LinkCapacitySupport failed");
+ if (v->ODMCombine4To1SupportCheckOK[i] == false)
+ dml_print("DML SUPPORT: DSC422NativeNotSupported failed");
+ if (v->NotEnoughDSCUnits[i] == true)
+ dml_print("DML SUPPORT: NotEnoughDSCUnits");
+ if (v->DTBCLKRequiredMoreThanSupported[i] == true)
+ dml_print("DML SUPPORT: DTBCLKRequiredMoreThanSupported");
+ if (v->ROBSupport[i][j] == false)
+ dml_print("DML SUPPORT: ROBSupport failed");
+ if (v->DISPCLK_DPPCLK_Support[i][j] == false)
+ dml_print("DML SUPPORT: DISPCLK_DPPCLK_Support failed");
+ if (v->TotalAvailablePipesSupport[i][j] == false)
+ dml_print("DML SUPPORT: DSC422NativeNotSupported failed");
+ if (EnoughWritebackUnits == false)
+ dml_print("DML SUPPORT: DSC422NativeNotSupported failed");
+ if (v->WritebackLatencySupport == false)
+ dml_print("DML SUPPORT: WritebackLatencySupport failed");
+ if (v->WritebackScaleRatioAndTapsSupport == false)
+ dml_print("DML SUPPORT: DSC422NativeNotSupported ");
+ if (v->CursorSupport == false)
+ dml_print("DML SUPPORT: DSC422NativeNotSupported failed");
+ if (v->PitchSupport == false)
+ dml_print("DML SUPPORT: PitchSupport failed");
+ if (ViewportExceedsSurface == true)
+ dml_print("DML SUPPORT: ViewportExceedsSurface failed");
+ if (v->PrefetchSupported[i][j] == false)
+ dml_print("DML SUPPORT: PrefetchSupported failed");
+ if (v->DynamicMetadataSupported[i][j] == false)
+ dml_print("DML SUPPORT: DSC422NativeNotSupported failed");
+ if (v->TotalVerticalActiveBandwidthSupport[i][j] == false)
+ dml_print("DML SUPPORT: TotalVerticalActiveBandwidthSupport failed");
+ if (v->VRatioInPrefetchSupported[i][j] == false)
+ dml_print("DML SUPPORT: VRatioInPrefetchSupported failed");
+ if (v->PTEBufferSizeNotExceeded[i][j] == false)
+ dml_print("DML SUPPORT: PTEBufferSizeNotExceeded failed");
+ if (v->NonsupportedDSCInputBPC == true)
+ dml_print("DML SUPPORT: NonsupportedDSCInputBPC failed");
+ if (!((v->HostVMEnable == false
+ && v->ImmediateFlipRequirement[0] != dm_immediate_flip_required)
+ || v->ImmediateFlipSupportedForState[i][j] == true))
+ dml_print("DML SUPPORT: ImmediateFlipRequirement failed");
+ if (FMTBufferExceeded == true)
+ dml_print("DML SUPPORT: FMTBufferExceeded failed");
+#endif
}
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn314/dcn314_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn314/dcn314_fpu.c
index 34b6c76..6a1cf6a 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn314/dcn314_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn314/dcn314_fpu.c
@@ -29,6 +29,7 @@
#include "dcn31/dcn31_hubbub.h"
#include "dcn314_fpu.h"
#include "dml/dcn20/dcn20_fpu.h"
+#include "dml/dcn31/dcn31_fpu.h"
#include "dml/display_mode_vba.h"
struct _vcs_dpi_ip_params_st dcn3_14_ip = {
@@ -148,8 +149,8 @@ struct _vcs_dpi_soc_bounding_box_st dcn3_14_soc = {
.num_states = 5,
.sr_exit_time_us = 16.5,
.sr_enter_plus_exit_time_us = 18.5,
- .sr_exit_z8_time_us = 442.0,
- .sr_enter_plus_exit_z8_time_us = 560.0,
+ .sr_exit_z8_time_us = 280.0,
+ .sr_enter_plus_exit_z8_time_us = 350.0,
.writeback_latency_us = 12.0,
.dram_channel_width_bytes = 4,
.round_trip_ping_latency_dcfclk_cycles = 106,
@@ -264,11 +265,8 @@ void dcn314_update_bw_bounding_box_fpu(struct dc *dc, struct clk_bw_params *bw_p
dc->dml.soc.dispclk_dppclk_vco_speed_mhz = max_dispclk_mhz * 2;
}
- if ((int)(dcn3_14_soc.dram_clock_change_latency_us * 1000)
- != dc->debug.dram_clock_change_latency_ns
- && dc->debug.dram_clock_change_latency_ns) {
- dcn3_14_soc.dram_clock_change_latency_us = dc->debug.dram_clock_change_latency_ns / 1000;
- }
+ dcn20_patch_bounding_box(dc, &dcn3_14_soc);
+
if (!IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment))
dml_init_instance(&dc->dml, &dcn3_14_soc, &dcn3_14_ip, DML_PROJECT_DCN314);
else
@@ -291,7 +289,7 @@ int dcn314_populate_dml_pipes_from_context_fpu(struct dc *dc, struct dc_state *c
dc_assert_fp_enabled();
- dcn20_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
+ dcn31x_populate_dml_pipes_from_context(dc, context, pipes, fast_validate);
for (i = 0, pipe_cnt = 0; i < dc->res_pool->pipe_count; i++) {
struct dc_crtc_timing *timing;
@@ -318,8 +316,6 @@ int dcn314_populate_dml_pipes_from_context_fpu(struct dc *dc, struct dc_state *c
pipes[pipe_cnt].pipe.src.immediate_flip = true;
pipes[pipe_cnt].pipe.src.unbounded_req_mode = false;
- pipes[pipe_cnt].pipe.src.hostvm = dc->res_pool->hubbub->riommu_active;
- pipes[pipe_cnt].pipe.src.gpuvm = true;
pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_luma = 0;
pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_chroma = 0;
pipes[pipe_cnt].pipe.dest.vfront_porch = timing->v_front_porch;
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn314/display_mode_vba_314.c b/drivers/gpu/drm/amd/display/dc/dml/dcn314/display_mode_vba_314.c
index 0d12fd0..950669f 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn314/display_mode_vba_314.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn314/display_mode_vba_314.c
@@ -1074,17 +1074,16 @@ static bool CalculatePrefetchSchedule(
else
bytes_pp = myPipe->BytePerPixelY + myPipe->BytePerPixelC;
/*rev 99*/
- prefetch_bw_pr = dml_min(1, bytes_pp * myPipe->PixelClock / (double) myPipe->DPPPerPlane);
+ prefetch_bw_pr = bytes_pp * myPipe->PixelClock / (double) myPipe->DPPPerPlane;
+ prefetch_bw_pr = dml_min(1, myPipe->VRatio) * prefetch_bw_pr;
max_Tsw = dml_max(PrefetchSourceLinesY, PrefetchSourceLinesC) * LineTime;
prefetch_sw_bytes = PrefetchSourceLinesY * swath_width_luma_ub * myPipe->BytePerPixelY + PrefetchSourceLinesC * swath_width_chroma_ub * myPipe->BytePerPixelC;
- prefetch_bw_oto = dml_max(bytes_pp * myPipe->PixelClock / myPipe->DPPPerPlane, prefetch_sw_bytes / (dml_max(PrefetchSourceLinesY, PrefetchSourceLinesC) * LineTime));
prefetch_bw_oto = dml_max(prefetch_bw_pr, prefetch_sw_bytes / max_Tsw);
min_Lsw = dml_max(1, dml_max(PrefetchSourceLinesY, PrefetchSourceLinesC) / max_vratio_pre);
Lsw_oto = dml_ceil(4 * dml_max(prefetch_sw_bytes / prefetch_bw_oto / LineTime, min_Lsw), 1) / 4;
Tsw_oto = Lsw_oto * LineTime;
- prefetch_bw_oto = (PrefetchSourceLinesY * swath_width_luma_ub * myPipe->BytePerPixelY + PrefetchSourceLinesC * swath_width_chroma_ub * myPipe->BytePerPixelC) / Tsw_oto;
#ifdef __DML_VBA_DEBUG__
dml_print("DML: HTotal: %d\n", myPipe->HTotal);
@@ -5180,7 +5179,7 @@ void dml314_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_
v->SwathHeightYThisState[k],
v->SwathHeightCThisState[k],
v->HTotal[k] / v->PixelClock[k],
- v->UrgentLatency,
+ v->UrgLatency[i],
v->CursorBufferSize,
v->CursorWidth[k][0],
v->CursorBPP[k][0],
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c
index 2abe396..f94abd1 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c
@@ -256,16 +256,24 @@ int dcn32_find_dummy_latency_index_for_fw_based_mclk_switch(struct dc *dc,
int vlevel)
{
const int max_latency_table_entries = 4;
- const struct vba_vars_st *vba = &context->bw_ctx.dml.vba;
+ struct vba_vars_st *vba = &context->bw_ctx.dml.vba;
int dummy_latency_index = 0;
+ enum clock_change_support temp_clock_change_support = vba->DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb];
dc_assert_fp_enabled();
while (dummy_latency_index < max_latency_table_entries) {
+ if (temp_clock_change_support != dm_dram_clock_change_unsupported)
+ vba->DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] = temp_clock_change_support;
context->bw_ctx.dml.soc.dram_clock_change_latency_us =
dc->clk_mgr->bw_params->dummy_pstate_table[dummy_latency_index].dummy_pstate_latency_us;
dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, false);
+ /* for subvp + DRR case, if subvp pipes are still present we support pstate */
+ if (vba->DRAMClockChangeSupport[vlevel][vba->maxMpcComb] == dm_dram_clock_change_unsupported &&
+ dcn32_subvp_in_use(dc, context))
+ vba->DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] = temp_clock_change_support;
+
if (vlevel < context->bw_ctx.dml.vba.soc.num_states &&
vba->DRAMClockChangeSupport[vlevel][vba->maxMpcComb] != dm_dram_clock_change_unsupported)
break;
@@ -531,9 +539,12 @@ void dcn32_set_phantom_stream_timing(struct dc *dc,
unsigned int i, pipe_idx;
struct pipe_ctx *pipe;
uint32_t phantom_vactive, phantom_bp, pstate_width_fw_delay_lines;
+ unsigned int num_dpp;
unsigned int vlevel = context->bw_ctx.dml.vba.VoltageLevel;
unsigned int dcfclk = context->bw_ctx.dml.vba.DCFCLKState[vlevel][context->bw_ctx.dml.vba.maxMpcComb];
unsigned int socclk = context->bw_ctx.dml.vba.SOCCLKPerState[vlevel];
+ struct vba_vars_st *vba = &context->bw_ctx.dml.vba;
+ struct dc_stream_state *main_stream = ref_pipe->stream;
dc_assert_fp_enabled();
@@ -569,13 +580,26 @@ void dcn32_set_phantom_stream_timing(struct dc *dc,
phantom_vactive = get_subviewport_lines_needed_in_mall(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx) +
pstate_width_fw_delay_lines + dc->caps.subvp_swath_height_margin_lines;
+ // W/A for DCC corruption with certain high resolution timings.
+ // Determing if pipesplit is used. If so, add meta_row_height to the phantom vactive.
+ num_dpp = vba->NoOfDPP[vba->VoltageLevel][vba->maxMpcComb][vba->pipe_plane[pipe_idx]];
+ phantom_vactive += num_dpp > 1 ? vba->meta_row_height[vba->pipe_plane[pipe_idx]] : 0;
+
+ /* dc->debug.subvp_extra_lines 0 by default*/
+ phantom_vactive += dc->debug.subvp_extra_lines;
+
// For backporch of phantom pipe, use vstartup of the main pipe
phantom_bp = get_vstartup(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
phantom_stream->dst.y = 0;
phantom_stream->dst.height = phantom_vactive;
+ /* When scaling, DML provides the end to end required number of lines for MALL.
+ * dst.height is always correct for this case, but src.height is not which causes a
+ * delta between main and phantom pipe scaling outputs. Need to adjust src.height on
+ * phantom for this case.
+ */
phantom_stream->src.y = 0;
- phantom_stream->src.height = phantom_vactive;
+ phantom_stream->src.height = (double)phantom_vactive * (double)main_stream->src.height / (double)main_stream->dst.height;
phantom_stream->timing.v_addressable = phantom_vactive;
phantom_stream->timing.v_front_porch = 1;
@@ -1128,7 +1152,7 @@ static void dcn32_full_validate_bw_helper(struct dc *dc,
context->bw_ctx.dml.soc.allow_for_pstate_or_stutter_in_vblank_final ==
dm_prefetch_support_uclk_fclk_and_stutter) {
context->bw_ctx.dml.soc.allow_for_pstate_or_stutter_in_vblank_final =
- dm_prefetch_support_stutter;
+ dm_prefetch_support_fclk_and_stutter;
/* There are params (such as FabricClock) that need to be recalculated
* after validation fails (otherwise it will be 0). Calculation for
* phantom vactive requires call into DML, so we must ensure all the
@@ -1179,7 +1203,7 @@ static void dcn32_full_validate_bw_helper(struct dc *dc,
// If SubVP pipe config is unsupported (or cannot be used for UCLK switching)
// remove phantom pipes and repopulate dml pipes
if (!found_supported_config) {
- dc->res_pool->funcs->remove_phantom_pipes(dc, context);
+ dc->res_pool->funcs->remove_phantom_pipes(dc, context, false);
vba->DRAMClockChangeSupport[*vlevel][vba->maxMpcComb] = dm_dram_clock_change_unsupported;
*pipe_cnt = dc->res_pool->funcs->populate_dml_pipes(dc, context, pipes, false);
@@ -1191,9 +1215,7 @@ static void dcn32_full_validate_bw_helper(struct dc *dc,
}
} else {
// Most populate phantom DLG params before programming hardware / timing for phantom pipe
- DC_FP_START();
dcn32_helper_populate_phantom_dlg_params(dc, context, pipes, *pipe_cnt);
- DC_FP_END();
/* Call validate_apply_pipe_split flags after calling DML getters for
* phantom dlg params, or some of the VBA params indicating pipe split
@@ -1230,7 +1252,7 @@ static void dcn32_calculate_dlg_params(struct dc *dc, struct dc_state *context,
display_e2e_pipe_params_st *pipes,
int pipe_cnt, int vlevel)
{
- int i, pipe_idx;
+ int i, pipe_idx, active_hubp_count = 0;
bool usr_retraining_support = false;
bool unbounded_req_enabled = false;
@@ -1275,6 +1297,8 @@ static void dcn32_calculate_dlg_params(struct dc *dc, struct dc_state *context,
for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
if (!context->res_ctx.pipe_ctx[i].stream)
continue;
+ if (context->res_ctx.pipe_ctx[i].plane_state)
+ active_hubp_count++;
pipes[pipe_idx].pipe.dest.vstartup_start = get_vstartup(&context->bw_ctx.dml, pipes, pipe_cnt,
pipe_idx);
pipes[pipe_idx].pipe.dest.vupdate_offset = get_vupdate_offset(&context->bw_ctx.dml, pipes, pipe_cnt,
@@ -1296,10 +1320,23 @@ static void dcn32_calculate_dlg_params(struct dc *dc, struct dc_state *context,
if (context->bw_ctx.bw.dcn.clk.dppclk_khz < pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000)
context->bw_ctx.bw.dcn.clk.dppclk_khz = pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000;
- context->res_ctx.pipe_ctx[i].plane_res.bw.dppclk_khz = pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000;
+ if (context->res_ctx.pipe_ctx[i].plane_state)
+ context->res_ctx.pipe_ctx[i].plane_res.bw.dppclk_khz = pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000;
+ else
+ context->res_ctx.pipe_ctx[i].plane_res.bw.dppclk_khz = 0;
context->res_ctx.pipe_ctx[i].pipe_dlg_param = pipes[pipe_idx].pipe.dest;
pipe_idx++;
}
+ /* If DCN isn't making memory requests we can allow pstate change and lower clocks */
+ if (!active_hubp_count) {
+ context->bw_ctx.bw.dcn.clk.socclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dramclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.fclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.p_state_change_support = true;
+ }
/*save a original dppclock copy*/
context->bw_ctx.bw.dcn.clk.bw_dppclk_khz = context->bw_ctx.bw.dcn.clk.dppclk_khz;
context->bw_ctx.bw.dcn.clk.bw_dispclk_khz = context->bw_ctx.bw.dcn.clk.dispclk_khz;
@@ -1481,7 +1518,7 @@ bool dcn32_internal_validate_bw(struct dc *dc,
return false;
// For each full update, remove all existing phantom pipes first
- dc->res_pool->funcs->remove_phantom_pipes(dc, context);
+ dc->res_pool->funcs->remove_phantom_pipes(dc, context, fast_validate);
dc->res_pool->funcs->update_soc_for_wm_a(dc, context);
@@ -1494,11 +1531,8 @@ bool dcn32_internal_validate_bw(struct dc *dc,
dml_log_pipe_params(&context->bw_ctx.dml, pipes, pipe_cnt);
- if (!fast_validate) {
- DC_FP_START();
+ if (!fast_validate)
dcn32_full_validate_bw_helper(dc, context, pipes, &vlevel, split, merge, &pipe_cnt);
- DC_FP_END();
- }
if (fast_validate ||
(dc->debug.dml_disallow_alternate_prefetch_modes &&
@@ -1734,6 +1768,10 @@ bool dcn32_internal_validate_bw(struct dc *dc,
}
if (repopulate_pipes) {
+ int flag_max_mpc_comb = vba->maxMpcComb;
+ int flag_vlevel = vlevel;
+ int i;
+
pipe_cnt = dc->res_pool->funcs->populate_dml_pipes(dc, context, pipes, fast_validate);
/* repopulate_pipes = 1 means the pipes were either split or merged. In this case
@@ -1741,10 +1779,28 @@ bool dcn32_internal_validate_bw(struct dc *dc,
* ensure all the params are calculated correctly. We do not need to run the
* pipe split check again after this call (pipes are already split / merged).
* */
- if (!fast_validate) {
- context->bw_ctx.dml.soc.allow_for_pstate_or_stutter_in_vblank_final =
- dm_prefetch_support_uclk_fclk_and_stutter_if_possible;
- vlevel = dml_get_voltage_level(&context->bw_ctx.dml, pipes, pipe_cnt);
+ context->bw_ctx.dml.soc.allow_for_pstate_or_stutter_in_vblank_final =
+ dm_prefetch_support_uclk_fclk_and_stutter_if_possible;
+ vlevel = dml_get_voltage_level(&context->bw_ctx.dml, pipes, pipe_cnt);
+ if (vlevel == context->bw_ctx.dml.soc.num_states) {
+ /* failed after DET size changes */
+ goto validate_fail;
+ } else if (flag_max_mpc_comb == 0 &&
+ flag_max_mpc_comb != context->bw_ctx.dml.vba.maxMpcComb) {
+ /* check the context constructed with pipe split flags is still valid*/
+ bool flags_valid = false;
+ for (i = flag_vlevel; i < context->bw_ctx.dml.soc.num_states; i++) {
+ if (vba->ModeSupport[i][flag_max_mpc_comb]) {
+ vba->maxMpcComb = flag_max_mpc_comb;
+ vba->VoltageLevel = i;
+ vlevel = i;
+ flags_valid = true;
+ }
+ }
+
+ /* this should never happen */
+ if (!flags_valid)
+ goto validate_fail;
}
}
*vlevel_out = vlevel;
@@ -1775,14 +1831,38 @@ void dcn32_calculate_wm_and_dlg_fpu(struct dc *dc, struct dc_state *context,
unsigned int dummy_latency_index = 0;
int maxMpcComb = context->bw_ctx.dml.vba.maxMpcComb;
unsigned int min_dram_speed_mts = context->bw_ctx.dml.vba.DRAMSpeed;
+ bool subvp_in_use = dcn32_subvp_in_use(dc, context);
unsigned int min_dram_speed_mts_margin;
+ bool need_fclk_lat_as_dummy = false;
+ bool is_subvp_p_drr = true;
dc_assert_fp_enabled();
- // Override DRAMClockChangeSupport for SubVP + DRR case where the DRR cannot switch without stretching it's VBLANK
- if (!pstate_en && dcn32_subvp_in_use(dc, context)) {
- context->bw_ctx.dml.vba.DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] = dm_dram_clock_change_vblank_w_mall_sub_vp;
- pstate_en = true;
+ /* need to find dummy latency index for subvp */
+ if (subvp_in_use) {
+ /* Override DRAMClockChangeSupport for SubVP + DRR case where the DRR cannot switch without stretching it's VBLANK */
+ if (!pstate_en) {
+ context->bw_ctx.dml.vba.DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] = dm_dram_clock_change_vblank_w_mall_sub_vp;
+ pstate_en = true;
+ is_subvp_p_drr = true;
+ }
+ dummy_latency_index = dcn32_find_dummy_latency_index_for_fw_based_mclk_switch(dc,
+ context, pipes, pipe_cnt, vlevel);
+
+ /* For DCN32/321 need to validate with fclk pstate change latency equal to dummy so prefetch is
+ * scheduled correctly to account for dummy pstate.
+ */
+ if (context->bw_ctx.dml.soc.fclk_change_latency_us < dc->clk_mgr->bw_params->dummy_pstate_table[dummy_latency_index].dummy_pstate_latency_us) {
+ need_fclk_lat_as_dummy = true;
+ context->bw_ctx.dml.soc.fclk_change_latency_us =
+ dc->clk_mgr->bw_params->dummy_pstate_table[dummy_latency_index].dummy_pstate_latency_us;
+ }
+ context->bw_ctx.dml.soc.dram_clock_change_latency_us =
+ dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.pstate_latency_us;
+ dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, false);
+ if (is_subvp_p_drr) {
+ context->bw_ctx.dml.vba.DRAMClockChangeSupport[vlevel][context->bw_ctx.dml.vba.maxMpcComb] = dm_dram_clock_change_vblank_w_mall_sub_vp;
+ }
}
context->bw_ctx.bw.dcn.clk.fw_based_mclk_switching = false;
@@ -1806,9 +1886,11 @@ void dcn32_calculate_wm_and_dlg_fpu(struct dc *dc, struct dc_state *context,
/* For DCN32/321 need to validate with fclk pstate change latency equal to dummy so
* prefetch is scheduled correctly to account for dummy pstate.
*/
- if (dummy_latency_index == 0)
+ if (context->bw_ctx.dml.soc.fclk_change_latency_us < dc->clk_mgr->bw_params->dummy_pstate_table[dummy_latency_index].dummy_pstate_latency_us) {
+ need_fclk_lat_as_dummy = true;
context->bw_ctx.dml.soc.fclk_change_latency_us =
dc->clk_mgr->bw_params->dummy_pstate_table[dummy_latency_index].dummy_pstate_latency_us;
+ }
dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, false);
maxMpcComb = context->bw_ctx.dml.vba.maxMpcComb;
dcfclk_from_fw_based_mclk_switching = context->bw_ctx.dml.vba.DCFCLKState[vlevel][context->bw_ctx.dml.vba.maxMpcComb];
@@ -1916,7 +1998,7 @@ void dcn32_calculate_wm_and_dlg_fpu(struct dc *dc, struct dc_state *context,
dc->clk_mgr->bw_params->clk_table.entries[min_dram_speed_mts_offset].memclk_mhz * 16;
}
- if (!context->bw_ctx.bw.dcn.clk.fw_based_mclk_switching) {
+ if (!context->bw_ctx.bw.dcn.clk.fw_based_mclk_switching && !subvp_in_use) {
/* find largest table entry that is lower than dram speed,
* but lower than DPM0 still uses DPM0
*/
@@ -1996,7 +2078,8 @@ void dcn32_calculate_wm_and_dlg_fpu(struct dc *dc, struct dc_state *context,
context->perf_params.stutter_period_us = context->bw_ctx.dml.vba.StutterPeriod;
- if (context->bw_ctx.bw.dcn.clk.fw_based_mclk_switching && dummy_latency_index == 0)
+ /* for proper prefetch calculations, if dummy lat > fclk lat, use fclk lat = dummy lat */
+ if (need_fclk_lat_as_dummy)
context->bw_ctx.dml.soc.fclk_change_latency_us =
dc->clk_mgr->bw_params->dummy_pstate_table[dummy_latency_index].dummy_pstate_latency_us;
@@ -2009,10 +2092,12 @@ void dcn32_calculate_wm_and_dlg_fpu(struct dc *dc, struct dc_state *context,
if (context->bw_ctx.bw.dcn.clk.fw_based_mclk_switching) {
dcn30_setup_mclk_switch_using_fw_based_vblank_stretch(dc, context);
- if (dummy_latency_index == 0)
- context->bw_ctx.dml.soc.fclk_change_latency_us =
- dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.fclk_change_latency_us;
}
+
+ /* revert fclk lat changes if required */
+ if (need_fclk_lat_as_dummy)
+ context->bw_ctx.dml.soc.fclk_change_latency_us =
+ dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.fclk_change_latency_us;
}
static void dcn32_get_optimal_dcfclk_fclk_for_uclk(unsigned int uclk_mts,
@@ -2159,9 +2244,7 @@ static int build_synthetic_soc_states(struct clk_bw_params *bw_params,
entry.fabricclk_mhz = 0;
entry.dram_speed_mts = 0;
- DC_FP_START();
insert_entry_into_table_sorted(table, num_entries, &entry);
- DC_FP_END();
}
// Insert the max DCFCLK
@@ -2169,9 +2252,7 @@ static int build_synthetic_soc_states(struct clk_bw_params *bw_params,
entry.fabricclk_mhz = 0;
entry.dram_speed_mts = 0;
- DC_FP_START();
insert_entry_into_table_sorted(table, num_entries, &entry);
- DC_FP_END();
// Insert the UCLK DPMS
for (i = 0; i < num_uclk_dpms; i++) {
@@ -2179,9 +2260,7 @@ static int build_synthetic_soc_states(struct clk_bw_params *bw_params,
entry.fabricclk_mhz = 0;
entry.dram_speed_mts = bw_params->clk_table.entries[i].memclk_mhz * 16;
- DC_FP_START();
insert_entry_into_table_sorted(table, num_entries, &entry);
- DC_FP_END();
}
// If FCLK is coarse grained, insert individual DPMs.
@@ -2191,9 +2270,7 @@ static int build_synthetic_soc_states(struct clk_bw_params *bw_params,
entry.fabricclk_mhz = bw_params->clk_table.entries[i].fclk_mhz;
entry.dram_speed_mts = 0;
- DC_FP_START();
insert_entry_into_table_sorted(table, num_entries, &entry);
- DC_FP_END();
}
}
// If FCLK fine grained, only insert max
@@ -2202,9 +2279,7 @@ static int build_synthetic_soc_states(struct clk_bw_params *bw_params,
entry.fabricclk_mhz = max_fclk_mhz;
entry.dram_speed_mts = 0;
- DC_FP_START();
insert_entry_into_table_sorted(table, num_entries, &entry);
- DC_FP_END();
}
// At this point, the table contains all "points of interest" based on
@@ -2539,3 +2614,11 @@ void dcn32_update_bw_bounding_box_fpu(struct dc *dc, struct clk_bw_params *bw_pa
}
}
+void dcn32_zero_pipe_dcc_fraction(display_e2e_pipe_params_st *pipes,
+ int pipe_cnt)
+{
+ dc_assert_fp_enabled();
+
+ pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_luma = 0;
+ pipes[pipe_cnt].pipe.src.dcc_fraction_of_zs_req_chroma = 0;
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.h b/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.h
index 3a3dc2c..ab010e7 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.h
@@ -73,4 +73,7 @@ int dcn32_find_dummy_latency_index_for_fw_based_mclk_switch(struct dc *dc,
void dcn32_patch_dpm_table(struct clk_bw_params *bw_params);
+void dcn32_zero_pipe_dcc_fraction(display_e2e_pipe_params_st *pipes,
+ int pipe_cnt);
+
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c
index 9afd9ba..4b8f5fa 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c
@@ -670,6 +670,25 @@ static void DISPCLKDPPCLKDCFCLKDeepSleepPrefetchParametersWatermarksAndPerforman
v->cursor_bw[k] = mode_lib->vba.NumberOfCursors[k] * mode_lib->vba.CursorWidth[k][0] * mode_lib->vba.CursorBPP[k][0] / 8 / (mode_lib->vba.HTotal[k] / mode_lib->vba.PixelClock[k]) * mode_lib->vba.VRatio[k];
}
+ v->NotEnoughDETSwathFillLatencyHiding = dml32_CalculateDETSwathFillLatencyHiding(
+ mode_lib->vba.NumberOfActiveSurfaces,
+ mode_lib->vba.ReturnBW,
+ v->UrgentLatency,
+ mode_lib->vba.SwathHeightY,
+ mode_lib->vba.SwathHeightC,
+ v->swath_width_luma_ub,
+ v->swath_width_chroma_ub,
+ v->BytePerPixelDETY,
+ v->BytePerPixelDETC,
+ mode_lib->vba.DETBufferSizeY,
+ mode_lib->vba.DETBufferSizeC,
+ mode_lib->vba.DPPPerPlane,
+ mode_lib->vba.HTotal,
+ mode_lib->vba.PixelClock,
+ mode_lib->vba.VRatio,
+ mode_lib->vba.VRatioChroma,
+ mode_lib->vba.UsesMALLForPStateChange);
+
for (k = 0; k < mode_lib->vba.NumberOfActiveSurfaces; ++k) {
v->MaxVStartupLines[k] = ((mode_lib->vba.Interlace[k] &&
!mode_lib->vba.ProgressiveToInterlaceUnitInOPP) ?
@@ -1665,6 +1684,8 @@ static void mode_support_configuration(struct vba_vars_st *v,
&& mode_lib->vba.DCCMetaBufferSizeNotExceeded[i][j] == true
&& mode_lib->vba.NonsupportedDSCInputBPC == false
&& !mode_lib->vba.ExceededMALLSize
+ && (mode_lib->vba.NotEnoughDETSwathFillLatencyHidingPerState[i][j] == false
+ || i == v->soc.num_states - 1)
&& ((mode_lib->vba.HostVMEnable == false
&& !mode_lib->vba.ImmediateFlipRequiredFinal)
|| mode_lib->vba.ImmediateFlipSupportedForState[i][j])
@@ -3158,6 +3179,25 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
mode_lib->vba.UrgentBurstFactorChroma,
mode_lib->vba.UrgentBurstFactorCursor);
+ mode_lib->vba.NotEnoughDETSwathFillLatencyHidingPerState[i][j] = dml32_CalculateDETSwathFillLatencyHiding(
+ mode_lib->vba.NumberOfActiveSurfaces,
+ mode_lib->vba.ReturnBWPerState[i][j],
+ mode_lib->vba.UrgLatency[i],
+ mode_lib->vba.SwathHeightYThisState,
+ mode_lib->vba.SwathHeightCThisState,
+ mode_lib->vba.swath_width_luma_ub_this_state,
+ mode_lib->vba.swath_width_chroma_ub_this_state,
+ mode_lib->vba.BytePerPixelInDETY,
+ mode_lib->vba.BytePerPixelInDETC,
+ mode_lib->vba.DETBufferSizeYThisState,
+ mode_lib->vba.DETBufferSizeCThisState,
+ mode_lib->vba.NoOfDPPThisState,
+ mode_lib->vba.HTotal,
+ mode_lib->vba.PixelClock,
+ mode_lib->vba.VRatio,
+ mode_lib->vba.VRatioChroma,
+ mode_lib->vba.UsesMALLForPStateChange);
+
v->dummy_vars.dml32_ModeSupportAndSystemConfigurationFull.VMDataOnlyReturnBWPerState = dml32_get_return_bw_mbps_vm_only(&mode_lib->vba.soc, i,
mode_lib->vba.DCFCLKState[i][j], mode_lib->vba.FabricClockPerState[i],
mode_lib->vba.DRAMSpeedPerState[i]);
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.c b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.c
index debe46b..5af601c 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.c
@@ -6228,3 +6228,72 @@ void dml32_CalculateImmediateFlipBandwithSupport(unsigned int NumberOfActiveSurf
*ImmediateFlipBandwidthSupport = (*TotalBandwidth <= ReturnBW);
*FractionOfUrgentBandwidth = *TotalBandwidth / ReturnBW;
}
+
+bool dml32_CalculateDETSwathFillLatencyHiding(unsigned int NumberOfActiveSurfaces,
+ double ReturnBW,
+ double UrgentLatency,
+ unsigned int SwathHeightY[],
+ unsigned int SwathHeightC[],
+ unsigned int SwathWidthY[],
+ unsigned int SwathWidthC[],
+ double BytePerPixelInDETY[],
+ double BytePerPixelInDETC[],
+ unsigned int DETBufferSizeY[],
+ unsigned int DETBufferSizeC[],
+ unsigned int NumOfDPP[],
+ unsigned int HTotal[],
+ double PixelClock[],
+ double VRatioY[],
+ double VRatioC[],
+ enum dm_use_mall_for_pstate_change_mode UsesMALLForPStateChange[DC__NUM_DPP__MAX])
+{
+ int k;
+ double SwathSizeAllSurfaces = 0;
+ double SwathSizeAllSurfacesInFetchTimeUs;
+ double DETSwathLatencyHidingUs;
+ double DETSwathLatencyHidingYUs;
+ double DETSwathLatencyHidingCUs;
+ double SwathSizePerSurfaceY[DC__NUM_DPP__MAX];
+ double SwathSizePerSurfaceC[DC__NUM_DPP__MAX];
+ bool NotEnoughDETSwathFillLatencyHiding = false;
+
+ /* calculate sum of single swath size for all pipes in bytes*/
+ for (k = 0; k < NumberOfActiveSurfaces; k++) {
+ SwathSizePerSurfaceY[k] += SwathHeightY[k] * SwathWidthY[k] * BytePerPixelInDETY[k] * NumOfDPP[k];
+
+ if (SwathHeightC[k] != 0)
+ SwathSizePerSurfaceC[k] += SwathHeightC[k] * SwathWidthC[k] * BytePerPixelInDETC[k] * NumOfDPP[k];
+ else
+ SwathSizePerSurfaceC[k] = 0;
+
+ SwathSizeAllSurfaces += SwathSizePerSurfaceY[k] + SwathSizePerSurfaceC[k];
+ }
+
+ SwathSizeAllSurfacesInFetchTimeUs = SwathSizeAllSurfaces / ReturnBW + UrgentLatency;
+
+ /* ensure all DET - 1 swath can hide a fetch for all surfaces */
+ for (k = 0; k < NumberOfActiveSurfaces; k++) {
+ double LineTime = HTotal[k] / PixelClock[k];
+
+ /* only care if surface is not phantom */
+ if (UsesMALLForPStateChange[k] != dm_use_mall_pstate_change_phantom_pipe) {
+ DETSwathLatencyHidingYUs = (dml_floor(DETBufferSizeY[k] / BytePerPixelInDETY[k] / SwathWidthY[k], 1.0) - SwathHeightY[k]) / VRatioY[k] * LineTime;
+
+ if (SwathHeightC[k] != 0) {
+ DETSwathLatencyHidingCUs = (dml_floor(DETBufferSizeC[k] / BytePerPixelInDETC[k] / SwathWidthC[k], 1.0) - SwathHeightC[k]) / VRatioC[k] * LineTime;
+
+ DETSwathLatencyHidingUs = dml_min(DETSwathLatencyHidingYUs, DETSwathLatencyHidingCUs);
+ } else {
+ DETSwathLatencyHidingUs = DETSwathLatencyHidingYUs;
+ }
+
+ /* DET must be able to hide time to fetch 1 swath for each surface */
+ if (DETSwathLatencyHidingUs < SwathSizeAllSurfacesInFetchTimeUs) {
+ NotEnoughDETSwathFillLatencyHiding = true;
+ break;
+ }
+ }
+ }
+
+ return NotEnoughDETSwathFillLatencyHiding;
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.h b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.h
index 3989c2a..779c680 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_util_32.h
@@ -1141,4 +1141,22 @@ void dml32_CalculateImmediateFlipBandwithSupport(unsigned int NumberOfActiveSurf
double *FractionOfUrgentBandwidth,
bool *ImmediateFlipBandwidthSupport);
+bool dml32_CalculateDETSwathFillLatencyHiding(unsigned int NumberOfActiveSurfaces,
+ double ReturnBW,
+ double UrgentLatency,
+ unsigned int SwathHeightY[],
+ unsigned int SwathHeightC[],
+ unsigned int SwathWidthY[],
+ unsigned int SwathWidthC[],
+ double BytePerPixelInDETY[],
+ double BytePerPixelInDETC[],
+ unsigned int DETBufferSizeY[],
+ unsigned int DETBufferSizeC[],
+ unsigned int NumOfDPP[],
+ unsigned int HTotal[],
+ double PixelClock[],
+ double VRatioY[],
+ double VRatioC[],
+ enum dm_use_mall_for_pstate_change_mode UsesMALLForPStateChange[DC__NUM_DPP__MAX]);
+
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/dml/display_mode_enums.h b/drivers/gpu/drm/amd/display/dc/dml/display_mode_enums.h
index f394b3f..0bffae9 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/display_mode_enums.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/display_mode_enums.h
@@ -105,14 +105,39 @@ enum source_macro_tile_size {
enum cursor_bpp {
dm_cur_2bit = 0, dm_cur_32bit = 1, dm_cur_64bit = 2
};
+
+/**
+ * @enum clock_change_support - It represents possible reasons to change the DRAM clock.
+ *
+ * DC may change the DRAM clock during its execution, and this enum tracks all
+ * the available methods. Note that every ASIC has their specific way to deal
+ * with these clock switch.
+ */
enum clock_change_support {
+ /**
+ * @dm_dram_clock_change_uninitialized: If you see this, we might have
+ * a code initialization issue
+ */
dm_dram_clock_change_uninitialized = 0,
+
+ /**
+ * @dm_dram_clock_change_vactive: Support DRAM switch in VActive
+ */
dm_dram_clock_change_vactive,
+
+ /**
+ * @dm_dram_clock_change_vblank: Support DRAM switch in VBlank
+ */
dm_dram_clock_change_vblank,
+
dm_dram_clock_change_vactive_w_mall_full_frame,
dm_dram_clock_change_vactive_w_mall_sub_vp,
dm_dram_clock_change_vblank_w_mall_full_frame,
dm_dram_clock_change_vblank_w_mall_sub_vp,
+
+ /**
+ * @dm_dram_clock_change_unsupported: Do not support DRAM switch
+ */
dm_dram_clock_change_unsupported
};
diff --git a/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.h b/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.h
index a0207a8..81e53e6 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.h
@@ -419,6 +419,15 @@ struct vba_vars_st {
double MinPixelChunkSizeBytes;
unsigned int DCCMetaBufferSizeBytes;
// Pipe/Plane Parameters
+
+ /** @VoltageLevel:
+ * Every ASIC has a fixed number of DPM states, and some devices might
+ * have some particular voltage configuration that does not map
+ * directly to the DPM states. This field tells how many states the
+ * target device supports; even though this field combines the DPM and
+ * special SOC voltages, it mostly matches the total number of DPM
+ * states.
+ */
int VoltageLevel;
double FabricClock;
double DRAMSpeed;
@@ -1041,6 +1050,7 @@ struct vba_vars_st {
double MinFullDETBufferingTime;
double AverageReadBandwidthGBytePerSecond;
bool FirstMainPlane;
+ bool NotEnoughDETSwathFillLatencyHiding;
unsigned int ViewportWidthChroma[DC__NUM_DPP__MAX];
unsigned int ViewportHeightChroma[DC__NUM_DPP__MAX];
@@ -1224,6 +1234,7 @@ struct vba_vars_st {
unsigned int BlockWidthC[DC__NUM_DPP__MAX];
unsigned int SubViewportLinesNeededInMALL[DC__NUM_DPP__MAX];
bool VActiveBandwithSupport[DC__VOLTAGE_STATES][2];
+ bool NotEnoughDETSwathFillLatencyHidingPerState[DC__VOLTAGE_STATES][2];
struct dummy_vars dummy_vars;
};
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dsc/qp_tables.h b/drivers/gpu/drm/amd/display/dc/dml/dsc/qp_tables.h
index e5fac9f..dcff0dd 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dsc/qp_tables.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/dsc/qp_tables.h
@@ -25,7 +25,7 @@
*/
-const qp_table qp_table_422_10bpc_min = {
+static const qp_table qp_table_422_10bpc_min = {
{ 6, { 0, 4, 5, 6, 6, 6, 6, 7, 7, 8, 9, 9, 9, 12, 16} },
{ 6.5, { 0, 4, 5, 6, 6, 6, 6, 7, 7, 8, 9, 9, 9, 12, 16} },
{ 7, { 0, 4, 5, 6, 6, 6, 6, 7, 7, 7, 9, 9, 9, 11, 15} },
@@ -58,7 +58,7 @@ const qp_table qp_table_422_10bpc_min = {
};
-const qp_table qp_table_444_8bpc_max = {
+static const qp_table qp_table_444_8bpc_max = {
{ 6, { 4, 6, 8, 8, 9, 9, 9, 10, 11, 12, 12, 12, 12, 13, 15} },
{ 6.5, { 4, 6, 7, 8, 8, 8, 9, 10, 11, 11, 12, 12, 12, 13, 15} },
{ 7, { 4, 5, 7, 7, 8, 8, 8, 9, 10, 11, 11, 12, 12, 13, 14} },
@@ -99,7 +99,7 @@ const qp_table qp_table_444_8bpc_max = {
};
-const qp_table qp_table_420_12bpc_max = {
+static const qp_table qp_table_420_12bpc_max = {
{ 4, {11, 12, 13, 14, 15, 15, 15, 16, 17, 18, 18, 19, 19, 21, 22} },
{ 4.5, {10, 11, 12, 13, 14, 15, 15, 16, 17, 18, 18, 19, 19, 20, 21} },
{ 5, { 9, 11, 12, 13, 14, 15, 15, 16, 17, 17, 18, 18, 19, 20, 21} },
@@ -132,7 +132,7 @@ const qp_table qp_table_420_12bpc_max = {
};
-const qp_table qp_table_444_10bpc_min = {
+static const qp_table qp_table_444_10bpc_min = {
{ 6, { 0, 4, 7, 7, 9, 9, 9, 9, 9, 10, 10, 10, 10, 12, 18} },
{ 6.5, { 0, 4, 6, 7, 8, 8, 9, 9, 9, 9, 10, 10, 10, 12, 18} },
{ 7, { 0, 4, 6, 6, 8, 8, 8, 8, 8, 9, 9, 10, 10, 12, 17} },
@@ -185,7 +185,7 @@ const qp_table qp_table_444_10bpc_min = {
};
-const qp_table qp_table_420_8bpc_max = {
+static const qp_table qp_table_420_8bpc_max = {
{ 4, { 4, 4, 5, 6, 7, 7, 7, 8, 9, 10, 10, 11, 11, 13, 14} },
{ 4.5, { 4, 4, 5, 6, 7, 7, 7, 8, 9, 10, 10, 11, 11, 12, 13} },
{ 5, { 3, 4, 5, 6, 7, 7, 7, 8, 9, 9, 10, 10, 11, 12, 13} },
@@ -206,7 +206,7 @@ const qp_table qp_table_420_8bpc_max = {
};
-const qp_table qp_table_444_8bpc_min = {
+static const qp_table qp_table_444_8bpc_min = {
{ 6, { 0, 1, 3, 3, 5, 5, 5, 5, 5, 6, 6, 6, 6, 9, 14} },
{ 6.5, { 0, 1, 2, 3, 4, 4, 5, 5, 5, 5, 6, 6, 6, 9, 14} },
{ 7, { 0, 0, 2, 2, 4, 4, 4, 4, 4, 5, 5, 6, 6, 9, 13} },
@@ -247,7 +247,7 @@ const qp_table qp_table_444_8bpc_min = {
};
-const qp_table qp_table_444_12bpc_min = {
+static const qp_table qp_table_444_12bpc_min = {
{ 6, { 0, 5, 11, 11, 13, 13, 13, 13, 13, 14, 14, 14, 14, 17, 22} },
{ 6.5, { 0, 5, 10, 11, 12, 12, 13, 13, 13, 13, 14, 14, 14, 17, 22} },
{ 7, { 0, 5, 10, 10, 12, 12, 12, 12, 12, 13, 13, 14, 14, 17, 21} },
@@ -312,7 +312,7 @@ const qp_table qp_table_444_12bpc_min = {
};
-const qp_table qp_table_420_12bpc_min = {
+static const qp_table qp_table_420_12bpc_min = {
{ 4, { 0, 4, 9, 10, 11, 11, 11, 11, 11, 11, 13, 13, 13, 15, 21} },
{ 4.5, { 0, 4, 8, 9, 10, 11, 11, 11, 11, 11, 13, 13, 13, 15, 20} },
{ 5, { 0, 4, 8, 9, 10, 11, 11, 11, 11, 11, 13, 13, 13, 15, 20} },
@@ -345,7 +345,7 @@ const qp_table qp_table_420_12bpc_min = {
};
-const qp_table qp_table_422_12bpc_min = {
+static const qp_table qp_table_422_12bpc_min = {
{ 6, { 0, 4, 9, 10, 11, 11, 11, 11, 11, 11, 13, 13, 13, 16, 20} },
{ 6.5, { 0, 4, 9, 10, 11, 11, 11, 11, 11, 11, 13, 13, 13, 16, 20} },
{ 7, { 0, 4, 9, 10, 11, 11, 11, 11, 11, 11, 13, 13, 13, 15, 19} },
@@ -386,7 +386,7 @@ const qp_table qp_table_422_12bpc_min = {
};
-const qp_table qp_table_422_12bpc_max = {
+static const qp_table qp_table_422_12bpc_max = {
{ 6, {12, 12, 13, 14, 15, 15, 15, 16, 17, 18, 18, 19, 19, 20, 21} },
{ 6.5, {12, 12, 13, 14, 15, 15, 15, 16, 17, 18, 18, 19, 19, 20, 21} },
{ 7, {11, 12, 13, 14, 15, 15, 15, 16, 17, 17, 18, 18, 19, 19, 20} },
@@ -427,7 +427,7 @@ const qp_table qp_table_422_12bpc_max = {
};
-const qp_table qp_table_444_12bpc_max = {
+static const qp_table qp_table_444_12bpc_max = {
{ 6, {12, 14, 16, 16, 17, 17, 17, 18, 19, 20, 20, 20, 20, 21, 23} },
{ 6.5, {12, 14, 15, 16, 16, 16, 17, 18, 19, 19, 20, 20, 20, 21, 23} },
{ 7, {12, 13, 15, 15, 16, 16, 16, 17, 18, 19, 19, 20, 20, 21, 22} },
@@ -492,7 +492,7 @@ const qp_table qp_table_444_12bpc_max = {
};
-const qp_table qp_table_420_8bpc_min = {
+static const qp_table qp_table_420_8bpc_min = {
{ 4, { 0, 0, 1, 1, 3, 3, 3, 3, 3, 3, 5, 5, 5, 9, 13} },
{ 4.5, { 0, 0, 1, 1, 3, 3, 3, 3, 3, 3, 5, 5, 5, 8, 12} },
{ 5, { 0, 0, 1, 1, 3, 3, 3, 3, 3, 3, 5, 5, 5, 8, 12} },
@@ -513,7 +513,7 @@ const qp_table qp_table_420_8bpc_min = {
};
-const qp_table qp_table_422_8bpc_min = {
+static const qp_table qp_table_422_8bpc_min = {
{ 6, { 0, 0, 1, 2, 3, 3, 3, 3, 3, 3, 5, 5, 5, 8, 12} },
{ 6.5, { 0, 0, 1, 2, 3, 3, 3, 3, 3, 3, 5, 5, 5, 8, 12} },
{ 7, { 0, 0, 1, 2, 3, 3, 3, 3, 3, 3, 5, 5, 5, 7, 11} },
@@ -538,7 +538,7 @@ const qp_table qp_table_422_8bpc_min = {
};
-const qp_table qp_table_422_10bpc_max = {
+static const qp_table qp_table_422_10bpc_max = {
{ 6, { 8, 8, 9, 10, 11, 11, 11, 12, 13, 14, 14, 15, 15, 16, 17} },
{ 6.5, { 8, 8, 9, 10, 11, 11, 11, 12, 13, 14, 14, 15, 15, 16, 17} },
{ 7, { 7, 8, 9, 10, 11, 11, 11, 12, 13, 13, 14, 14, 15, 15, 16} },
@@ -571,7 +571,7 @@ const qp_table qp_table_422_10bpc_max = {
};
-const qp_table qp_table_420_10bpc_max = {
+static const qp_table qp_table_420_10bpc_max = {
{ 4, { 8, 8, 9, 10, 11, 11, 11, 12, 13, 14, 14, 15, 15, 17, 18} },
{ 4.5, { 8, 8, 9, 10, 11, 11, 11, 12, 13, 14, 14, 15, 15, 16, 17} },
{ 5, { 7, 8, 9, 10, 11, 11, 11, 12, 13, 13, 14, 14, 15, 16, 17} },
@@ -598,7 +598,7 @@ const qp_table qp_table_420_10bpc_max = {
};
-const qp_table qp_table_420_10bpc_min = {
+static const qp_table qp_table_420_10bpc_min = {
{ 4, { 0, 4, 4, 5, 7, 7, 7, 7, 7, 7, 9, 9, 9, 13, 17} },
{ 4.5, { 0, 4, 4, 5, 7, 7, 7, 7, 7, 7, 9, 9, 9, 12, 16} },
{ 5, { 0, 4, 4, 5, 7, 7, 7, 7, 7, 7, 9, 9, 9, 12, 16} },
@@ -625,7 +625,7 @@ const qp_table qp_table_420_10bpc_min = {
};
-const qp_table qp_table_444_10bpc_max = {
+static const qp_table qp_table_444_10bpc_max = {
{ 6, { 8, 10, 12, 12, 13, 13, 13, 14, 15, 16, 16, 16, 16, 17, 19} },
{ 6.5, { 8, 10, 11, 12, 12, 12, 13, 14, 15, 15, 16, 16, 16, 17, 19} },
{ 7, { 8, 9, 11, 11, 12, 12, 12, 13, 14, 15, 15, 16, 16, 17, 18} },
@@ -678,7 +678,7 @@ const qp_table qp_table_444_10bpc_max = {
};
-const qp_table qp_table_422_8bpc_max = {
+static const qp_table qp_table_422_8bpc_max = {
{ 6, { 4, 4, 5, 6, 7, 7, 7, 8, 9, 10, 10, 11, 11, 12, 13} },
{ 6.5, { 4, 4, 5, 6, 7, 7, 7, 8, 9, 10, 10, 11, 11, 12, 13} },
{ 7, { 3, 4, 5, 6, 7, 7, 7, 8, 9, 9, 10, 10, 11, 11, 12} },
diff --git a/drivers/gpu/drm/amd/display/dc/gpio/dcn32/hw_factory_dcn32.c b/drivers/gpu/drm/amd/display/dc/gpio/dcn32/hw_factory_dcn32.c
index 0ea52ba..9fd8b26 100644
--- a/drivers/gpu/drm/amd/display/dc/gpio/dcn32/hw_factory_dcn32.c
+++ b/drivers/gpu/drm/amd/display/dc/gpio/dcn32/hw_factory_dcn32.c
@@ -256,8 +256,8 @@ static const struct hw_factory_funcs funcs = {
*/
void dal_hw_factory_dcn32_init(struct hw_factory *factory)
{
- factory->number_of_pins[GPIO_ID_DDC_DATA] = 6;
- factory->number_of_pins[GPIO_ID_DDC_CLOCK] = 6;
+ factory->number_of_pins[GPIO_ID_DDC_DATA] = 8;
+ factory->number_of_pins[GPIO_ID_DDC_CLOCK] = 8;
factory->number_of_pins[GPIO_ID_GENERIC] = 4;
factory->number_of_pins[GPIO_ID_HPD] = 5;
factory->number_of_pins[GPIO_ID_GPIO_PAD] = 28;
diff --git a/drivers/gpu/drm/amd/display/dc/inc/core_types.h b/drivers/gpu/drm/amd/display/dc/inc/core_types.h
index 9498105..525f8f0 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/core_types.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/core_types.h
@@ -115,6 +115,13 @@ struct resource_funcs {
int vlevel);
void (*update_soc_for_wm_a)(
struct dc *dc, struct dc_state *context);
+
+ /**
+ * @populate_dml_pipes - Populate pipe data struct
+ *
+ * Returns:
+ * Total of pipes available in the specific ASIC.
+ */
int (*populate_dml_pipes)(
struct dc *dc,
struct dc_state *context,
@@ -233,8 +240,11 @@ struct resource_funcs {
unsigned int pipe_cnt,
unsigned int index);
- bool (*remove_phantom_pipes)(struct dc *dc, struct dc_state *context);
+ bool (*remove_phantom_pipes)(struct dc *dc, struct dc_state *context, bool fast_update);
+ void (*retain_phantom_pipes)(struct dc *dc, struct dc_state *context);
void (*get_panel_config_defaults)(struct dc_panel_config *panel_config);
+ void (*save_mall_state)(struct dc *dc, struct dc_state *context, struct mall_temp_config *temp_config);
+ void (*restore_mall_state)(struct dc *dc, struct dc_state *context, struct mall_temp_config *temp_config);
};
struct audio_support{
@@ -413,7 +423,10 @@ struct pipe_ctx {
struct pll_settings pll_settings;
- /* link config records software decision for what link config should be
+ /**
+ * @link_config:
+ *
+ * link config records software decision for what link config should be
* enabled given current link capability and stream during hw resource
* mapping. This is to decouple the dependency on link capability during
* dc commit or update.
@@ -507,33 +520,62 @@ struct bw_context {
union bw_output bw;
struct display_mode_lib dml;
};
+
/**
- * struct dc_state - The full description of a state requested by a user
- *
- * @streams: Stream properties
- * @stream_status: The planes on a given stream
- * @res_ctx: Persistent state of resources
- * @bw_ctx: The output from bandwidth and watermark calculations and the DML
- * @pp_display_cfg: PowerPlay clocks and settings
- * @dcn_bw_vars: non-stack memory to support bandwidth calculations
- *
+ * struct dc_state - The full description of a state requested by users
*/
struct dc_state {
+ /**
+ * @streams: Stream state properties
+ */
struct dc_stream_state *streams[MAX_PIPES];
+
+ /**
+ * @stream_status: Planes status on a given stream
+ */
struct dc_stream_status stream_status[MAX_PIPES];
+
+ /**
+ * @stream_count: Total of streams in use
+ */
uint8_t stream_count;
uint8_t stream_mask;
+ /**
+ * @res_ctx: Persistent state of resources
+ */
struct resource_context res_ctx;
+ /**
+ * @bw_ctx: The output from bandwidth and watermark calculations and the DML
+ *
+ * Each context must have its own instance of VBA, and in order to
+ * initialize and obtain IP and SOC, the base DML instance from DC is
+ * initially copied into every context.
+ */
struct bw_context bw_ctx;
- /* Note: these are big structures, do *not* put on stack! */
+ /**
+ * @pp_display_cfg: PowerPlay clocks and settings
+ * Note: this is a big struct, do *not* put on stack!
+ */
struct dm_pp_display_configuration pp_display_cfg;
+
+ /**
+ * @dcn_bw_vars: non-stack memory to support bandwidth calculations
+ * Note: this is a big struct, do *not* put on stack!
+ */
struct dcn_bw_internal_vars dcn_bw_vars;
struct clk_mgr *clk_mgr;
+ /**
+ * @refcount: refcount reference
+ *
+ * Notice that dc_state is used around the code to capture the current
+ * context, so we need to pass it everywhere. That's why we want to use
+ * kref in this struct.
+ */
struct kref refcount;
struct {
diff --git a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
index b304d45..e8d8c5c 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
@@ -193,7 +193,7 @@ enum dc_status dpcd_configure_lttpr_mode(
struct link_training_settings *lt_settings);
enum dp_link_encoding dp_get_link_encoding_format(const struct dc_link_settings *link_settings);
-bool dp_retrieve_lttpr_cap(struct dc_link *link);
+enum dc_status dp_retrieve_lttpr_cap(struct dc_link *link);
bool dp_is_lttpr_present(struct dc_link *link);
enum lttpr_mode dp_decide_lttpr_mode(struct dc_link *link, struct dc_link_settings *link_setting);
void dp_get_lttpr_mode_override(struct dc_link *link, enum lttpr_mode *override);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/dchubbub.h b/drivers/gpu/drm/amd/display/dc/inc/hw/dchubbub.h
index e7571c6f5..f2e1fcb 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/dchubbub.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/dchubbub.h
@@ -167,10 +167,26 @@ struct hubbub_funcs {
void (*force_pstate_change_control)(struct hubbub *hubbub, bool force, bool allow);
void (*init_watermarks)(struct hubbub *hubbub);
+
+ /**
+ * @program_det_size:
+ *
+ * DE-Tile buffers (DET) is a memory that is used to convert the tiled
+ * data into linear, which the rest of the display can use to generate
+ * the graphics output. One of the main features of this component is
+ * that each pipe has a configurable DET buffer which means that when a
+ * pipe is not enabled, the device can assign the memory to other
+ * enabled pipes to try to be more efficient.
+ *
+ * DET logic is handled by dchubbub. Some ASICs provide a feature named
+ * Configurable Return Buffer (CRB) segments which can be allocated to
+ * compressed or detiled buffers.
+ */
void (*program_det_size)(struct hubbub *hubbub, int hubp_inst, unsigned det_buffer_size_in_kbyte);
void (*program_compbuf_size)(struct hubbub *hubbub, unsigned compbuf_size_kb, bool safe_to_increase);
void (*init_crb)(struct hubbub *hubbub);
void (*force_usr_retraining_allow)(struct hubbub *hubbub, bool allow);
+ void (*set_request_limit)(struct hubbub *hubbub, int memory_channel_count, int words_per_channel);
};
struct hubbub {
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h b/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h
index 8df2765..de3113e 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h
@@ -56,20 +56,6 @@ struct dmcu {
bool auto_load_dmcu;
};
-#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
-struct crc_region {
- uint16_t x_start;
- uint16_t y_start;
- uint16_t x_end;
- uint16_t y_end;
-};
-
-struct otg_phy_mux {
- uint8_t phy_output_num;
- uint8_t otg_output_num;
-};
-#endif
-
struct dmcu_funcs {
bool (*dmcu_init)(struct dmcu *dmcu);
bool (*load_iram)(struct dmcu *dmcu,
@@ -100,7 +86,7 @@ struct dmcu_funcs {
bool (*recv_edid_cea_ack)(struct dmcu *dmcu, int *offset);
#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
void (*forward_crc_window)(struct dmcu *dmcu,
- struct crc_region *crc_win,
+ struct rect *rect,
struct otg_phy_mux *mux_mapping);
void (*stop_crc_win_update)(struct dmcu *dmcu,
struct otg_phy_mux *mux_mapping);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/dpp.h b/drivers/gpu/drm/amd/display/dc/inc/hw/dpp.h
index dcb80c4..131fcfa 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/dpp.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/dpp.h
@@ -83,10 +83,15 @@ static const struct dpp_input_csc_matrix __maybe_unused dpp_input_csc_matrix[] =
{COLOR_SPACE_YCBCR709,
{0x3265, 0x2000, 0, 0xe6ce, 0xf105, 0x2000, 0xfa01, 0xa7d, 0,
0x2000, 0x3b61, 0xe24f} },
-
{COLOR_SPACE_YCBCR709_LIMITED,
{0x39a6, 0x2568, 0, 0xe0d6, 0xeedd, 0x2568, 0xf925, 0x9a8, 0,
- 0x2568, 0x43ee, 0xdbb2} }
+ 0x2568, 0x43ee, 0xdbb2} },
+ {COLOR_SPACE_2020_YCBCR,
+ {0x2F30, 0x2000, 0, 0xE869, 0xEDB7, 0x2000, 0xFABC, 0xBC6, 0,
+ 0x2000, 0x3C34, 0xE1E6} },
+ {COLOR_SPACE_2020_RGB_LIMITEDRANGE,
+ {0x35E0, 0x255F, 0, 0xE2B3, 0xEB20, 0x255F, 0xF9FD, 0xB1E, 0,
+ 0x255F, 0x44BD, 0xDB43} }
};
struct dpp_grph_csc_adjustment {
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h b/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h
index cd2be72..a819f0f 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h
@@ -35,6 +35,13 @@
******************************************************************************/
#define MAX_AUDIOS 7
+
+/**
+ * @MAX_PIPES:
+ *
+ * Every ASIC support a fixed number of pipes; MAX_PIPES defines a large number
+ * to be used inside loops and for determining array sizes.
+ */
#define MAX_PIPES 6
#define MAX_DIG_LINK_ENCODERS 7
#define MAX_DWB_PIPES 1
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h b/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h
index 42afa19..42db4b7 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h
@@ -243,6 +243,9 @@ struct stream_encoder_funcs {
uint32_t hubp_requestor_id,
enum dynamic_metadata_mode dmdata_mode);
+ /**
+ * @dp_set_odm_combine: Sets up DP stream encoder for ODM.
+ */
void (*dp_set_odm_combine)(
struct stream_encoder *enc,
bool odm_combine);
@@ -317,9 +320,6 @@ struct hpo_dp_stream_encoder_funcs {
uint32_t stream_enc_inst,
uint32_t link_enc_inst);
- void (*audio_mute_control)(
- struct hpo_dp_stream_encoder *enc, bool mute);
-
void (*dp_audio_setup)(
struct hpo_dp_stream_encoder *enc,
unsigned int az_inst,
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h b/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
index 25a1df4..0e42e721 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
@@ -185,6 +185,7 @@ struct timing_generator_funcs {
#ifdef CONFIG_DRM_AMD_DC_DCN
void (*phantom_crtc_post_enable)(struct timing_generator *tg);
#endif
+ void (*disable_phantom_crtc)(struct timing_generator *tg);
bool (*immediate_disable_crtc)(struct timing_generator *tg);
bool (*is_counter_moving)(struct timing_generator *tg);
void (*get_position)(struct timing_generator *tg,
@@ -301,6 +302,11 @@ struct timing_generator_funcs {
void (*get_dsc_status)(struct timing_generator *optc,
uint32_t *dsc_mode);
void (*set_odm_bypass)(struct timing_generator *optc, const struct dc_crtc_timing *dc_crtc_timing);
+
+ /**
+ * @set_odm_combine: Set up the ODM block to read from the correct
+ * OPP(s) and turn on/off ODM memory.
+ */
void (*set_odm_combine)(struct timing_generator *optc, int *opp_id, int opp_cnt,
struct dc_crtc_timing *timing);
void (*set_h_timing_div_manual_mode)(struct timing_generator *optc, bool manual_mode);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
index d04b68d..c43523f 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
@@ -263,6 +263,7 @@ struct hw_sequencer_funcs {
void (*update_phantom_vp_position)(struct dc *dc,
struct dc_state *context,
struct pipe_ctx *phantom_pipe);
+ void (*apply_update_flags_for_phantom)(struct pipe_ctx *phantom_pipe);
void (*commit_subvp_config)(struct dc *dc, struct dc_state *context);
void (*subvp_pipe_control_lock)(struct dc *dc,
diff --git a/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h b/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h
index 89964c9..0f69946 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/link_hwss.h
@@ -38,6 +38,7 @@ struct link_resource;
struct pipe_ctx;
struct encoder_set_dp_phy_pattern_param;
struct link_mst_stream_allocation_table;
+struct audio_output;
struct link_hwss_ext {
/* function pointers below may require to check for NULL if caller
@@ -79,6 +80,10 @@ struct link_hwss {
void (*disable_link_output)(struct dc_link *link,
const struct link_resource *link_res,
enum signal_type signal);
+ void (*setup_audio_output)(struct pipe_ctx *pipe_ctx,
+ struct audio_output *audio_output, uint32_t audio_inst);
+ void (*enable_audio_packet)(struct pipe_ctx *pipe_ctx);
+ void (*disable_audio_packet)(struct pipe_ctx *pipe_ctx);
};
#endif /* __DC_LINK_HWSS_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c b/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c
index 45f9935..5f4f6dd 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.c
@@ -28,20 +28,19 @@
#include "include/logger_interface.h"
#include "../dce110/irq_service_dce110.h"
+#include "irq_service_dcn201.h"
#include "dcn/dcn_2_0_3_offset.h"
#include "dcn/dcn_2_0_3_sh_mask.h"
#include "cyan_skillfish_ip_offset.h"
#include "soc15_hw_ip.h"
-
-#include "irq_service_dcn201.h"
-
#include "ivsrcid/dcn/irqsrcs_dcn_1_0.h"
-static enum dc_irq_source to_dal_irq_source_dcn201(struct irq_service *irq_service,
- uint32_t src_id,
- uint32_t ext_id)
+enum dc_irq_source to_dal_irq_source_dcn201(
+ struct irq_service *irq_service,
+ uint32_t src_id,
+ uint32_t ext_id)
{
switch (src_id) {
case DCN_1_0__SRCID__DC_D1_OTG_VSTARTUP:
@@ -79,7 +78,6 @@ static enum dc_irq_source to_dal_irq_source_dcn201(struct irq_service *irq_servi
default:
return DC_IRQ_SOURCE_INVALID;
}
- return DC_IRQ_SOURCE_INVALID;
}
static bool hpd_ack(
@@ -138,6 +136,11 @@ static const struct irq_source_info_funcs vupdate_no_lock_irq_info_funcs = {
.ack = NULL
};
+static const struct irq_source_info_funcs dmub_outbox_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
#undef BASE_INNER
#define BASE_INNER(seg) DMU_BASE__INST0_SEG ## seg
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.h b/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.h
index 8e27c5e..0cfd2f2 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.h
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn201/irq_service_dcn201.h
@@ -1,5 +1,5 @@
/*
- * Copyright 2018 Advanced Micro Devices, Inc.
+ * Copyright 2022 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
diff --git a/drivers/gpu/drm/amd/display/dc/irq/irq_service.c b/drivers/gpu/drm/amd/display/dc/irq/irq_service.c
index 7bad39b..d100edae 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/irq_service.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/irq_service.c
@@ -112,8 +112,15 @@ bool dal_irq_service_set(
dal_irq_service_ack(irq_service, source);
- if (info->funcs && info->funcs->set)
+ if (info->funcs && info->funcs->set) {
+ if (info->funcs->set == dal_irq_service_dummy_set) {
+ DC_LOG_WARNING("%s: src: %d, st: %d\n", __func__,
+ source, enable);
+ ASSERT(0);
+ }
+
return info->funcs->set(irq_service, info, enable);
+ }
dal_irq_service_set_generic(irq_service, info, enable);
@@ -146,8 +153,14 @@ bool dal_irq_service_ack(
return false;
}
- if (info->funcs && info->funcs->ack)
+ if (info->funcs && info->funcs->ack) {
+ if (info->funcs->ack == dal_irq_service_dummy_ack) {
+ DC_LOG_WARNING("%s: src: %d\n", __func__, source);
+ ASSERT(0);
+ }
+
return info->funcs->ack(irq_service, info);
+ }
dal_irq_service_ack_generic(irq_service, info);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c b/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.c
similarity index 61%
copy from drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
copy to drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.c
index 217268f..801a95b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
+++ b/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.c
@@ -1,5 +1,6 @@
+
/*
- * Copyright 2018 Red Hat Inc.
+ * Copyright 2022 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -18,28 +19,10 @@
* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
*/
-#include "user.h"
-
-static int
-tu102_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc,
- enum nvkm_object_map *type, u64 *addr, u64 *size)
-{
- struct nvkm_device *device = object->engine->subdev.device;
- *addr = 0xbb0000 + device->func->resource_addr(device, 0);
- *size = 0x010000;
- *type = NVKM_OBJECT_MAP_IO;
- return 0;
-}
-
-static const struct nvkm_object_func
-tu102_fifo_user = {
- .map = tu102_fifo_user_map,
-};
-
-int
-tu102_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
- struct nvkm_object **pobject)
-{
- return nvkm_object_new_(&tu102_fifo_user, oclass, argv, argc, pobject);
-}
+/*********************************************************************/
+// USB4 DPIA BANDWIDTH ALLOCATION LOGIC
+/*********************************************************************/
diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.h b/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.h
new file mode 100644
index 0000000..669e995f8
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.h
@@ -0,0 +1,69 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef DC_INC_LINK_DP_DPIA_BW_H_
+#define DC_INC_LINK_DP_DPIA_BW_H_
+
+// XXX: TODO: Re-add for Phase 2
+/* Number of Host Routers per motherboard is 2 and 2 DPIA per host router */
+#define MAX_HR_NUM 2
+
+struct dc_host_router_bw_alloc {
+ int max_bw[MAX_HR_NUM]; // The Max BW that each Host Router has available to be shared btw DPIAs
+ int total_estimated_bw[MAX_HR_NUM]; // The Total Verified and available BW that Host Router has
+};
+
+/*
+ * Enable BW Allocation Mode Support from the DP-Tx side
+ *
+ * @link: pointer to the dc_link struct instance
+ *
+ * return: SUCCESS or FAILURE
+ */
+bool set_dptx_usb4_bw_alloc_support(struct dc_link *link);
+
+/*
+ * Send a request from DP-Tx requesting to allocate BW remotely after
+ * allocating it locally. This will get processed by CM and a CB function
+ * will be called.
+ *
+ * @link: pointer to the dc_link struct instance
+ * @req_bw: The requested bw in Kbyte to allocated
+ *
+ * return: none
+ */
+void set_usb4_req_bw_req(struct dc_link *link, int req_bw);
+
+/*
+ * CB function for when the status of the Req above is complete. We will
+ * find out the result of allocating on CM and update structs accordingly
+ *
+ * @link: pointer to the dc_link struct instance
+ *
+ * return: none
+ */
+void get_usb4_req_bw_resp(struct dc_link *link);
+
+#endif /* DC_INC_LINK_DP_DPIA_BW_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c b/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c
index 4227adb..33148b7 100644
--- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c
+++ b/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c
@@ -170,11 +170,63 @@ static void update_dio_stream_allocation_table(struct dc_link *link,
link_enc->funcs->update_mst_stream_allocation_table(link_enc, table);
}
+void setup_dio_audio_output(struct pipe_ctx *pipe_ctx,
+ struct audio_output *audio_output, uint32_t audio_inst)
+{
+ if (dc_is_dp_signal(pipe_ctx->stream->signal))
+ pipe_ctx->stream_res.stream_enc->funcs->dp_audio_setup(
+ pipe_ctx->stream_res.stream_enc,
+ audio_inst,
+ &pipe_ctx->stream->audio_info);
+ else
+ pipe_ctx->stream_res.stream_enc->funcs->hdmi_audio_setup(
+ pipe_ctx->stream_res.stream_enc,
+ audio_inst,
+ &pipe_ctx->stream->audio_info,
+ &audio_output->crtc_info);
+}
+
+void enable_dio_audio_packet(struct pipe_ctx *pipe_ctx)
+{
+ if (dc_is_dp_signal(pipe_ctx->stream->signal))
+ pipe_ctx->stream_res.stream_enc->funcs->dp_audio_enable(
+ pipe_ctx->stream_res.stream_enc);
+
+ pipe_ctx->stream_res.stream_enc->funcs->audio_mute_control(
+ pipe_ctx->stream_res.stream_enc, false);
+
+ if (dc_is_dp_signal(pipe_ctx->stream->signal))
+ dp_source_sequence_trace(pipe_ctx->stream->link,
+ DPCD_SOURCE_SEQ_AFTER_ENABLE_AUDIO_STREAM);
+}
+
+void disable_dio_audio_packet(struct pipe_ctx *pipe_ctx)
+{
+ pipe_ctx->stream_res.stream_enc->funcs->audio_mute_control(
+ pipe_ctx->stream_res.stream_enc, true);
+
+ if (pipe_ctx->stream_res.audio) {
+ if (dc_is_dp_signal(pipe_ctx->stream->signal))
+ pipe_ctx->stream_res.stream_enc->funcs->dp_audio_disable(
+ pipe_ctx->stream_res.stream_enc);
+ else
+ pipe_ctx->stream_res.stream_enc->funcs->hdmi_audio_disable(
+ pipe_ctx->stream_res.stream_enc);
+ }
+
+ if (dc_is_dp_signal(pipe_ctx->stream->signal))
+ dp_source_sequence_trace(pipe_ctx->stream->link,
+ DPCD_SOURCE_SEQ_AFTER_DISABLE_AUDIO_STREAM);
+}
+
static const struct link_hwss dio_link_hwss = {
.setup_stream_encoder = setup_dio_stream_encoder,
.reset_stream_encoder = reset_dio_stream_encoder,
.setup_stream_attribute = setup_dio_stream_attribute,
.disable_link_output = disable_dio_link_output,
+ .setup_audio_output = setup_dio_audio_output,
+ .enable_audio_packet = enable_dio_audio_packet,
+ .disable_audio_packet = disable_dio_audio_packet,
.ext = {
.set_throttled_vcp_size = set_dio_throttled_vcp_size,
.enable_dp_link_output = enable_dio_dp_link_output,
diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h b/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h
index 126d37f..9a108c3 100644
--- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h
+++ b/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h
@@ -50,5 +50,9 @@ void set_dio_dp_lane_settings(struct dc_link *link,
const struct link_resource *link_res,
const struct dc_link_settings *link_settings,
const struct dc_lane_settings lane_settings[LANE_COUNT_DP_MAX]);
+void setup_dio_audio_output(struct pipe_ctx *pipe_ctx,
+ struct audio_output *audio_output, uint32_t audio_inst);
+void enable_dio_audio_packet(struct pipe_ctx *pipe_ctx);
+void disable_dio_audio_packet(struct pipe_ctx *pipe_ctx);
#endif /* __LINK_HWSS_DIO_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.c b/drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.c
index 64f7ea6..861f3cd 100644
--- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.c
+++ b/drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.c
@@ -57,6 +57,9 @@ static const struct link_hwss dpia_link_hwss = {
.reset_stream_encoder = reset_dio_stream_encoder,
.setup_stream_attribute = setup_dio_stream_attribute,
.disable_link_output = disable_dio_link_output,
+ .setup_audio_output = setup_dio_audio_output,
+ .enable_audio_packet = enable_dio_audio_packet,
+ .disable_audio_packet = disable_dio_audio_packet,
.ext = {
.set_throttled_vcp_size = set_dio_throttled_vcp_size,
.enable_dp_link_output = enable_dio_dp_link_output,
diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c b/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c
index 153a883..2f46e1a 100644
--- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c
+++ b/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c
@@ -262,11 +262,36 @@ static void update_hpo_dp_stream_allocation_table(struct dc_link *link,
table);
}
+static void setup_hpo_dp_audio_output(struct pipe_ctx *pipe_ctx,
+ struct audio_output *audio_output, uint32_t audio_inst)
+{
+ pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_audio_setup(
+ pipe_ctx->stream_res.hpo_dp_stream_enc,
+ audio_inst,
+ &pipe_ctx->stream->audio_info);
+}
+
+static void enable_hpo_dp_audio_packet(struct pipe_ctx *pipe_ctx)
+{
+ pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_audio_enable(
+ pipe_ctx->stream_res.hpo_dp_stream_enc);
+}
+
+static void disable_hpo_dp_audio_packet(struct pipe_ctx *pipe_ctx)
+{
+ if (pipe_ctx->stream_res.audio)
+ pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_audio_disable(
+ pipe_ctx->stream_res.hpo_dp_stream_enc);
+}
+
static const struct link_hwss hpo_dp_link_hwss = {
.setup_stream_encoder = setup_hpo_dp_stream_encoder,
.reset_stream_encoder = reset_hpo_dp_stream_encoder,
.setup_stream_attribute = setup_hpo_dp_stream_attribute,
.disable_link_output = disable_hpo_dp_link_output,
+ .setup_audio_output = setup_hpo_dp_audio_output,
+ .enable_audio_packet = enable_hpo_dp_audio_packet,
+ .disable_audio_packet = disable_hpo_dp_audio_packet,
.ext = {
.set_throttled_vcp_size = set_hpo_dp_throttled_vcp_size,
.set_hblank_min_symbol_width = set_hpo_dp_hblank_min_symbol_width,
diff --git a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
index 7a8f615..33907fe 100644
--- a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
+++ b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
@@ -225,6 +225,12 @@ union dmub_psr_debug_flags {
* Use TPS3 signal when restore main link.
*/
uint32_t force_wakeup_by_tps3 : 1;
+
+ /**
+ * Back to back flip, therefore cannot power down PHY
+ */
+ uint32_t back_to_back_flip : 1;
+
} bitfields;
/**
@@ -401,8 +407,9 @@ union dmub_fw_boot_options {
uint32_t gpint_scratch8: 1; /* 1 if GPINT is in scratch8*/
uint32_t usb4_cm_version: 1; /**< 1 CM support */
uint32_t dpia_hpd_int_enable_supported: 1; /* 1 if dpia hpd int enable supported */
+ uint32_t usb4_dpia_bw_alloc_supported: 1; /* 1 if USB4 dpia BW allocation supported */
- uint32_t reserved : 16; /**< reserved */
+ uint32_t reserved : 15; /**< reserved */
} bits; /**< boot bits */
uint32_t all; /**< 32-bit access to bits */
};
@@ -731,6 +738,11 @@ enum dmub_cmd_type {
*/
/**
+ * Command type used for all SECURE_DISPLAY commands.
+ */
+ DMUB_CMD__SECURE_DISPLAY = 85,
+
+ /**
* Command type used to set DPIA HPD interrupt state
*/
DMUB_CMD__DPIA_HPD_INT_ENABLE = 86,
@@ -1017,13 +1029,14 @@ struct dmub_cmd_fw_assisted_mclk_switch_pipe_data_v2 {
uint16_t vtotal;
uint16_t htotal;
uint8_t vblank_pipe_index;
- uint8_t padding[2];
+ uint8_t padding[1];
struct {
uint8_t drr_in_use;
uint8_t drr_window_size_ms; // Indicates largest VMIN/VMAX adjustment per frame
uint16_t min_vtotal_supported; // Min VTOTAL that supports switching in VBLANK
uint16_t max_vtotal_supported; // Max VTOTAL that can support SubVP static scheduling
uint8_t use_ramping; // Use ramping or not
+ uint8_t drr_vblank_start_margin;
} drr_info; // DRR considered as part of SubVP + VBLANK case
} vblank_data;
} pipe_config;
@@ -1866,9 +1879,13 @@ struct dmub_cmd_psr_copy_settings_data {
*/
uint8_t use_phy_fsm;
/**
+ * frame delay for frame re-lock
+ */
+ uint8_t relock_delay_frame_cnt;
+ /**
* Explicit padding to 2 byte boundary.
*/
- uint8_t pad3[2];
+ uint8_t pad3;
};
/**
@@ -3144,6 +3161,33 @@ struct dmub_rb_cmd_get_usbc_cable_id {
};
/**
+ * Command type of a DMUB_CMD__SECURE_DISPLAY command
+ */
+enum dmub_cmd_secure_display_type {
+ DMUB_CMD__SECURE_DISPLAY_TEST_CMD = 0, /* test command to only check if inbox message works */
+ DMUB_CMD__SECURE_DISPLAY_CRC_STOP_UPDATE,
+ DMUB_CMD__SECURE_DISPLAY_CRC_WIN_NOTIFY
+};
+
+/**
+ * Definition of a DMUB_CMD__SECURE_DISPLAY command
+ */
+struct dmub_rb_cmd_secure_display {
+ struct dmub_cmd_header header;
+ /**
+ * Data passed from driver to dmub firmware.
+ */
+ struct dmub_cmd_roi_info {
+ uint16_t x_start;
+ uint16_t x_end;
+ uint16_t y_start;
+ uint16_t y_end;
+ uint8_t otg_id;
+ uint8_t phy_id;
+ } roi_info;
+};
+
+/**
* union dmub_rb_cmd - DMUB inbox command.
*/
union dmub_rb_cmd {
@@ -3348,6 +3392,11 @@ union dmub_rb_cmd {
*/
struct dmub_rb_cmd_query_hpd_state query_hpd;
/**
+ * Definition of a DMUB_CMD__SECURE_DISPLAY command.
+ */
+ struct dmub_rb_cmd_secure_display secure_display;
+
+ /**
* Definition of a DMUB_CMD__DPIA_HPD_INT_ENABLE command.
*/
struct dmub_rb_cmd_dpia_hpd_int_enable dpia_hpd_int_enable;
diff --git a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
index 447a0ec..f603421 100644
--- a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
+++ b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
@@ -61,7 +61,7 @@ static const int32_t numerator01[] = { 31308, 180000, 0, 0, 0};
static const int32_t numerator02[] = { 12920, 4500, 0, 0, 0};
static const int32_t numerator03[] = { 55, 99, 0, 0, 0};
static const int32_t numerator04[] = { 55, 99, 0, 0, 0};
-static const int32_t numerator05[] = { 2400, 2200, 2200, 2400, 2600};
+static const int32_t numerator05[] = { 2400, 2222, 2200, 2400, 2600};
/* one-time setup of X points */
void setup_x_points_distribution(void)
diff --git a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
index 0f39ab9..c2e00f7 100644
--- a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
+++ b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
@@ -688,10 +688,10 @@ static void build_vrr_infopacket_fs2_data(enum color_transfer_func app_tf,
if (app_tf != TRANSFER_FUNC_UNKNOWN) {
infopacket->valid = true;
- infopacket->sb[6] |= 0x08; // PB6 = [Bit 3 = Native Color Active]
-
- if (app_tf == TRANSFER_FUNC_GAMMA_22) {
- infopacket->sb[9] |= 0x04; // PB6 = [Bit 2 = Gamma 2.2 EOTF Active]
+ if (app_tf != TRANSFER_FUNC_PQ2084) {
+ infopacket->sb[6] |= 0x08; // PB6 = [Bit 3 = Native Color Active]
+ if (app_tf == TRANSFER_FUNC_GAMMA_22)
+ infopacket->sb[9] |= 0x04; // PB6 = [Bit 2 = Gamma 2.2 EOTF Active]
}
}
}
diff --git a/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h b/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h
index 1d8b746..edf5845 100644
--- a/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h
+++ b/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h
@@ -35,7 +35,8 @@ struct mod_vrr_params;
void mod_build_vsc_infopacket(const struct dc_stream_state *stream,
struct dc_info_packet *info_packet,
- enum dc_color_space cs);
+ enum dc_color_space cs,
+ enum color_transfer_func tf);
void mod_build_hf_vsif_infopacket(const struct dc_stream_state *stream,
struct dc_info_packet *info_packet);
diff --git a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c
index 27ceba9..6969105 100644
--- a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c
+++ b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c
@@ -132,7 +132,8 @@ enum ColorimetryYCCDP {
void mod_build_vsc_infopacket(const struct dc_stream_state *stream,
struct dc_info_packet *info_packet,
- enum dc_color_space cs)
+ enum dc_color_space cs,
+ enum color_transfer_func tf)
{
unsigned int vsc_packet_revision = vsc_packet_undefined;
unsigned int i;
@@ -382,6 +383,9 @@ void mod_build_vsc_infopacket(const struct dc_stream_state *stream,
colorimetryFormat = ColorimetryYCC_DP_AdobeYCC;
else if (cs == COLOR_SPACE_2020_YCBCR)
colorimetryFormat = ColorimetryYCC_DP_ITU2020YCbCr;
+
+ if (cs == COLOR_SPACE_2020_YCBCR && tf == TRANSFER_FUNC_GAMMA_22)
+ colorimetryFormat = ColorimetryYCC_DP_ITU709;
break;
default:
diff --git a/drivers/gpu/drm/amd/display/modules/power/power_helpers.c b/drivers/gpu/drm/amd/display/modules/power/power_helpers.c
index 235259d6..9b5d9b2 100644
--- a/drivers/gpu/drm/amd/display/modules/power/power_helpers.c
+++ b/drivers/gpu/drm/amd/display/modules/power/power_helpers.c
@@ -102,9 +102,18 @@ static const struct abm_parameters abm_settings_config1[abm_defines_max_level] =
{0x82, 0x4d, 0x20, 0x00, 0x00, 0xff, 0xb3, 0x70, 0x70, 0xcccc, 0xcccc},
};
+static const struct abm_parameters abm_settings_config2[abm_defines_max_level] = {
+// min_red max_red bright_pos dark_pos bright_gain contrast dev min_knee max_knee blRed blStart
+ {0xf0, 0xbf, 0x20, 0x00, 0x88, 0x99, 0xb3, 0x40, 0xe0, 0x0000, 0xcccc},
+ {0xd8, 0x85, 0x20, 0x00, 0x70, 0x90, 0xa8, 0x40, 0xc8, 0x0700, 0xb333},
+ {0xb8, 0x58, 0x20, 0x00, 0x64, 0x88, 0x78, 0x70, 0xa0, 0x7000, 0x9999},
+ {0x82, 0x40, 0x20, 0x00, 0x00, 0xb8, 0xb3, 0x70, 0x70, 0xc333, 0xb333},
+};
+
static const struct abm_parameters * const abm_settings[] = {
abm_settings_config0,
abm_settings_config1,
+ abm_settings_config2,
};
#define NUM_AMBI_LEVEL 5
diff --git a/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h
index 483769f..537aee0 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: MIT
#ifndef _dcn_3_0_0_OFFSET_HEADER
#define _dcn_3_0_0_OFFSET_HEADER
diff --git a/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_sh_mask.h
index b79be3a..f9d90b0 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_sh_mask.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_sh_mask.h
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: MIT
#ifndef _dcn_3_0_0_SH_MASK_HEADER
#define _dcn_3_0_0_SH_MASK_HEADER
diff --git a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_offset.h b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_offset.h
index 363d213..db7e227 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_offset.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_offset.h
@@ -993,7 +993,8 @@
#define mmUVD_RAS_VCPU_VCODEC_STATUS_BASE_IDX 1
#define mmUVD_RAS_MMSCH_FATAL_ERROR 0x0058
#define mmUVD_RAS_MMSCH_FATAL_ERROR_BASE_IDX 1
-
+#define mmVCN_RAS_CNTL 0x04b9
+#define mmVCN_RAS_CNTL_BASE_IDX 1
/* JPEG 2_6_0 regs */
#define mmUVD_RAS_JPEG0_STATUS 0x0059
diff --git a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_sh_mask.h
index 8de883b7..874a8b7 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_sh_mask.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_5_sh_mask.h
@@ -3618,6 +3618,33 @@
#define UVD_RAS_MMSCH_FATAL_ERROR__POISONED_VF_MASK 0x7FFFFFFFL
#define UVD_RAS_MMSCH_FATAL_ERROR__POISONED_PF_MASK 0x80000000L
+//VCN 2_6_0 VCN_RAS_CNTL
+#define VCN_RAS_CNTL__VCPU_VCODEC_IH_EN__SHIFT 0x0
+#define VCN_RAS_CNTL__MMSCH_FATAL_ERROR_EN__SHIFT 0x1
+#define VCN_RAS_CNTL__VCPU_VCODEC_PMI_EN__SHIFT 0x4
+#define VCN_RAS_CNTL__MMSCH_PMI_EN__SHIFT 0x5
+#define VCN_RAS_CNTL__VCPU_VCODEC_REARM__SHIFT 0x8
+#define VCN_RAS_CNTL__MMSCH_REARM__SHIFT 0x9
+#define VCN_RAS_CNTL__VCPU_VCODEC_STALL_EN__SHIFT 0xc
+#define VCN_RAS_CNTL__VCPU_VCODEC_READY__SHIFT 0x10
+#define VCN_RAS_CNTL__MMSCH_READY__SHIFT 0x11
+#define VCN_RAS_CNTL__VCPU_VCODEC_IH_EN_MASK 0x00000001L
+#define VCN_RAS_CNTL__MMSCH_FATAL_ERROR_EN_MASK 0x00000002L
+#define VCN_RAS_CNTL__VCPU_VCODEC_PMI_EN_MASK 0x00000010L
+#define VCN_RAS_CNTL__MMSCH_PMI_EN_MASK 0x00000020L
+#define VCN_RAS_CNTL__VCPU_VCODEC_REARM_MASK 0x00000100L
+#define VCN_RAS_CNTL__MMSCH_REARM_MASK 0x00000200L
+#define VCN_RAS_CNTL__VCPU_VCODEC_STALL_EN_MASK 0x00001000L
+#define VCN_RAS_CNTL__VCPU_VCODEC_READY_MASK 0x00010000L
+#define VCN_RAS_CNTL__MMSCH_READY_MASK 0x00020000L
+
+//VCN 2_6_0 UVD_VCPU_INT_EN
+#define UVD_VCPU_INT_EN__RASCNTL_VCPU_VCODEC_EN__SHIFT 0x16
+#define UVD_VCPU_INT_EN__RASCNTL_VCPU_VCODEC_EN_MASK 0x00400000L
+
+//VCN 2_6_0 UVD_SYS_INT_EN
+#define UVD_SYS_INT_EN__RASCNTL_VCPU_VCODEC_EN_MASK 0x04000000L
+
/* JPEG 2_6_0 UVD_RAS_JPEG0_STATUS */
#define UVD_RAS_JPEG0_STATUS__POISONED_VF__SHIFT 0x0
#define UVD_RAS_JPEG0_STATUS__POISONED_PF__SHIFT 0x1f
diff --git a/drivers/gpu/drm/amd/include/atombios.h b/drivers/gpu/drm/amd/include/atombios.h
index 15943bc..b78360a 100644
--- a/drivers/gpu/drm/amd/include/atombios.h
+++ b/drivers/gpu/drm/amd/include/atombios.h
@@ -4107,7 +4107,7 @@ typedef struct _ATOM_FAKE_EDID_PATCH_RECORD
{
UCHAR ucRecordType;
UCHAR ucFakeEDIDLength; // = 128 means EDID length is 128 bytes, otherwise the EDID length = ucFakeEDIDLength*128
- UCHAR ucFakeEDIDString[1]; // This actually has ucFakeEdidLength elements.
+ UCHAR ucFakeEDIDString[]; // This actually has ucFakeEdidLength elements.
} ATOM_FAKE_EDID_PATCH_RECORD;
typedef struct _ATOM_PANEL_RESOLUTION_PATCH_RECORD
@@ -4386,7 +4386,7 @@ typedef struct _ATOM_GPIO_PIN_ASSIGNMENT
typedef struct _ATOM_GPIO_PIN_LUT
{
ATOM_COMMON_TABLE_HEADER sHeader;
- ATOM_GPIO_PIN_ASSIGNMENT asGPIO_Pin[1];
+ ATOM_GPIO_PIN_ASSIGNMENT asGPIO_Pin[];
}ATOM_GPIO_PIN_LUT;
/****************************************************************************/
@@ -4513,7 +4513,7 @@ typedef struct _ATOM_DISPLAY_OBJECT_PATH
USHORT usSize; //the size of ATOM_DISPLAY_OBJECT_PATH
USHORT usConnObjectId; //Connector Object ID
USHORT usGPUObjectId; //GPU ID
- USHORT usGraphicObjIds[1]; //1st Encoder Obj source from GPU to last Graphic Obj destinate to connector.
+ USHORT usGraphicObjIds[]; //1st Encoder Obj source from GPU to last Graphic Obj destinate to connector.
}ATOM_DISPLAY_OBJECT_PATH;
typedef struct _ATOM_DISPLAY_EXTERNAL_OBJECT_PATH
@@ -4530,7 +4530,7 @@ typedef struct _ATOM_DISPLAY_OBJECT_PATH_TABLE
UCHAR ucNumOfDispPath;
UCHAR ucVersion;
UCHAR ucPadding[2];
- ATOM_DISPLAY_OBJECT_PATH asDispPath[1];
+ ATOM_DISPLAY_OBJECT_PATH asDispPath[];
}ATOM_DISPLAY_OBJECT_PATH_TABLE;
typedef struct _ATOM_OBJECT //each object has this structure
@@ -4545,7 +4545,7 @@ typedef struct _ATOM_OBJECT_TABLE //Above 4 object table
{
UCHAR ucNumberOfObjects;
UCHAR ucPadding[3];
- ATOM_OBJECT asObjects[1];
+ ATOM_OBJECT asObjects[];
}ATOM_OBJECT_TABLE;
typedef struct _ATOM_SRC_DST_TABLE_FOR_ONE_OBJECT //usSrcDstTableOffset pointing to this structure
@@ -4733,7 +4733,7 @@ typedef struct _ATOM_CONNECTOR_DEVICE_TAG_RECORD
ATOM_COMMON_RECORD_HEADER sheader;
UCHAR ucNumberOfDevice;
UCHAR ucReserved;
- ATOM_CONNECTOR_DEVICE_TAG asDeviceTag[1]; //This Id is same as "ATOM_DEVICE_XXX_SUPPORT", 1 is only for allocation
+ ATOM_CONNECTOR_DEVICE_TAG asDeviceTag[]; //This Id is same as "ATOM_DEVICE_XXX_SUPPORT"
}ATOM_CONNECTOR_DEVICE_TAG_RECORD;
@@ -4793,7 +4793,7 @@ typedef struct _ATOM_OBJECT_GPIO_CNTL_RECORD
ATOM_COMMON_RECORD_HEADER sheader;
UCHAR ucFlags; // Future expnadibility
UCHAR ucNumberOfPins; // Number of GPIO pins used to control the object
- ATOM_GPIO_PIN_CONTROL_PAIR asGpio[1]; // the real gpio pin pair determined by number of pins ucNumberOfPins
+ ATOM_GPIO_PIN_CONTROL_PAIR asGpio[]; // the real gpio pin pair determined by number of pins ucNumberOfPins
}ATOM_OBJECT_GPIO_CNTL_RECORD;
//Definitions for GPIO pin state
@@ -4982,7 +4982,7 @@ typedef struct _ATOM_BRACKET_LAYOUT_RECORD
UCHAR ucWidth;
UCHAR ucConnNum;
UCHAR ucReserved;
- ATOM_CONNECTOR_LAYOUT_INFO asConnInfo[1];
+ ATOM_CONNECTOR_LAYOUT_INFO asConnInfo[];
}ATOM_BRACKET_LAYOUT_RECORD;
@@ -5146,7 +5146,7 @@ typedef struct _ATOM_I2C_VOLTAGE_OBJECT_V3
UCHAR ucVoltageControlOffset;
UCHAR ucVoltageControlFlag; // Bit0: 0 - One byte data; 1 - Two byte data
UCHAR ulReserved[3];
- VOLTAGE_LUT_ENTRY asVolI2cLut[1]; // end with 0xff
+ VOLTAGE_LUT_ENTRY asVolI2cLut[]; // end with 0xff
}ATOM_I2C_VOLTAGE_OBJECT_V3;
// ATOM_I2C_VOLTAGE_OBJECT_V3.ucVoltageControlFlag
@@ -5161,7 +5161,7 @@ typedef struct _ATOM_GPIO_VOLTAGE_OBJECT_V3
UCHAR ucPhaseDelay; // phase delay in unit of micro second
UCHAR ucReserved;
ULONG ulGpioMaskVal; // GPIO Mask value
- VOLTAGE_LUT_ENTRY_V2 asVolGpioLut[1];
+ VOLTAGE_LUT_ENTRY_V2 asVolGpioLut[];
}ATOM_GPIO_VOLTAGE_OBJECT_V3;
typedef struct _ATOM_LEAKAGE_VOLTAGE_OBJECT_V3
@@ -5171,7 +5171,7 @@ typedef struct _ATOM_LEAKAGE_VOLTAGE_OBJECT_V3
UCHAR ucLeakageEntryNum; // indicate the entry number of LeakageId/Voltage Lut table
UCHAR ucReserved[2];
ULONG ulMaxVoltageLevel;
- LEAKAGE_VOLTAGE_LUT_ENTRY_V2 asLeakageIdLut[1];
+ LEAKAGE_VOLTAGE_LUT_ENTRY_V2 asLeakageIdLut[];
}ATOM_LEAKAGE_VOLTAGE_OBJECT_V3;
@@ -6599,7 +6599,7 @@ typedef struct _ATOM_FUSION_SYSTEM_INFO_V3
typedef struct _ATOM_I2C_DATA_RECORD
{
UCHAR ucNunberOfBytes; //Indicates how many bytes SW needs to write to the external ASIC for one block, besides to "Start" and "Stop"
- UCHAR ucI2CData[1]; //I2C data in bytes, should be less than 16 bytes usually
+ UCHAR ucI2CData[]; //I2C data in bytes, should be less than 16 bytes usually
}ATOM_I2C_DATA_RECORD;
@@ -6610,14 +6610,14 @@ typedef struct _ATOM_I2C_DEVICE_SETUP_INFO
UCHAR ucSSChipID; //SS chip being used
UCHAR ucSSChipSlaveAddr; //Slave Address to set up this SS chip
UCHAR ucNumOfI2CDataRecords; //number of data block
- ATOM_I2C_DATA_RECORD asI2CData[1];
+ ATOM_I2C_DATA_RECORD asI2CData[];
}ATOM_I2C_DEVICE_SETUP_INFO;
//==========================================================================================
typedef struct _ATOM_ASIC_MVDD_INFO
{
ATOM_COMMON_TABLE_HEADER sHeader;
- ATOM_I2C_DEVICE_SETUP_INFO asI2CSetup[1];
+ ATOM_I2C_DEVICE_SETUP_INFO asI2CSetup[];
}ATOM_ASIC_MVDD_INFO;
//==========================================================================================
@@ -6679,7 +6679,7 @@ typedef struct _ATOM_ASIC_INTERNAL_SS_INFO
typedef struct _ATOM_ASIC_INTERNAL_SS_INFO_V2
{
ATOM_COMMON_TABLE_HEADER sHeader;
- ATOM_ASIC_SS_ASSIGNMENT_V2 asSpreadSpectrum[1]; //this is point only.
+ ATOM_ASIC_SS_ASSIGNMENT_V2 asSpreadSpectrum[]; //this is point only.
}ATOM_ASIC_INTERNAL_SS_INFO_V2;
typedef struct _ATOM_ASIC_SS_ASSIGNMENT_V3
@@ -6701,7 +6701,7 @@ typedef struct _ATOM_ASIC_SS_ASSIGNMENT_V3
typedef struct _ATOM_ASIC_INTERNAL_SS_INFO_V3
{
ATOM_COMMON_TABLE_HEADER sHeader;
- ATOM_ASIC_SS_ASSIGNMENT_V3 asSpreadSpectrum[1]; //this is pointer only.
+ ATOM_ASIC_SS_ASSIGNMENT_V3 asSpreadSpectrum[]; //this is pointer only.
}ATOM_ASIC_INTERNAL_SS_INFO_V3;
@@ -9292,7 +9292,7 @@ typedef struct {
typedef struct {
VFCT_IMAGE_HEADER VbiosHeader;
- UCHAR VbiosContent[1];
+ UCHAR VbiosContent[];
}GOP_VBIOS_CONTENT;
typedef struct {
diff --git a/drivers/gpu/drm/amd/include/atomfirmware.h b/drivers/gpu/drm/amd/include/atomfirmware.h
index ff855cb..bbe1337 100644
--- a/drivers/gpu/drm/amd/include/atomfirmware.h
+++ b/drivers/gpu/drm/amd/include/atomfirmware.h
@@ -705,20 +705,65 @@ struct atom_gpio_pin_lut_v2_1
};
-/*
- ***************************************************************************
- Data Table vram_usagebyfirmware structure
- ***************************************************************************
-*/
+/*
+ * VBIOS/PRE-OS always reserve a FB region at the top of frame buffer. driver should not write
+ * access that region. driver can allocate their own reservation region as long as it does not
+ * overlap firwmare's reservation region.
+ * if (pre-NV1X) atom data table firmwareInfoTable version < 3.3:
+ * in this case, atom data table vram_usagebyfirmwareTable version always <= 2.1
+ * if VBIOS/UEFI GOP is posted:
+ * VBIOS/UEFIGOP update used_by_firmware_in_kb = total reserved size by VBIOS
+ * update start_address_in_kb = total_mem_size_in_kb - used_by_firmware_in_kb;
+ * ( total_mem_size_in_kb = reg(CONFIG_MEMSIZE)<<10)
+ * driver can allocate driver reservation region under firmware reservation,
+ * used_by_driver_in_kb = driver reservation size
+ * driver reservation start address = (start_address_in_kb - used_by_driver_in_kb)
+ * Comment1[hchan]: There is only one reservation at the beginning of the FB reserved by
+ * host driver. Host driver would overwrite the table with the following
+ * used_by_firmware_in_kb = total reserved size for pf-vf info exchange and
+ * set SRIOV_MSG_SHARE_RESERVATION mask start_address_in_kb = 0
+ * else there is no VBIOS reservation region:
+ * driver must allocate driver reservation region at top of FB.
+ * driver set used_by_driver_in_kb = driver reservation size
+ * driver reservation start address = (total_mem_size_in_kb - used_by_driver_in_kb)
+ * same as Comment1
+ * else (NV1X and after):
+ * if VBIOS/UEFI GOP is posted:
+ * VBIOS/UEFIGOP update:
+ * used_by_firmware_in_kb = atom_firmware_Info_v3_3.fw_reserved_size_in_kb;
+ * start_address_in_kb = total_mem_size_in_kb - used_by_firmware_in_kb;
+ * (total_mem_size_in_kb = reg(CONFIG_MEMSIZE)<<10)
+ * if vram_usagebyfirmwareTable version <= 2.1:
+ * driver can allocate driver reservation region under firmware reservation,
+ * driver set used_by_driver_in_kb = driver reservation size
+ * driver reservation start address = start_address_in_kb - used_by_driver_in_kb
+ * same as Comment1
+ * else driver can:
+ * allocate it reservation any place as long as it does overlap pre-OS FW reservation area
+ * set used_by_driver_region0_in_kb = driver reservation size
+ * set driver_region0_start_address_in_kb = driver reservation region start address
+ * Comment2[hchan]: Host driver can set used_by_firmware_in_kb and start_address_in_kb to
+ * zero as the reservation for VF as it doesn’t exist. And Host driver should also
+ * update atom_firmware_Info table to remove the same VBIOS reservation as well.
+ */
struct vram_usagebyfirmware_v2_1
{
- struct atom_common_table_header table_header;
- uint32_t start_address_in_kb;
- uint16_t used_by_firmware_in_kb;
- uint16_t used_by_driver_in_kb;
+ struct atom_common_table_header table_header;
+ uint32_t start_address_in_kb;
+ uint16_t used_by_firmware_in_kb;
+ uint16_t used_by_driver_in_kb;
};
+struct vram_usagebyfirmware_v2_2 {
+ struct atom_common_table_header table_header;
+ uint32_t fw_region_start_address_in_kb;
+ uint16_t used_by_firmware_in_kb;
+ uint16_t reserved;
+ uint32_t driver_region0_start_address_in_kb;
+ uint32_t used_by_driver_region0_in_kb;
+ uint32_t reserved32[7];
+};
/*
***************************************************************************
diff --git a/drivers/gpu/drm/amd/include/ivsrcid/vcn/irqsrcs_vcn_4_0.h b/drivers/gpu/drm/amd/include/ivsrcid/vcn/irqsrcs_vcn_4_0.h
index a81138c..03cfa05 100644
--- a/drivers/gpu/drm/amd/include/ivsrcid/vcn/irqsrcs_vcn_4_0.h
+++ b/drivers/gpu/drm/amd/include/ivsrcid/vcn/irqsrcs_vcn_4_0.h
@@ -38,4 +38,7 @@
#define VCN_4_0__SRCID__JPEG6_DECODE 174 // 0xae JRBC6 Decode interrupt
#define VCN_4_0__SRCID__JPEG7_DECODE 175 // 0xaf JRBC7 Decode interrupt
+#define VCN_4_0__SRCID_UVD_POISON 160
+#define VCN_4_0__SRCID_DJPEG0_POISON 161
+#define VCN_4_0__SRCID_EJPEG0_POISON 162
#endif
diff --git a/drivers/gpu/drm/amd/include/kgd_pp_interface.h b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
index a40ead4..d18162e 100644
--- a/drivers/gpu/drm/amd/include/kgd_pp_interface.h
+++ b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
@@ -354,7 +354,8 @@ struct amd_pm_funcs {
int (*get_power_profile_mode)(void *handle, char *buf);
int (*set_power_profile_mode)(void *handle, long *input, uint32_t size);
int (*set_fine_grain_clk_vol)(void *handle, uint32_t type, long *input, uint32_t size);
- int (*odn_edit_dpm_table)(void *handle, uint32_t type, long *input, uint32_t size);
+ int (*odn_edit_dpm_table)(void *handle, enum PP_OD_DPM_TABLE_COMMAND type,
+ long *input, uint32_t size);
int (*set_mp1_state)(void *handle, enum pp_mp1_state mp1_state);
int (*smu_i2c_bus_access)(void *handle, bool acquire);
int (*gfx_state_change_set)(void *handle, uint32_t state);
diff --git a/drivers/gpu/drm/amd/include/mes_v11_api_def.h b/drivers/gpu/drm/amd/include/mes_v11_api_def.h
index 7e85cdc..dc694cb 100644
--- a/drivers/gpu/drm/amd/include/mes_v11_api_def.h
+++ b/drivers/gpu/drm/amd/include/mes_v11_api_def.h
@@ -222,7 +222,11 @@ union MESAPI_SET_HW_RESOURCES {
uint32_t apply_grbm_remote_register_dummy_read_wa : 1;
uint32_t second_gfx_pipe_enabled : 1;
uint32_t enable_level_process_quantum_check : 1;
- uint32_t reserved : 25;
+ uint32_t legacy_sch_mode : 1;
+ uint32_t disable_add_queue_wptr_mc_addr : 1;
+ uint32_t enable_mes_event_int_logging : 1;
+ uint32_t enable_reg_active_poll : 1;
+ uint32_t reserved : 21;
};
uint32_t uint32_t_all;
};
diff --git a/drivers/gpu/drm/amd/include/yellow_carp_offset.h b/drivers/gpu/drm/amd/include/yellow_carp_offset.h
index 28a56b5..0fea6a7 100644
--- a/drivers/gpu/drm/amd/include/yellow_carp_offset.h
+++ b/drivers/gpu/drm/amd/include/yellow_carp_offset.h
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: MIT
#ifndef YELLOW_CARP_OFFSET_H
#define YELLOW_CARP_OFFSET_H
diff --git a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
index ec05585..304190d 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
@@ -838,7 +838,8 @@ static int pp_set_fine_grain_clk_vol(void *handle, uint32_t type, long *input, u
return hwmgr->hwmgr_func->set_fine_grain_clk_vol(hwmgr, type, input, size);
}
-static int pp_odn_edit_dpm_table(void *handle, uint32_t type, long *input, uint32_t size)
+static int pp_odn_edit_dpm_table(void *handle, enum PP_OD_DPM_TABLE_COMMAND type,
+ long *input, uint32_t size)
{
struct pp_hwmgr *hwmgr = handle;
@@ -1507,7 +1508,7 @@ static void pp_pm_compute_clocks(void *handle)
struct pp_hwmgr *hwmgr = handle;
struct amdgpu_device *adev = hwmgr->adev;
- if (!amdgpu_device_has_dc_support(adev)) {
+ if (!adev->dc_enabled) {
amdgpu_dpm_get_active_displays(adev);
adev->pm.pm_display_cfg.num_display = adev->pm.dpm.new_active_crtc_count;
adev->pm.pm_display_cfg.vrefresh = amdgpu_dpm_get_vrefresh(adev);
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/pp_psm.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/pp_psm.c
index 67d7da0..1d82940 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/pp_psm.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/pp_psm.c
@@ -75,8 +75,10 @@ int psm_init_power_state_table(struct pp_hwmgr *hwmgr)
for (i = 0; i < table_entries; i++) {
result = hwmgr->hwmgr_func->get_pp_table_entry(hwmgr, i, state);
if (result) {
+ kfree(hwmgr->current_ps);
kfree(hwmgr->request_ps);
kfree(hwmgr->ps);
+ hwmgr->current_ps = NULL;
hwmgr->request_ps = NULL;
hwmgr->ps = NULL;
return -EINVAL;
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c
index 97b3ad3..b30684c 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c
@@ -2961,7 +2961,8 @@ static int vega20_odn_edit_dpm_table(struct pp_hwmgr *hwmgr,
data->od8_settings.od8_settings_array;
OverDriveTable_t *od_table =
&(data->smc_state_table.overdrive_table);
- int32_t input_index, input_clk, input_vol, i;
+ int32_t input_clk, input_vol, i;
+ uint32_t input_index;
int od8_id;
int ret;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index b880f4d..ca3beb5 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -161,7 +161,7 @@ int smu_get_dpm_freq_range(struct smu_context *smu,
int smu_set_gfx_power_up_by_imu(struct smu_context *smu)
{
- if (!smu->ppt_funcs && !smu->ppt_funcs->set_gfx_power_up_by_imu)
+ if (!smu->ppt_funcs || !smu->ppt_funcs->set_gfx_power_up_by_imu)
return -EOPNOTSUPP;
return smu->ppt_funcs->set_gfx_power_up_by_imu(smu);
@@ -585,6 +585,7 @@ static int smu_set_funcs(struct amdgpu_device *adev)
yellow_carp_set_ppt_funcs(smu);
break;
case IP_VERSION(13, 0, 4):
+ case IP_VERSION(13, 0, 11):
smu_v13_0_4_set_ppt_funcs(smu);
break;
case IP_VERSION(13, 0, 5):
@@ -1448,6 +1449,7 @@ static int smu_disable_dpms(struct smu_context *smu)
switch (adev->ip_versions[MP1_HWIP][0]) {
case IP_VERSION(13, 0, 0):
case IP_VERSION(13, 0, 7):
+ case IP_VERSION(13, 0, 10):
return 0;
default:
break;
@@ -1516,7 +1518,7 @@ static int smu_disable_dpms(struct smu_context *smu)
}
if (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(9, 4, 2) &&
- adev->gfx.rlc.funcs->stop)
+ !amdgpu_sriov_vf(adev) && adev->gfx.rlc.funcs->stop)
adev->gfx.rlc.funcs->stop(adev);
return ret;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
index f816b1d..3bc4128 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
@@ -168,6 +168,7 @@ struct smu_temperature_range {
int mem_crit_max;
int mem_emergency_max;
int software_shutdown_temp;
+ int software_shutdown_temp_offset;
};
struct smu_state_validation_block {
@@ -568,6 +569,10 @@ struct smu_context
u32 param_reg;
u32 msg_reg;
u32 resp_reg;
+
+ u32 debug_param_reg;
+ u32 debug_msg_reg;
+ u32 debug_resp_reg;
};
struct i2c_adapter;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_0_ppsmc.h b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_0_ppsmc.h
index 9ebb8f3..8b82668 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_0_ppsmc.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_0_ppsmc.h
@@ -131,7 +131,13 @@
#define PPSMC_MSG_EnableAudioStutterWA 0x44
#define PPSMC_MSG_PowerUpUmsch 0x45
#define PPSMC_MSG_PowerDownUmsch 0x46
-#define PPSMC_Message_Count 0x47
+#define PPSMC_MSG_SetDcsArch 0x47
+#define PPSMC_MSG_TriggerVFFLR 0x48
+#define PPSMC_MSG_SetNumBadMemoryPagesRetired 0x49
+#define PPSMC_MSG_SetBadMemoryPagesRetiredFlagsPerChannel 0x4A
+#define PPSMC_MSG_SetPriorityDeltaGain 0x4B
+#define PPSMC_MSG_AllowIHHostInterrupt 0x4C
+#define PPSMC_Message_Count 0x4D
//Debug Dump Message
#define DEBUGSMC_MSG_TestMessage 0x1
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
index 58098b8..a4e3425 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h
@@ -239,7 +239,9 @@
__SMU_DUMMY_MAP(DriverMode2Reset), \
__SMU_DUMMY_MAP(GetGfxOffStatus), \
__SMU_DUMMY_MAP(GetGfxOffEntryCount), \
- __SMU_DUMMY_MAP(LogGfxOffResidency),
+ __SMU_DUMMY_MAP(LogGfxOffResidency), \
+ __SMU_DUMMY_MAP(SetNumBadMemoryPagesRetired), \
+ __SMU_DUMMY_MAP(SetBadMemoryPagesRetiredFlagsPerChannel),
#undef __SMU_DUMMY_MAP
#define __SMU_DUMMY_MAP(type) SMU_MSG_##type
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c
index 70b5607..ad66d57 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c
@@ -79,6 +79,17 @@ MODULE_FIRMWARE("amdgpu/beige_goby_smc.bin");
#define mmTHM_BACO_CNTL_ARCT 0xA7
#define mmTHM_BACO_CNTL_ARCT_BASE_IDX 0
+static void smu_v11_0_poll_baco_exit(struct smu_context *smu)
+{
+ struct amdgpu_device *adev = smu->adev;
+ uint32_t data, loop = 0;
+
+ do {
+ usleep_range(1000, 1100);
+ data = RREG32_SOC15(THM, 0, mmTHM_BACO_CNTL);
+ } while ((data & 0x100) && (++loop < 100));
+}
+
int smu_v11_0_init_microcode(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
@@ -1588,6 +1599,10 @@ bool smu_v11_0_baco_is_support(struct smu_context *smu)
if (amdgpu_sriov_vf(smu->adev) || !smu_baco->platform_support)
return false;
+ /* return true if ASIC is in BACO state already */
+ if (smu_v11_0_baco_get_state(smu) == SMU_BACO_STATE_ENTER)
+ return true;
+
/* Arcturus does not support this bit mask */
if (smu_cmn_feature_is_supported(smu, SMU_FEATURE_BACO_BIT) &&
!smu_cmn_feature_is_enabled(smu, SMU_FEATURE_BACO_BIT))
@@ -1685,7 +1700,18 @@ int smu_v11_0_baco_enter(struct smu_context *smu)
int smu_v11_0_baco_exit(struct smu_context *smu)
{
- return smu_v11_0_baco_set_state(smu, SMU_BACO_STATE_EXIT);
+ int ret;
+
+ ret = smu_v11_0_baco_set_state(smu, SMU_BACO_STATE_EXIT);
+ if (!ret) {
+ /*
+ * Poll BACO exit status to ensure FW has completed
+ * BACO exit process to avoid timing issues.
+ */
+ smu_v11_0_poll_baco_exit(smu);
+ }
+
+ return ret;
}
int smu_v11_0_mode1_reset(struct smu_context *smu)
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
index 89f0f6e..f5e90e0 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
@@ -250,6 +250,7 @@ int smu_v13_0_check_fw_status(struct smu_context *smu)
switch (adev->ip_versions[MP1_HWIP][0]) {
case IP_VERSION(13, 0, 4):
+ case IP_VERSION(13, 0, 11):
mp1_fw_flags = RREG32_PCIE(MP1_Public |
(smnMP1_V13_0_4_FIRMWARE_FLAGS & 0xffffffff));
break;
@@ -301,6 +302,7 @@ int smu_v13_0_check_fw_version(struct smu_context *smu)
smu->smc_driver_if_version = SMU13_DRIVER_IF_VERSION_YELLOW_CARP;
break;
case IP_VERSION(13, 0, 4):
+ case IP_VERSION(13, 0, 11):
smu->smc_driver_if_version = SMU13_DRIVER_IF_VERSION_SMU_V13_0_4;
break;
case IP_VERSION(13, 0, 5):
@@ -841,6 +843,7 @@ int smu_v13_0_gfx_off_control(struct smu_context *smu, bool enable)
case IP_VERSION(13, 0, 7):
case IP_VERSION(13, 0, 8):
case IP_VERSION(13, 0, 10):
+ case IP_VERSION(13, 0, 11):
if (!(adev->pm.pp_feature & PP_GFXOFF_MASK))
return 0;
if (enable)
@@ -1376,6 +1379,7 @@ static int smu_v13_0_irq_process(struct amdgpu_device *adev,
*/
uint32_t ctxid = entry->src_data[0];
uint32_t data;
+ uint32_t high;
if (client_id == SOC15_IH_CLIENTID_THM) {
switch (src_id) {
@@ -1432,6 +1436,36 @@ static int smu_v13_0_irq_process(struct amdgpu_device *adev,
schedule_work(&smu->throttling_logging_work);
break;
+ case 0x8:
+ high = smu->thermal_range.software_shutdown_temp +
+ smu->thermal_range.software_shutdown_temp_offset;
+ high = min_t(typeof(high),
+ SMU_THERMAL_MAXIMUM_ALERT_TEMP,
+ high);
+ dev_emerg(adev->dev, "Reduce soft CTF limit to %d (by an offset %d)\n",
+ high,
+ smu->thermal_range.software_shutdown_temp_offset);
+
+ data = RREG32_SOC15(THM, 0, regTHM_THERMAL_INT_CTRL);
+ data = REG_SET_FIELD(data, THM_THERMAL_INT_CTRL,
+ DIG_THERM_INTH,
+ (high & 0xff));
+ data = data & (~THM_THERMAL_INT_CTRL__THERM_TRIGGER_MASK_MASK);
+ WREG32_SOC15(THM, 0, regTHM_THERMAL_INT_CTRL, data);
+ break;
+ case 0x9:
+ high = min_t(typeof(high),
+ SMU_THERMAL_MAXIMUM_ALERT_TEMP,
+ smu->thermal_range.software_shutdown_temp);
+ dev_emerg(adev->dev, "Recover soft CTF limit to %d\n", high);
+
+ data = RREG32_SOC15(THM, 0, regTHM_THERMAL_INT_CTRL);
+ data = REG_SET_FIELD(data, THM_THERMAL_INT_CTRL,
+ DIG_THERM_INTH,
+ (high & 0xff));
+ data = data & (~THM_THERMAL_INT_CTRL__THERM_TRIGGER_MASK_MASK);
+ WREG32_SOC15(THM, 0, regTHM_THERMAL_INT_CTRL, data);
+ break;
}
}
}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
index f0121d1..87d7c66 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
@@ -70,6 +70,26 @@
#define MP0_MP1_DATA_REGION_SIZE_COMBOPPTABLE 0x4000
+#define mmMP1_SMN_C2PMSG_66 0x0282
+#define mmMP1_SMN_C2PMSG_66_BASE_IDX 0
+
+#define mmMP1_SMN_C2PMSG_82 0x0292
+#define mmMP1_SMN_C2PMSG_82_BASE_IDX 0
+
+#define mmMP1_SMN_C2PMSG_90 0x029a
+#define mmMP1_SMN_C2PMSG_90_BASE_IDX 0
+
+#define mmMP1_SMN_C2PMSG_75 0x028b
+#define mmMP1_SMN_C2PMSG_75_BASE_IDX 0
+
+#define mmMP1_SMN_C2PMSG_53 0x0275
+#define mmMP1_SMN_C2PMSG_53_BASE_IDX 0
+
+#define mmMP1_SMN_C2PMSG_54 0x0276
+#define mmMP1_SMN_C2PMSG_54_BASE_IDX 0
+
+#define DEBUGSMC_MSG_Mode1Reset 2
+
static struct cmn2asic_msg_mapping smu_v13_0_0_message_map[SMU_MSG_MAX_COUNT] = {
MSG_MAP(TestMessage, PPSMC_MSG_TestMessage, 1),
MSG_MAP(GetSmuVersion, PPSMC_MSG_GetSmuVersion, 1),
@@ -121,6 +141,9 @@ static struct cmn2asic_msg_mapping smu_v13_0_0_message_map[SMU_MSG_MAX_COUNT] =
MSG_MAP(PrepareMp1ForUnload, PPSMC_MSG_PrepareMp1ForUnload, 0),
MSG_MAP(DFCstateControl, PPSMC_MSG_SetExternalClientDfCstateAllow, 0),
MSG_MAP(ArmD3, PPSMC_MSG_ArmD3, 0),
+ MSG_MAP(SetNumBadMemoryPagesRetired, PPSMC_MSG_SetNumBadMemoryPagesRetired, 0),
+ MSG_MAP(SetBadMemoryPagesRetiredFlagsPerChannel,
+ PPSMC_MSG_SetBadMemoryPagesRetiredFlagsPerChannel, 0),
};
static struct cmn2asic_mapping smu_v13_0_0_clk_map[SMU_CLK_COUNT] = {
@@ -1789,6 +1812,69 @@ static int smu_v13_0_0_set_df_cstate(struct smu_context *smu,
NULL);
}
+static int smu_v13_0_0_mode1_reset(struct smu_context *smu)
+{
+ int ret;
+ struct amdgpu_device *adev = smu->adev;
+
+ if (adev->ip_versions[MP1_HWIP][0] == IP_VERSION(13, 0, 10))
+ ret = smu_cmn_send_debug_smc_msg(smu, DEBUGSMC_MSG_Mode1Reset);
+ else
+ ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, NULL);
+
+ if (!ret)
+ msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS);
+
+ return ret;
+}
+
+static void smu_v13_0_0_set_smu_mailbox_registers(struct smu_context *smu)
+{
+ struct amdgpu_device *adev = smu->adev;
+
+ smu->param_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_82);
+ smu->msg_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_66);
+ smu->resp_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_90);
+
+ smu->debug_param_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_53);
+ smu->debug_msg_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_75);
+ smu->debug_resp_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_54);
+}
+
+static int smu_v13_0_0_smu_send_bad_mem_page_num(struct smu_context *smu,
+ uint32_t size)
+{
+ int ret = 0;
+
+ /* message SMU to update the bad page number on SMUBUS */
+ ret = smu_cmn_send_smc_msg_with_param(smu,
+ SMU_MSG_SetNumBadMemoryPagesRetired,
+ size, NULL);
+ if (ret)
+ dev_err(smu->adev->dev,
+ "[%s] failed to message SMU to update bad memory pages number\n",
+ __func__);
+
+ return ret;
+}
+
+static int smu_v13_0_0_send_bad_mem_channel_flag(struct smu_context *smu,
+ uint32_t size)
+{
+ int ret = 0;
+
+ /* message SMU to update the bad channel info on SMUBUS */
+ ret = smu_cmn_send_smc_msg_with_param(smu,
+ SMU_MSG_SetBadMemoryPagesRetiredFlagsPerChannel,
+ size, NULL);
+ if (ret)
+ dev_err(smu->adev->dev,
+ "[%s] failed to message SMU to update bad memory pages channel info\n",
+ __func__);
+
+ return ret;
+}
+
static const struct pptable_funcs smu_v13_0_0_ppt_funcs = {
.get_allowed_feature_mask = smu_v13_0_0_get_allowed_feature_mask,
.set_default_dpm_table = smu_v13_0_0_set_default_dpm_table,
@@ -1856,9 +1942,11 @@ static const struct pptable_funcs smu_v13_0_0_ppt_funcs = {
.baco_enter = smu_v13_0_0_baco_enter,
.baco_exit = smu_v13_0_0_baco_exit,
.mode1_reset_is_support = smu_v13_0_0_is_mode1_reset_supported,
- .mode1_reset = smu_v13_0_mode1_reset,
+ .mode1_reset = smu_v13_0_0_mode1_reset,
.set_mp1_state = smu_v13_0_0_set_mp1_state,
.set_df_cstate = smu_v13_0_0_set_df_cstate,
+ .send_hbm_bad_pages_num = smu_v13_0_0_smu_send_bad_mem_page_num,
+ .send_hbm_bad_channel_flag = smu_v13_0_0_send_bad_mem_channel_flag,
};
void smu_v13_0_0_set_ppt_funcs(struct smu_context *smu)
@@ -1870,5 +1958,5 @@ void smu_v13_0_0_set_ppt_funcs(struct smu_context *smu)
smu->table_map = smu_v13_0_0_table_map;
smu->pwr_src_map = smu_v13_0_0_pwr_src_map;
smu->workload_map = smu_v13_0_0_workload_map;
- smu_v13_0_set_smu_mailbox_registers(smu);
+ smu_v13_0_0_set_smu_mailbox_registers(smu);
}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_4_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_4_ppt.c
index 97e1d55..8fa9a36 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_4_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_4_ppt.c
@@ -1026,6 +1026,15 @@ static const struct pptable_funcs smu_v13_0_4_ppt_funcs = {
.set_gfx_power_up_by_imu = smu_v13_0_set_gfx_power_up_by_imu,
};
+static void smu_v13_0_4_set_smu_mailbox_registers(struct smu_context *smu)
+{
+ struct amdgpu_device *adev = smu->adev;
+
+ smu->param_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_82);
+ smu->msg_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_66);
+ smu->resp_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_90);
+}
+
void smu_v13_0_4_set_ppt_funcs(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
@@ -1035,7 +1044,9 @@ void smu_v13_0_4_set_ppt_funcs(struct smu_context *smu)
smu->feature_map = smu_v13_0_4_feature_mask_map;
smu->table_map = smu_v13_0_4_table_map;
smu->is_apu = true;
- smu->param_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_82);
- smu->msg_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_66);
- smu->resp_reg = SOC15_REG_OFFSET(MP1, 0, mmMP1_SMN_C2PMSG_90);
+
+ if (adev->ip_versions[MP1_HWIP][0] == IP_VERSION(13, 0, 4))
+ smu_v13_0_4_set_smu_mailbox_registers(smu);
+ else
+ smu_v13_0_set_smu_mailbox_registers(smu);
}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
index d74debc..c3c9ef5 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
@@ -1223,6 +1223,7 @@ static int smu_v13_0_7_get_thermal_temperature_range(struct smu_context *smu,
range->mem_emergency_max = (pptable->SkuTable.TemperatureLimit[TEMP_MEM] + CTF_OFFSET_MEM)*
SMU_TEMPERATURE_UNITS_PER_CENTIGRADES;
range->software_shutdown_temp = powerplay_table->software_shutdown_temp;
+ range->software_shutdown_temp_offset = pptable->SkuTable.FanAbnormalTempLimitOffset;
return 0;
}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c b/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c
index e4f8f90a..768b6e7 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c
@@ -233,6 +233,18 @@ static void __smu_cmn_send_msg(struct smu_context *smu,
WREG32(smu->msg_reg, msg);
}
+static int __smu_cmn_send_debug_msg(struct smu_context *smu,
+ u32 msg,
+ u32 param)
+{
+ struct amdgpu_device *adev = smu->adev;
+
+ WREG32(smu->debug_param_reg, param);
+ WREG32(smu->debug_msg_reg, msg);
+ WREG32(smu->debug_resp_reg, 0);
+
+ return 0;
+}
/**
* smu_cmn_send_msg_without_waiting -- send the message; don't wait for status
* @smu: pointer to an SMU context
@@ -386,6 +398,12 @@ int smu_cmn_send_smc_msg(struct smu_context *smu,
read_arg);
}
+int smu_cmn_send_debug_smc_msg(struct smu_context *smu,
+ uint32_t msg)
+{
+ return __smu_cmn_send_debug_msg(smu, msg, 0);
+}
+
int smu_cmn_to_asic_specific_index(struct smu_context *smu,
enum smu_cmn2asic_mapping_type type,
uint32_t index)
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.h b/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.h
index 1526ce0..f82cf76 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu_cmn.h
@@ -42,6 +42,9 @@ int smu_cmn_send_smc_msg(struct smu_context *smu,
enum smu_message_type msg,
uint32_t *read_arg);
+int smu_cmn_send_debug_smc_msg(struct smu_context *smu,
+ uint32_t msg);
+
int smu_cmn_wait_for_response(struct smu_context *smu);
int smu_cmn_to_asic_specific_index(struct smu_context *smu,
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_drv.c b/drivers/gpu/drm/arm/display/komeda/komeda_drv.c
index 9fce423..3f4e719 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_drv.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_drv.c
@@ -9,7 +9,7 @@
#include <linux/platform_device.h>
#include <linux/component.h>
#include <linux/pm_runtime.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_module.h>
#include <drm/drm_of.h>
#include "komeda_dev.h"
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_kms.c b/drivers/gpu/drm/arm/display/komeda/komeda_kms.c
index 451746e..62dc645 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_kms.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_kms.c
@@ -10,7 +10,6 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_managed.h>
@@ -59,7 +58,6 @@ static irqreturn_t komeda_kms_irq_handler(int irq, void *data)
static const struct drm_driver komeda_kms_driver = {
.driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC,
- .lastclose = drm_fb_helper_lastclose,
DRM_GEM_DMA_DRIVER_OPS_WITH_DUMB_CREATE(komeda_gem_dma_dumb_create),
.fops = &komeda_cma_fops,
.name = "komeda",
diff --git a/drivers/gpu/drm/arm/hdlcd_crtc.c b/drivers/gpu/drm/arm/hdlcd_crtc.c
index 7030339..3cfefad 100644
--- a/drivers/gpu/drm/arm/hdlcd_crtc.c
+++ b/drivers/gpu/drm/arm/hdlcd_crtc.c
@@ -19,7 +19,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_of.h>
@@ -275,7 +274,7 @@ static void hdlcd_plane_atomic_update(struct drm_plane *plane,
dest_h = drm_rect_height(&new_plane_state->dst);
scanout_start = drm_fb_dma_get_gem_addr(fb, new_plane_state, 0);
- hdlcd = plane->dev->dev_private;
+ hdlcd = drm_to_hdlcd_priv(plane->dev);
hdlcd_write(hdlcd, HDLCD_REG_FB_LINE_LENGTH, fb->pitches[0]);
hdlcd_write(hdlcd, HDLCD_REG_FB_LINE_PITCH, fb->pitches[0]);
hdlcd_write(hdlcd, HDLCD_REG_FB_LINE_COUNT, dest_h - 1);
@@ -290,7 +289,6 @@ static const struct drm_plane_helper_funcs hdlcd_plane_helper_funcs = {
static const struct drm_plane_funcs hdlcd_plane_funcs = {
.update_plane = drm_atomic_helper_update_plane,
.disable_plane = drm_atomic_helper_disable_plane,
- .destroy = drm_plane_cleanup,
.reset = drm_atomic_helper_plane_reset,
.atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
.atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
@@ -298,24 +296,19 @@ static const struct drm_plane_funcs hdlcd_plane_funcs = {
static struct drm_plane *hdlcd_plane_init(struct drm_device *drm)
{
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
+ struct hdlcd_drm_private *hdlcd = drm_to_hdlcd_priv(drm);
struct drm_plane *plane = NULL;
u32 formats[ARRAY_SIZE(supported_formats)], i;
- int ret;
-
- plane = devm_kzalloc(drm->dev, sizeof(*plane), GFP_KERNEL);
- if (!plane)
- return ERR_PTR(-ENOMEM);
for (i = 0; i < ARRAY_SIZE(supported_formats); i++)
formats[i] = supported_formats[i].fourcc;
- ret = drm_universal_plane_init(drm, plane, 0xff, &hdlcd_plane_funcs,
- formats, ARRAY_SIZE(formats),
- NULL,
- DRM_PLANE_TYPE_PRIMARY, NULL);
- if (ret)
- return ERR_PTR(ret);
+ plane = drmm_universal_plane_alloc(drm, struct drm_plane, dev, 0xff,
+ &hdlcd_plane_funcs,
+ formats, ARRAY_SIZE(formats),
+ NULL, DRM_PLANE_TYPE_PRIMARY, NULL);
+ if (IS_ERR(plane))
+ return plane;
drm_plane_helper_add(plane, &hdlcd_plane_helper_funcs);
hdlcd->plane = plane;
@@ -325,7 +318,7 @@ static struct drm_plane *hdlcd_plane_init(struct drm_device *drm)
int hdlcd_setup_crtc(struct drm_device *drm)
{
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
+ struct hdlcd_drm_private *hdlcd = drm_to_hdlcd_priv(drm);
struct drm_plane *primary;
int ret;
diff --git a/drivers/gpu/drm/arm/hdlcd_drv.c b/drivers/gpu/drm/arm/hdlcd_drv.c
index a032003..7043d1c 100644
--- a/drivers/gpu/drm/arm/hdlcd_drv.c
+++ b/drivers/gpu/drm/arm/hdlcd_drv.c
@@ -26,7 +26,7 @@
#include <drm/drm_crtc.h>
#include <drm/drm_debugfs.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_modeset_helper.h>
@@ -98,7 +98,7 @@ static void hdlcd_irq_uninstall(struct hdlcd_drm_private *hdlcd)
static int hdlcd_load(struct drm_device *drm, unsigned long flags)
{
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
+ struct hdlcd_drm_private *hdlcd = drm_to_hdlcd_priv(drm);
struct platform_device *pdev = to_platform_device(drm->dev);
struct resource *res;
u32 version;
@@ -175,14 +175,21 @@ static const struct drm_mode_config_funcs hdlcd_mode_config_funcs = {
.atomic_commit = drm_atomic_helper_commit,
};
-static void hdlcd_setup_mode_config(struct drm_device *drm)
+static int hdlcd_setup_mode_config(struct drm_device *drm)
{
- drm_mode_config_init(drm);
+ int ret;
+
+ ret = drmm_mode_config_init(drm);
+ if (ret)
+ return ret;
+
drm->mode_config.min_width = 0;
drm->mode_config.min_height = 0;
drm->mode_config.max_width = HDLCD_MAX_XRES;
drm->mode_config.max_height = HDLCD_MAX_YRES;
drm->mode_config.funcs = &hdlcd_mode_config_funcs;
+
+ return 0;
}
#ifdef CONFIG_DEBUG_FS
@@ -190,7 +197,7 @@ static int hdlcd_show_underrun_count(struct seq_file *m, void *arg)
{
struct drm_info_node *node = (struct drm_info_node *)m->private;
struct drm_device *drm = node->minor->dev;
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
+ struct hdlcd_drm_private *hdlcd = drm_to_hdlcd_priv(drm);
seq_printf(m, "underrun : %d\n", atomic_read(&hdlcd->buffer_underrun_count));
seq_printf(m, "dma_end : %d\n", atomic_read(&hdlcd->dma_end_count));
@@ -203,7 +210,7 @@ static int hdlcd_show_pxlclock(struct seq_file *m, void *arg)
{
struct drm_info_node *node = (struct drm_info_node *)m->private;
struct drm_device *drm = node->minor->dev;
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
+ struct hdlcd_drm_private *hdlcd = drm_to_hdlcd_priv(drm);
unsigned long clkrate = clk_get_rate(hdlcd->clk);
unsigned long mode_clock = hdlcd->crtc.mode.crtc_clock * 1000;
@@ -247,18 +254,18 @@ static int hdlcd_drm_bind(struct device *dev)
struct hdlcd_drm_private *hdlcd;
int ret;
- hdlcd = devm_kzalloc(dev, sizeof(*hdlcd), GFP_KERNEL);
- if (!hdlcd)
- return -ENOMEM;
+ hdlcd = devm_drm_dev_alloc(dev, &hdlcd_driver, typeof(*hdlcd), base);
+ if (IS_ERR(hdlcd))
+ return PTR_ERR(hdlcd);
- drm = drm_dev_alloc(&hdlcd_driver, dev);
- if (IS_ERR(drm))
- return PTR_ERR(drm);
+ drm = &hdlcd->base;
- drm->dev_private = hdlcd;
dev_set_drvdata(dev, drm);
- hdlcd_setup_mode_config(drm);
+ ret = hdlcd_setup_mode_config(drm);
+ if (ret)
+ goto err_free;
+
ret = hdlcd_load(drm, 0);
if (ret)
goto err_free;
@@ -317,17 +324,14 @@ static int hdlcd_drm_bind(struct device *dev)
hdlcd_irq_uninstall(hdlcd);
of_reserved_mem_device_release(drm->dev);
err_free:
- drm_mode_config_cleanup(drm);
dev_set_drvdata(dev, NULL);
- drm_dev_put(drm);
-
return ret;
}
static void hdlcd_drm_unbind(struct device *dev)
{
struct drm_device *drm = dev_get_drvdata(dev);
- struct hdlcd_drm_private *hdlcd = drm->dev_private;
+ struct hdlcd_drm_private *hdlcd = drm_to_hdlcd_priv(drm);
drm_dev_unregister(drm);
drm_kms_helper_poll_fini(drm);
@@ -341,10 +345,7 @@ static void hdlcd_drm_unbind(struct device *dev)
if (pm_runtime_enabled(dev))
pm_runtime_disable(dev);
of_reserved_mem_device_release(dev);
- drm_mode_config_cleanup(drm);
- drm->dev_private = NULL;
dev_set_drvdata(dev, NULL);
- drm_dev_put(drm);
}
static const struct component_master_ops hdlcd_master_ops = {
diff --git a/drivers/gpu/drm/arm/hdlcd_drv.h b/drivers/gpu/drm/arm/hdlcd_drv.h
index 909c39c..f1c1da2a 100644
--- a/drivers/gpu/drm/arm/hdlcd_drv.h
+++ b/drivers/gpu/drm/arm/hdlcd_drv.h
@@ -7,6 +7,7 @@
#define __HDLCD_DRV_H__
struct hdlcd_drm_private {
+ struct drm_device base;
void __iomem *mmio;
struct clk *clk;
struct drm_crtc crtc;
@@ -20,6 +21,7 @@ struct hdlcd_drm_private {
#endif
};
+#define drm_to_hdlcd_priv(x) container_of(x, struct hdlcd_drm_private, base)
#define crtc_to_hdlcd_priv(x) container_of(x, struct hdlcd_drm_private, crtc)
static inline void hdlcd_write(struct hdlcd_drm_private *hdlcd,
diff --git a/drivers/gpu/drm/arm/malidp_crtc.c b/drivers/gpu/drm/arm/malidp_crtc.c
index 96273077..dc01c43 100644
--- a/drivers/gpu/drm/arm/malidp_crtc.c
+++ b/drivers/gpu/drm/arm/malidp_crtc.c
@@ -514,7 +514,6 @@ static void malidp_crtc_disable_vblank(struct drm_crtc *crtc)
}
static const struct drm_crtc_funcs malidp_crtc_funcs = {
- .destroy = drm_crtc_cleanup,
.set_config = drm_atomic_helper_set_config,
.page_flip = drm_atomic_helper_page_flip,
.reset = malidp_crtc_reset,
@@ -526,7 +525,7 @@ static const struct drm_crtc_funcs malidp_crtc_funcs = {
int malidp_crtc_init(struct drm_device *drm)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct drm_plane *primary = NULL, *plane;
int ret;
@@ -548,8 +547,8 @@ int malidp_crtc_init(struct drm_device *drm)
return -EINVAL;
}
- ret = drm_crtc_init_with_planes(drm, &malidp->crtc, primary, NULL,
- &malidp_crtc_funcs, NULL);
+ ret = drmm_crtc_init_with_planes(drm, &malidp->crtc, primary, NULL,
+ &malidp_crtc_funcs, NULL);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/arm/malidp_drv.c b/drivers/gpu/drm/arm/malidp_drv.c
index 1d0b0c5..589c1c6 100644
--- a/drivers/gpu/drm/arm/malidp_drv.c
+++ b/drivers/gpu/drm/arm/malidp_drv.c
@@ -19,10 +19,11 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
+#include <drm/drm_managed.h>
#include <drm/drm_modeset_helper.h>
#include <drm/drm_module.h>
#include <drm/drm_of.h>
@@ -168,7 +169,7 @@ static void malidp_atomic_commit_se_config(struct drm_crtc *crtc,
*/
static int malidp_set_and_wait_config_valid(struct drm_device *drm)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
int ret;
@@ -189,7 +190,7 @@ static int malidp_set_and_wait_config_valid(struct drm_device *drm)
static void malidp_atomic_commit_hw_done(struct drm_atomic_state *state)
{
struct drm_device *drm = state->dev;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
int loop = 5;
malidp->event = malidp->crtc.state->event;
@@ -230,7 +231,7 @@ static void malidp_atomic_commit_hw_done(struct drm_atomic_state *state)
static void malidp_atomic_commit_tail(struct drm_atomic_state *state)
{
struct drm_device *drm = state->dev;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct drm_crtc *crtc;
struct drm_crtc_state *old_crtc_state;
int i;
@@ -392,10 +393,12 @@ static const struct drm_mode_config_funcs malidp_mode_config_funcs = {
static int malidp_init(struct drm_device *drm)
{
int ret;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
- drm_mode_config_init(drm);
+ ret = drmm_mode_config_init(drm);
+ if (ret)
+ goto out;
drm->mode_config.min_width = hwdev->min_line_size;
drm->mode_config.min_height = hwdev->min_line_size;
@@ -406,29 +409,21 @@ static int malidp_init(struct drm_device *drm)
ret = malidp_crtc_init(drm);
if (ret)
- goto crtc_fail;
+ goto out;
ret = malidp_mw_connector_init(drm);
if (ret)
- goto crtc_fail;
+ goto out;
- return 0;
-
-crtc_fail:
- drm_mode_config_cleanup(drm);
+out:
return ret;
}
-static void malidp_fini(struct drm_device *drm)
-{
- drm_mode_config_cleanup(drm);
-}
-
static int malidp_irq_init(struct platform_device *pdev)
{
int irq_de, irq_se, ret = 0;
struct drm_device *drm = dev_get_drvdata(&pdev->dev);
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
/* fetch the interrupts from DT */
@@ -462,7 +457,7 @@ static int malidp_dumb_create(struct drm_file *file_priv,
struct drm_device *drm,
struct drm_mode_create_dumb *args)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
/* allocate for the worst case scenario, i.e. rotated buffers */
u8 alignment = malidp_hw_get_pitch_align(malidp->dev, 1);
@@ -508,7 +503,7 @@ static void malidp_error_stats_dump(const char *prefix,
static int malidp_show_stats(struct seq_file *m, void *arg)
{
struct drm_device *drm = m->private;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
unsigned long irqflags;
struct malidp_error_stats de_errors, se_errors;
@@ -531,7 +526,7 @@ static ssize_t malidp_debugfs_write(struct file *file, const char __user *ubuf,
{
struct seq_file *m = file->private_data;
struct drm_device *drm = m->private;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
unsigned long irqflags;
spin_lock_irqsave(&malidp->errors_lock, irqflags);
@@ -552,7 +547,7 @@ static const struct file_operations malidp_debugfs_fops = {
static void malidp_debugfs_init(struct drm_minor *minor)
{
- struct malidp_drm *malidp = minor->dev->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(minor->dev);
malidp_error_stats_init(&malidp->de_errors);
malidp_error_stats_init(&malidp->se_errors);
@@ -652,7 +647,7 @@ static ssize_t core_id_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
struct drm_device *drm = dev_get_drvdata(dev);
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
return snprintf(buf, PAGE_SIZE, "%08x\n", malidp->core_id);
}
@@ -670,7 +665,7 @@ ATTRIBUTE_GROUPS(mali_dp);
static int malidp_runtime_pm_suspend(struct device *dev)
{
struct drm_device *drm = dev_get_drvdata(dev);
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
/* we can only suspend if the hardware is in config mode */
@@ -689,7 +684,7 @@ static int malidp_runtime_pm_suspend(struct device *dev)
static int malidp_runtime_pm_resume(struct device *dev)
{
struct drm_device *drm = dev_get_drvdata(dev);
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
clk_prepare_enable(hwdev->pclk);
@@ -716,11 +711,13 @@ static int malidp_bind(struct device *dev)
int ret = 0, i;
u32 version, out_depth = 0;
- malidp = devm_kzalloc(dev, sizeof(*malidp), GFP_KERNEL);
- if (!malidp)
- return -ENOMEM;
+ malidp = devm_drm_dev_alloc(dev, &malidp_driver, typeof(*malidp), base);
+ if (IS_ERR(malidp))
+ return PTR_ERR(malidp);
- hwdev = devm_kzalloc(dev, sizeof(*hwdev), GFP_KERNEL);
+ drm = &malidp->base;
+
+ hwdev = drmm_kzalloc(drm, sizeof(*hwdev), GFP_KERNEL);
if (!hwdev)
return -ENOMEM;
@@ -753,13 +750,6 @@ static int malidp_bind(struct device *dev)
if (ret && ret != -ENODEV)
return ret;
- drm = drm_dev_alloc(&malidp_driver, dev);
- if (IS_ERR(drm)) {
- ret = PTR_ERR(drm);
- goto alloc_fail;
- }
-
- drm->dev_private = malidp;
dev_set_drvdata(dev, drm);
/* Enable power management */
@@ -878,17 +868,13 @@ static int malidp_bind(struct device *dev)
bind_fail:
of_node_put(malidp->crtc.port);
malidp->crtc.port = NULL;
- malidp_fini(drm);
query_hw_fail:
pm_runtime_put(dev);
if (pm_runtime_enabled(dev))
pm_runtime_disable(dev);
else
malidp_runtime_pm_suspend(dev);
- drm->dev_private = NULL;
dev_set_drvdata(dev, NULL);
- drm_dev_put(drm);
-alloc_fail:
of_reserved_mem_device_release(dev);
return ret;
@@ -897,7 +883,7 @@ static int malidp_bind(struct device *dev)
static void malidp_unbind(struct device *dev)
{
struct drm_device *drm = dev_get_drvdata(dev);
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
drm_dev_unregister(drm);
@@ -909,15 +895,12 @@ static void malidp_unbind(struct device *dev)
component_unbind_all(dev, drm);
of_node_put(malidp->crtc.port);
malidp->crtc.port = NULL;
- malidp_fini(drm);
pm_runtime_put(dev);
if (pm_runtime_enabled(dev))
pm_runtime_disable(dev);
else
malidp_runtime_pm_suspend(dev);
- drm->dev_private = NULL;
dev_set_drvdata(dev, NULL);
- drm_dev_put(drm);
of_reserved_mem_device_release(dev);
}
diff --git a/drivers/gpu/drm/arm/malidp_drv.h b/drivers/gpu/drm/arm/malidp_drv.h
index cdfddfa..bc03878 100644
--- a/drivers/gpu/drm/arm/malidp_drv.h
+++ b/drivers/gpu/drm/arm/malidp_drv.h
@@ -29,6 +29,7 @@ struct malidp_error_stats {
};
struct malidp_drm {
+ struct drm_device base;
struct malidp_hw_device *dev;
struct drm_crtc crtc;
struct drm_writeback_connector mw_connector;
@@ -44,6 +45,7 @@ struct malidp_drm {
#endif
};
+#define drm_to_malidp(x) container_of(x, struct malidp_drm, base)
#define crtc_to_malidp_device(x) container_of(x, struct malidp_drm, crtc)
struct malidp_plane {
diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index e9de542..9b845d3 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -1168,7 +1168,7 @@ static void malidp_hw_clear_irq(struct malidp_hw_device *hwdev, u8 block, u32 ir
static irqreturn_t malidp_de_irq(int irq, void *arg)
{
struct drm_device *drm = arg;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev;
struct malidp_hw *hw;
const struct malidp_irq_map *de;
@@ -1226,7 +1226,7 @@ static irqreturn_t malidp_de_irq(int irq, void *arg)
static irqreturn_t malidp_de_irq_thread_handler(int irq, void *arg)
{
struct drm_device *drm = arg;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
wake_up(&malidp->wq);
@@ -1252,7 +1252,7 @@ void malidp_de_irq_hw_init(struct malidp_hw_device *hwdev)
int malidp_de_irq_init(struct drm_device *drm, int irq)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
int ret;
@@ -1286,7 +1286,7 @@ void malidp_de_irq_fini(struct malidp_hw_device *hwdev)
static irqreturn_t malidp_se_irq(int irq, void *arg)
{
struct drm_device *drm = arg;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
struct malidp_hw *hw = hwdev->hw;
const struct malidp_irq_map *se = &hw->map.se_irq_map;
@@ -1363,7 +1363,7 @@ static irqreturn_t malidp_se_irq_thread_handler(int irq, void *arg)
int malidp_se_irq_init(struct drm_device *drm, int irq)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct malidp_hw_device *hwdev = malidp->dev;
int ret;
diff --git a/drivers/gpu/drm/arm/malidp_mw.c b/drivers/gpu/drm/arm/malidp_mw.c
index ef76d0e..626709b 100644
--- a/drivers/gpu/drm/arm/malidp_mw.c
+++ b/drivers/gpu/drm/arm/malidp_mw.c
@@ -129,7 +129,7 @@ malidp_mw_encoder_atomic_check(struct drm_encoder *encoder,
struct drm_connector_state *conn_state)
{
struct malidp_mw_connector_state *mw_state = to_mw_state(conn_state);
- struct malidp_drm *malidp = encoder->dev->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(encoder->dev);
struct drm_framebuffer *fb;
int i, n_planes;
@@ -207,7 +207,7 @@ static u32 *get_writeback_formats(struct malidp_drm *malidp, int *n_formats)
int malidp_mw_connector_init(struct drm_device *drm)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
u32 *formats;
int ret, n_formats;
@@ -236,7 +236,7 @@ int malidp_mw_connector_init(struct drm_device *drm)
void malidp_mw_atomic_commit(struct drm_device *drm,
struct drm_atomic_state *old_state)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
struct drm_writeback_connector *mw_conn = &malidp->mw_connector;
struct drm_connector_state *conn_state = mw_conn->base.state;
struct malidp_hw_device *hwdev = malidp->dev;
diff --git a/drivers/gpu/drm/arm/malidp_planes.c b/drivers/gpu/drm/arm/malidp_planes.c
index 45f5e35..34547ed 100644
--- a/drivers/gpu/drm/arm/malidp_planes.c
+++ b/drivers/gpu/drm/arm/malidp_planes.c
@@ -68,14 +68,6 @@
/* readahead for partial-frame prefetch */
#define MALIDP_MMU_PREFETCH_READAHEAD 8
-static void malidp_de_plane_destroy(struct drm_plane *plane)
-{
- struct malidp_plane *mp = to_malidp_plane(plane);
-
- drm_plane_cleanup(plane);
- kfree(mp);
-}
-
/*
* Replicate what the default ->reset hook does: free the state pointer and
* allocate a new empty object. We just need enough space to store
@@ -151,7 +143,7 @@ bool malidp_format_mod_supported(struct drm_device *drm,
{
const struct drm_format_info *info;
const u64 *modifiers;
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
const struct malidp_hw_regmap *map = &malidp->dev->hw->map;
if (WARN_ON(modifier == DRM_FORMAT_MOD_INVALID))
@@ -260,7 +252,6 @@ static bool malidp_format_mod_supported_per_plane(struct drm_plane *plane,
static const struct drm_plane_funcs malidp_de_plane_funcs = {
.update_plane = drm_atomic_helper_update_plane,
.disable_plane = drm_atomic_helper_disable_plane,
- .destroy = malidp_de_plane_destroy,
.reset = malidp_plane_reset,
.atomic_duplicate_state = malidp_duplicate_plane_state,
.atomic_destroy_state = malidp_destroy_plane_state,
@@ -931,7 +922,7 @@ static const uint64_t linear_only_modifiers[] = {
int malidp_de_planes_init(struct drm_device *drm)
{
- struct malidp_drm *malidp = drm->dev_private;
+ struct malidp_drm *malidp = drm_to_malidp(drm);
const struct malidp_hw_regmap *map = &malidp->dev->hw->map;
struct malidp_plane *plane = NULL;
enum drm_plane_type plane_type;
@@ -972,12 +963,6 @@ int malidp_de_planes_init(struct drm_device *drm)
for (i = 0; i < map->n_layers; i++) {
u8 id = map->layers[i].id;
- plane = kzalloc(sizeof(*plane), GFP_KERNEL);
- if (!plane) {
- ret = -ENOMEM;
- goto cleanup;
- }
-
/* build the list of DRM supported formats based on the map */
for (n = 0, j = 0; j < map->n_pixel_formats; j++) {
if ((map->pixel_formats[j].layer & id) == id)
@@ -990,13 +975,14 @@ int malidp_de_planes_init(struct drm_device *drm)
/*
* All the layers except smart layer supports AFBC modifiers.
*/
- ret = drm_universal_plane_init(drm, &plane->base, crtcs,
- &malidp_de_plane_funcs, formats, n,
- (id == DE_SMART) ? linear_only_modifiers : modifiers,
- plane_type, NULL);
-
- if (ret < 0)
+ plane = drmm_universal_plane_alloc(drm, struct malidp_plane, base,
+ crtcs, &malidp_de_plane_funcs, formats, n,
+ (id == DE_SMART) ? linear_only_modifiers :
+ modifiers, plane_type, NULL);
+ if (IS_ERR(plane)) {
+ ret = PTR_ERR(plane);
goto cleanup;
+ }
drm_plane_helper_add(&plane->base,
&malidp_de_plane_helper_funcs);
diff --git a/drivers/gpu/drm/armada/armada_fbdev.c b/drivers/gpu/drm/armada/armada_fbdev.c
index 38f5170..584cee1 100644
--- a/drivers/gpu/drm/armada/armada_fbdev.c
+++ b/drivers/gpu/drm/armada/armada_fbdev.c
@@ -19,6 +19,8 @@
static const struct fb_ops armada_fb_ops = {
.owner = THIS_MODULE,
DRM_FB_HELPER_DEFAULT_OPS,
+ .fb_read = drm_fb_helper_cfb_read,
+ .fb_write = drm_fb_helper_cfb_write,
.fb_fillrect = drm_fb_helper_cfb_fillrect,
.fb_copyarea = drm_fb_helper_cfb_copyarea,
.fb_imageblit = drm_fb_helper_cfb_imageblit,
@@ -72,7 +74,7 @@ static int armada_fbdev_create(struct drm_fb_helper *fbh,
if (IS_ERR(dfb))
return PTR_ERR(dfb);
- info = drm_fb_helper_alloc_fbi(fbh);
+ info = drm_fb_helper_alloc_info(fbh);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
goto err_fballoc;
@@ -155,7 +157,7 @@ void armada_fbdev_fini(struct drm_device *dev)
struct drm_fb_helper *fbh = priv->fbdev;
if (fbh) {
- drm_fb_helper_unregister_fbi(fbh);
+ drm_fb_helper_unregister_info(fbh);
drm_fb_helper_fini(fbh);
diff --git a/drivers/gpu/drm/armada/armada_gem.c b/drivers/gpu/drm/armada/armada_gem.c
index 5430265..26d1006 100644
--- a/drivers/gpu/drm/armada/armada_gem.c
+++ b/drivers/gpu/drm/armada/armada_gem.c
@@ -66,8 +66,8 @@ void armada_gem_free_object(struct drm_gem_object *obj)
if (dobj->obj.import_attach) {
/* We only ever display imported data */
if (dobj->sgt)
- dma_buf_unmap_attachment(dobj->obj.import_attach,
- dobj->sgt, DMA_TO_DEVICE);
+ dma_buf_unmap_attachment_unlocked(dobj->obj.import_attach,
+ dobj->sgt, DMA_TO_DEVICE);
drm_prime_gem_destroy(&dobj->obj, NULL);
}
@@ -539,8 +539,8 @@ int armada_gem_map_import(struct armada_gem_object *dobj)
{
int ret;
- dobj->sgt = dma_buf_map_attachment(dobj->obj.import_attach,
- DMA_TO_DEVICE);
+ dobj->sgt = dma_buf_map_attachment_unlocked(dobj->obj.import_attach,
+ DMA_TO_DEVICE);
if (IS_ERR(dobj->sgt)) {
ret = PTR_ERR(dobj->sgt);
dobj->sgt = NULL;
diff --git a/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c b/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c
index a94f1a9..718119e1 100644
--- a/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c
+++ b/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c
@@ -16,7 +16,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_device.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/ast/Kconfig b/drivers/gpu/drm/ast/Kconfig
index fbcf2f45..d367a90 100644
--- a/drivers/gpu/drm/ast/Kconfig
+++ b/drivers/gpu/drm/ast/Kconfig
@@ -2,10 +2,8 @@
config DRM_AST
tristate "AST server chips"
depends on DRM && PCI && MMU
+ select DRM_GEM_SHMEM_HELPER
select DRM_KMS_HELPER
- select DRM_VRAM_HELPER
- select DRM_TTM
- select DRM_TTM_HELPER
help
Say yes for experimental AST GPU driver. Do not enable
this driver without having a working -modesetting,
diff --git a/drivers/gpu/drm/ast/ast_drv.c b/drivers/gpu/drm/ast/ast_drv.c
index b9392f3..420fc75 100644
--- a/drivers/gpu/drm/ast/ast_drv.c
+++ b/drivers/gpu/drm/ast/ast_drv.c
@@ -33,7 +33,8 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_gem_vram_helper.h>
+#include <drm/drm_fbdev_generic.h>
+#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_module.h>
#include <drm/drm_probe_helper.h>
@@ -63,7 +64,7 @@ static const struct drm_driver ast_driver = {
.minor = DRIVER_MINOR,
.patchlevel = DRIVER_PATCHLEVEL,
- DRM_GEM_VRAM_DRIVER
+ DRM_GEM_SHMEM_DRIVER_OPS
};
/*
diff --git a/drivers/gpu/drm/ast/ast_drv.h b/drivers/gpu/drm/ast/ast_drv.h
index 2e44b97..d51b81f 100644
--- a/drivers/gpu/drm/ast/ast_drv.h
+++ b/drivers/gpu/drm/ast/ast_drv.h
@@ -38,7 +38,6 @@
#include <drm/drm_encoder.h>
#include <drm/drm_mode.h>
#include <drm/drm_framebuffer.h>
-#include <drm/drm_fb_helper.h>
#define DRIVER_AUTHOR "Dave Airlie"
@@ -87,7 +86,7 @@ enum ast_tx_chip {
#define AST_DRAM_8Gx16 8
/*
- * Cursor plane
+ * Hardware cursor
*/
#define AST_MAX_HWC_WIDTH 64
@@ -96,8 +95,6 @@ enum ast_tx_chip {
#define AST_HWC_SIZE (AST_MAX_HWC_WIDTH * AST_MAX_HWC_HEIGHT * 2)
#define AST_HWC_SIGNATURE_SIZE 32
-#define AST_DEFAULT_HWC_NUM 2
-
/* define for signature structure */
#define AST_HWC_SIGNATURE_CHECKSUM 0x00
#define AST_HWC_SIGNATURE_SizeX 0x04
@@ -107,22 +104,21 @@ enum ast_tx_chip {
#define AST_HWC_SIGNATURE_HOTSPOTX 0x14
#define AST_HWC_SIGNATURE_HOTSPOTY 0x18
-struct ast_cursor_plane {
+/*
+ * Planes
+ */
+
+struct ast_plane {
struct drm_plane base;
- struct {
- struct drm_gem_vram_object *gbo;
- struct iosys_map map;
- u64 off;
- } hwc[AST_DEFAULT_HWC_NUM];
-
- unsigned int next_hwc_index;
+ void __iomem *vaddr;
+ u64 offset;
+ unsigned long size;
};
-static inline struct ast_cursor_plane *
-to_ast_cursor_plane(struct drm_plane *plane)
+static inline struct ast_plane *to_ast_plane(struct drm_plane *plane)
{
- return container_of(plane, struct ast_cursor_plane, base);
+ return container_of(plane, struct ast_plane, base);
}
/*
@@ -175,8 +171,13 @@ struct ast_private {
uint32_t dram_type;
uint32_t mclk;
- struct drm_plane primary_plane;
- struct ast_cursor_plane cursor_plane;
+ void __iomem *vram;
+ unsigned long vram_base;
+ unsigned long vram_size;
+ unsigned long vram_fb_available;
+
+ struct ast_plane primary_plane;
+ struct ast_plane cursor_plane;
struct drm_crtc crtc;
struct {
struct {
diff --git a/drivers/gpu/drm/ast/ast_main.c b/drivers/gpu/drm/ast/ast_main.c
index 0674532..bffa310 100644
--- a/drivers/gpu/drm/ast/ast_main.c
+++ b/drivers/gpu/drm/ast/ast_main.c
@@ -32,7 +32,6 @@
#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_gem.h>
-#include <drm/drm_gem_vram_helper.h>
#include <drm/drm_managed.h>
#include "ast_drv.h"
@@ -461,8 +460,8 @@ struct ast_private *ast_device_create(const struct drm_driver *drv,
/* map reserved buffer */
ast->dp501_fw_buf = NULL;
- if (dev->vram_mm->vram_size < pci_resource_len(pdev, 0)) {
- ast->dp501_fw_buf = pci_iomap_range(pdev, 0, dev->vram_mm->vram_size, 0);
+ if (ast->vram_size < pci_resource_len(pdev, 0)) {
+ ast->dp501_fw_buf = pci_iomap_range(pdev, 0, ast->vram_size, 0);
if (!ast->dp501_fw_buf)
drm_info(dev, "failed to map reserved buffer!\n");
}
diff --git a/drivers/gpu/drm/ast/ast_mm.c b/drivers/gpu/drm/ast/ast_mm.c
index 6e99940..248284a 100644
--- a/drivers/gpu/drm/ast/ast_mm.c
+++ b/drivers/gpu/drm/ast/ast_mm.c
@@ -28,7 +28,6 @@
#include <linux/pci.h>
-#include <drm/drm_gem_vram_helper.h>
#include <drm/drm_managed.h>
#include <drm/drm_print.h>
@@ -80,7 +79,6 @@ int ast_mm_init(struct ast_private *ast)
struct pci_dev *pdev = to_pci_dev(dev->dev);
resource_size_t base, size;
u32 vram_size;
- int ret;
base = pci_resource_start(pdev, 0);
size = pci_resource_len(pdev, 0);
@@ -91,11 +89,13 @@ int ast_mm_init(struct ast_private *ast)
vram_size = ast_get_vram_size(ast);
- ret = drmm_vram_helper_init(dev, base, vram_size);
- if (ret) {
- drm_err(dev, "Error initializing VRAM MM; %d\n", ret);
- return ret;
- }
+ ast->vram = devm_ioremap_wc(dev->dev, base, vram_size);
+ if (!ast->vram)
+ return -ENOMEM;
+
+ ast->vram_base = base;
+ ast->vram_size = vram_size;
+ ast->vram_fb_available = vram_size;
return 0;
}
diff --git a/drivers/gpu/drm/ast/ast_mode.c b/drivers/gpu/drm/ast/ast_mode.c
index 1bc0220..c744331 100644
--- a/drivers/gpu/drm/ast/ast_mode.c
+++ b/drivers/gpu/drm/ast/ast_mode.c
@@ -36,11 +36,13 @@
#include <drm/drm_atomic_state_helper.h>
#include <drm/drm_crtc.h>
#include <drm/drm_crtc_helper.h>
+#include <drm/drm_damage_helper.h>
#include <drm/drm_edid.h>
+#include <drm/drm_format_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
-#include <drm/drm_gem_vram_helper.h>
+#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_managed.h>
#include <drm/drm_probe_helper.h>
#include <drm/drm_simple_kms_helper.h>
@@ -48,6 +50,8 @@
#include "ast_drv.h"
#include "ast_tables.h"
+#define AST_LUT_SIZE 256
+
static inline void ast_load_palette_index(struct ast_private *ast,
u8 index, u8 red, u8 green,
u8 blue)
@@ -62,20 +66,46 @@ static inline void ast_load_palette_index(struct ast_private *ast,
ast_io_read8(ast, AST_IO_SEQ_PORT);
}
-static void ast_crtc_load_lut(struct ast_private *ast, struct drm_crtc *crtc)
+static void ast_crtc_set_gamma_linear(struct ast_private *ast,
+ const struct drm_format_info *format)
{
- u16 *r, *g, *b;
int i;
- if (!crtc->enabled)
- return;
+ switch (format->format) {
+ case DRM_FORMAT_C8: /* In this case, gamma table is used as color palette */
+ case DRM_FORMAT_RGB565:
+ case DRM_FORMAT_XRGB8888:
+ for (i = 0; i < AST_LUT_SIZE; i++)
+ ast_load_palette_index(ast, i, i, i, i);
+ break;
+ default:
+ drm_warn_once(&ast->base, "Unsupported format %p4cc for gamma correction\n",
+ &format->format);
+ break;
+ }
+}
- r = crtc->gamma_store;
- g = r + crtc->gamma_size;
- b = g + crtc->gamma_size;
+static void ast_crtc_set_gamma(struct ast_private *ast,
+ const struct drm_format_info *format,
+ struct drm_color_lut *lut)
+{
+ int i;
- for (i = 0; i < 256; i++)
- ast_load_palette_index(ast, i, *r++ >> 8, *g++ >> 8, *b++ >> 8);
+ switch (format->format) {
+ case DRM_FORMAT_C8: /* In this case, gamma table is used as color palette */
+ case DRM_FORMAT_RGB565:
+ case DRM_FORMAT_XRGB8888:
+ for (i = 0; i < AST_LUT_SIZE; i++)
+ ast_load_palette_index(ast, i,
+ lut[i].red >> 8,
+ lut[i].green >> 8,
+ lut[i].blue >> 8);
+ break;
+ default:
+ drm_warn_once(&ast->base, "Unsupported format %p4cc for gamma correction\n",
+ &format->format);
+ break;
+ }
}
static bool ast_get_vbios_mode_info(const struct drm_format_info *format,
@@ -538,6 +568,29 @@ static void ast_wait_for_vretrace(struct ast_private *ast)
}
/*
+ * Planes
+ */
+
+static int ast_plane_init(struct drm_device *dev, struct ast_plane *ast_plane,
+ void __iomem *vaddr, u64 offset, unsigned long size,
+ uint32_t possible_crtcs,
+ const struct drm_plane_funcs *funcs,
+ const uint32_t *formats, unsigned int format_count,
+ const uint64_t *format_modifiers,
+ enum drm_plane_type type)
+{
+ struct drm_plane *plane = &ast_plane->base;
+
+ ast_plane->vaddr = vaddr;
+ ast_plane->offset = offset;
+ ast_plane->size = size;
+
+ return drm_universal_plane_init(dev, plane, possible_crtcs, funcs,
+ formats, format_count, format_modifiers,
+ type, NULL);
+}
+
+/*
* Primary plane
*/
@@ -550,52 +603,62 @@ static const uint32_t ast_primary_plane_formats[] = {
static int ast_primary_plane_helper_atomic_check(struct drm_plane *plane,
struct drm_atomic_state *state)
{
- struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
- plane);
- struct drm_crtc_state *crtc_state;
- struct ast_crtc_state *ast_crtc_state;
+ struct drm_device *dev = plane->dev;
+ struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state, plane);
+ struct drm_crtc_state *new_crtc_state = NULL;
+ struct ast_crtc_state *new_ast_crtc_state;
int ret;
- if (!new_plane_state->crtc)
- return 0;
+ if (new_plane_state->crtc)
+ new_crtc_state = drm_atomic_get_new_crtc_state(state, new_plane_state->crtc);
- crtc_state = drm_atomic_get_new_crtc_state(state,
- new_plane_state->crtc);
-
- ret = drm_atomic_helper_check_plane_state(new_plane_state, crtc_state,
+ ret = drm_atomic_helper_check_plane_state(new_plane_state, new_crtc_state,
DRM_PLANE_NO_SCALING,
DRM_PLANE_NO_SCALING,
false, true);
- if (ret)
+ if (ret) {
return ret;
+ } else if (!new_plane_state->visible) {
+ if (drm_WARN_ON(dev, new_plane_state->crtc)) /* cannot legally happen */
+ return -EINVAL;
+ else
+ return 0;
+ }
- if (!new_plane_state->visible)
- return 0;
+ new_ast_crtc_state = to_ast_crtc_state(new_crtc_state);
- ast_crtc_state = to_ast_crtc_state(crtc_state);
-
- ast_crtc_state->format = new_plane_state->fb->format;
+ new_ast_crtc_state->format = new_plane_state->fb->format;
return 0;
}
-static void
-ast_primary_plane_helper_atomic_update(struct drm_plane *plane,
- struct drm_atomic_state *state)
+static void ast_handle_damage(struct ast_plane *ast_plane, struct iosys_map *src,
+ struct drm_framebuffer *fb,
+ const struct drm_rect *clip)
{
- struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state,
- plane);
+ struct iosys_map dst = IOSYS_MAP_INIT_VADDR(ast_plane->vaddr);
+
+ iosys_map_incr(&dst, drm_fb_clip_offset(fb->pitches[0], fb->format, clip));
+ drm_fb_memcpy(&dst, fb->pitches, src, fb, clip);
+}
+
+static void ast_primary_plane_helper_atomic_update(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
struct drm_device *dev = plane->dev;
struct ast_private *ast = to_ast_private(dev);
- struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
- plane);
- struct drm_gem_vram_object *gbo;
- s64 gpu_addr;
- struct drm_framebuffer *fb = new_state->fb;
- struct drm_framebuffer *old_fb = old_state->fb;
+ struct drm_plane_state *plane_state = drm_atomic_get_new_plane_state(state, plane);
+ struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(plane_state);
+ struct drm_framebuffer *fb = plane_state->fb;
+ struct drm_plane_state *old_plane_state = drm_atomic_get_old_plane_state(state, plane);
+ struct drm_framebuffer *old_fb = old_plane_state->fb;
+ struct ast_plane *ast_plane = to_ast_plane(plane);
+ struct drm_rect damage;
+ struct drm_atomic_helper_damage_iter iter;
if (!old_fb || (fb->format != old_fb->format)) {
- struct drm_crtc_state *crtc_state = new_state->crtc->state;
+ struct drm_crtc *crtc = plane_state->crtc;
+ struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
struct ast_crtc_state *ast_crtc_state = to_ast_crtc_state(crtc_state);
struct ast_vbios_mode_info *vbios_mode_info = &ast_crtc_state->vbios_mode_info;
@@ -603,20 +666,28 @@ ast_primary_plane_helper_atomic_update(struct drm_plane *plane,
ast_set_vbios_color_reg(ast, fb->format, vbios_mode_info);
}
- gbo = drm_gem_vram_of_gem(fb->obj[0]);
- gpu_addr = drm_gem_vram_offset(gbo);
- if (drm_WARN_ON_ONCE(dev, gpu_addr < 0))
- return; /* Bug: we didn't pin the BO to VRAM in prepare_fb. */
+ drm_atomic_helper_damage_iter_init(&iter, old_plane_state, plane_state);
+ drm_atomic_for_each_plane_damage(&iter, &damage) {
+ ast_handle_damage(ast_plane, shadow_plane_state->data, fb, &damage);
+ }
- ast_set_offset_reg(ast, fb);
- ast_set_start_address_crt1(ast, (u32)gpu_addr);
-
- ast_set_index_reg_mask(ast, AST_IO_SEQ_PORT, 0x1, 0xdf, 0x00);
+ /*
+ * Some BMCs stop scanning out the video signal after the driver
+ * reprogrammed the offset or scanout address. This stalls display
+ * output for several seconds and makes the display unusable.
+ * Therefore only update the offset if it changes and reprogram the
+ * address after enabling the plane.
+ */
+ if (!old_fb || old_fb->pitches[0] != fb->pitches[0])
+ ast_set_offset_reg(ast, fb);
+ if (!old_fb) {
+ ast_set_start_address_crt1(ast, (u32)ast_plane->offset);
+ ast_set_index_reg_mask(ast, AST_IO_SEQ_PORT, 0x1, 0xdf, 0x00);
+ }
}
-static void
-ast_primary_plane_helper_atomic_disable(struct drm_plane *plane,
- struct drm_atomic_state *state)
+static void ast_primary_plane_helper_atomic_disable(struct drm_plane *plane,
+ struct drm_atomic_state *state)
{
struct ast_private *ast = to_ast_private(plane->dev);
@@ -624,7 +695,7 @@ ast_primary_plane_helper_atomic_disable(struct drm_plane *plane,
}
static const struct drm_plane_helper_funcs ast_primary_plane_helper_funcs = {
- DRM_GEM_VRAM_PLANE_HELPER_FUNCS,
+ DRM_GEM_SHADOW_PLANE_HELPER_FUNCS,
.atomic_check = ast_primary_plane_helper_atomic_check,
.atomic_update = ast_primary_plane_helper_atomic_update,
.atomic_disable = ast_primary_plane_helper_atomic_disable,
@@ -634,27 +705,30 @@ static const struct drm_plane_funcs ast_primary_plane_funcs = {
.update_plane = drm_atomic_helper_update_plane,
.disable_plane = drm_atomic_helper_disable_plane,
.destroy = drm_plane_cleanup,
- .reset = drm_atomic_helper_plane_reset,
- .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+ DRM_GEM_SHADOW_PLANE_FUNCS,
};
static int ast_primary_plane_init(struct ast_private *ast)
{
struct drm_device *dev = &ast->base;
- struct drm_plane *primary_plane = &ast->primary_plane;
+ struct ast_plane *ast_primary_plane = &ast->primary_plane;
+ struct drm_plane *primary_plane = &ast_primary_plane->base;
+ void __iomem *vaddr = ast->vram;
+ u64 offset = ast->vram_base;
+ unsigned long cursor_size = roundup(AST_HWC_SIZE + AST_HWC_SIGNATURE_SIZE, PAGE_SIZE);
+ unsigned long size = ast->vram_fb_available - cursor_size;
int ret;
- ret = drm_universal_plane_init(dev, primary_plane, 0x01,
- &ast_primary_plane_funcs,
- ast_primary_plane_formats,
- ARRAY_SIZE(ast_primary_plane_formats),
- NULL, DRM_PLANE_TYPE_PRIMARY, NULL);
+ ret = ast_plane_init(dev, ast_primary_plane, vaddr, offset, size,
+ 0x01, &ast_primary_plane_funcs,
+ ast_primary_plane_formats, ARRAY_SIZE(ast_primary_plane_formats),
+ NULL, DRM_PLANE_TYPE_PRIMARY);
if (ret) {
- drm_err(dev, "drm_universal_plane_init() failed: %d\n", ret);
+ drm_err(dev, "ast_plane_init() failed: %d\n", ret);
return ret;
}
drm_plane_helper_add(primary_plane, &ast_primary_plane_helper_funcs);
+ drm_plane_enable_fb_damage_clips(primary_plane);
return 0;
}
@@ -774,99 +848,79 @@ static const uint32_t ast_cursor_plane_formats[] = {
static int ast_cursor_plane_helper_atomic_check(struct drm_plane *plane,
struct drm_atomic_state *state)
{
- struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
- plane);
- struct drm_framebuffer *fb = new_plane_state->fb;
- struct drm_crtc_state *crtc_state;
+ struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state, plane);
+ struct drm_framebuffer *new_fb = new_plane_state->fb;
+ struct drm_crtc_state *new_crtc_state = NULL;
int ret;
- if (!new_plane_state->crtc)
- return 0;
+ if (new_plane_state->crtc)
+ new_crtc_state = drm_atomic_get_new_crtc_state(state, new_plane_state->crtc);
- crtc_state = drm_atomic_get_new_crtc_state(state,
- new_plane_state->crtc);
-
- ret = drm_atomic_helper_check_plane_state(new_plane_state, crtc_state,
+ ret = drm_atomic_helper_check_plane_state(new_plane_state, new_crtc_state,
DRM_PLANE_NO_SCALING,
DRM_PLANE_NO_SCALING,
true, true);
- if (ret)
+ if (ret || !new_plane_state->visible)
return ret;
- if (!new_plane_state->visible)
- return 0;
-
- if (fb->width > AST_MAX_HWC_WIDTH || fb->height > AST_MAX_HWC_HEIGHT)
+ if (new_fb->width > AST_MAX_HWC_WIDTH || new_fb->height > AST_MAX_HWC_HEIGHT)
return -EINVAL;
return 0;
}
-static void
-ast_cursor_plane_helper_atomic_update(struct drm_plane *plane,
- struct drm_atomic_state *state)
+static void ast_cursor_plane_helper_atomic_update(struct drm_plane *plane,
+ struct drm_atomic_state *state)
{
- struct ast_cursor_plane *ast_cursor_plane = to_ast_cursor_plane(plane);
- struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state,
- plane);
- struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
- plane);
- struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(new_state);
- struct drm_framebuffer *fb = new_state->fb;
+ struct ast_plane *ast_plane = to_ast_plane(plane);
+ struct drm_plane_state *plane_state = drm_atomic_get_new_plane_state(state, plane);
+ struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(plane_state);
+ struct drm_framebuffer *fb = plane_state->fb;
+ struct drm_plane_state *old_plane_state = drm_atomic_get_old_plane_state(state, plane);
struct ast_private *ast = to_ast_private(plane->dev);
- struct iosys_map dst_map =
- ast_cursor_plane->hwc[ast_cursor_plane->next_hwc_index].map;
- u64 dst_off =
- ast_cursor_plane->hwc[ast_cursor_plane->next_hwc_index].off;
struct iosys_map src_map = shadow_plane_state->data[0];
+ struct drm_rect damage;
+ const u8 *src = src_map.vaddr; /* TODO: Use mapping abstraction properly */
+ u64 dst_off = ast_plane->offset;
+ u8 __iomem *dst = ast_plane->vaddr; /* TODO: Use mapping abstraction properly */
+ u8 __iomem *sig = dst + AST_HWC_SIZE; /* TODO: Use mapping abstraction properly */
unsigned int offset_x, offset_y;
u16 x, y;
u8 x_offset, y_offset;
- u8 __iomem *dst;
- u8 __iomem *sig;
- const u8 *src;
-
- src = src_map.vaddr; /* TODO: Use mapping abstraction properly */
- dst = dst_map.vaddr_iomem; /* TODO: Use mapping abstraction properly */
- sig = dst + AST_HWC_SIZE; /* TODO: Use mapping abstraction properly */
/*
- * Do data transfer to HW cursor BO. If a new cursor image was installed,
- * point the scanout engine to dst_gbo's offset and page-flip the HWC buffers.
+ * Do data transfer to hardware buffer and point the scanout
+ * engine to the offset.
*/
- ast_update_cursor_image(dst, src, fb->width, fb->height);
-
- if (new_state->fb != old_state->fb) {
+ if (drm_atomic_helper_damage_merged(old_plane_state, plane_state, &damage)) {
+ ast_update_cursor_image(dst, src, fb->width, fb->height);
ast_set_cursor_base(ast, dst_off);
-
- ++ast_cursor_plane->next_hwc_index;
- ast_cursor_plane->next_hwc_index %= ARRAY_SIZE(ast_cursor_plane->hwc);
}
/*
* Update location in HWC signature and registers.
*/
- writel(new_state->crtc_x, sig + AST_HWC_SIGNATURE_X);
- writel(new_state->crtc_y, sig + AST_HWC_SIGNATURE_Y);
+ writel(plane_state->crtc_x, sig + AST_HWC_SIGNATURE_X);
+ writel(plane_state->crtc_y, sig + AST_HWC_SIGNATURE_Y);
offset_x = AST_MAX_HWC_WIDTH - fb->width;
offset_y = AST_MAX_HWC_HEIGHT - fb->height;
- if (new_state->crtc_x < 0) {
- x_offset = (-new_state->crtc_x) + offset_x;
+ if (plane_state->crtc_x < 0) {
+ x_offset = (-plane_state->crtc_x) + offset_x;
x = 0;
} else {
x_offset = offset_x;
- x = new_state->crtc_x;
+ x = plane_state->crtc_x;
}
- if (new_state->crtc_y < 0) {
- y_offset = (-new_state->crtc_y) + offset_y;
+ if (plane_state->crtc_y < 0) {
+ y_offset = (-plane_state->crtc_y) + offset_y;
y = 0;
} else {
y_offset = offset_y;
- y = new_state->crtc_y;
+ y = plane_state->crtc_y;
}
ast_set_cursor_location(ast, x, y, x_offset, y_offset);
@@ -875,9 +929,8 @@ ast_cursor_plane_helper_atomic_update(struct drm_plane *plane,
ast_set_cursor_enabled(ast, true);
}
-static void
-ast_cursor_plane_helper_atomic_disable(struct drm_plane *plane,
- struct drm_atomic_state *state)
+static void ast_cursor_plane_helper_atomic_disable(struct drm_plane *plane,
+ struct drm_atomic_state *state)
{
struct ast_private *ast = to_ast_private(plane->dev);
@@ -891,41 +944,22 @@ static const struct drm_plane_helper_funcs ast_cursor_plane_helper_funcs = {
.atomic_disable = ast_cursor_plane_helper_atomic_disable,
};
-static void ast_cursor_plane_destroy(struct drm_plane *plane)
-{
- struct ast_cursor_plane *ast_cursor_plane = to_ast_cursor_plane(plane);
- size_t i;
- struct drm_gem_vram_object *gbo;
- struct iosys_map map;
-
- for (i = 0; i < ARRAY_SIZE(ast_cursor_plane->hwc); ++i) {
- gbo = ast_cursor_plane->hwc[i].gbo;
- map = ast_cursor_plane->hwc[i].map;
- drm_gem_vram_vunmap(gbo, &map);
- drm_gem_vram_unpin(gbo);
- drm_gem_vram_put(gbo);
- }
-
- drm_plane_cleanup(plane);
-}
-
static const struct drm_plane_funcs ast_cursor_plane_funcs = {
.update_plane = drm_atomic_helper_update_plane,
.disable_plane = drm_atomic_helper_disable_plane,
- .destroy = ast_cursor_plane_destroy,
+ .destroy = drm_plane_cleanup,
DRM_GEM_SHADOW_PLANE_FUNCS,
};
static int ast_cursor_plane_init(struct ast_private *ast)
{
struct drm_device *dev = &ast->base;
- struct ast_cursor_plane *ast_cursor_plane = &ast->cursor_plane;
+ struct ast_plane *ast_cursor_plane = &ast->cursor_plane;
struct drm_plane *cursor_plane = &ast_cursor_plane->base;
- size_t size, i;
- struct drm_gem_vram_object *gbo;
- struct iosys_map map;
+ size_t size;
+ void __iomem *vaddr;
+ u64 offset;
int ret;
- s64 off;
/*
* Allocate backing storage for cursors. The BOs are permanently
@@ -934,60 +968,26 @@ static int ast_cursor_plane_init(struct ast_private *ast)
size = roundup(AST_HWC_SIZE + AST_HWC_SIGNATURE_SIZE, PAGE_SIZE);
- for (i = 0; i < ARRAY_SIZE(ast_cursor_plane->hwc); ++i) {
- gbo = drm_gem_vram_create(dev, size, 0);
- if (IS_ERR(gbo)) {
- ret = PTR_ERR(gbo);
- goto err_hwc;
- }
- ret = drm_gem_vram_pin(gbo, DRM_GEM_VRAM_PL_FLAG_VRAM |
- DRM_GEM_VRAM_PL_FLAG_TOPDOWN);
- if (ret)
- goto err_drm_gem_vram_put;
- ret = drm_gem_vram_vmap(gbo, &map);
- if (ret)
- goto err_drm_gem_vram_unpin;
- off = drm_gem_vram_offset(gbo);
- if (off < 0) {
- ret = off;
- goto err_drm_gem_vram_vunmap;
- }
- ast_cursor_plane->hwc[i].gbo = gbo;
- ast_cursor_plane->hwc[i].map = map;
- ast_cursor_plane->hwc[i].off = off;
- }
+ if (ast->vram_fb_available < size)
+ return -ENOMEM;
- /*
- * Create the cursor plane. The plane's destroy callback will release
- * the backing storages' BO memory.
- */
+ vaddr = ast->vram + ast->vram_fb_available - size;
+ offset = ast->vram_base + ast->vram_fb_available - size;
- ret = drm_universal_plane_init(dev, cursor_plane, 0x01,
- &ast_cursor_plane_funcs,
- ast_cursor_plane_formats,
- ARRAY_SIZE(ast_cursor_plane_formats),
- NULL, DRM_PLANE_TYPE_CURSOR, NULL);
+ ret = ast_plane_init(dev, ast_cursor_plane, vaddr, offset, size,
+ 0x01, &ast_cursor_plane_funcs,
+ ast_cursor_plane_formats, ARRAY_SIZE(ast_cursor_plane_formats),
+ NULL, DRM_PLANE_TYPE_CURSOR);
if (ret) {
- drm_err(dev, "drm_universal_plane failed(): %d\n", ret);
- goto err_hwc;
+ drm_err(dev, "ast_plane_init() failed: %d\n", ret);
+ return ret;
}
drm_plane_helper_add(cursor_plane, &ast_cursor_plane_helper_funcs);
+ drm_plane_enable_fb_damage_clips(cursor_plane);
+
+ ast->vram_fb_available -= size;
return 0;
-
-err_hwc:
- while (i) {
- --i;
- gbo = ast_cursor_plane->hwc[i].gbo;
- map = ast_cursor_plane->hwc[i].map;
-err_drm_gem_vram_vunmap:
- drm_gem_vram_vunmap(gbo, &map);
-err_drm_gem_vram_unpin:
- drm_gem_vram_unpin(gbo);
-err_drm_gem_vram_put:
- drm_gem_vram_put(gbo);
- }
- return ret;
}
/*
@@ -1026,9 +1026,11 @@ static void ast_crtc_dpms(struct drm_crtc *crtc, int mode)
ast_set_color_reg(ast, format);
ast_set_vbios_color_reg(ast, format, vbios_mode_info);
+ if (crtc->state->gamma_lut)
+ ast_crtc_set_gamma(ast, format, crtc->state->gamma_lut->data);
+ else
+ ast_crtc_set_gamma_linear(ast, format);
}
-
- ast_crtc_load_lut(ast, crtc);
break;
case DRM_MODE_DPMS_STANDBY:
case DRM_MODE_DPMS_SUSPEND:
@@ -1123,47 +1125,50 @@ static int ast_crtc_helper_atomic_check(struct drm_crtc *crtc,
struct drm_atomic_state *state)
{
struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
+ struct drm_crtc_state *old_crtc_state = drm_atomic_get_old_crtc_state(state, crtc);
+ struct ast_crtc_state *old_ast_crtc_state = to_ast_crtc_state(old_crtc_state);
struct drm_device *dev = crtc->dev;
struct ast_crtc_state *ast_state;
const struct drm_format_info *format;
bool succ;
int ret;
- ret = drm_atomic_helper_check_crtc_state(crtc_state, false);
+ if (!crtc_state->enable)
+ return 0;
+
+ ret = drm_atomic_helper_check_crtc_primary_plane(crtc_state);
if (ret)
return ret;
- if (!crtc_state->enable)
- goto out;
-
ast_state = to_ast_crtc_state(crtc_state);
format = ast_state->format;
if (drm_WARN_ON_ONCE(dev, !format))
return -EINVAL; /* BUG: We didn't set format in primary check(). */
+ /*
+ * The gamma LUT has to be reloaded after changing the primary
+ * plane's color format.
+ */
+ if (old_ast_crtc_state->format != format)
+ crtc_state->color_mgmt_changed = true;
+
+ if (crtc_state->color_mgmt_changed && crtc_state->gamma_lut) {
+ if (crtc_state->gamma_lut->length !=
+ AST_LUT_SIZE * sizeof(struct drm_color_lut)) {
+ drm_err(dev, "Wrong size for gamma_lut %zu\n",
+ crtc_state->gamma_lut->length);
+ return -EINVAL;
+ }
+ }
+
succ = ast_get_vbios_mode_info(format, &crtc_state->mode,
&crtc_state->adjusted_mode,
&ast_state->vbios_mode_info);
if (!succ)
return -EINVAL;
-out:
- return drm_atomic_add_affected_planes(state, crtc);
-}
-
-static void ast_crtc_helper_atomic_begin(struct drm_crtc *crtc, struct drm_atomic_state *state)
-{
- struct drm_device *dev = crtc->dev;
- struct ast_private *ast = to_ast_private(dev);
-
- /*
- * Concurrent operations could possibly trigger a call to
- * drm_connector_helper_funcs.get_modes by trying to read the
- * display modes. Protect access to I/O registers by acquiring
- * the I/O-register lock. Released in atomic_flush().
- */
- mutex_lock(&ast->ioregs_lock);
+ return 0;
}
static void
@@ -1172,35 +1177,34 @@ ast_crtc_helper_atomic_flush(struct drm_crtc *crtc,
{
struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state,
crtc);
- struct drm_crtc_state *old_crtc_state = drm_atomic_get_old_crtc_state(state,
- crtc);
struct drm_device *dev = crtc->dev;
struct ast_private *ast = to_ast_private(dev);
struct ast_crtc_state *ast_crtc_state = to_ast_crtc_state(crtc_state);
- struct ast_crtc_state *old_ast_crtc_state = to_ast_crtc_state(old_crtc_state);
struct ast_vbios_mode_info *vbios_mode_info = &ast_crtc_state->vbios_mode_info;
/*
* The gamma LUT has to be reloaded after changing the primary
* plane's color format.
*/
- if (old_ast_crtc_state->format != ast_crtc_state->format)
- ast_crtc_load_lut(ast, crtc);
+ if (crtc_state->enable && crtc_state->color_mgmt_changed) {
+ if (crtc_state->gamma_lut)
+ ast_crtc_set_gamma(ast,
+ ast_crtc_state->format,
+ crtc_state->gamma_lut->data);
+ else
+ ast_crtc_set_gamma_linear(ast, ast_crtc_state->format);
+ }
//Set Aspeed Display-Port
if (ast->tx_chip_types & AST_TX_ASTDP_BIT)
ast_dp_set_mode(crtc, vbios_mode_info);
-
- mutex_unlock(&ast->ioregs_lock);
}
-static void
-ast_crtc_helper_atomic_enable(struct drm_crtc *crtc,
- struct drm_atomic_state *state)
+static void ast_crtc_helper_atomic_enable(struct drm_crtc *crtc, struct drm_atomic_state *state)
{
struct drm_device *dev = crtc->dev;
struct ast_private *ast = to_ast_private(dev);
- struct drm_crtc_state *crtc_state = crtc->state;
+ struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
struct ast_crtc_state *ast_crtc_state = to_ast_crtc_state(crtc_state);
struct ast_vbios_mode_info *vbios_mode_info =
&ast_crtc_state->vbios_mode_info;
@@ -1217,12 +1221,9 @@ ast_crtc_helper_atomic_enable(struct drm_crtc *crtc,
ast_crtc_dpms(crtc, DRM_MODE_DPMS_ON);
}
-static void
-ast_crtc_helper_atomic_disable(struct drm_crtc *crtc,
- struct drm_atomic_state *state)
+static void ast_crtc_helper_atomic_disable(struct drm_crtc *crtc, struct drm_atomic_state *state)
{
- struct drm_crtc_state *old_crtc_state = drm_atomic_get_old_crtc_state(state,
- crtc);
+ struct drm_crtc_state *old_crtc_state = drm_atomic_get_old_crtc_state(state, crtc);
struct drm_device *dev = crtc->dev;
struct ast_private *ast = to_ast_private(dev);
@@ -1250,7 +1251,6 @@ ast_crtc_helper_atomic_disable(struct drm_crtc *crtc,
static const struct drm_crtc_helper_funcs ast_crtc_helper_funcs = {
.mode_valid = ast_crtc_helper_mode_valid,
.atomic_check = ast_crtc_helper_atomic_check,
- .atomic_begin = ast_crtc_helper_atomic_begin,
.atomic_flush = ast_crtc_helper_atomic_flush,
.atomic_enable = ast_crtc_helper_atomic_enable,
.atomic_disable = ast_crtc_helper_atomic_disable,
@@ -1317,13 +1317,15 @@ static int ast_crtc_init(struct drm_device *dev)
struct drm_crtc *crtc = &ast->crtc;
int ret;
- ret = drm_crtc_init_with_planes(dev, crtc, &ast->primary_plane,
+ ret = drm_crtc_init_with_planes(dev, crtc, &ast->primary_plane.base,
&ast->cursor_plane.base, &ast_crtc_funcs,
NULL);
if (ret)
return ret;
- drm_mode_crtc_set_gamma_size(crtc, 256);
+ drm_mode_crtc_set_gamma_size(crtc, AST_LUT_SIZE);
+ drm_crtc_enable_color_mgmt(crtc, 0, false, AST_LUT_SIZE);
+
drm_crtc_helper_add(crtc, &ast_crtc_helper_funcs);
return 0;
@@ -1718,13 +1720,46 @@ static int ast_astdp_output_init(struct ast_private *ast)
* Mode config
*/
+static void ast_mode_config_helper_atomic_commit_tail(struct drm_atomic_state *state)
+{
+ struct ast_private *ast = to_ast_private(state->dev);
+
+ /*
+ * Concurrent operations could possibly trigger a call to
+ * drm_connector_helper_funcs.get_modes by trying to read the
+ * display modes. Protect access to I/O registers by acquiring
+ * the I/O-register lock. Released in atomic_flush().
+ */
+ mutex_lock(&ast->ioregs_lock);
+ drm_atomic_helper_commit_tail_rpm(state);
+ mutex_unlock(&ast->ioregs_lock);
+}
+
static const struct drm_mode_config_helper_funcs ast_mode_config_helper_funcs = {
- .atomic_commit_tail = drm_atomic_helper_commit_tail_rpm,
+ .atomic_commit_tail = ast_mode_config_helper_atomic_commit_tail,
};
+static enum drm_mode_status ast_mode_config_mode_valid(struct drm_device *dev,
+ const struct drm_display_mode *mode)
+{
+ static const unsigned long max_bpp = 4; /* DRM_FORMAT_XRGB8888 */
+ struct ast_private *ast = to_ast_private(dev);
+ unsigned long fbsize, fbpages, max_fbpages;
+
+ max_fbpages = (ast->vram_fb_available) >> PAGE_SHIFT;
+
+ fbsize = mode->hdisplay * mode->vdisplay * max_bpp;
+ fbpages = DIV_ROUND_UP(fbsize, PAGE_SIZE);
+
+ if (fbpages > max_fbpages)
+ return MODE_MEM;
+
+ return MODE_OK;
+}
+
static const struct drm_mode_config_funcs ast_mode_config_funcs = {
- .fb_create = drm_gem_fb_create,
- .mode_valid = drm_vram_helper_mode_valid,
+ .fb_create = drm_gem_fb_create_with_dirty,
+ .mode_valid = ast_mode_config_mode_valid,
.atomic_check = drm_atomic_helper_check,
.atomic_commit = drm_atomic_helper_commit,
};
@@ -1732,7 +1767,6 @@ static const struct drm_mode_config_funcs ast_mode_config_funcs = {
int ast_mode_config_init(struct ast_private *ast)
{
struct drm_device *dev = &ast->base;
- struct pci_dev *pdev = to_pci_dev(dev->dev);
int ret;
ret = drmm_mode_config_init(dev);
@@ -1743,8 +1777,6 @@ int ast_mode_config_init(struct ast_private *ast)
dev->mode_config.min_width = 0;
dev->mode_config.min_height = 0;
dev->mode_config.preferred_depth = 24;
- dev->mode_config.prefer_shadow = 1;
- dev->mode_config.fb_base = pci_resource_start(pdev, 0);
if (ast->chip == AST2100 ||
ast->chip == AST2200 ||
@@ -1761,7 +1793,6 @@ int ast_mode_config_init(struct ast_private *ast)
dev->mode_config.helper_private = &ast_mode_config_helper_funcs;
-
ret = ast_primary_plane_init(ast);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c
index f7e7f4e..a2bb5b9 100644
--- a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c
+++ b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c
@@ -19,7 +19,7 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/bridge/adv7511/adv7511.h b/drivers/gpu/drm/bridge/adv7511/adv7511.h
index 94de73c..1744580 100644
--- a/drivers/gpu/drm/bridge/adv7511/adv7511.h
+++ b/drivers/gpu/drm/bridge/adv7511/adv7511.h
@@ -402,7 +402,8 @@ static inline int adv7511_cec_init(struct device *dev, struct adv7511 *adv7511)
void adv7533_dsi_power_on(struct adv7511 *adv);
void adv7533_dsi_power_off(struct adv7511 *adv);
-void adv7533_mode_set(struct adv7511 *adv, const struct drm_display_mode *mode);
+enum drm_mode_status adv7533_mode_valid(struct adv7511 *adv,
+ const struct drm_display_mode *mode);
int adv7533_patch_registers(struct adv7511 *adv);
int adv7533_patch_cec_registers(struct adv7511 *adv);
int adv7533_attach_dsi(struct adv7511 *adv);
diff --git a/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c b/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c
index f887200..e7a6e456 100644
--- a/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c
+++ b/drivers/gpu/drm/bridge/adv7511/adv7511_drv.c
@@ -697,7 +697,7 @@ adv7511_detect(struct adv7511 *adv7511, struct drm_connector *connector)
}
static enum drm_mode_status adv7511_mode_valid(struct adv7511 *adv7511,
- struct drm_display_mode *mode)
+ const struct drm_display_mode *mode)
{
if (mode->clock > 165000)
return MODE_CLOCK_HIGH;
@@ -791,9 +791,6 @@ static void adv7511_mode_set(struct adv7511 *adv7511,
regmap_update_bits(adv7511->regmap, 0x17,
0x60, (vsync_polarity << 6) | (hsync_polarity << 5));
- if (adv7511->type == ADV7533 || adv7511->type == ADV7535)
- adv7533_mode_set(adv7511, adj_mode);
-
drm_mode_copy(&adv7511->curr_mode, adj_mode);
/*
@@ -913,6 +910,18 @@ static void adv7511_bridge_mode_set(struct drm_bridge *bridge,
adv7511_mode_set(adv, mode, adj_mode);
}
+static enum drm_mode_status adv7511_bridge_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *info,
+ const struct drm_display_mode *mode)
+{
+ struct adv7511 *adv = bridge_to_adv7511(bridge);
+
+ if (adv->type == ADV7533 || adv->type == ADV7535)
+ return adv7533_mode_valid(adv, mode);
+ else
+ return adv7511_mode_valid(adv, mode);
+}
+
static int adv7511_bridge_attach(struct drm_bridge *bridge,
enum drm_bridge_attach_flags flags)
{
@@ -960,6 +969,7 @@ static const struct drm_bridge_funcs adv7511_bridge_funcs = {
.enable = adv7511_bridge_enable,
.disable = adv7511_bridge_disable,
.mode_set = adv7511_bridge_mode_set,
+ .mode_valid = adv7511_bridge_mode_valid,
.attach = adv7511_bridge_attach,
.detect = adv7511_bridge_detect,
.get_edid = adv7511_bridge_get_edid,
@@ -1209,10 +1219,8 @@ static int adv7511_probe(struct i2c_client *i2c, const struct i2c_device_id *id)
return ret;
ret = adv7511_init_regulators(adv7511);
- if (ret) {
- dev_err(dev, "failed to init regulators\n");
- return ret;
- }
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to init regulators\n");
/*
* The power down GPIO is optional. If present, toggle it from active to
diff --git a/drivers/gpu/drm/bridge/adv7511/adv7533.c b/drivers/gpu/drm/bridge/adv7511/adv7533.c
index ef62708..fdfeadc 100644
--- a/drivers/gpu/drm/bridge/adv7511/adv7533.c
+++ b/drivers/gpu/drm/bridge/adv7511/adv7533.c
@@ -100,26 +100,27 @@ void adv7533_dsi_power_off(struct adv7511 *adv)
regmap_write(adv->regmap_cec, 0x27, 0x0b);
}
-void adv7533_mode_set(struct adv7511 *adv, const struct drm_display_mode *mode)
+enum drm_mode_status adv7533_mode_valid(struct adv7511 *adv,
+ const struct drm_display_mode *mode)
{
+ int lanes;
struct mipi_dsi_device *dsi = adv->dsi;
- int lanes, ret;
-
- if (adv->num_dsi_lanes != 4)
- return;
if (mode->clock > 80000)
lanes = 4;
else
lanes = 3;
- if (lanes != dsi->lanes) {
- mipi_dsi_detach(dsi);
- dsi->lanes = lanes;
- ret = mipi_dsi_attach(dsi);
- if (ret)
- dev_err(&dsi->dev, "failed to change host lanes\n");
- }
+ /*
+ * TODO: add support for dynamic switching of lanes
+ * by using the bridge pre_enable() op . Till then filter
+ * out the modes which shall need different number of lanes
+ * than what was configured in the device tree.
+ */
+ if (lanes != dsi->lanes)
+ return MODE_BAD;
+
+ return MODE_OK;
}
int adv7533_patch_registers(struct adv7511 *adv)
@@ -148,16 +149,14 @@ int adv7533_attach_dsi(struct adv7511 *adv)
};
host = of_find_mipi_dsi_host_by_node(adv->host_node);
- if (!host) {
- dev_err(dev, "failed to find dsi host\n");
- return -EPROBE_DEFER;
- }
+ if (!host)
+ return dev_err_probe(dev, -EPROBE_DEFER,
+ "failed to find dsi host\n");
dsi = devm_mipi_dsi_device_register_full(dev, host, &info);
- if (IS_ERR(dsi)) {
- dev_err(dev, "failed to create dsi device\n");
- return PTR_ERR(dsi);
- }
+ if (IS_ERR(dsi))
+ return dev_err_probe(dev, PTR_ERR(dsi),
+ "failed to create dsi device\n");
adv->dsi = dsi;
@@ -167,10 +166,8 @@ int adv7533_attach_dsi(struct adv7511 *adv)
MIPI_DSI_MODE_NO_EOT_PACKET | MIPI_DSI_MODE_VIDEO_HSE;
ret = devm_mipi_dsi_attach(dev, dsi);
- if (ret < 0) {
- dev_err(dev, "failed to attach dsi to host\n");
- return ret;
- }
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "failed to attach dsi to host\n");
return 0;
}
diff --git a/drivers/gpu/drm/bridge/ite-it6505.c b/drivers/gpu/drm/bridge/ite-it6505.c
index dfe4351..21a9b84 100644
--- a/drivers/gpu/drm/bridge/ite-it6505.c
+++ b/drivers/gpu/drm/bridge/ite-it6505.c
@@ -412,6 +412,7 @@ struct it6505 {
* Mutex protects extcon and interrupt functions from interfering
* each other.
*/
+ struct mutex irq_lock;
struct mutex extcon_lock;
struct mutex mode_lock; /* used to bridge_detect */
struct mutex aux_lock; /* used to aux data transfers */
@@ -421,6 +422,7 @@ struct it6505 {
struct notifier_block event_nb;
struct extcon_dev *extcon;
struct work_struct extcon_wq;
+ int extcon_state;
enum drm_connector_status connector_status;
enum link_train_status link_state;
struct work_struct link_works;
@@ -439,7 +441,7 @@ struct it6505 {
enum hdcp_state hdcp_status;
struct delayed_work hdcp_work;
struct work_struct hdcp_wait_ksv_list;
- struct completion wait_edid_complete;
+ struct completion extcon_completion;
u8 auto_train_retry;
bool hdcp_desired;
bool is_repeater;
@@ -724,28 +726,6 @@ static void it6505_calc_video_info(struct it6505 *it6505)
DRM_MODE_ARG(&it6505->video_info));
}
-static int it6505_drm_dp_link_probe(struct drm_dp_aux *aux,
- struct it6505_drm_dp_link *link)
-{
- u8 values[3];
- int err;
-
- memset(link, 0, sizeof(*link));
-
- err = drm_dp_dpcd_read(aux, DP_DPCD_REV, values, sizeof(values));
- if (err < 0)
- return err;
-
- link->revision = values[0];
- link->rate = drm_dp_bw_code_to_link_rate(values[1]);
- link->num_lanes = values[2] & DP_MAX_LANE_COUNT_MASK;
-
- if (values[2] & DP_ENHANCED_FRAME_CAP)
- link->capabilities = DP_ENHANCED_FRAME_CAP;
-
- return 0;
-}
-
static int it6505_drm_dp_link_set_power(struct drm_dp_aux *aux,
struct it6505_drm_dp_link *link,
u8 mode)
@@ -1455,11 +1435,19 @@ static void it6505_parse_link_capabilities(struct it6505 *it6505)
int bcaps;
if (it6505->dpcd[0] == 0) {
- it6505_aux_on(it6505);
- it6505_get_dpcd(it6505, DP_DPCD_REV, it6505->dpcd,
- ARRAY_SIZE(it6505->dpcd));
+ dev_err(dev, "DPCD is not initialized");
+ return;
}
+ memset(link, 0, sizeof(*link));
+
+ link->revision = it6505->dpcd[0];
+ link->rate = drm_dp_bw_code_to_link_rate(it6505->dpcd[1]);
+ link->num_lanes = it6505->dpcd[2] & DP_MAX_LANE_COUNT_MASK;
+
+ if (it6505->dpcd[2] & DP_ENHANCED_FRAME_CAP)
+ link->capabilities = DP_ENHANCED_FRAME_CAP;
+
DRM_DEV_DEBUG_DRIVER(dev, "DPCD Rev.: %d.%d",
link->revision >> 4, link->revision & 0x0F);
@@ -2322,19 +2310,32 @@ static int it6505_process_hpd_irq(struct it6505 *it6505)
static void it6505_irq_hpd(struct it6505 *it6505)
{
struct device *dev = &it6505->client->dev;
+ int dp_sink_count;
it6505->hpd_state = it6505_get_sink_hpd_status(it6505);
DRM_DEV_DEBUG_DRIVER(dev, "hpd change interrupt, change to %s",
it6505->hpd_state ? "high" : "low");
- if (it6505->bridge.dev)
- drm_helper_hpd_irq_event(it6505->bridge.dev);
- DRM_DEV_DEBUG_DRIVER(dev, "it6505->sink_count: %d",
- it6505->sink_count);
-
if (it6505->hpd_state) {
- wait_for_completion_timeout(&it6505->wait_edid_complete,
- msecs_to_jiffies(6000));
+ wait_for_completion_timeout(&it6505->extcon_completion,
+ msecs_to_jiffies(1000));
+ it6505_aux_on(it6505);
+ if (it6505->dpcd[0] == 0) {
+ it6505_get_dpcd(it6505, DP_DPCD_REV, it6505->dpcd,
+ ARRAY_SIZE(it6505->dpcd));
+ it6505_variable_config(it6505);
+ it6505_parse_link_capabilities(it6505);
+ }
+ it6505->auto_train_retry = AUTO_TRAIN_RETRY;
+
+ it6505_drm_dp_link_set_power(&it6505->aux, &it6505->link,
+ DP_SET_POWER_D0);
+ dp_sink_count = it6505_dpcd_read(it6505, DP_SINK_COUNT);
+ it6505->sink_count = DP_GET_SINK_COUNT(dp_sink_count);
+
+ DRM_DEV_DEBUG_DRIVER(dev, "it6505->sink_count: %d",
+ it6505->sink_count);
+
it6505_lane_termination_on(it6505);
it6505_lane_power_on(it6505);
@@ -2362,6 +2363,9 @@ static void it6505_irq_hpd(struct it6505 *it6505)
it6505_lane_off(it6505);
it6505_link_reset_step_train(it6505);
}
+
+ if (it6505->bridge.dev)
+ drm_helper_hpd_irq_event(it6505->bridge.dev);
}
static void it6505_irq_hpd_irq(struct it6505 *it6505)
@@ -2490,8 +2494,7 @@ static irqreturn_t it6505_int_threaded_handler(int unused, void *data)
};
int int_status[3], i;
- msleep(100);
- mutex_lock(&it6505->extcon_lock);
+ mutex_lock(&it6505->irq_lock);
if (it6505->enable_drv_hold || !it6505->powered)
goto unlock;
@@ -2521,7 +2524,7 @@ static irqreturn_t it6505_int_threaded_handler(int unused, void *data)
}
unlock:
- mutex_unlock(&it6505->extcon_lock);
+ mutex_unlock(&it6505->irq_lock);
return IRQ_HANDLED;
}
@@ -2624,26 +2627,14 @@ static enum drm_connector_status it6505_detect(struct it6505 *it6505)
goto unlock;
if (it6505->enable_drv_hold) {
- status = it6505_get_sink_hpd_status(it6505) ?
- connector_status_connected :
- connector_status_disconnected;
+ status = it6505->hpd_state ? connector_status_connected :
+ connector_status_disconnected;
goto unlock;
}
- if (it6505_get_sink_hpd_status(it6505)) {
- it6505_aux_on(it6505);
- it6505_drm_dp_link_probe(&it6505->aux, &it6505->link);
+ if (it6505->hpd_state) {
it6505_drm_dp_link_set_power(&it6505->aux, &it6505->link,
DP_SET_POWER_D0);
- it6505->auto_train_retry = AUTO_TRAIN_RETRY;
-
- if (it6505->dpcd[0] == 0) {
- it6505_get_dpcd(it6505, DP_DPCD_REV, it6505->dpcd,
- ARRAY_SIZE(it6505->dpcd));
- it6505_variable_config(it6505);
- it6505_parse_link_capabilities(it6505);
- }
-
dp_sink_count = it6505_dpcd_read(it6505, DP_SINK_COUNT);
it6505->sink_count = DP_GET_SINK_COUNT(dp_sink_count);
DRM_DEV_DEBUG_DRIVER(dev, "it6505->sink_count:%d branch:%d",
@@ -2685,31 +2676,44 @@ static void it6505_extcon_work(struct work_struct *work)
{
struct it6505 *it6505 = container_of(work, struct it6505, extcon_wq);
struct device *dev = &it6505->client->dev;
- int state = extcon_get_state(it6505->extcon, EXTCON_DISP_DP);
- unsigned int pwroffretry = 0;
+ int state, ret;
if (it6505->enable_drv_hold)
return;
mutex_lock(&it6505->extcon_lock);
+ state = extcon_get_state(it6505->extcon, EXTCON_DISP_DP);
DRM_DEV_DEBUG_DRIVER(dev, "EXTCON_DISP_DP = 0x%02x", state);
- if (state > 0) {
+
+ if (state == it6505->extcon_state || unlikely(state < 0))
+ goto unlock;
+ it6505->extcon_state = state;
+ if (state) {
DRM_DEV_DEBUG_DRIVER(dev, "start to power on");
msleep(100);
- it6505_poweron(it6505);
+ ret = pm_runtime_get_sync(dev);
+
+ /*
+ * On system resume, extcon_work can be triggered before
+ * pm_runtime_force_resume re-enables runtime power management.
+ * Handling the error here to make sure the bridge is powered on.
+ */
+ if (ret < 0)
+ it6505_poweron(it6505);
+
+ complete_all(&it6505->extcon_completion);
} else {
DRM_DEV_DEBUG_DRIVER(dev, "start to power off");
- while (it6505_poweroff(it6505) && pwroffretry++ < 5) {
- DRM_DEV_DEBUG_DRIVER(dev, "power off fail %d times",
- pwroffretry);
- }
+ pm_runtime_put_sync(dev);
+ reinit_completion(&it6505->extcon_completion);
drm_helper_hpd_irq_event(it6505->bridge.dev);
memset(it6505->dpcd, 0, sizeof(it6505->dpcd));
DRM_DEV_DEBUG_DRIVER(dev, "power off it6505 success!");
}
+unlock:
mutex_unlock(&it6505->extcon_lock);
}
@@ -2860,10 +2864,7 @@ static int it6505_bridge_attach(struct drm_bridge *bridge,
}
/* Register aux channel */
- it6505->aux.name = "DP-AUX";
- it6505->aux.dev = dev;
it6505->aux.drm_dev = bridge->dev;
- it6505->aux.transfer = it6505_aux_transfer;
ret = drm_dp_aux_register(&it6505->aux);
@@ -2980,6 +2981,28 @@ static void it6505_bridge_atomic_disable(struct drm_bridge *bridge,
}
}
+static void it6505_bridge_atomic_pre_enable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_state)
+{
+ struct it6505 *it6505 = bridge_to_it6505(bridge);
+ struct device *dev = &it6505->client->dev;
+
+ DRM_DEV_DEBUG_DRIVER(dev, "start");
+
+ pm_runtime_get_sync(dev);
+}
+
+static void it6505_bridge_atomic_post_disable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_state)
+{
+ struct it6505 *it6505 = bridge_to_it6505(bridge);
+ struct device *dev = &it6505->client->dev;
+
+ DRM_DEV_DEBUG_DRIVER(dev, "start");
+
+ pm_runtime_put_sync(dev);
+}
+
static enum drm_connector_status
it6505_bridge_detect(struct drm_bridge *bridge)
{
@@ -3014,6 +3037,8 @@ static const struct drm_bridge_funcs it6505_bridge_funcs = {
.mode_valid = it6505_bridge_mode_valid,
.atomic_enable = it6505_bridge_atomic_enable,
.atomic_disable = it6505_bridge_atomic_disable,
+ .atomic_pre_enable = it6505_bridge_atomic_pre_enable,
+ .atomic_post_disable = it6505_bridge_atomic_post_disable,
.detect = it6505_bridge_detect,
.get_edid = it6505_bridge_get_edid,
};
@@ -3032,8 +3057,10 @@ static __maybe_unused int it6505_bridge_suspend(struct device *dev)
return it6505_poweroff(it6505);
}
-static SIMPLE_DEV_PM_OPS(it6505_bridge_pm_ops, it6505_bridge_suspend,
- it6505_bridge_resume);
+static const struct dev_pm_ops it6505_bridge_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(it6505_bridge_suspend, it6505_bridge_resume, NULL)
+};
static int it6505_init_pdata(struct it6505 *it6505)
{
@@ -3250,6 +3277,7 @@ static int it6505_i2c_probe(struct i2c_client *client,
if (!it6505)
return -ENOMEM;
+ mutex_init(&it6505->irq_lock);
mutex_init(&it6505->extcon_lock);
mutex_init(&it6505->mode_lock);
mutex_init(&it6505->aux_lock);
@@ -3305,7 +3333,7 @@ static int it6505_i2c_probe(struct i2c_client *client,
INIT_WORK(&it6505->link_works, it6505_link_training_work);
INIT_WORK(&it6505->hdcp_wait_ksv_list, it6505_hdcp_wait_ksv_list);
INIT_DELAYED_WORK(&it6505->hdcp_work, it6505_hdcp_work);
- init_completion(&it6505->wait_edid_complete);
+ init_completion(&it6505->extcon_completion);
memset(it6505->dpcd, 0, sizeof(it6505->dpcd));
it6505->powered = false;
it6505->enable_drv_hold = DEFAULT_DRV_HOLD;
@@ -3315,6 +3343,12 @@ static int it6505_i2c_probe(struct i2c_client *client,
DRM_DEV_DEBUG_DRIVER(dev, "it6505 device name: %s", dev_name(dev));
debugfs_init(it6505);
+ pm_runtime_enable(dev);
+
+ it6505->aux.name = "DP-AUX";
+ it6505->aux.dev = dev;
+ it6505->aux.transfer = it6505_aux_transfer;
+ drm_dp_aux_init(&it6505->aux);
it6505->bridge.funcs = &it6505_bridge_funcs;
it6505->bridge.type = DRM_MODE_CONNECTOR_DisplayPort;
diff --git a/drivers/gpu/drm/bridge/parade-ps8640.c b/drivers/gpu/drm/bridge/parade-ps8640.c
index 083337a..6a614e5 100644
--- a/drivers/gpu/drm/bridge/parade-ps8640.c
+++ b/drivers/gpu/drm/bridge/parade-ps8640.c
@@ -304,7 +304,6 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux,
}
switch (data & SWAUX_STATUS_MASK) {
- /* Ignore the DEFER cases as they are already handled in hardware */
case SWAUX_STATUS_NACK:
case SWAUX_STATUS_I2C_NACK:
/*
@@ -321,6 +320,14 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux,
case SWAUX_STATUS_ACKM:
len = data & SWAUX_M_MASK;
break;
+ case SWAUX_STATUS_DEFER:
+ case SWAUX_STATUS_I2C_DEFER:
+ if (is_native_aux)
+ msg->reply |= DP_AUX_NATIVE_REPLY_DEFER;
+ else
+ msg->reply |= DP_AUX_I2C_REPLY_DEFER;
+ len = data & SWAUX_M_MASK;
+ break;
case SWAUX_STATUS_INVALID:
return -EOPNOTSUPP;
case SWAUX_STATUS_TIMEOUT:
diff --git a/drivers/gpu/drm/bridge/tc358762.c b/drivers/gpu/drm/bridge/tc358762.c
index 7f4fce1a..0b6a284 100644
--- a/drivers/gpu/drm/bridge/tc358762.c
+++ b/drivers/gpu/drm/bridge/tc358762.c
@@ -11,6 +11,7 @@
*/
#include <linux/delay.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/of_graph.h>
#include <linux/regulator/consumer.h>
@@ -19,7 +20,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_mipi_dsi.h>
#include <drm/drm_of.h>
#include <drm/drm_panel.h>
diff --git a/drivers/gpu/drm/bridge/tc358775.c b/drivers/gpu/drm/bridge/tc358775.c
index 02dc12b..3ceb0e9 100644
--- a/drivers/gpu/drm/bridge/tc358775.c
+++ b/drivers/gpu/drm/bridge/tc358775.c
@@ -408,7 +408,7 @@ static void tc_bridge_enable(struct drm_bridge *bridge)
(val >> 8) & 0xFF, val & 0xFF);
d2l_write(tc->i2c, SYSRST, SYS_RST_REG | SYS_RST_DSIRX | SYS_RST_BM |
- SYS_RST_LCD | SYS_RST_I2CM | SYS_RST_I2CS);
+ SYS_RST_LCD | SYS_RST_I2CM);
usleep_range(30000, 40000);
d2l_write(tc->i2c, PPI_TX_RX_TA, TTA_GET | TTA_SURE);
diff --git a/drivers/gpu/drm/display/Makefile b/drivers/gpu/drm/display/Makefile
index 52cdda1..17ac4a1 100644
--- a/drivers/gpu/drm/display/Makefile
+++ b/drivers/gpu/drm/display/Makefile
@@ -3,13 +3,15 @@
obj-$(CONFIG_DRM_DP_AUX_BUS) += drm_dp_aux_bus.o
drm_display_helper-y := drm_display_helper_mod.o
-drm_display_helper-$(CONFIG_DRM_DISPLAY_DP_HELPER) += drm_dp_dual_mode_helper.o \
- drm_dp_helper.o \
- drm_dp_mst_topology.o \
- drm_dsc_helper.o
+drm_display_helper-$(CONFIG_DRM_DISPLAY_DP_HELPER) += \
+ drm_dp_dual_mode_helper.o \
+ drm_dp_helper.o \
+ drm_dp_mst_topology.o \
+ drm_dsc_helper.o
drm_display_helper-$(CONFIG_DRM_DISPLAY_HDCP_HELPER) += drm_hdcp_helper.o
-drm_display_helper-$(CONFIG_DRM_DISPLAY_HDMI_HELPER) += drm_hdmi_helper.o \
- drm_scdc_helper.o
+drm_display_helper-$(CONFIG_DRM_DISPLAY_HDMI_HELPER) += \
+ drm_hdmi_helper.o \
+ drm_scdc_helper.o
drm_display_helper-$(CONFIG_DRM_DP_AUX_CHARDEV) += drm_dp_aux_dev.o
drm_display_helper-$(CONFIG_DRM_DP_CEC) += drm_dp_cec.o
diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
index 98cc313..d579fd8f 100644
--- a/drivers/gpu/drm/drm_atomic_helper.c
+++ b/drivers/gpu/drm/drm_atomic_helper.c
@@ -924,59 +924,35 @@ int drm_atomic_helper_check_plane_state(struct drm_plane_state *plane_state,
EXPORT_SYMBOL(drm_atomic_helper_check_plane_state);
/**
- * drm_atomic_helper_check_crtc_state() - Check CRTC state for validity
+ * drm_atomic_helper_check_crtc_primary_plane() - Check CRTC state for primary plane
* @crtc_state: CRTC state to check
- * @can_disable_primary_planes: can the CRTC be enabled without a primary plane?
*
- * Checks that a desired CRTC update is valid. Drivers that provide
- * their own CRTC handling rather than helper-provided implementations may
- * still wish to call this function to avoid duplication of error checking
- * code.
- *
- * Note that @can_disable_primary_planes only tests if the CRTC can be
- * enabled without a primary plane. To test if a primary plane can be updated
- * without a CRTC, use drm_atomic_helper_check_plane_state() in the plane's
- * atomic check.
+ * Checks that a CRTC has at least one primary plane attached to it, which is
+ * a requirement on some hardware. Note that this only involves the CRTC side
+ * of the test. To test if the primary plane is visible or if it can be updated
+ * without the CRTC being enabled, use drm_atomic_helper_check_plane_state() in
+ * the plane's atomic check.
*
* RETURNS:
- * Zero if update appears valid, error code on failure
+ * 0 if a primary plane is attached to the CRTC, or an error code otherwise
*/
-int drm_atomic_helper_check_crtc_state(struct drm_crtc_state *crtc_state,
- bool can_disable_primary_planes)
+int drm_atomic_helper_check_crtc_primary_plane(struct drm_crtc_state *crtc_state)
{
- struct drm_device *dev = crtc_state->crtc->dev;
- struct drm_atomic_state *state = crtc_state->state;
-
- if (!crtc_state->enable)
- return 0;
+ struct drm_crtc *crtc = crtc_state->crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_plane *plane;
/* needs at least one primary plane to be enabled */
- if (!can_disable_primary_planes) {
- bool has_primary_plane = false;
- struct drm_plane *plane;
-
- drm_for_each_plane_mask(plane, dev, crtc_state->plane_mask) {
- struct drm_plane_state *plane_state;
-
- if (plane->type != DRM_PLANE_TYPE_PRIMARY)
- continue;
- plane_state = drm_atomic_get_plane_state(state, plane);
- if (IS_ERR(plane_state))
- return PTR_ERR(plane_state);
- if (plane_state->fb && plane_state->crtc) {
- has_primary_plane = true;
- break;
- }
- }
- if (!has_primary_plane) {
- drm_dbg_kms(dev, "Cannot enable CRTC without a primary plane.\n");
- return -EINVAL;
- }
+ drm_for_each_plane_mask(plane, dev, crtc_state->plane_mask) {
+ if (plane->type == DRM_PLANE_TYPE_PRIMARY)
+ return 0;
}
- return 0;
+ drm_dbg_atomic(dev, "[CRTC:%d:%s] primary plane missing\n", crtc->base.id, crtc->name);
+
+ return -EINVAL;
}
-EXPORT_SYMBOL(drm_atomic_helper_check_crtc_state);
+EXPORT_SYMBOL(drm_atomic_helper_check_crtc_primary_plane);
/**
* drm_atomic_helper_check_planes - validate state object for planes changes
@@ -2560,7 +2536,7 @@ int drm_atomic_helper_prepare_planes(struct drm_device *dev,
if (funcs->prepare_fb) {
ret = funcs->prepare_fb(plane, new_plane_state);
if (ret)
- goto fail;
+ goto fail_prepare_fb;
} else {
WARN_ON_ONCE(funcs->cleanup_fb);
@@ -2569,13 +2545,34 @@ int drm_atomic_helper_prepare_planes(struct drm_device *dev,
ret = drm_gem_plane_helper_prepare_fb(plane, new_plane_state);
if (ret)
- goto fail;
+ goto fail_prepare_fb;
+ }
+ }
+
+ for_each_new_plane_in_state(state, plane, new_plane_state, i) {
+ const struct drm_plane_helper_funcs *funcs = plane->helper_private;
+
+ if (funcs->begin_fb_access) {
+ ret = funcs->begin_fb_access(plane, new_plane_state);
+ if (ret)
+ goto fail_begin_fb_access;
}
}
return 0;
-fail:
+fail_begin_fb_access:
+ for_each_new_plane_in_state(state, plane, new_plane_state, j) {
+ const struct drm_plane_helper_funcs *funcs = plane->helper_private;
+
+ if (j >= i)
+ continue;
+
+ if (funcs->end_fb_access)
+ funcs->end_fb_access(plane, new_plane_state);
+ }
+ i = j; /* set i to upper limit to cleanup all planes */
+fail_prepare_fb:
for_each_new_plane_in_state(state, plane, new_plane_state, j) {
const struct drm_plane_helper_funcs *funcs;
@@ -2852,6 +2849,13 @@ void drm_atomic_helper_cleanup_planes(struct drm_device *dev,
int i;
for_each_oldnew_plane_in_state(old_state, plane, old_plane_state, new_plane_state, i) {
+ const struct drm_plane_helper_funcs *funcs = plane->helper_private;
+
+ if (funcs->end_fb_access)
+ funcs->end_fb_access(plane, new_plane_state);
+ }
+
+ for_each_oldnew_plane_in_state(old_state, plane, old_plane_state, new_plane_state, i) {
const struct drm_plane_helper_funcs *funcs;
struct drm_plane_state *plane_state;
diff --git a/drivers/gpu/drm/drm_atomic_state_helper.c b/drivers/gpu/drm/drm_atomic_state_helper.c
index bf31b9d..dfb5721 100644
--- a/drivers/gpu/drm/drm_atomic_state_helper.c
+++ b/drivers/gpu/drm/drm_atomic_state_helper.c
@@ -464,12 +464,12 @@ void drm_atomic_helper_connector_reset(struct drm_connector *connector)
EXPORT_SYMBOL(drm_atomic_helper_connector_reset);
/**
- * drm_atomic_helper_connector_tv_reset - Resets TV connector properties
+ * drm_atomic_helper_connector_tv_margins_reset - Resets TV connector properties
* @connector: DRM connector
*
* Resets the TV-related properties attached to a connector.
*/
-void drm_atomic_helper_connector_tv_reset(struct drm_connector *connector)
+void drm_atomic_helper_connector_tv_margins_reset(struct drm_connector *connector)
{
struct drm_cmdline_mode *cmdline = &connector->cmdline_mode;
struct drm_connector_state *state = connector->state;
@@ -479,7 +479,7 @@ void drm_atomic_helper_connector_tv_reset(struct drm_connector *connector)
state->tv.margins.top = cmdline->tv_margins.top;
state->tv.margins.bottom = cmdline->tv_margins.bottom;
}
-EXPORT_SYMBOL(drm_atomic_helper_connector_tv_reset);
+EXPORT_SYMBOL(drm_atomic_helper_connector_tv_margins_reset);
/**
* __drm_atomic_helper_connector_duplicate_state - copy atomic connector state
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index 79730fa..c06d063 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -687,6 +687,8 @@ static int drm_atomic_connector_set_property(struct drm_connector *connector,
*/
return -EINVAL;
} else if (property == config->tv_select_subconnector_property) {
+ state->tv.select_subconnector = val;
+ } else if (property == config->tv_subconnector_property) {
state->tv.subconnector = val;
} else if (property == config->tv_left_margin_property) {
state->tv.margins.left = val;
@@ -795,6 +797,8 @@ drm_atomic_connector_get_property(struct drm_connector *connector,
else
*val = connector->dpms;
} else if (property == config->tv_select_subconnector_property) {
+ *val = state->tv.select_subconnector;
+ } else if (property == config->tv_subconnector_property) {
*val = state->tv.subconnector;
} else if (property == config->tv_left_margin_property) {
*val = state->tv.margins.left;
diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
index 2b230b4..fd67efe 100644
--- a/drivers/gpu/drm/drm_client.c
+++ b/drivers/gpu/drm/drm_client.c
@@ -235,10 +235,10 @@ static void drm_client_buffer_delete(struct drm_client_buffer *buffer)
{
struct drm_device *dev = buffer->client->dev;
- drm_gem_vunmap(buffer->gem, &buffer->map);
-
- if (buffer->gem)
+ if (buffer->gem) {
+ drm_gem_vunmap_unlocked(buffer->gem, &buffer->map);
drm_gem_object_put(buffer->gem);
+ }
if (buffer->handle)
drm_mode_destroy_dumb(dev, buffer->handle, buffer->client->file);
@@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
* fd_install step out of the driver backend hooks, to make that
* final step optional for internal users.
*/
- ret = drm_gem_vmap(buffer->gem, map);
+ ret = drm_gem_vmap_unlocked(buffer->gem, map);
if (ret)
return ret;
@@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
{
struct iosys_map *map = &buffer->map;
- drm_gem_vunmap(buffer->gem, map);
+ drm_gem_vunmap_unlocked(buffer->gem, map);
}
EXPORT_SYMBOL(drm_client_buffer_vunmap);
diff --git a/drivers/gpu/drm/drm_client_modeset.c b/drivers/gpu/drm/drm_client_modeset.c
index bbc535c..d553e79 100644
--- a/drivers/gpu/drm/drm_client_modeset.c
+++ b/drivers/gpu/drm/drm_client_modeset.c
@@ -1237,3 +1237,7 @@ int drm_client_modeset_dpms(struct drm_client_dev *client, int mode)
return ret;
}
EXPORT_SYMBOL(drm_client_modeset_dpms);
+
+#ifdef CONFIG_DRM_KUNIT_TEST
+#include "tests/drm_client_modeset_test.c"
+#endif
diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
index 61c29ce..547356e 100644
--- a/drivers/gpu/drm/drm_connector.c
+++ b/drivers/gpu/drm/drm_connector.c
@@ -274,6 +274,7 @@ static int __drm_connector_init(struct drm_device *dev,
INIT_LIST_HEAD(&connector->probed_modes);
INIT_LIST_HEAD(&connector->modes);
mutex_init(&connector->mutex);
+ mutex_init(&connector->edid_override_mutex);
connector->edid_blob_ptr = NULL;
connector->epoch_counter = 0;
connector->tile_blob_ptr = NULL;
@@ -582,6 +583,9 @@ void drm_connector_cleanup(struct drm_connector *connector)
mutex_destroy(&connector->mutex);
memset(connector, 0, sizeof(*connector));
+
+ if (dev->registered)
+ drm_sysfs_hotplug_event(dev);
}
EXPORT_SYMBOL(drm_connector_cleanup);
diff --git a/drivers/gpu/drm/drm_crtc_helper.c b/drivers/gpu/drm/drm_crtc_helper.c
index 7d86020..a209659 100644
--- a/drivers/gpu/drm/drm_crtc_helper.c
+++ b/drivers/gpu/drm/drm_crtc_helper.c
@@ -43,7 +43,6 @@
#include <drm/drm_drv.h>
#include <drm/drm_edid.h>
#include <drm/drm_encoder.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_print.h>
@@ -434,6 +433,32 @@ bool drm_crtc_helper_set_mode(struct drm_crtc *crtc,
}
EXPORT_SYMBOL(drm_crtc_helper_set_mode);
+/**
+ * drm_crtc_helper_atomic_check() - Helper to check CRTC atomic-state
+ * @crtc: CRTC to check
+ * @state: atomic state object
+ *
+ * Provides a default CRTC-state check handler for CRTCs that only have
+ * one primary plane attached to it.
+ *
+ * This is often the case for the CRTC of simple framebuffers. See also
+ * drm_plane_helper_atomic_check() for the respective plane-state check
+ * helper function.
+ *
+ * RETURNS:
+ * Zero on success, or an errno code otherwise.
+ */
+int drm_crtc_helper_atomic_check(struct drm_crtc *crtc, struct drm_atomic_state *state)
+{
+ struct drm_crtc_state *new_crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
+
+ if (!new_crtc_state->enable)
+ return 0;
+
+ return drm_atomic_helper_check_crtc_primary_plane(new_crtc_state);
+}
+EXPORT_SYMBOL(drm_crtc_helper_atomic_check);
+
static void
drm_crtc_helper_disable(struct drm_crtc *crtc)
{
diff --git a/drivers/gpu/drm/drm_crtc_internal.h b/drivers/gpu/drm/drm_crtc_internal.h
index 56041b60..501a10e 100644
--- a/drivers/gpu/drm/drm_crtc_internal.h
+++ b/drivers/gpu/drm/drm_crtc_internal.h
@@ -56,9 +56,10 @@ struct drm_plane;
struct drm_plane_state;
struct drm_property;
struct edid;
-struct kref;
-struct work_struct;
struct fwnode_handle;
+struct kref;
+struct seq_file;
+struct work_struct;
/* drm_crtc.c */
int drm_mode_crtc_set_obj_prop(struct drm_mode_object *obj,
@@ -286,5 +287,17 @@ int drm_mode_page_flip_ioctl(struct drm_device *dev,
/* drm_edid.c */
void drm_mode_fixup_1366x768(struct drm_display_mode *mode);
+int drm_edid_override_show(struct drm_connector *connector, struct seq_file *m);
int drm_edid_override_set(struct drm_connector *connector, const void *edid, size_t size);
int drm_edid_override_reset(struct drm_connector *connector);
+
+/* drm_edid_load.c */
+#ifdef CONFIG_DRM_LOAD_EDID_FIRMWARE
+const struct drm_edid *drm_edid_load_firmware(struct drm_connector *connector);
+#else
+static inline const struct drm_edid *
+drm_edid_load_firmware(struct drm_connector *connector)
+{
+ return ERR_PTR(-ENOENT);
+}
+#endif
diff --git a/drivers/gpu/drm/drm_debugfs.c b/drivers/gpu/drm/drm_debugfs.c
index 01ee3fe..ee445f4 100644
--- a/drivers/gpu/drm/drm_debugfs.c
+++ b/drivers/gpu/drm/drm_debugfs.c
@@ -328,13 +328,7 @@ static ssize_t connector_write(struct file *file, const char __user *ubuf,
static int edid_show(struct seq_file *m, void *data)
{
- struct drm_connector *connector = m->private;
- struct drm_property_blob *edid = connector->edid_blob_ptr;
-
- if (connector->override_edid && edid)
- seq_write(m, edid->data, edid->length);
-
- return 0;
+ return drm_edid_override_show(m->private, m);
}
static int edid_open(struct inode *inode, struct file *file)
diff --git a/drivers/gpu/drm/drm_drv.c b/drivers/gpu/drm/drm_drv.c
index 203bf8d..73b845a 100644
--- a/drivers/gpu/drm/drm_drv.c
+++ b/drivers/gpu/drm/drm_drv.c
@@ -35,6 +35,7 @@
#include <linux/slab.h>
#include <linux/srcu.h>
+#include <drm/drm_accel.h>
#include <drm/drm_cache.h>
#include <drm/drm_client.h>
#include <drm/drm_color_mgmt.h>
@@ -90,6 +91,8 @@ static struct drm_minor **drm_minor_get_slot(struct drm_device *dev,
return &dev->primary;
case DRM_MINOR_RENDER:
return &dev->render;
+ case DRM_MINOR_ACCEL:
+ return &dev->accel;
default:
BUG();
}
@@ -104,9 +107,13 @@ static void drm_minor_alloc_release(struct drm_device *dev, void *data)
put_device(minor->kdev);
- spin_lock_irqsave(&drm_minor_lock, flags);
- idr_remove(&drm_minors_idr, minor->index);
- spin_unlock_irqrestore(&drm_minor_lock, flags);
+ if (minor->type == DRM_MINOR_ACCEL) {
+ accel_minor_remove(minor->index);
+ } else {
+ spin_lock_irqsave(&drm_minor_lock, flags);
+ idr_remove(&drm_minors_idr, minor->index);
+ spin_unlock_irqrestore(&drm_minor_lock, flags);
+ }
}
static int drm_minor_alloc(struct drm_device *dev, unsigned int type)
@@ -123,13 +130,17 @@ static int drm_minor_alloc(struct drm_device *dev, unsigned int type)
minor->dev = dev;
idr_preload(GFP_KERNEL);
- spin_lock_irqsave(&drm_minor_lock, flags);
- r = idr_alloc(&drm_minors_idr,
- NULL,
- 64 * type,
- 64 * (type + 1),
- GFP_NOWAIT);
- spin_unlock_irqrestore(&drm_minor_lock, flags);
+ if (type == DRM_MINOR_ACCEL) {
+ r = accel_minor_alloc();
+ } else {
+ spin_lock_irqsave(&drm_minor_lock, flags);
+ r = idr_alloc(&drm_minors_idr,
+ NULL,
+ 64 * type,
+ 64 * (type + 1),
+ GFP_NOWAIT);
+ spin_unlock_irqrestore(&drm_minor_lock, flags);
+ }
idr_preload_end();
if (r < 0)
@@ -161,10 +172,14 @@ static int drm_minor_register(struct drm_device *dev, unsigned int type)
if (!minor)
return 0;
- ret = drm_debugfs_init(minor, minor->index, drm_debugfs_root);
- if (ret) {
- DRM_ERROR("DRM: Failed to initialize /sys/kernel/debug/dri.\n");
- goto err_debugfs;
+ if (minor->type == DRM_MINOR_ACCEL) {
+ accel_debugfs_init(minor, minor->index);
+ } else {
+ ret = drm_debugfs_init(minor, minor->index, drm_debugfs_root);
+ if (ret) {
+ DRM_ERROR("DRM: Failed to initialize /sys/kernel/debug/dri.\n");
+ goto err_debugfs;
+ }
}
ret = device_add(minor->kdev);
@@ -172,9 +187,13 @@ static int drm_minor_register(struct drm_device *dev, unsigned int type)
goto err_debugfs;
/* replace NULL with @minor so lookups will succeed from now on */
- spin_lock_irqsave(&drm_minor_lock, flags);
- idr_replace(&drm_minors_idr, minor, minor->index);
- spin_unlock_irqrestore(&drm_minor_lock, flags);
+ if (minor->type == DRM_MINOR_ACCEL) {
+ accel_minor_replace(minor, minor->index);
+ } else {
+ spin_lock_irqsave(&drm_minor_lock, flags);
+ idr_replace(&drm_minors_idr, minor, minor->index);
+ spin_unlock_irqrestore(&drm_minor_lock, flags);
+ }
DRM_DEBUG("new minor registered %d\n", minor->index);
return 0;
@@ -194,9 +213,13 @@ static void drm_minor_unregister(struct drm_device *dev, unsigned int type)
return;
/* replace @minor with NULL so lookups will fail from now on */
- spin_lock_irqsave(&drm_minor_lock, flags);
- idr_replace(&drm_minors_idr, NULL, minor->index);
- spin_unlock_irqrestore(&drm_minor_lock, flags);
+ if (minor->type == DRM_MINOR_ACCEL) {
+ accel_minor_replace(NULL, minor->index);
+ } else {
+ spin_lock_irqsave(&drm_minor_lock, flags);
+ idr_replace(&drm_minors_idr, NULL, minor->index);
+ spin_unlock_irqrestore(&drm_minor_lock, flags);
+ }
device_del(minor->kdev);
dev_set_drvdata(minor->kdev, NULL); /* safety belt */
@@ -603,6 +626,13 @@ static int drm_dev_init(struct drm_device *dev,
/* no per-device feature limits by default */
dev->driver_features = ~0u;
+ if (drm_core_check_feature(dev, DRIVER_COMPUTE_ACCEL) &&
+ (drm_core_check_feature(dev, DRIVER_RENDER) ||
+ drm_core_check_feature(dev, DRIVER_MODESET))) {
+ DRM_ERROR("DRM driver can't be both a compute acceleration and graphics driver\n");
+ return -EINVAL;
+ }
+
drm_legacy_init_members(dev);
INIT_LIST_HEAD(&dev->filelist);
INIT_LIST_HEAD(&dev->filelist_internal);
@@ -628,16 +658,22 @@ static int drm_dev_init(struct drm_device *dev,
dev->anon_inode = inode;
- if (drm_core_check_feature(dev, DRIVER_RENDER)) {
- ret = drm_minor_alloc(dev, DRM_MINOR_RENDER);
+ if (drm_core_check_feature(dev, DRIVER_COMPUTE_ACCEL)) {
+ ret = drm_minor_alloc(dev, DRM_MINOR_ACCEL);
+ if (ret)
+ goto err;
+ } else {
+ if (drm_core_check_feature(dev, DRIVER_RENDER)) {
+ ret = drm_minor_alloc(dev, DRM_MINOR_RENDER);
+ if (ret)
+ goto err;
+ }
+
+ ret = drm_minor_alloc(dev, DRM_MINOR_PRIMARY);
if (ret)
goto err;
}
- ret = drm_minor_alloc(dev, DRM_MINOR_PRIMARY);
- if (ret)
- goto err;
-
ret = drm_legacy_create_map_hash(dev);
if (ret)
goto err;
@@ -883,6 +919,10 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags)
if (ret)
goto err_minors;
+ ret = drm_minor_register(dev, DRM_MINOR_ACCEL);
+ if (ret)
+ goto err_minors;
+
ret = create_compat_control_link(dev);
if (ret)
goto err_minors;
@@ -902,12 +942,13 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags)
driver->name, driver->major, driver->minor,
driver->patchlevel, driver->date,
dev->dev ? dev_name(dev->dev) : "virtual device",
- dev->primary->index);
+ dev->primary ? dev->primary->index : dev->accel->index);
goto out_unlock;
err_minors:
remove_compat_control_link(dev);
+ drm_minor_unregister(dev, DRM_MINOR_ACCEL);
drm_minor_unregister(dev, DRM_MINOR_PRIMARY);
drm_minor_unregister(dev, DRM_MINOR_RENDER);
out_unlock:
@@ -950,6 +991,7 @@ void drm_dev_unregister(struct drm_device *dev)
drm_legacy_rmmaps(dev);
remove_compat_control_link(dev);
+ drm_minor_unregister(dev, DRM_MINOR_ACCEL);
drm_minor_unregister(dev, DRM_MINOR_PRIMARY);
drm_minor_unregister(dev, DRM_MINOR_RENDER);
}
@@ -1034,6 +1076,7 @@ static const struct file_operations drm_stub_fops = {
static void drm_core_exit(void)
{
drm_privacy_screen_lookup_exit();
+ accel_core_exit();
unregister_chrdev(DRM_MAJOR, "drm");
debugfs_remove(drm_debugfs_root);
drm_sysfs_destroy();
@@ -1061,6 +1104,10 @@ static int __init drm_core_init(void)
if (ret < 0)
goto error;
+ ret = accel_core_init();
+ if (ret < 0)
+ goto error;
+
drm_privacy_screen_lookup_init();
drm_core_init_complete = true;
diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c
index 4005dab..3841aba 100644
--- a/drivers/gpu/drm/drm_edid.c
+++ b/drivers/gpu/drm/drm_edid.c
@@ -87,6 +87,8 @@ static int oui(u8 first, u8 second, u8 third)
#define EDID_QUIRK_FORCE_10BPC (1 << 11)
/* Non desktop display (i.e. HMD) */
#define EDID_QUIRK_NON_DESKTOP (1 << 12)
+/* Cap the DSC target bitrate to 15bpp */
+#define EDID_QUIRK_CAP_DSC_15BPP (1 << 13)
#define MICROSOFT_IEEE_OUI 0xca125c
@@ -147,6 +149,12 @@ static const struct edid_quirk {
EDID_QUIRK('F', 'C', 'M', 13600, EDID_QUIRK_PREFER_LARGE_75 |
EDID_QUIRK_DETAILED_IN_CM),
+ /* LG 27GP950 */
+ EDID_QUIRK('G', 'S', 'M', 0x5bbf, EDID_QUIRK_CAP_DSC_15BPP),
+
+ /* LG 27GN950 */
+ EDID_QUIRK('G', 'S', 'M', 0x5b9a, EDID_QUIRK_CAP_DSC_15BPP),
+
/* LGD panel of HP zBook 17 G2, eDP 10 bpc, but reports unknown bpc */
EDID_QUIRK('L', 'G', 'D', 764, EDID_QUIRK_FORCE_10BPC),
@@ -1572,15 +1580,6 @@ struct drm_edid {
const struct edid *edid;
};
-static bool version_greater(const struct drm_edid *drm_edid,
- u8 version, u8 revision)
-{
- const struct edid *edid = drm_edid->edid;
-
- return edid->version > version ||
- (edid->version == version && edid->revision > revision);
-}
-
static int edid_hfeeodb_extension_block_count(const struct edid *edid);
static int edid_hfeeodb_block_count(const struct edid *edid)
@@ -1622,7 +1621,8 @@ static const void *edid_extension_block_data(const struct edid *edid, int index)
return edid_block_data(edid, index + 1);
}
-static int drm_edid_block_count(const struct drm_edid *drm_edid)
+/* EDID block count indicated in EDID, may exceed allocated size */
+static int __drm_edid_block_count(const struct drm_edid *drm_edid)
{
int num_blocks;
@@ -1642,12 +1642,18 @@ static int drm_edid_block_count(const struct drm_edid *drm_edid)
num_blocks = eeodb;
}
- /* Limit by allocated size */
- num_blocks = min(num_blocks, (int)drm_edid->size / EDID_LENGTH);
-
return num_blocks;
}
+/* EDID block count, limited by allocated size */
+static int drm_edid_block_count(const struct drm_edid *drm_edid)
+{
+ /* Limit by allocated size */
+ return min(__drm_edid_block_count(drm_edid),
+ (int)drm_edid->size / EDID_LENGTH);
+}
+
+/* EDID extension block count, limited by allocated size */
static int drm_edid_extension_block_count(const struct drm_edid *drm_edid)
{
return drm_edid_block_count(drm_edid) - 1;
@@ -1981,7 +1987,7 @@ bool drm_edid_block_valid(u8 *_block, int block_num, bool print_bad_edid,
status = edid_block_check(block, is_base_block);
if (status == EDID_BLOCK_HEADER_REPAIR) {
- DRM_DEBUG("Fixing EDID header, your hardware may be failing\n");
+ DRM_DEBUG_KMS("Fixing EDID header, your hardware may be failing\n");
edid_header_fix(block);
/* Retry with fixed header, update status if that worked. */
@@ -2042,6 +2048,36 @@ bool drm_edid_is_valid(struct edid *edid)
}
EXPORT_SYMBOL(drm_edid_is_valid);
+/**
+ * drm_edid_valid - sanity check EDID data
+ * @drm_edid: EDID data
+ *
+ * Sanity check an EDID. Cross check block count against allocated size and
+ * checksum the blocks.
+ *
+ * Return: True if the EDID data is valid, false otherwise.
+ */
+bool drm_edid_valid(const struct drm_edid *drm_edid)
+{
+ int i;
+
+ if (!drm_edid)
+ return false;
+
+ if (edid_size_by_blocks(__drm_edid_block_count(drm_edid)) != drm_edid->size)
+ return false;
+
+ for (i = 0; i < drm_edid_block_count(drm_edid); i++) {
+ const void *block = drm_edid_block_data(drm_edid, i);
+
+ if (!edid_block_valid(block, i == 0))
+ return false;
+ }
+
+ return true;
+}
+EXPORT_SYMBOL(drm_edid_valid);
+
static struct edid *edid_filter_invalid_blocks(struct edid *edid,
size_t *alloc_size)
{
@@ -2168,58 +2204,91 @@ static void connector_bad_edid(struct drm_connector *connector,
if (connector->bad_edid_counter++ && !drm_debug_enabled(DRM_UT_KMS))
return;
- drm_dbg_kms(connector->dev, "%s: EDID is invalid:\n", connector->name);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] EDID is invalid:\n",
+ connector->base.id, connector->name);
for (i = 0; i < num_blocks; i++)
edid_block_dump(KERN_DEBUG, edid + i, i);
}
/* Get override or firmware EDID */
-static struct edid *drm_get_override_edid(struct drm_connector *connector,
- size_t *alloc_size)
+static const struct drm_edid *drm_edid_override_get(struct drm_connector *connector)
{
- struct edid *override = NULL;
+ const struct drm_edid *override = NULL;
- if (connector->override_edid)
- override = drm_edid_duplicate(connector->edid_blob_ptr->data);
+ mutex_lock(&connector->edid_override_mutex);
+
+ if (connector->edid_override)
+ override = drm_edid_dup(connector->edid_override);
+
+ mutex_unlock(&connector->edid_override_mutex);
if (!override)
- override = drm_load_edid_firmware(connector);
-
- /* FIXME: Get alloc size from deeper down the stack */
- if (!IS_ERR_OR_NULL(override) && alloc_size)
- *alloc_size = edid_size(override);
+ override = drm_edid_load_firmware(connector);
return IS_ERR(override) ? NULL : override;
}
/* For debugfs edid_override implementation */
+int drm_edid_override_show(struct drm_connector *connector, struct seq_file *m)
+{
+ const struct drm_edid *drm_edid;
+
+ mutex_lock(&connector->edid_override_mutex);
+
+ drm_edid = connector->edid_override;
+ if (drm_edid)
+ seq_write(m, drm_edid->edid, drm_edid->size);
+
+ mutex_unlock(&connector->edid_override_mutex);
+
+ return 0;
+}
+
+/* For debugfs edid_override implementation */
int drm_edid_override_set(struct drm_connector *connector, const void *edid,
size_t size)
{
- int ret;
+ const struct drm_edid *drm_edid;
- if (size < EDID_LENGTH || edid_size(edid) > size)
+ drm_edid = drm_edid_alloc(edid, size);
+ if (!drm_edid_valid(drm_edid)) {
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] EDID override invalid\n",
+ connector->base.id, connector->name);
+ drm_edid_free(drm_edid);
return -EINVAL;
+ }
- connector->override_edid = false;
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] EDID override set\n",
+ connector->base.id, connector->name);
- ret = drm_connector_update_edid_property(connector, edid);
- if (!ret)
- connector->override_edid = true;
+ mutex_lock(&connector->edid_override_mutex);
- return ret;
+ drm_edid_free(connector->edid_override);
+ connector->edid_override = drm_edid;
+
+ mutex_unlock(&connector->edid_override_mutex);
+
+ return 0;
}
/* For debugfs edid_override implementation */
int drm_edid_override_reset(struct drm_connector *connector)
{
- connector->override_edid = false;
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] EDID override reset\n",
+ connector->base.id, connector->name);
- return drm_connector_update_edid_property(connector, NULL);
+ mutex_lock(&connector->edid_override_mutex);
+
+ drm_edid_free(connector->edid_override);
+ connector->edid_override = NULL;
+
+ mutex_unlock(&connector->edid_override_mutex);
+
+ return 0;
}
/**
- * drm_add_override_edid_modes - add modes from override/firmware EDID
+ * drm_edid_override_connector_update - add modes from override/firmware EDID
* @connector: connector we're probing
*
* Add modes from the override/firmware EDID, if available. Only to be used from
@@ -2229,24 +2298,25 @@ int drm_edid_override_reset(struct drm_connector *connector)
*
* Return: The number of modes added or 0 if we couldn't find any.
*/
-int drm_add_override_edid_modes(struct drm_connector *connector)
+int drm_edid_override_connector_update(struct drm_connector *connector)
{
- struct edid *override;
+ const struct drm_edid *override;
int num_modes = 0;
- override = drm_get_override_edid(connector, NULL);
+ override = drm_edid_override_get(connector);
if (override) {
- drm_connector_update_edid_property(connector, override);
- num_modes = drm_add_edid_modes(connector, override);
- kfree(override);
+ num_modes = drm_edid_connector_update(connector, override);
- DRM_DEBUG_KMS("[CONNECTOR:%d:%s] adding %d modes via fallback override/firmware EDID\n",
- connector->base.id, connector->name, num_modes);
+ drm_edid_free(override);
+
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] adding %d modes via fallback override/firmware EDID\n",
+ connector->base.id, connector->name, num_modes);
}
return num_modes;
}
-EXPORT_SYMBOL(drm_add_override_edid_modes);
+EXPORT_SYMBOL(drm_edid_override_connector_update);
typedef int read_block_fn(void *context, u8 *buf, unsigned int block, size_t len);
@@ -2289,12 +2359,19 @@ static struct edid *_drm_do_get_edid(struct drm_connector *connector,
{
enum edid_block_status status;
int i, num_blocks, invalid_blocks = 0;
+ const struct drm_edid *override;
struct edid *edid, *new;
size_t alloc_size = EDID_LENGTH;
- edid = drm_get_override_edid(connector, &alloc_size);
- if (edid)
+ override = drm_edid_override_get(connector);
+ if (override) {
+ alloc_size = override->size;
+ edid = kmemdup(override->edid, alloc_size, GFP_KERNEL);
+ drm_edid_free(override);
+ if (!edid)
+ return NULL;
goto ok;
+ }
edid = kmalloc(alloc_size, GFP_KERNEL);
if (!edid)
@@ -2397,7 +2474,7 @@ static struct edid *_drm_do_get_edid(struct drm_connector *connector,
* adapter and use drm_get_edid() instead of abusing this function.
*
* The EDID may be overridden using debugfs override_edid or firmware EDID
- * (drm_load_edid_firmware() and drm.edid_firmware parameter), in this priority
+ * (drm_edid_load_firmware() and drm.edid_firmware parameter), in this priority
* order. Having either of them bypasses actual EDID reads.
*
* Return: Pointer to valid EDID or NULL if we couldn't find any.
@@ -2575,7 +2652,7 @@ EXPORT_SYMBOL(drm_get_edid);
* this function.
*
* The EDID may be overridden using debugfs override_edid or firmware EDID
- * (drm_load_edid_firmware() and drm.edid_firmware parameter), in this priority
+ * (drm_edid_load_firmware() and drm.edid_firmware parameter), in this priority
* order. Having either of them bypasses actual EDID reads.
*
* The returned pointer must be freed using drm_edid_free().
@@ -2613,7 +2690,7 @@ EXPORT_SYMBOL(drm_edid_read_custom);
* Read EDID using the given I2C adapter.
*
* The EDID may be overridden using debugfs override_edid or firmware EDID
- * (drm_load_edid_firmware() and drm.edid_firmware parameter), in this priority
+ * (drm_edid_load_firmware() and drm.edid_firmware parameter), in this priority
* order. Having either of them bypasses actual EDID reads.
*
* Prefer initializing connector->ddc with drm_connector_init_with_ddc() and
@@ -2649,7 +2726,7 @@ EXPORT_SYMBOL(drm_edid_read_ddc);
* Read EDID using the connector's I2C adapter.
*
* The EDID may be overridden using debugfs override_edid or firmware EDID
- * (drm_load_edid_firmware() and drm.edid_firmware parameter), in this priority
+ * (drm_edid_load_firmware() and drm.edid_firmware parameter), in this priority
* order. Having either of them bypasses actual EDID reads.
*
* The returned pointer must be freed using drm_edid_free().
@@ -2730,6 +2807,8 @@ u32 drm_edid_get_panel_id(struct i2c_adapter *adapter)
if (edid_block_status_valid(status, edid_block_tag(base_block)))
panel_id = edid_extract_panel_id(base_block);
+ else
+ edid_block_dump(KERN_NOTICE, base_block, 0);
kfree(base_block);
@@ -2984,7 +3063,7 @@ is_rb(const struct detailed_timing *descriptor, void *data)
BUILD_BUG_ON(offsetof(typeof(*descriptor), data.other_data.data.range.formula.cvt.flags) != 15);
if (descriptor->data.other_data.data.range.flags == DRM_EDID_CVT_SUPPORT_FLAG &&
- descriptor->data.other_data.data.range.formula.cvt.flags & 0x10)
+ descriptor->data.other_data.data.range.formula.cvt.flags & DRM_EDID_CVT_FLAGS_REDUCED_BLANKING)
*res = true;
}
@@ -3012,7 +3091,7 @@ find_gtf2(const struct detailed_timing *descriptor, void *data)
BUILD_BUG_ON(offsetof(typeof(*descriptor), data.other_data.data.range.flags) != 10);
- if (descriptor->data.other_data.data.range.flags == 0x02)
+ if (descriptor->data.other_data.data.range.flags == DRM_EDID_SECONDARY_GTF_SUPPORT_FLAG)
*res = descriptor;
}
@@ -3077,20 +3156,53 @@ drm_gtf2_2j(const struct drm_edid *drm_edid)
return descriptor ? descriptor->data.other_data.data.range.formula.gtf2.j : 0;
}
+static void
+get_timing_level(const struct detailed_timing *descriptor, void *data)
+{
+ int *res = data;
+
+ if (!is_display_descriptor(descriptor, EDID_DETAIL_MONITOR_RANGE))
+ return;
+
+ BUILD_BUG_ON(offsetof(typeof(*descriptor), data.other_data.data.range.flags) != 10);
+
+ switch (descriptor->data.other_data.data.range.flags) {
+ case DRM_EDID_DEFAULT_GTF_SUPPORT_FLAG:
+ *res = LEVEL_GTF;
+ break;
+ case DRM_EDID_SECONDARY_GTF_SUPPORT_FLAG:
+ *res = LEVEL_GTF2;
+ break;
+ case DRM_EDID_CVT_SUPPORT_FLAG:
+ *res = LEVEL_CVT;
+ break;
+ default:
+ break;
+ }
+}
+
/* Get standard timing level (CVT/GTF/DMT). */
static int standard_timing_level(const struct drm_edid *drm_edid)
{
const struct edid *edid = drm_edid->edid;
- if (edid->revision >= 2) {
- if (edid->revision >= 4 && (edid->features & DRM_EDID_FEATURE_DEFAULT_GTF))
- return LEVEL_CVT;
- if (drm_gtf2_hbreak(drm_edid))
- return LEVEL_GTF2;
- if (edid->features & DRM_EDID_FEATURE_DEFAULT_GTF)
- return LEVEL_GTF;
+ if (edid->revision >= 4) {
+ /*
+ * If the range descriptor doesn't
+ * indicate otherwise default to CVT
+ */
+ int ret = LEVEL_CVT;
+
+ drm_for_each_detailed_block(drm_edid, get_timing_level, &ret);
+
+ return ret;
+ } else if (edid->revision >= 3 && drm_gtf2_hbreak(drm_edid)) {
+ return LEVEL_GTF2;
+ } else if (edid->revision >= 2) {
+ return LEVEL_GTF;
+ } else {
+ return LEVEL_DMT;
}
- return LEVEL_DMT;
}
/*
@@ -3113,6 +3225,35 @@ static int drm_mode_hsync(const struct drm_display_mode *mode)
return DIV_ROUND_CLOSEST(mode->clock, mode->htotal);
}
+static struct drm_display_mode *
+drm_gtf2_mode(struct drm_device *dev,
+ const struct drm_edid *drm_edid,
+ int hsize, int vsize, int vrefresh_rate)
+{
+ struct drm_display_mode *mode;
+
+ /*
+ * This is potentially wrong if there's ever a monitor with
+ * more than one ranges section, each claiming a different
+ * secondary GTF curve. Please don't do that.
+ */
+ mode = drm_gtf_mode(dev, hsize, vsize, vrefresh_rate, 0, 0);
+ if (!mode)
+ return NULL;
+
+ if (drm_mode_hsync(mode) > drm_gtf2_hbreak(drm_edid)) {
+ drm_mode_destroy(dev, mode);
+ mode = drm_gtf_mode_complex(dev, hsize, vsize,
+ vrefresh_rate, 0, 0,
+ drm_gtf2_m(drm_edid),
+ drm_gtf2_2c(drm_edid),
+ drm_gtf2_k(drm_edid),
+ drm_gtf2_2j(drm_edid));
+ }
+
+ return mode;
+}
+
/*
* Take the standard timing params (in this case width, aspect, and refresh)
* and convert them into a real mode using CVT/GTF/DMT.
@@ -3201,23 +3342,7 @@ static struct drm_display_mode *drm_mode_std(struct drm_connector *connector,
mode = drm_gtf_mode(dev, hsize, vsize, vrefresh_rate, 0, 0);
break;
case LEVEL_GTF2:
- /*
- * This is potentially wrong if there's ever a monitor with
- * more than one ranges section, each claiming a different
- * secondary GTF curve. Please don't do that.
- */
- mode = drm_gtf_mode(dev, hsize, vsize, vrefresh_rate, 0, 0);
- if (!mode)
- return NULL;
- if (drm_mode_hsync(mode) > drm_gtf2_hbreak(drm_edid)) {
- drm_mode_destroy(dev, mode);
- mode = drm_gtf_mode_complex(dev, hsize, vsize,
- vrefresh_rate, 0, 0,
- drm_gtf2_m(drm_edid),
- drm_gtf2_2c(drm_edid),
- drm_gtf2_k(drm_edid),
- drm_gtf2_2j(drm_edid));
- }
+ mode = drm_gtf2_mode(dev, drm_edid, hsize, vsize, vrefresh_rate);
break;
case LEVEL_CVT:
mode = drm_cvt_mode(dev, hsize, vsize, vrefresh_rate, 0, 0,
@@ -3274,11 +3399,12 @@ drm_mode_do_interlace_quirk(struct drm_display_mode *mode,
* timing block contains enough info for us to create and return a new struct
* drm_display_mode.
*/
-static struct drm_display_mode *drm_mode_detailed(struct drm_device *dev,
+static struct drm_display_mode *drm_mode_detailed(struct drm_connector *connector,
const struct drm_edid *drm_edid,
const struct detailed_timing *timing,
u32 quirks)
{
+ struct drm_device *dev = connector->dev;
struct drm_display_mode *mode;
const struct detailed_pixel_timing *pt = &timing->data.pixel_data;
unsigned hactive = (pt->hactive_hblank_hi & 0xf0) << 4 | pt->hactive_lo;
@@ -3295,17 +3421,19 @@ static struct drm_display_mode *drm_mode_detailed(struct drm_device *dev,
return NULL;
if (pt->misc & DRM_EDID_PT_STEREO) {
- DRM_DEBUG_KMS("stereo mode not supported\n");
+ drm_dbg_kms(dev, "[CONNECTOR:%d:%s] Stereo mode not supported\n",
+ connector->base.id, connector->name);
return NULL;
}
if (!(pt->misc & DRM_EDID_PT_SEPARATE_SYNC)) {
- DRM_DEBUG_KMS("composite sync not supported\n");
+ drm_dbg_kms(dev, "[CONNECTOR:%d:%s] Composite sync not supported\n",
+ connector->base.id, connector->name);
}
/* it is incorrect if hsync/vsync width is zero */
if (!hsync_pulse_width || !vsync_pulse_width) {
- DRM_DEBUG_KMS("Incorrect Detailed timing. "
- "Wrong Hsync/Vsync pulse width\n");
+ drm_dbg_kms(dev, "[CONNECTOR:%d:%s] Incorrect Detailed timing. Wrong Hsync/Vsync pulse width\n",
+ connector->base.id, connector->name);
return NULL;
}
@@ -3415,7 +3543,7 @@ range_pixel_clock(const struct edid *edid, const u8 *t)
return 0;
/* 1.4 with CVT support gives us real precision, yay */
- if (edid->revision >= 4 && t[10] == 0x04)
+ if (edid->revision >= 4 && t[10] == DRM_EDID_CVT_SUPPORT_FLAG)
return (t[9] * 10000) - ((t[12] >> 2) * 250);
/* 1.3 is pathetic, so fuzz up a bit */
@@ -3441,7 +3569,7 @@ static bool mode_in_range(const struct drm_display_mode *mode,
return false;
/* 1.4 max horizontal check */
- if (edid->revision >= 4 && t[10] == 0x04)
+ if (edid->revision >= 4 && t[10] == DRM_EDID_CVT_SUPPORT_FLAG)
if (t[13] && mode->hdisplay > 8 * (t[13] + (256 * (t[12]&0x3))))
return false;
@@ -3533,6 +3661,35 @@ static int drm_gtf_modes_for_range(struct drm_connector *connector,
return modes;
}
+static int drm_gtf2_modes_for_range(struct drm_connector *connector,
+ const struct drm_edid *drm_edid,
+ const struct detailed_timing *timing)
+{
+ int i, modes = 0;
+ struct drm_display_mode *newmode;
+ struct drm_device *dev = connector->dev;
+
+ for (i = 0; i < ARRAY_SIZE(extra_modes); i++) {
+ const struct minimode *m = &extra_modes[i];
+
+ newmode = drm_gtf2_mode(dev, drm_edid, m->w, m->h, m->r);
+ if (!newmode)
+ return modes;
+
+ drm_mode_fixup_1366x768(newmode);
+ if (!mode_in_range(newmode, drm_edid, timing) ||
+ !valid_inferred_mode(connector, newmode)) {
+ drm_mode_destroy(dev, newmode);
+ continue;
+ }
+
+ drm_mode_probed_add(connector, newmode);
+ modes++;
+ }
+
+ return modes;
+}
+
static int drm_cvt_modes_for_range(struct drm_connector *connector,
const struct drm_edid *drm_edid,
const struct detailed_timing *timing)
@@ -3577,25 +3734,29 @@ do_inferred_modes(const struct detailed_timing *timing, void *c)
closure->drm_edid,
timing);
- if (!version_greater(closure->drm_edid, 1, 1))
+ if (closure->drm_edid->edid->revision < 2)
return; /* GTF not defined yet */
switch (range->flags) {
- case 0x02: /* secondary gtf, XXX could do more */
- case 0x00: /* default gtf */
+ case DRM_EDID_SECONDARY_GTF_SUPPORT_FLAG:
+ closure->modes += drm_gtf2_modes_for_range(closure->connector,
+ closure->drm_edid,
+ timing);
+ break;
+ case DRM_EDID_DEFAULT_GTF_SUPPORT_FLAG:
closure->modes += drm_gtf_modes_for_range(closure->connector,
closure->drm_edid,
timing);
break;
- case 0x04: /* cvt, only in 1.4+ */
- if (!version_greater(closure->drm_edid, 1, 3))
+ case DRM_EDID_CVT_SUPPORT_FLAG:
+ if (closure->drm_edid->edid->revision < 4)
break;
closure->modes += drm_cvt_modes_for_range(closure->connector,
closure->drm_edid,
timing);
break;
- case 0x01: /* just the ranges, no formula */
+ case DRM_EDID_RANGE_LIMITS_ONLY_FLAG:
default:
break;
}
@@ -3609,7 +3770,7 @@ static int add_inferred_modes(struct drm_connector *connector,
.drm_edid = drm_edid,
};
- if (version_greater(drm_edid, 1, 0))
+ if (drm_edid->edid->revision >= 1)
drm_for_each_detailed_block(drm_edid, do_inferred_modes, &closure);
return closure.modes;
@@ -3686,7 +3847,7 @@ static int add_established_modes(struct drm_connector *connector,
}
}
- if (version_greater(drm_edid, 1, 0))
+ if (edid->revision >= 1)
drm_for_each_detailed_block(drm_edid, do_established_modes,
&closure);
@@ -3741,7 +3902,7 @@ static int add_standard_modes(struct drm_connector *connector,
}
}
- if (version_greater(drm_edid, 1, 0))
+ if (drm_edid->edid->revision >= 1)
drm_for_each_detailed_block(drm_edid, do_standard_modes,
&closure);
@@ -3821,7 +3982,7 @@ add_cvt_modes(struct drm_connector *connector, const struct drm_edid *drm_edid)
.drm_edid = drm_edid,
};
- if (version_greater(drm_edid, 1, 2))
+ if (drm_edid->edid->revision >= 3)
drm_for_each_detailed_block(drm_edid, do_cvt_mode, &closure);
/* XXX should also look for CVT codes in VTB blocks */
@@ -3829,7 +3990,8 @@ add_cvt_modes(struct drm_connector *connector, const struct drm_edid *drm_edid)
return closure.modes;
}
-static void fixup_detailed_cea_mode_clock(struct drm_display_mode *mode);
+static void fixup_detailed_cea_mode_clock(struct drm_connector *connector,
+ struct drm_display_mode *mode);
static void
do_detailed_mode(const struct detailed_timing *timing, void *c)
@@ -3840,7 +4002,7 @@ do_detailed_mode(const struct detailed_timing *timing, void *c)
if (!is_detailed_timing_descriptor(timing))
return;
- newmode = drm_mode_detailed(closure->connector->dev,
+ newmode = drm_mode_detailed(closure->connector,
closure->drm_edid, timing,
closure->quirks);
if (!newmode)
@@ -3854,7 +4016,7 @@ do_detailed_mode(const struct detailed_timing *timing, void *c)
* so fix up anything that looks like CEA/HDMI mode, but the clock
* is just slightly off.
*/
- fixup_detailed_cea_mode_clock(newmode);
+ fixup_detailed_cea_mode_clock(closure->connector, newmode);
drm_mode_probed_add(closure->connector, newmode);
closure->modes++;
@@ -3873,13 +4035,14 @@ static int add_detailed_modes(struct drm_connector *connector,
struct detailed_mode_closure closure = {
.connector = connector,
.drm_edid = drm_edid,
- .preferred = true,
.quirks = quirks,
};
- if (closure.preferred && !version_greater(drm_edid, 1, 3))
+ if (drm_edid->edid->revision >= 4)
+ closure.preferred = true; /* first detailed timing is always preferred */
+ else
closure.preferred =
- (drm_edid->edid->features & DRM_EDID_FEATURE_PREFERRED_TIMING);
+ drm_edid->edid->features & DRM_EDID_FEATURE_PREFERRED_TIMING;
drm_for_each_detailed_block(drm_edid, do_detailed_mode, &closure);
@@ -4515,7 +4678,8 @@ static int add_hdmi_mode(struct drm_connector *connector, u8 vic)
struct drm_display_mode *newmode;
if (!drm_valid_hdmi_vic(vic)) {
- DRM_ERROR("Unknown HDMI VIC: %d\n", vic);
+ drm_err(connector->dev, "[CONNECTOR:%d:%s] Unknown HDMI VIC: %d\n",
+ connector->base.id, connector->name, vic);
return 0;
}
@@ -5122,7 +5286,8 @@ static int add_cea_modes(struct drm_connector *connector,
return modes;
}
-static void fixup_detailed_cea_mode_clock(struct drm_display_mode *mode)
+static void fixup_detailed_cea_mode_clock(struct drm_connector *connector,
+ struct drm_display_mode *mode)
{
const struct drm_display_mode *cea_mode;
int clock1, clock2, clock;
@@ -5160,8 +5325,10 @@ static void fixup_detailed_cea_mode_clock(struct drm_display_mode *mode)
if (mode->clock == clock)
return;
- DRM_DEBUG("detailed mode matches %s VIC %d, adjusting clock %d -> %d\n",
- type, vic, mode->clock, clock);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] detailed mode matches %s VIC %d, adjusting clock %d -> %d\n",
+ connector->base.id, connector->name,
+ type, vic, mode->clock, clock);
mode->clock = clock;
}
@@ -5269,15 +5436,12 @@ drm_parse_hdmi_vsdb_audio(struct drm_connector *connector, const u8 *db)
if (len >= 12)
connector->audio_latency[1] = db[12];
- DRM_DEBUG_KMS("HDMI: latency present %d %d, "
- "video latency %d %d, "
- "audio latency %d %d\n",
- connector->latency_present[0],
- connector->latency_present[1],
- connector->video_latency[0],
- connector->video_latency[1],
- connector->audio_latency[0],
- connector->audio_latency[1]);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] HDMI: latency present %d %d, video latency %d %d, audio latency %d %d\n",
+ connector->base.id, connector->name,
+ connector->latency_present[0], connector->latency_present[1],
+ connector->video_latency[0], connector->video_latency[1],
+ connector->audio_latency[0], connector->audio_latency[1]);
}
static void
@@ -5375,7 +5539,9 @@ static void drm_edid_to_eld(struct drm_connector *connector,
return;
mnl = get_monitor_name(drm_edid, &eld[DRM_ELD_MONITOR_NAME_STRING]);
- DRM_DEBUG_KMS("ELD monitor %s\n", &eld[DRM_ELD_MONITOR_NAME_STRING]);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] ELD monitor %s\n",
+ connector->base.id, connector->name,
+ &eld[DRM_ELD_MONITOR_NAME_STRING]);
eld[DRM_ELD_CEA_EDID_VER_MNL] = info->cea_rev << DRM_ELD_CEA_EDID_VER_SHIFT;
eld[DRM_ELD_CEA_EDID_VER_MNL] |= mnl;
@@ -5429,8 +5595,9 @@ static void drm_edid_to_eld(struct drm_connector *connector,
eld[DRM_ELD_BASELINE_ELD_LEN] =
DIV_ROUND_UP(drm_eld_calc_baseline_block_size(eld), 4);
- DRM_DEBUG_KMS("ELD size %d, SAD count %d\n",
- drm_eld_size(eld), total_sad_count);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] ELD size %d, SAD count %d\n",
+ connector->base.id, connector->name,
+ drm_eld_size(eld), total_sad_count);
}
static int _drm_edid_to_sad(const struct drm_edid *drm_edid,
@@ -5701,7 +5868,8 @@ static void drm_parse_vcdb(struct drm_connector *connector, const u8 *db)
{
struct drm_display_info *info = &connector->display_info;
- DRM_DEBUG_KMS("CEA VCDB 0x%02x\n", db[2]);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] CEA VCDB 0x%02x\n",
+ connector->base.id, connector->name, db[2]);
if (db[2] & EDID_CEA_VCDB_QS)
info->rgb_quant_range_selectable = true;
@@ -5752,12 +5920,87 @@ static void drm_parse_ycbcr420_deep_color_info(struct drm_connector *connector,
hdmi->y420_dc_modes = dc_mask;
}
+static void drm_parse_dsc_info(struct drm_hdmi_dsc_cap *hdmi_dsc,
+ const u8 *hf_scds)
+{
+ hdmi_dsc->v_1p2 = hf_scds[11] & DRM_EDID_DSC_1P2;
+
+ if (!hdmi_dsc->v_1p2)
+ return;
+
+ hdmi_dsc->native_420 = hf_scds[11] & DRM_EDID_DSC_NATIVE_420;
+ hdmi_dsc->all_bpp = hf_scds[11] & DRM_EDID_DSC_ALL_BPP;
+
+ if (hf_scds[11] & DRM_EDID_DSC_16BPC)
+ hdmi_dsc->bpc_supported = 16;
+ else if (hf_scds[11] & DRM_EDID_DSC_12BPC)
+ hdmi_dsc->bpc_supported = 12;
+ else if (hf_scds[11] & DRM_EDID_DSC_10BPC)
+ hdmi_dsc->bpc_supported = 10;
+ else
+ /* Supports min 8 BPC if DSC 1.2 is supported*/
+ hdmi_dsc->bpc_supported = 8;
+
+ if (cea_db_payload_len(hf_scds) >= 12 && hf_scds[12]) {
+ u8 dsc_max_slices;
+ u8 dsc_max_frl_rate;
+
+ dsc_max_frl_rate = (hf_scds[12] & DRM_EDID_DSC_MAX_FRL_RATE_MASK) >> 4;
+ drm_get_max_frl_rate(dsc_max_frl_rate, &hdmi_dsc->max_lanes,
+ &hdmi_dsc->max_frl_rate_per_lane);
+
+ dsc_max_slices = hf_scds[12] & DRM_EDID_DSC_MAX_SLICES;
+
+ switch (dsc_max_slices) {
+ case 1:
+ hdmi_dsc->max_slices = 1;
+ hdmi_dsc->clk_per_slice = 340;
+ break;
+ case 2:
+ hdmi_dsc->max_slices = 2;
+ hdmi_dsc->clk_per_slice = 340;
+ break;
+ case 3:
+ hdmi_dsc->max_slices = 4;
+ hdmi_dsc->clk_per_slice = 340;
+ break;
+ case 4:
+ hdmi_dsc->max_slices = 8;
+ hdmi_dsc->clk_per_slice = 340;
+ break;
+ case 5:
+ hdmi_dsc->max_slices = 8;
+ hdmi_dsc->clk_per_slice = 400;
+ break;
+ case 6:
+ hdmi_dsc->max_slices = 12;
+ hdmi_dsc->clk_per_slice = 400;
+ break;
+ case 7:
+ hdmi_dsc->max_slices = 16;
+ hdmi_dsc->clk_per_slice = 400;
+ break;
+ case 0:
+ default:
+ hdmi_dsc->max_slices = 0;
+ hdmi_dsc->clk_per_slice = 0;
+ }
+ }
+
+ if (cea_db_payload_len(hf_scds) >= 13 && hf_scds[13])
+ hdmi_dsc->total_chunk_kbytes = hf_scds[13] & DRM_EDID_DSC_TOTAL_CHUNK_KBYTES;
+}
+
/* Sink Capability Data Structure */
static void drm_parse_hdmi_forum_scds(struct drm_connector *connector,
const u8 *hf_scds)
{
struct drm_display_info *display = &connector->display_info;
struct drm_hdmi_info *hdmi = &display->hdmi;
+ struct drm_hdmi_dsc_cap *hdmi_dsc = &hdmi->dsc_cap;
+ int max_tmds_clock = 0;
+ u8 max_frl_rate = 0;
+ bool dsc_support = false;
display->has_hdmi_infoframe = true;
@@ -5777,14 +6020,13 @@ static void drm_parse_hdmi_forum_scds(struct drm_connector *connector,
*/
if (hf_scds[5]) {
- /* max clock is 5000 KHz times block value */
- u32 max_tmds_clock = hf_scds[5] * 5000;
struct drm_scdc *scdc = &hdmi->scdc;
+ /* max clock is 5000 KHz times block value */
+ max_tmds_clock = hf_scds[5] * 5000;
+
if (max_tmds_clock > 340000) {
display->max_tmds_clock = max_tmds_clock;
- DRM_DEBUG_KMS("HF-VSDB: max TMDS clock %d kHz\n",
- display->max_tmds_clock);
}
if (scdc->supported) {
@@ -5797,74 +6039,22 @@ static void drm_parse_hdmi_forum_scds(struct drm_connector *connector,
}
if (hf_scds[7]) {
- u8 max_frl_rate;
- u8 dsc_max_frl_rate;
- u8 dsc_max_slices;
- struct drm_hdmi_dsc_cap *hdmi_dsc = &hdmi->dsc_cap;
-
- DRM_DEBUG_KMS("hdmi_21 sink detected. parsing edid\n");
max_frl_rate = (hf_scds[7] & DRM_EDID_MAX_FRL_RATE_MASK) >> 4;
drm_get_max_frl_rate(max_frl_rate, &hdmi->max_lanes,
&hdmi->max_frl_rate_per_lane);
- hdmi_dsc->v_1p2 = hf_scds[11] & DRM_EDID_DSC_1P2;
-
- if (hdmi_dsc->v_1p2) {
- hdmi_dsc->native_420 = hf_scds[11] & DRM_EDID_DSC_NATIVE_420;
- hdmi_dsc->all_bpp = hf_scds[11] & DRM_EDID_DSC_ALL_BPP;
-
- if (hf_scds[11] & DRM_EDID_DSC_16BPC)
- hdmi_dsc->bpc_supported = 16;
- else if (hf_scds[11] & DRM_EDID_DSC_12BPC)
- hdmi_dsc->bpc_supported = 12;
- else if (hf_scds[11] & DRM_EDID_DSC_10BPC)
- hdmi_dsc->bpc_supported = 10;
- else
- hdmi_dsc->bpc_supported = 0;
-
- dsc_max_frl_rate = (hf_scds[12] & DRM_EDID_DSC_MAX_FRL_RATE_MASK) >> 4;
- drm_get_max_frl_rate(dsc_max_frl_rate, &hdmi_dsc->max_lanes,
- &hdmi_dsc->max_frl_rate_per_lane);
- hdmi_dsc->total_chunk_kbytes = hf_scds[13] & DRM_EDID_DSC_TOTAL_CHUNK_KBYTES;
-
- dsc_max_slices = hf_scds[12] & DRM_EDID_DSC_MAX_SLICES;
- switch (dsc_max_slices) {
- case 1:
- hdmi_dsc->max_slices = 1;
- hdmi_dsc->clk_per_slice = 340;
- break;
- case 2:
- hdmi_dsc->max_slices = 2;
- hdmi_dsc->clk_per_slice = 340;
- break;
- case 3:
- hdmi_dsc->max_slices = 4;
- hdmi_dsc->clk_per_slice = 340;
- break;
- case 4:
- hdmi_dsc->max_slices = 8;
- hdmi_dsc->clk_per_slice = 340;
- break;
- case 5:
- hdmi_dsc->max_slices = 8;
- hdmi_dsc->clk_per_slice = 400;
- break;
- case 6:
- hdmi_dsc->max_slices = 12;
- hdmi_dsc->clk_per_slice = 400;
- break;
- case 7:
- hdmi_dsc->max_slices = 16;
- hdmi_dsc->clk_per_slice = 400;
- break;
- case 0:
- default:
- hdmi_dsc->max_slices = 0;
- hdmi_dsc->clk_per_slice = 0;
- }
- }
}
drm_parse_ycbcr420_deep_color_info(connector, hf_scds);
+
+ if (cea_db_payload_len(hf_scds) >= 11 && hf_scds[11]) {
+ drm_parse_dsc_info(hdmi_dsc, hf_scds);
+ dsc_support = true;
+ }
+
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] HF-VSDB: max TMDS clock: %d KHz, HDMI 2.1 support: %s, DSC 1.2 support: %s\n",
+ connector->base.id, connector->name,
+ max_tmds_clock, str_yes_no(max_frl_rate), str_yes_no(dsc_support));
}
static void drm_parse_hdmi_deep_color_info(struct drm_connector *connector,
@@ -5882,39 +6072,39 @@ static void drm_parse_hdmi_deep_color_info(struct drm_connector *connector,
if (hdmi[6] & DRM_EDID_HDMI_DC_30) {
dc_bpc = 10;
info->edid_hdmi_rgb444_dc_modes |= DRM_EDID_HDMI_DC_30;
- DRM_DEBUG("%s: HDMI sink does deep color 30.\n",
- connector->name);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] HDMI sink does deep color 30.\n",
+ connector->base.id, connector->name);
}
if (hdmi[6] & DRM_EDID_HDMI_DC_36) {
dc_bpc = 12;
info->edid_hdmi_rgb444_dc_modes |= DRM_EDID_HDMI_DC_36;
- DRM_DEBUG("%s: HDMI sink does deep color 36.\n",
- connector->name);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] HDMI sink does deep color 36.\n",
+ connector->base.id, connector->name);
}
if (hdmi[6] & DRM_EDID_HDMI_DC_48) {
dc_bpc = 16;
info->edid_hdmi_rgb444_dc_modes |= DRM_EDID_HDMI_DC_48;
- DRM_DEBUG("%s: HDMI sink does deep color 48.\n",
- connector->name);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] HDMI sink does deep color 48.\n",
+ connector->base.id, connector->name);
}
if (dc_bpc == 0) {
- DRM_DEBUG("%s: No deep color support on this HDMI sink.\n",
- connector->name);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] No deep color support on this HDMI sink.\n",
+ connector->base.id, connector->name);
return;
}
- DRM_DEBUG("%s: Assigning HDMI sink color depth as %d bpc.\n",
- connector->name, dc_bpc);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] Assigning HDMI sink color depth as %d bpc.\n",
+ connector->base.id, connector->name, dc_bpc);
info->bpc = dc_bpc;
/* YCRCB444 is optional according to spec. */
if (hdmi[6] & DRM_EDID_HDMI_DC_Y444) {
info->edid_hdmi_ycbcr444_dc_modes = info->edid_hdmi_rgb444_dc_modes;
- DRM_DEBUG("%s: HDMI sink does YCRCB444 in deep color.\n",
- connector->name);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] HDMI sink does YCRCB444 in deep color.\n",
+ connector->base.id, connector->name);
}
/*
@@ -5922,8 +6112,8 @@ static void drm_parse_hdmi_deep_color_info(struct drm_connector *connector,
* then deep color 36 bit must be supported.
*/
if (!(hdmi[6] & DRM_EDID_HDMI_DC_36)) {
- DRM_DEBUG("%s: HDMI sink should do DC_36, but does not!\n",
- connector->name);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] HDMI sink should do DC_36, but does not!\n",
+ connector->base.id, connector->name);
}
}
@@ -5940,10 +6130,9 @@ drm_parse_hdmi_vsdb_video(struct drm_connector *connector, const u8 *db)
if (len >= 7)
info->max_tmds_clock = db[7] * 5000;
- DRM_DEBUG_KMS("HDMI: DVI dual %d, "
- "max TMDS clock %d kHz\n",
- info->dvi_dual,
- info->max_tmds_clock);
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] HDMI: DVI dual %d, max TMDS clock %d kHz\n",
+ connector->base.id, connector->name,
+ info->dvi_dual, info->max_tmds_clock);
drm_parse_hdmi_deep_color_info(connector, db);
}
@@ -5963,8 +6152,9 @@ static void drm_parse_microsoft_vsdb(struct drm_connector *connector,
if (version == 1 || version == 2 || (version == 3 && !desktop_usage))
info->non_desktop = true;
- drm_dbg_kms(connector->dev, "HMD or specialized display VSDB version %u: 0x%02x\n",
- version, db[5]);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] HMD or specialized display VSDB version %u: 0x%02x\n",
+ connector->base.id, connector->name, version, db[5]);
}
static void drm_parse_cea_ext(struct drm_connector *connector,
@@ -5985,8 +6175,10 @@ static void drm_parse_cea_ext(struct drm_connector *connector,
info->cea_rev = edid_ext[1];
if (info->cea_rev != edid_ext[1])
- DRM_DEBUG_KMS("CEA extension version mismatch %u != %u\n",
- info->cea_rev, edid_ext[1]);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] CEA extension version mismatch %u != %u\n",
+ connector->base.id, connector->name,
+ info->cea_rev, edid_ext[1]);
/* The existence of a CTA extension should imply RGB support */
info->color_formats = DRM_COLOR_FORMAT_RGB444;
@@ -6033,10 +6225,13 @@ void get_monitor_range(const struct detailed_timing *timing, void *c)
return;
/*
- * Check for flag range limits only. If flag == 1 then
- * no additional timing information provided.
- * Default GTF, GTF Secondary curve and CVT are not
- * supported
+ * These limits are used to determine the VRR refresh
+ * rate range. Only the "range limits only" variant
+ * of the range descriptor seems to guarantee that
+ * any and all timings are accepted by the sink, as
+ * opposed to just timings conforming to the indicated
+ * formula (GTF/GTF2/CVT). Thus other variants of the
+ * range descriptor are not accepted here.
*/
if (range->flags != DRM_EDID_RANGE_LIMITS_ONLY_FLAG)
return;
@@ -6061,14 +6256,18 @@ static void drm_get_monitor_range(struct drm_connector *connector,
.drm_edid = drm_edid,
};
- if (!version_greater(drm_edid, 1, 1))
+ if (drm_edid->edid->revision < 4)
+ return;
+
+ if (!(drm_edid->edid->features & DRM_EDID_FEATURE_CONTINUOUS_FREQ))
return;
drm_for_each_detailed_block(drm_edid, get_monitor_range, &closure);
- DRM_DEBUG_KMS("Supported Monitor Refresh rate range is %d Hz - %d Hz\n",
- info->monitor_range.min_vfreq,
- info->monitor_range.max_vfreq);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] Supported Monitor Refresh rate range is %d Hz - %d Hz\n",
+ connector->base.id, connector->name,
+ info->monitor_range.min_vfreq, info->monitor_range.max_vfreq);
}
static void drm_parse_vesa_mso_data(struct drm_connector *connector,
@@ -6079,8 +6278,9 @@ static void drm_parse_vesa_mso_data(struct drm_connector *connector,
struct drm_display_info *info = &connector->display_info;
if (block->num_bytes < 3) {
- drm_dbg_kms(connector->dev, "Unexpected vendor block size %u\n",
- block->num_bytes);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] Unexpected vendor block size %u\n",
+ connector->base.id, connector->name, block->num_bytes);
return;
}
@@ -6088,13 +6288,16 @@ static void drm_parse_vesa_mso_data(struct drm_connector *connector,
return;
if (sizeof(*vesa) != sizeof(*block) + block->num_bytes) {
- drm_dbg_kms(connector->dev, "Unexpected VESA vendor block size\n");
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] Unexpected VESA vendor block size\n",
+ connector->base.id, connector->name);
return;
}
switch (FIELD_GET(DISPLAYID_VESA_MSO_MODE, vesa->mso)) {
default:
- drm_dbg_kms(connector->dev, "Reserved MSO mode value\n");
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] Reserved MSO mode value\n",
+ connector->base.id, connector->name);
fallthrough;
case 0:
info->mso_stream_count = 0;
@@ -6114,12 +6317,16 @@ static void drm_parse_vesa_mso_data(struct drm_connector *connector,
info->mso_pixel_overlap = FIELD_GET(DISPLAYID_VESA_MSO_OVERLAP, vesa->mso);
if (info->mso_pixel_overlap > 8) {
- drm_dbg_kms(connector->dev, "Reserved MSO pixel overlap value %u\n",
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] Reserved MSO pixel overlap value %u\n",
+ connector->base.id, connector->name,
info->mso_pixel_overlap);
info->mso_pixel_overlap = 8;
}
- drm_dbg_kms(connector->dev, "MSO stream count %u, pixel overlap %u\n",
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] MSO stream count %u, pixel overlap %u\n",
+ connector->base.id, connector->name,
info->mso_stream_count, info->mso_pixel_overlap);
}
@@ -6166,6 +6373,7 @@ static void drm_reset_display_info(struct drm_connector *connector)
info->mso_stream_count = 0;
info->mso_pixel_overlap = 0;
+ info->max_dsc_bpp = 0;
}
static u32 update_display_info(struct drm_connector *connector,
@@ -6202,8 +6410,9 @@ static u32 update_display_info(struct drm_connector *connector,
if (info->bpc == 0 && edid->revision == 3 &&
edid->input & DRM_EDID_DIGITAL_DFP_1_X) {
info->bpc = 8;
- DRM_DEBUG("%s: Assigning DFP sink color depth as %d bpc.\n",
- connector->name, info->bpc);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] Assigning DFP sink color depth as %d bpc.\n",
+ connector->base.id, connector->name, info->bpc);
}
/* Only defined for 1.4 with digital displays */
@@ -6235,8 +6444,9 @@ static u32 update_display_info(struct drm_connector *connector,
break;
}
- DRM_DEBUG("%s: Assigning EDID-1.4 digital sink color depth as %d bpc.\n",
- connector->name, info->bpc);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] Assigning EDID-1.4 digital sink color depth as %d bpc.\n",
+ connector->base.id, connector->name, info->bpc);
if (edid->features & DRM_EDID_FEATURE_RGB_YCRCB444)
info->color_formats |= DRM_COLOR_FORMAT_YCBCR444;
@@ -6247,11 +6457,15 @@ static u32 update_display_info(struct drm_connector *connector,
out:
if (quirks & EDID_QUIRK_NON_DESKTOP) {
- drm_dbg_kms(connector->dev, "Non-desktop display%s\n",
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] Non-desktop display%s\n",
+ connector->base.id, connector->name,
info->non_desktop ? " (redundant quirk)" : "");
info->non_desktop = true;
}
+ if (quirks & EDID_QUIRK_CAP_DSC_15BPP)
+ info->max_dsc_bpp = 15;
+
return quirks;
}
@@ -6390,7 +6604,7 @@ static int _drm_edid_connector_update(struct drm_connector *connector,
num_modes += add_cea_modes(connector, drm_edid);
num_modes += add_alternate_cea_modes(connector, drm_edid);
num_modes += add_displayid_detailed_modes(connector, drm_edid);
- if (drm_edid->edid->features & DRM_EDID_FEATURE_DEFAULT_GTF)
+ if (drm_edid->edid->features & DRM_EDID_FEATURE_CONTINUOUS_FREQ)
num_modes += add_inferred_modes(connector, drm_edid);
if (quirks & (EDID_QUIRK_PREFER_LARGE_60 | EDID_QUIRK_PREFER_LARGE_75))
@@ -6482,23 +6696,6 @@ int drm_edid_connector_update(struct drm_connector *connector,
{
int count;
- /*
- * FIXME: Reconcile the differences in override_edid handling between
- * this and drm_connector_update_edid_property().
- *
- * If override_edid is set, and the EDID passed in here originates from
- * drm_edid_read() and friends, it will be the override EDID, and there
- * are no issues. drm_connector_update_edid_property() ignoring requests
- * to set the EDID dates back to a time when override EDID was not
- * handled at the low level EDID read.
- *
- * The only way the EDID passed in here can be different from the
- * override EDID is when a driver passes in an EDID that does *not*
- * originate from drm_edid_read() and friends, or passes in a stale
- * cached version. This, in turn, is a question of when an override EDID
- * set via debugfs should take effect.
- */
-
count = _drm_edid_connector_update(connector, drm_edid);
_drm_update_tile_info(connector, drm_edid);
@@ -6513,10 +6710,6 @@ EXPORT_SYMBOL(drm_edid_connector_update);
static int _drm_connector_update_edid_property(struct drm_connector *connector,
const struct drm_edid *drm_edid)
{
- /* ignore requests to set edid when overridden */
- if (connector->override_edid)
- return 0;
-
/*
* Set the display info, using edid if available, otherwise resetting
* the values to defaults. This duplicates the work done in
@@ -6579,8 +6772,8 @@ int drm_add_edid_modes(struct drm_connector *connector, struct edid *edid)
struct drm_edid drm_edid;
if (edid && !drm_edid_is_valid(edid)) {
- drm_warn(connector->dev, "%s: EDID invalid.\n",
- connector->name);
+ drm_warn(connector->dev, "[CONNECTOR:%d:%s] EDID invalid.\n",
+ connector->base.id, connector->name);
edid = NULL;
}
@@ -6837,7 +7030,7 @@ drm_hdmi_avi_infoframe_quant_range(struct hdmi_avi_infoframe *frame,
* by non-zero YQ when receiving RGB. There doesn't seem to be any
* good way to tell which version of CEA-861 the sink supports, so
* we limit non-zero YQ to HDMI 2.0 sinks only as HDMI 2.0 is based
- * on on CEA-861-F.
+ * on CEA-861-F.
*/
if (!is_hdmi2_sink(connector) ||
rgb_quant_range == HDMI_QUANTIZATION_RANGE_LIMITED)
@@ -6956,11 +7149,14 @@ static void drm_parse_tiled_block(struct drm_connector *connector,
connector->tile_h_size = w + 1;
connector->tile_v_size = h + 1;
- DRM_DEBUG_KMS("tile cap 0x%x\n", tile->tile_cap);
- DRM_DEBUG_KMS("tile_size %d x %d\n", w + 1, h + 1);
- DRM_DEBUG_KMS("topo num tiles %dx%d, location %dx%d\n",
- num_h_tile + 1, num_v_tile + 1, tile_h_loc, tile_v_loc);
- DRM_DEBUG_KMS("vend %c%c%c\n", tile->topology_id[0], tile->topology_id[1], tile->topology_id[2]);
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] tile cap 0x%x, size %dx%d, num tiles %dx%d, location %dx%d, vend %c%c%c",
+ connector->base.id, connector->name,
+ tile->tile_cap,
+ connector->tile_h_size, connector->tile_v_size,
+ connector->num_h_tile, connector->num_v_tile,
+ connector->tile_h_loc, connector->tile_v_loc,
+ tile->topology_id[0], tile->topology_id[1], tile->topology_id[2]);
tg = drm_mode_get_tile_group(connector->dev, tile->topology_id);
if (!tg)
diff --git a/drivers/gpu/drm/drm_edid_load.c b/drivers/gpu/drm/drm_edid_load.c
index 37d8ba3..5d9ef26 100644
--- a/drivers/gpu/drm/drm_edid_load.c
+++ b/drivers/gpu/drm/drm_edid_load.c
@@ -11,12 +11,13 @@
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_crtc_helper.h>
+#include <drm/drm_connector.h>
#include <drm/drm_drv.h>
#include <drm/drm_edid.h>
#include <drm/drm_print.h>
+#include "drm_crtc_internal.h"
+
static char edid_firmware[PATH_MAX];
module_param_string(edid_firmware, edid_firmware, sizeof(edid_firmware), 0644);
MODULE_PARM_DESC(edid_firmware, "Do not probe monitor, use specified EDID blob "
@@ -159,44 +160,26 @@ static const u8 generic_edid[GENERIC_EDIDS][128] = {
},
};
-static int edid_size(const u8 *edid, int data_size)
-{
- if (data_size < EDID_LENGTH)
- return 0;
-
- return (edid[0x7e] + 1) * EDID_LENGTH;
-}
-
-static void *edid_load(struct drm_connector *connector, const char *name,
- const char *connector_name)
+static const struct drm_edid *edid_load(struct drm_connector *connector, const char *name)
{
const struct firmware *fw = NULL;
const u8 *fwdata;
- u8 *edid;
+ const struct drm_edid *drm_edid;
int fwsize, builtin;
- int i, valid_extensions = 0;
- bool print_bad_edid = !connector->bad_edid_counter || drm_debug_enabled(DRM_UT_KMS);
builtin = match_string(generic_edid_name, GENERIC_EDIDS, name);
if (builtin >= 0) {
fwdata = generic_edid[builtin];
fwsize = sizeof(generic_edid[builtin]);
} else {
- struct platform_device *pdev;
int err;
- pdev = platform_device_register_simple(connector_name, -1, NULL, 0);
- if (IS_ERR(pdev)) {
- DRM_ERROR("Failed to register EDID firmware platform device "
- "for connector \"%s\"\n", connector_name);
- return ERR_CAST(pdev);
- }
-
- err = request_firmware(&fw, name, &pdev->dev);
- platform_device_unregister(pdev);
+ err = request_firmware(&fw, name, connector->dev->dev);
if (err) {
- DRM_ERROR("Requesting EDID firmware \"%s\" failed (err=%d)\n",
- name, err);
+ drm_err(connector->dev,
+ "[CONNECTOR:%d:%s] Requesting EDID firmware \"%s\" failed (err=%d)\n",
+ connector->base.id, connector->name,
+ name, err);
return ERR_PTR(err);
}
@@ -204,70 +187,26 @@ static void *edid_load(struct drm_connector *connector, const char *name,
fwsize = fw->size;
}
- if (edid_size(fwdata, fwsize) != fwsize) {
- DRM_ERROR("Size of EDID firmware \"%s\" is invalid "
- "(expected %d, got %d\n", name,
- edid_size(fwdata, fwsize), (int)fwsize);
- edid = ERR_PTR(-EINVAL);
- goto out;
+ drm_dbg_kms(connector->dev, "[CONNECTOR:%d:%s] Loaded %s firmware EDID \"%s\"\n",
+ connector->base.id, connector->name,
+ builtin >= 0 ? "built-in" : "external", name);
+
+ drm_edid = drm_edid_alloc(fwdata, fwsize);
+ if (!drm_edid_valid(drm_edid)) {
+ drm_err(connector->dev, "Invalid firmware EDID \"%s\"\n", name);
+ drm_edid_free(drm_edid);
+ drm_edid = ERR_PTR(-EINVAL);
}
- edid = kmemdup(fwdata, fwsize, GFP_KERNEL);
- if (edid == NULL) {
- edid = ERR_PTR(-ENOMEM);
- goto out;
- }
-
- if (!drm_edid_block_valid(edid, 0, print_bad_edid,
- &connector->edid_corrupt)) {
- connector->bad_edid_counter++;
- DRM_ERROR("Base block of EDID firmware \"%s\" is invalid ",
- name);
- kfree(edid);
- edid = ERR_PTR(-EINVAL);
- goto out;
- }
-
- for (i = 1; i <= edid[0x7e]; i++) {
- if (i != valid_extensions + 1)
- memcpy(edid + (valid_extensions + 1) * EDID_LENGTH,
- edid + i * EDID_LENGTH, EDID_LENGTH);
- if (drm_edid_block_valid(edid + i * EDID_LENGTH, i,
- print_bad_edid,
- NULL))
- valid_extensions++;
- }
-
- if (valid_extensions != edid[0x7e]) {
- u8 *new_edid;
-
- edid[EDID_LENGTH-1] += edid[0x7e] - valid_extensions;
- DRM_INFO("Found %d valid extensions instead of %d in EDID data "
- "\"%s\" for connector \"%s\"\n", valid_extensions,
- edid[0x7e], name, connector_name);
- edid[0x7e] = valid_extensions;
-
- new_edid = krealloc(edid, (valid_extensions + 1) * EDID_LENGTH,
- GFP_KERNEL);
- if (new_edid)
- edid = new_edid;
- }
-
- DRM_INFO("Got %s EDID base block and %d extension%s from "
- "\"%s\" for connector \"%s\"\n", (builtin >= 0) ? "built-in" :
- "external", valid_extensions, valid_extensions == 1 ? "" : "s",
- name, connector_name);
-
-out:
release_firmware(fw);
- return edid;
+
+ return drm_edid;
}
-struct edid *drm_load_edid_firmware(struct drm_connector *connector)
+const struct drm_edid *drm_edid_load_firmware(struct drm_connector *connector)
{
- const char *connector_name = connector->name;
char *edidname, *last, *colon, *fwstr, *edidstr, *fallback = NULL;
- struct edid *edid;
+ const struct drm_edid *drm_edid;
if (edid_firmware[0] == '\0')
return ERR_PTR(-ENOENT);
@@ -288,7 +227,7 @@ struct edid *drm_load_edid_firmware(struct drm_connector *connector)
while ((edidname = strsep(&edidstr, ","))) {
colon = strchr(edidname, ':');
if (colon != NULL) {
- if (strncmp(connector_name, edidname, colon - edidname))
+ if (strncmp(connector->name, edidname, colon - edidname))
continue;
edidname = colon + 1;
break;
@@ -310,8 +249,9 @@ struct edid *drm_load_edid_firmware(struct drm_connector *connector)
if (*last == '\n')
*last = '\0';
- edid = edid_load(connector, edidname, connector_name);
+ drm_edid = edid_load(connector, edidname);
+
kfree(fwstr);
- return edid;
+ return drm_edid;
}
diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 71edb80..b3a731b 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -30,24 +30,17 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/console.h>
-#include <linux/dma-buf.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/sysrq.h>
-#include <linux/vmalloc.h>
#include <drm/drm_atomic.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
+#include <drm/drm_modeset_helper_vtables.h>
#include <drm/drm_print.h>
#include <drm/drm_vblank.h>
-#include "drm_crtc_helper_internal.h"
#include "drm_internal.h"
static bool drm_fbdev_emulation = true;
@@ -74,7 +67,7 @@ MODULE_PARM_DESC(drm_fbdev_overalloc,
* considered as a broken and legacy behaviour from a modern fbdev device.
*/
#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM)
-static bool drm_leak_fbdev_smem = false;
+static bool drm_leak_fbdev_smem;
module_param_unsafe(drm_leak_fbdev_smem, bool, 0600);
MODULE_PARM_DESC(drm_leak_fbdev_smem,
"Allow unsafe leaking fbdev physical smem address [default=false]");
@@ -96,11 +89,13 @@ static DEFINE_MUTEX(kernel_fb_helper_lock);
* It will automatically set up deferred I/O if the driver requires a shadow
* buffer.
*
- * At runtime drivers should restore the fbdev console by using
+ * Existing fbdev implementations should restore the fbdev console by using
* drm_fb_helper_lastclose() as their &drm_driver.lastclose callback.
* They should also notify the fb helper code from updates to the output
* configuration by using drm_fb_helper_output_poll_changed() as their
- * &drm_mode_config_funcs.output_poll_changed callback.
+ * &drm_mode_config_funcs.output_poll_changed callback. New implementations
+ * of fbdev should be build on top of struct &drm_client_funcs, which handles
+ * this automatically. Setting the old callbacks should be avoided.
*
* For suspend/resume consider using drm_mode_config_helper_suspend() and
* drm_mode_config_helper_resume() which takes care of fbdev as well.
@@ -368,115 +363,30 @@ static void drm_fb_helper_resume_worker(struct work_struct *work)
resume_work);
console_lock();
- fb_set_suspend(helper->fbdev, 0);
+ fb_set_suspend(helper->info, 0);
console_unlock();
}
-static void drm_fb_helper_damage_blit_real(struct drm_fb_helper *fb_helper,
- struct drm_clip_rect *clip,
- struct iosys_map *dst)
+static void drm_fb_helper_fb_dirty(struct drm_fb_helper *helper)
{
- struct drm_framebuffer *fb = fb_helper->fb;
- size_t offset = clip->y1 * fb->pitches[0];
- size_t len = clip->x2 - clip->x1;
- unsigned int y;
- void *src;
-
- switch (drm_format_info_bpp(fb->format, 0)) {
- case 1:
- offset += clip->x1 / 8;
- len = DIV_ROUND_UP(len + clip->x1 % 8, 8);
- break;
- case 2:
- offset += clip->x1 / 4;
- len = DIV_ROUND_UP(len + clip->x1 % 4, 4);
- break;
- case 4:
- offset += clip->x1 / 2;
- len = DIV_ROUND_UP(len + clip->x1 % 2, 2);
- break;
- default:
- offset += clip->x1 * fb->format->cpp[0];
- len *= fb->format->cpp[0];
- break;
- }
-
- src = fb_helper->fbdev->screen_buffer + offset;
- iosys_map_incr(dst, offset); /* go to first pixel within clip rect */
-
- for (y = clip->y1; y < clip->y2; y++) {
- iosys_map_memcpy_to(dst, 0, src, len);
- iosys_map_incr(dst, fb->pitches[0]);
- src += fb->pitches[0];
- }
-}
-
-static int drm_fb_helper_damage_blit(struct drm_fb_helper *fb_helper,
- struct drm_clip_rect *clip)
-{
- struct drm_client_buffer *buffer = fb_helper->buffer;
- struct iosys_map map, dst;
- int ret;
-
- /*
- * We have to pin the client buffer to its current location while
- * flushing the shadow buffer. In the general case, concurrent
- * modesetting operations could try to move the buffer and would
- * fail. The modeset has to be serialized by acquiring the reservation
- * object of the underlying BO here.
- *
- * For fbdev emulation, we only have to protect against fbdev modeset
- * operations. Nothing else will involve the client buffer's BO. So it
- * is sufficient to acquire struct drm_fb_helper.lock here.
- */
- mutex_lock(&fb_helper->lock);
-
- ret = drm_client_buffer_vmap(buffer, &map);
- if (ret)
- goto out;
-
- dst = map;
- drm_fb_helper_damage_blit_real(fb_helper, clip, &dst);
-
- drm_client_buffer_vunmap(buffer);
-
-out:
- mutex_unlock(&fb_helper->lock);
-
- return ret;
-}
-
-static void drm_fb_helper_damage_work(struct work_struct *work)
-{
- struct drm_fb_helper *helper = container_of(work, struct drm_fb_helper,
- damage_work);
struct drm_device *dev = helper->dev;
struct drm_clip_rect *clip = &helper->damage_clip;
struct drm_clip_rect clip_copy;
unsigned long flags;
int ret;
+ if (drm_WARN_ON_ONCE(dev, !helper->funcs->fb_dirty))
+ return;
+
spin_lock_irqsave(&helper->damage_lock, flags);
clip_copy = *clip;
clip->x1 = clip->y1 = ~0;
clip->x2 = clip->y2 = 0;
spin_unlock_irqrestore(&helper->damage_lock, flags);
- /* Call damage handlers only if necessary */
- if (!(clip_copy.x1 < clip_copy.x2 && clip_copy.y1 < clip_copy.y2))
- return;
-
- if (helper->buffer) {
- ret = drm_fb_helper_damage_blit(helper, &clip_copy);
- if (drm_WARN_ONCE(dev, ret, "Damage blitter failed: ret=%d\n", ret))
- goto err;
- }
-
- if (helper->fb->funcs->dirty) {
- ret = helper->fb->funcs->dirty(helper->fb, NULL, 0, 0, &clip_copy, 1);
- if (drm_WARN_ONCE(dev, ret, "Dirty helper failed: ret=%d\n", ret))
- goto err;
- }
+ ret = helper->funcs->fb_dirty(helper, &clip_copy);
+ if (ret)
+ goto err;
return;
@@ -493,6 +403,13 @@ static void drm_fb_helper_damage_work(struct work_struct *work)
spin_unlock_irqrestore(&helper->damage_lock, flags);
}
+static void drm_fb_helper_damage_work(struct work_struct *work)
+{
+ struct drm_fb_helper *helper = container_of(work, struct drm_fb_helper, damage_work);
+
+ drm_fb_helper_fb_dirty(helper);
+}
+
/**
* drm_fb_helper_prepare - setup a drm_fb_helper structure
* @dev: DRM device
@@ -536,11 +453,6 @@ int drm_fb_helper_init(struct drm_device *dev,
{
int ret;
- if (!drm_fbdev_emulation) {
- dev->fb_helper = fb_helper;
- return 0;
- }
-
/*
* If this is not the generic fbdev client, initialize a drm_client
* without callbacks so we can use the modesets.
@@ -558,7 +470,7 @@ int drm_fb_helper_init(struct drm_device *dev,
EXPORT_SYMBOL(drm_fb_helper_init);
/**
- * drm_fb_helper_alloc_fbi - allocate fb_info and some of its members
+ * drm_fb_helper_alloc_info - allocate fb_info and some of its members
* @fb_helper: driver-allocated fbdev helper
*
* A helper to alloc fb_info and the members cmap and apertures. Called
@@ -570,7 +482,7 @@ EXPORT_SYMBOL(drm_fb_helper_init);
* fb_info pointer if things went okay, pointer containing error code
* otherwise
*/
-struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
+struct fb_info *drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper)
{
struct device *dev = fb_helper->dev->dev;
struct fb_info *info;
@@ -598,7 +510,7 @@ struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
goto err_free_cmap;
}
- fb_helper->fbdev = info;
+ fb_helper->info = info;
info->skip_vt_switch = true;
return info;
@@ -609,22 +521,22 @@ struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
framebuffer_release(info);
return ERR_PTR(ret);
}
-EXPORT_SYMBOL(drm_fb_helper_alloc_fbi);
+EXPORT_SYMBOL(drm_fb_helper_alloc_info);
/**
- * drm_fb_helper_unregister_fbi - unregister fb_info framebuffer device
+ * drm_fb_helper_unregister_info - unregister fb_info framebuffer device
* @fb_helper: driver-allocated fbdev helper, can be NULL
*
* A wrapper around unregister_framebuffer, to release the fb_info
* framebuffer device. This must be called before releasing all resources for
* @fb_helper by calling drm_fb_helper_fini().
*/
-void drm_fb_helper_unregister_fbi(struct drm_fb_helper *fb_helper)
+void drm_fb_helper_unregister_info(struct drm_fb_helper *fb_helper)
{
- if (fb_helper && fb_helper->fbdev)
- unregister_framebuffer(fb_helper->fbdev);
+ if (fb_helper && fb_helper->info)
+ unregister_framebuffer(fb_helper->info);
}
-EXPORT_SYMBOL(drm_fb_helper_unregister_fbi);
+EXPORT_SYMBOL(drm_fb_helper_unregister_info);
/**
* drm_fb_helper_fini - finialize a &struct drm_fb_helper
@@ -647,13 +559,13 @@ void drm_fb_helper_fini(struct drm_fb_helper *fb_helper)
cancel_work_sync(&fb_helper->resume_work);
cancel_work_sync(&fb_helper->damage_work);
- info = fb_helper->fbdev;
+ info = fb_helper->info;
if (info) {
if (info->cmap.len)
fb_dealloc_cmap(&info->cmap);
framebuffer_release(info);
}
- fb_helper->fbdev = NULL;
+ fb_helper->info = NULL;
mutex_lock(&kernel_fb_helper_lock);
if (!list_empty(&fb_helper->kernel_fb_list)) {
@@ -670,32 +582,24 @@ void drm_fb_helper_fini(struct drm_fb_helper *fb_helper)
}
EXPORT_SYMBOL(drm_fb_helper_fini);
-static bool drm_fbdev_use_shadow_fb(struct drm_fb_helper *fb_helper)
+static void drm_fb_helper_add_damage_clip(struct drm_fb_helper *helper, u32 x, u32 y,
+ u32 width, u32 height)
{
- struct drm_device *dev = fb_helper->dev;
- struct drm_framebuffer *fb = fb_helper->fb;
-
- return dev->mode_config.prefer_shadow_fbdev ||
- dev->mode_config.prefer_shadow ||
- fb->funcs->dirty;
-}
-
-static void drm_fb_helper_damage(struct fb_info *info, u32 x, u32 y,
- u32 width, u32 height)
-{
- struct drm_fb_helper *helper = info->par;
struct drm_clip_rect *clip = &helper->damage_clip;
unsigned long flags;
- if (!drm_fbdev_use_shadow_fb(helper))
- return;
-
spin_lock_irqsave(&helper->damage_lock, flags);
clip->x1 = min_t(u32, clip->x1, x);
clip->y1 = min_t(u32, clip->y1, y);
clip->x2 = max_t(u32, clip->x2, x + width);
clip->y2 = max_t(u32, clip->y2, y + height);
spin_unlock_irqrestore(&helper->damage_lock, flags);
+}
+
+static void drm_fb_helper_damage(struct drm_fb_helper *helper, u32 x, u32 y,
+ u32 width, u32 height)
+{
+ drm_fb_helper_add_damage_clip(helper, x, y, width, height);
schedule_work(&helper->damage_work);
}
@@ -739,6 +643,7 @@ static void drm_fb_helper_memory_range_to_clip(struct fb_info *info, off_t off,
*/
void drm_fb_helper_deferred_io(struct fb_info *info, struct list_head *pagereflist)
{
+ struct drm_fb_helper *helper = info->par;
unsigned long start, end, min_off, max_off;
struct fb_deferred_io_pageref *pageref;
struct drm_rect damage_area;
@@ -751,8 +656,6 @@ void drm_fb_helper_deferred_io(struct fb_info *info, struct list_head *pagerefli
min_off = min(min_off, start);
max_off = max(max_off, end);
}
- if (min_off >= max_off)
- return;
/*
* As we can only track pages, we might reach beyond the end
@@ -761,53 +664,160 @@ void drm_fb_helper_deferred_io(struct fb_info *info, struct list_head *pagerefli
*/
max_off = min(max_off, info->screen_size);
- drm_fb_helper_memory_range_to_clip(info, min_off, max_off - min_off, &damage_area);
- drm_fb_helper_damage(info, damage_area.x1, damage_area.y1,
- drm_rect_width(&damage_area),
- drm_rect_height(&damage_area));
+ if (min_off < max_off) {
+ drm_fb_helper_memory_range_to_clip(info, min_off, max_off - min_off, &damage_area);
+ drm_fb_helper_damage(helper, damage_area.x1, damage_area.y1,
+ drm_rect_width(&damage_area),
+ drm_rect_height(&damage_area));
+ }
}
EXPORT_SYMBOL(drm_fb_helper_deferred_io);
+typedef ssize_t (*drm_fb_helper_read_screen)(struct fb_info *info, char __user *buf,
+ size_t count, loff_t pos);
+
+static ssize_t __drm_fb_helper_read(struct fb_info *info, char __user *buf, size_t count,
+ loff_t *ppos, drm_fb_helper_read_screen read_screen)
+{
+ loff_t pos = *ppos;
+ size_t total_size;
+ ssize_t ret;
+
+ if (info->screen_size)
+ total_size = info->screen_size;
+ else
+ total_size = info->fix.smem_len;
+
+ if (pos >= total_size)
+ return 0;
+ if (count >= total_size)
+ count = total_size;
+ if (total_size - count < pos)
+ count = total_size - pos;
+
+ if (info->fbops->fb_sync)
+ info->fbops->fb_sync(info);
+
+ ret = read_screen(info, buf, count, pos);
+ if (ret > 0)
+ *ppos += ret;
+
+ return ret;
+}
+
+typedef ssize_t (*drm_fb_helper_write_screen)(struct fb_info *info, const char __user *buf,
+ size_t count, loff_t pos);
+
+static ssize_t __drm_fb_helper_write(struct fb_info *info, const char __user *buf, size_t count,
+ loff_t *ppos, drm_fb_helper_write_screen write_screen)
+{
+ loff_t pos = *ppos;
+ size_t total_size;
+ ssize_t ret;
+ int err = 0;
+
+ if (info->screen_size)
+ total_size = info->screen_size;
+ else
+ total_size = info->fix.smem_len;
+
+ if (pos > total_size)
+ return -EFBIG;
+ if (count > total_size) {
+ err = -EFBIG;
+ count = total_size;
+ }
+ if (total_size - count < pos) {
+ if (!err)
+ err = -ENOSPC;
+ count = total_size - pos;
+ }
+
+ if (info->fbops->fb_sync)
+ info->fbops->fb_sync(info);
+
+ /*
+ * Copy to framebuffer even if we already logged an error. Emulates
+ * the behavior of the original fbdev implementation.
+ */
+ ret = write_screen(info, buf, count, pos);
+ if (ret < 0)
+ return ret; /* return last error, if any */
+ else if (!ret)
+ return err; /* return previous error, if any */
+
+ *ppos += ret;
+
+ return ret;
+}
+
+static ssize_t drm_fb_helper_read_screen_buffer(struct fb_info *info, char __user *buf,
+ size_t count, loff_t pos)
+{
+ const char *src = info->screen_buffer + pos;
+
+ if (copy_to_user(buf, src, count))
+ return -EFAULT;
+
+ return count;
+}
+
/**
- * drm_fb_helper_sys_read - wrapper around fb_sys_read
+ * drm_fb_helper_sys_read - Implements struct &fb_ops.fb_read for system memory
* @info: fb_info struct pointer
* @buf: userspace buffer to read from framebuffer memory
* @count: number of bytes to read from framebuffer memory
* @ppos: read offset within framebuffer memory
*
- * A wrapper around fb_sys_read implemented by fbdev core
+ * Returns:
+ * The number of bytes read on success, or an error code otherwise.
*/
ssize_t drm_fb_helper_sys_read(struct fb_info *info, char __user *buf,
size_t count, loff_t *ppos)
{
- return fb_sys_read(info, buf, count, ppos);
+ return __drm_fb_helper_read(info, buf, count, ppos, drm_fb_helper_read_screen_buffer);
}
EXPORT_SYMBOL(drm_fb_helper_sys_read);
+static ssize_t drm_fb_helper_write_screen_buffer(struct fb_info *info, const char __user *buf,
+ size_t count, loff_t pos)
+{
+ char *dst = info->screen_buffer + pos;
+
+ if (copy_from_user(dst, buf, count))
+ return -EFAULT;
+
+ return count;
+}
+
/**
- * drm_fb_helper_sys_write - wrapper around fb_sys_write
+ * drm_fb_helper_sys_write - Implements struct &fb_ops.fb_write for system memory
* @info: fb_info struct pointer
* @buf: userspace buffer to write to framebuffer memory
* @count: number of bytes to write to framebuffer memory
* @ppos: write offset within framebuffer memory
*
- * A wrapper around fb_sys_write implemented by fbdev core
+ * Returns:
+ * The number of bytes written on success, or an error code otherwise.
*/
ssize_t drm_fb_helper_sys_write(struct fb_info *info, const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct drm_fb_helper *helper = info->par;
loff_t pos = *ppos;
ssize_t ret;
struct drm_rect damage_area;
- ret = fb_sys_write(info, buf, count, ppos);
+ ret = __drm_fb_helper_write(info, buf, count, ppos, drm_fb_helper_write_screen_buffer);
if (ret <= 0)
return ret;
- drm_fb_helper_memory_range_to_clip(info, pos, ret, &damage_area);
- drm_fb_helper_damage(info, damage_area.x1, damage_area.y1,
- drm_rect_width(&damage_area),
- drm_rect_height(&damage_area));
+ if (helper->funcs->fb_dirty) {
+ drm_fb_helper_memory_range_to_clip(info, pos, ret, &damage_area);
+ drm_fb_helper_damage(helper, damage_area.x1, damage_area.y1,
+ drm_rect_width(&damage_area),
+ drm_rect_height(&damage_area));
+ }
return ret;
}
@@ -823,8 +833,12 @@ EXPORT_SYMBOL(drm_fb_helper_sys_write);
void drm_fb_helper_sys_fillrect(struct fb_info *info,
const struct fb_fillrect *rect)
{
+ struct drm_fb_helper *helper = info->par;
+
sys_fillrect(info, rect);
- drm_fb_helper_damage(info, rect->dx, rect->dy, rect->width, rect->height);
+
+ if (helper->funcs->fb_dirty)
+ drm_fb_helper_damage(helper, rect->dx, rect->dy, rect->width, rect->height);
}
EXPORT_SYMBOL(drm_fb_helper_sys_fillrect);
@@ -838,8 +852,12 @@ EXPORT_SYMBOL(drm_fb_helper_sys_fillrect);
void drm_fb_helper_sys_copyarea(struct fb_info *info,
const struct fb_copyarea *area)
{
+ struct drm_fb_helper *helper = info->par;
+
sys_copyarea(info, area);
- drm_fb_helper_damage(info, area->dx, area->dy, area->width, area->height);
+
+ if (helper->funcs->fb_dirty)
+ drm_fb_helper_damage(helper, area->dx, area->dy, area->width, area->height);
}
EXPORT_SYMBOL(drm_fb_helper_sys_copyarea);
@@ -853,11 +871,131 @@ EXPORT_SYMBOL(drm_fb_helper_sys_copyarea);
void drm_fb_helper_sys_imageblit(struct fb_info *info,
const struct fb_image *image)
{
+ struct drm_fb_helper *helper = info->par;
+
sys_imageblit(info, image);
- drm_fb_helper_damage(info, image->dx, image->dy, image->width, image->height);
+
+ if (helper->funcs->fb_dirty)
+ drm_fb_helper_damage(helper, image->dx, image->dy, image->width, image->height);
}
EXPORT_SYMBOL(drm_fb_helper_sys_imageblit);
+static ssize_t fb_read_screen_base(struct fb_info *info, char __user *buf, size_t count,
+ loff_t pos)
+{
+ const char __iomem *src = info->screen_base + pos;
+ size_t alloc_size = min_t(size_t, count, PAGE_SIZE);
+ ssize_t ret = 0;
+ int err = 0;
+ char *tmp;
+
+ tmp = kmalloc(alloc_size, GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ while (count) {
+ size_t c = min_t(size_t, count, alloc_size);
+
+ memcpy_fromio(tmp, src, c);
+ if (copy_to_user(buf, tmp, c)) {
+ err = -EFAULT;
+ break;
+ }
+
+ src += c;
+ buf += c;
+ ret += c;
+ count -= c;
+ }
+
+ kfree(tmp);
+
+ return ret ? ret : err;
+}
+
+/**
+ * drm_fb_helper_cfb_read - Implements struct &fb_ops.fb_read for I/O memory
+ * @info: fb_info struct pointer
+ * @buf: userspace buffer to read from framebuffer memory
+ * @count: number of bytes to read from framebuffer memory
+ * @ppos: read offset within framebuffer memory
+ *
+ * Returns:
+ * The number of bytes read on success, or an error code otherwise.
+ */
+ssize_t drm_fb_helper_cfb_read(struct fb_info *info, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ return __drm_fb_helper_read(info, buf, count, ppos, fb_read_screen_base);
+}
+EXPORT_SYMBOL(drm_fb_helper_cfb_read);
+
+static ssize_t fb_write_screen_base(struct fb_info *info, const char __user *buf, size_t count,
+ loff_t pos)
+{
+ char __iomem *dst = info->screen_base + pos;
+ size_t alloc_size = min_t(size_t, count, PAGE_SIZE);
+ ssize_t ret = 0;
+ int err = 0;
+ u8 *tmp;
+
+ tmp = kmalloc(alloc_size, GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ while (count) {
+ size_t c = min_t(size_t, count, alloc_size);
+
+ if (copy_from_user(tmp, buf, c)) {
+ err = -EFAULT;
+ break;
+ }
+ memcpy_toio(dst, tmp, c);
+
+ dst += c;
+ buf += c;
+ ret += c;
+ count -= c;
+ }
+
+ kfree(tmp);
+
+ return ret ? ret : err;
+}
+
+/**
+ * drm_fb_helper_cfb_write - Implements struct &fb_ops.fb_write for I/O memory
+ * @info: fb_info struct pointer
+ * @buf: userspace buffer to write to framebuffer memory
+ * @count: number of bytes to write to framebuffer memory
+ * @ppos: write offset within framebuffer memory
+ *
+ * Returns:
+ * The number of bytes written on success, or an error code otherwise.
+ */
+ssize_t drm_fb_helper_cfb_write(struct fb_info *info, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ struct drm_fb_helper *helper = info->par;
+ loff_t pos = *ppos;
+ ssize_t ret;
+ struct drm_rect damage_area;
+
+ ret = __drm_fb_helper_write(info, buf, count, ppos, fb_write_screen_base);
+ if (ret <= 0)
+ return ret;
+
+ if (helper->funcs->fb_dirty) {
+ drm_fb_helper_memory_range_to_clip(info, pos, ret, &damage_area);
+ drm_fb_helper_damage(helper, damage_area.x1, damage_area.y1,
+ drm_rect_width(&damage_area),
+ drm_rect_height(&damage_area));
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL(drm_fb_helper_cfb_write);
+
/**
* drm_fb_helper_cfb_fillrect - wrapper around cfb_fillrect
* @info: fbdev registered by the helper
@@ -868,8 +1006,12 @@ EXPORT_SYMBOL(drm_fb_helper_sys_imageblit);
void drm_fb_helper_cfb_fillrect(struct fb_info *info,
const struct fb_fillrect *rect)
{
+ struct drm_fb_helper *helper = info->par;
+
cfb_fillrect(info, rect);
- drm_fb_helper_damage(info, rect->dx, rect->dy, rect->width, rect->height);
+
+ if (helper->funcs->fb_dirty)
+ drm_fb_helper_damage(helper, rect->dx, rect->dy, rect->width, rect->height);
}
EXPORT_SYMBOL(drm_fb_helper_cfb_fillrect);
@@ -883,8 +1025,12 @@ EXPORT_SYMBOL(drm_fb_helper_cfb_fillrect);
void drm_fb_helper_cfb_copyarea(struct fb_info *info,
const struct fb_copyarea *area)
{
+ struct drm_fb_helper *helper = info->par;
+
cfb_copyarea(info, area);
- drm_fb_helper_damage(info, area->dx, area->dy, area->width, area->height);
+
+ if (helper->funcs->fb_dirty)
+ drm_fb_helper_damage(helper, area->dx, area->dy, area->width, area->height);
}
EXPORT_SYMBOL(drm_fb_helper_cfb_copyarea);
@@ -898,8 +1044,12 @@ EXPORT_SYMBOL(drm_fb_helper_cfb_copyarea);
void drm_fb_helper_cfb_imageblit(struct fb_info *info,
const struct fb_image *image)
{
+ struct drm_fb_helper *helper = info->par;
+
cfb_imageblit(info, image);
- drm_fb_helper_damage(info, image->dx, image->dy, image->width, image->height);
+
+ if (helper->funcs->fb_dirty)
+ drm_fb_helper_damage(helper, image->dx, image->dy, image->width, image->height);
}
EXPORT_SYMBOL(drm_fb_helper_cfb_imageblit);
@@ -914,8 +1064,8 @@ EXPORT_SYMBOL(drm_fb_helper_cfb_imageblit);
*/
void drm_fb_helper_set_suspend(struct drm_fb_helper *fb_helper, bool suspend)
{
- if (fb_helper && fb_helper->fbdev)
- fb_set_suspend(fb_helper->fbdev, suspend);
+ if (fb_helper && fb_helper->info)
+ fb_set_suspend(fb_helper->info, suspend);
}
EXPORT_SYMBOL(drm_fb_helper_set_suspend);
@@ -938,20 +1088,20 @@ EXPORT_SYMBOL(drm_fb_helper_set_suspend);
void drm_fb_helper_set_suspend_unlocked(struct drm_fb_helper *fb_helper,
bool suspend)
{
- if (!fb_helper || !fb_helper->fbdev)
+ if (!fb_helper || !fb_helper->info)
return;
/* make sure there's no pending/ongoing resume */
flush_work(&fb_helper->resume_work);
if (suspend) {
- if (fb_helper->fbdev->state != FBINFO_STATE_RUNNING)
+ if (fb_helper->info->state != FBINFO_STATE_RUNNING)
return;
console_lock();
} else {
- if (fb_helper->fbdev->state == FBINFO_STATE_RUNNING)
+ if (fb_helper->info->state == FBINFO_STATE_RUNNING)
return;
if (!console_trylock()) {
@@ -960,7 +1110,7 @@ void drm_fb_helper_set_suspend_unlocked(struct drm_fb_helper *fb_helper,
}
}
- fb_set_suspend(fb_helper->fbdev, suspend);
+ fb_set_suspend(fb_helper->info, suspend);
console_unlock();
}
EXPORT_SYMBOL(drm_fb_helper_set_suspend_unlocked);
@@ -1749,6 +1899,10 @@ static int drm_fb_helper_single_fb_probe(struct drm_fb_helper *fb_helper,
sizes.surface_height = config->max_height;
}
+#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM)
+ fb_helper->hint_leak_smem_start = drm_leak_fbdev_smem;
+#endif
+
/* push down into drivers */
ret = (*fb_helper->funcs->fb_probe)(fb_helper, &sizes);
if (ret < 0)
@@ -1850,7 +2004,7 @@ EXPORT_SYMBOL(drm_fb_helper_fill_info);
/*
* This is a continuation of drm_setup_crtcs() that sets up anything related
* to the framebuffer. During initialization, drm_setup_crtcs() is called before
- * the framebuffer has been allocated (fb_helper->fb and fb_helper->fbdev).
+ * the framebuffer has been allocated (fb_helper->fb and fb_helper->info).
* So, any setup that touches those fields needs to be done here instead of in
* drm_setup_crtcs().
*/
@@ -1858,7 +2012,7 @@ static void drm_setup_crtcs_fb(struct drm_fb_helper *fb_helper)
{
struct drm_client_dev *client = &fb_helper->client;
struct drm_connector_list_iter conn_iter;
- struct fb_info *info = fb_helper->fbdev;
+ struct fb_info *info = fb_helper->info;
unsigned int rotation, sw_rotations = 0;
struct drm_connector *connector;
struct drm_mode_set *modeset;
@@ -1942,11 +2096,11 @@ __drm_fb_helper_initial_config_and_unlock(struct drm_fb_helper *fb_helper,
fb_helper->deferred_setup = false;
- info = fb_helper->fbdev;
+ info = fb_helper->info;
info->var.pixclock = 0;
/* Shamelessly allow physical address leaking to userspace */
#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM)
- if (!drm_leak_fbdev_smem)
+ if (!fb_helper->hint_leak_smem_start)
#endif
/* don't leak any physical addresses to userspace */
info->flags |= FBINFO_HIDE_SMEM_START;
@@ -2077,7 +2231,7 @@ int drm_fb_helper_hotplug_event(struct drm_fb_helper *fb_helper)
drm_setup_crtcs_fb(fb_helper);
mutex_unlock(&fb_helper->lock);
- drm_fb_helper_set_par(fb_helper->fbdev);
+ drm_fb_helper_set_par(fb_helper->info);
return 0;
}
@@ -2103,530 +2257,10 @@ EXPORT_SYMBOL(drm_fb_helper_lastclose);
*
* This function can be used as the
* &drm_mode_config_funcs.output_poll_changed callback for drivers that only
- * need to call drm_fb_helper_hotplug_event().
+ * need to call drm_fbdev.hotplug_event().
*/
void drm_fb_helper_output_poll_changed(struct drm_device *dev)
{
drm_fb_helper_hotplug_event(dev->fb_helper);
}
EXPORT_SYMBOL(drm_fb_helper_output_poll_changed);
-
-/* @user: 1=userspace, 0=fbcon */
-static int drm_fbdev_fb_open(struct fb_info *info, int user)
-{
- struct drm_fb_helper *fb_helper = info->par;
-
- /* No need to take a ref for fbcon because it unbinds on unregister */
- if (user && !try_module_get(fb_helper->dev->driver->fops->owner))
- return -ENODEV;
-
- return 0;
-}
-
-static int drm_fbdev_fb_release(struct fb_info *info, int user)
-{
- struct drm_fb_helper *fb_helper = info->par;
-
- if (user)
- module_put(fb_helper->dev->driver->fops->owner);
-
- return 0;
-}
-
-static void drm_fbdev_cleanup(struct drm_fb_helper *fb_helper)
-{
- struct fb_info *fbi = fb_helper->fbdev;
- void *shadow = NULL;
-
- if (!fb_helper->dev)
- return;
-
- if (fbi) {
- if (fbi->fbdefio)
- fb_deferred_io_cleanup(fbi);
- if (drm_fbdev_use_shadow_fb(fb_helper))
- shadow = fbi->screen_buffer;
- }
-
- drm_fb_helper_fini(fb_helper);
-
- if (shadow)
- vfree(shadow);
- else if (fb_helper->buffer)
- drm_client_buffer_vunmap(fb_helper->buffer);
-
- drm_client_framebuffer_delete(fb_helper->buffer);
-}
-
-static void drm_fbdev_release(struct drm_fb_helper *fb_helper)
-{
- drm_fbdev_cleanup(fb_helper);
- drm_client_release(&fb_helper->client);
- kfree(fb_helper);
-}
-
-/*
- * fb_ops.fb_destroy is called by the last put_fb_info() call at the end of
- * unregister_framebuffer() or fb_release().
- */
-static void drm_fbdev_fb_destroy(struct fb_info *info)
-{
- drm_fbdev_release(info->par);
-}
-
-static int drm_fbdev_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
-{
- struct drm_fb_helper *fb_helper = info->par;
-
- if (drm_fbdev_use_shadow_fb(fb_helper))
- return fb_deferred_io_mmap(info, vma);
- else if (fb_helper->dev->driver->gem_prime_mmap)
- return fb_helper->dev->driver->gem_prime_mmap(fb_helper->buffer->gem, vma);
- else
- return -ENODEV;
-}
-
-static bool drm_fbdev_use_iomem(struct fb_info *info)
-{
- struct drm_fb_helper *fb_helper = info->par;
- struct drm_client_buffer *buffer = fb_helper->buffer;
-
- return !drm_fbdev_use_shadow_fb(fb_helper) && buffer->map.is_iomem;
-}
-
-static ssize_t fb_read_screen_base(struct fb_info *info, char __user *buf, size_t count,
- loff_t pos)
-{
- const char __iomem *src = info->screen_base + pos;
- size_t alloc_size = min_t(size_t, count, PAGE_SIZE);
- ssize_t ret = 0;
- int err = 0;
- char *tmp;
-
- tmp = kmalloc(alloc_size, GFP_KERNEL);
- if (!tmp)
- return -ENOMEM;
-
- while (count) {
- size_t c = min_t(size_t, count, alloc_size);
-
- memcpy_fromio(tmp, src, c);
- if (copy_to_user(buf, tmp, c)) {
- err = -EFAULT;
- break;
- }
-
- src += c;
- buf += c;
- ret += c;
- count -= c;
- }
-
- kfree(tmp);
-
- return ret ? ret : err;
-}
-
-static ssize_t fb_read_screen_buffer(struct fb_info *info, char __user *buf, size_t count,
- loff_t pos)
-{
- const char *src = info->screen_buffer + pos;
-
- if (copy_to_user(buf, src, count))
- return -EFAULT;
-
- return count;
-}
-
-static ssize_t drm_fbdev_fb_read(struct fb_info *info, char __user *buf,
- size_t count, loff_t *ppos)
-{
- loff_t pos = *ppos;
- size_t total_size;
- ssize_t ret;
-
- if (info->screen_size)
- total_size = info->screen_size;
- else
- total_size = info->fix.smem_len;
-
- if (pos >= total_size)
- return 0;
- if (count >= total_size)
- count = total_size;
- if (total_size - count < pos)
- count = total_size - pos;
-
- if (drm_fbdev_use_iomem(info))
- ret = fb_read_screen_base(info, buf, count, pos);
- else
- ret = fb_read_screen_buffer(info, buf, count, pos);
-
- if (ret > 0)
- *ppos += ret;
-
- return ret;
-}
-
-static ssize_t fb_write_screen_base(struct fb_info *info, const char __user *buf, size_t count,
- loff_t pos)
-{
- char __iomem *dst = info->screen_base + pos;
- size_t alloc_size = min_t(size_t, count, PAGE_SIZE);
- ssize_t ret = 0;
- int err = 0;
- u8 *tmp;
-
- tmp = kmalloc(alloc_size, GFP_KERNEL);
- if (!tmp)
- return -ENOMEM;
-
- while (count) {
- size_t c = min_t(size_t, count, alloc_size);
-
- if (copy_from_user(tmp, buf, c)) {
- err = -EFAULT;
- break;
- }
- memcpy_toio(dst, tmp, c);
-
- dst += c;
- buf += c;
- ret += c;
- count -= c;
- }
-
- kfree(tmp);
-
- return ret ? ret : err;
-}
-
-static ssize_t fb_write_screen_buffer(struct fb_info *info, const char __user *buf, size_t count,
- loff_t pos)
-{
- char *dst = info->screen_buffer + pos;
-
- if (copy_from_user(dst, buf, count))
- return -EFAULT;
-
- return count;
-}
-
-static ssize_t drm_fbdev_fb_write(struct fb_info *info, const char __user *buf,
- size_t count, loff_t *ppos)
-{
- loff_t pos = *ppos;
- size_t total_size;
- ssize_t ret;
- struct drm_rect damage_area;
- int err = 0;
-
- if (info->screen_size)
- total_size = info->screen_size;
- else
- total_size = info->fix.smem_len;
-
- if (pos > total_size)
- return -EFBIG;
- if (count > total_size) {
- err = -EFBIG;
- count = total_size;
- }
- if (total_size - count < pos) {
- if (!err)
- err = -ENOSPC;
- count = total_size - pos;
- }
-
- /*
- * Copy to framebuffer even if we already logged an error. Emulates
- * the behavior of the original fbdev implementation.
- */
- if (drm_fbdev_use_iomem(info))
- ret = fb_write_screen_base(info, buf, count, pos);
- else
- ret = fb_write_screen_buffer(info, buf, count, pos);
-
- if (ret < 0)
- return ret; /* return last error, if any */
- else if (!ret)
- return err; /* return previous error, if any */
-
- *ppos += ret;
-
- drm_fb_helper_memory_range_to_clip(info, pos, ret, &damage_area);
- drm_fb_helper_damage(info, damage_area.x1, damage_area.y1,
- drm_rect_width(&damage_area),
- drm_rect_height(&damage_area));
-
- return ret;
-}
-
-static void drm_fbdev_fb_fillrect(struct fb_info *info,
- const struct fb_fillrect *rect)
-{
- if (drm_fbdev_use_iomem(info))
- drm_fb_helper_cfb_fillrect(info, rect);
- else
- drm_fb_helper_sys_fillrect(info, rect);
-}
-
-static void drm_fbdev_fb_copyarea(struct fb_info *info,
- const struct fb_copyarea *area)
-{
- if (drm_fbdev_use_iomem(info))
- drm_fb_helper_cfb_copyarea(info, area);
- else
- drm_fb_helper_sys_copyarea(info, area);
-}
-
-static void drm_fbdev_fb_imageblit(struct fb_info *info,
- const struct fb_image *image)
-{
- if (drm_fbdev_use_iomem(info))
- drm_fb_helper_cfb_imageblit(info, image);
- else
- drm_fb_helper_sys_imageblit(info, image);
-}
-
-static const struct fb_ops drm_fbdev_fb_ops = {
- .owner = THIS_MODULE,
- DRM_FB_HELPER_DEFAULT_OPS,
- .fb_open = drm_fbdev_fb_open,
- .fb_release = drm_fbdev_fb_release,
- .fb_destroy = drm_fbdev_fb_destroy,
- .fb_mmap = drm_fbdev_fb_mmap,
- .fb_read = drm_fbdev_fb_read,
- .fb_write = drm_fbdev_fb_write,
- .fb_fillrect = drm_fbdev_fb_fillrect,
- .fb_copyarea = drm_fbdev_fb_copyarea,
- .fb_imageblit = drm_fbdev_fb_imageblit,
-};
-
-static struct fb_deferred_io drm_fbdev_defio = {
- .delay = HZ / 20,
- .deferred_io = drm_fb_helper_deferred_io,
-};
-
-/*
- * This function uses the client API to create a framebuffer backed by a dumb buffer.
- *
- * The _sys_ versions are used for &fb_ops.fb_read, fb_write, fb_fillrect,
- * fb_copyarea, fb_imageblit.
- */
-static int drm_fb_helper_generic_probe(struct drm_fb_helper *fb_helper,
- struct drm_fb_helper_surface_size *sizes)
-{
- struct drm_client_dev *client = &fb_helper->client;
- struct drm_device *dev = fb_helper->dev;
- struct drm_client_buffer *buffer;
- struct drm_framebuffer *fb;
- struct fb_info *fbi;
- u32 format;
- struct iosys_map map;
- int ret;
-
- drm_dbg_kms(dev, "surface width(%d), height(%d) and bpp(%d)\n",
- sizes->surface_width, sizes->surface_height,
- sizes->surface_bpp);
-
- format = drm_mode_legacy_fb_format(sizes->surface_bpp, sizes->surface_depth);
- buffer = drm_client_framebuffer_create(client, sizes->surface_width,
- sizes->surface_height, format);
- if (IS_ERR(buffer))
- return PTR_ERR(buffer);
-
- fb_helper->buffer = buffer;
- fb_helper->fb = buffer->fb;
- fb = buffer->fb;
-
- fbi = drm_fb_helper_alloc_fbi(fb_helper);
- if (IS_ERR(fbi))
- return PTR_ERR(fbi);
-
- fbi->fbops = &drm_fbdev_fb_ops;
- fbi->screen_size = sizes->surface_height * fb->pitches[0];
- fbi->fix.smem_len = fbi->screen_size;
- fbi->flags = FBINFO_DEFAULT;
-
- drm_fb_helper_fill_info(fbi, fb_helper, sizes);
-
- if (drm_fbdev_use_shadow_fb(fb_helper)) {
- fbi->screen_buffer = vzalloc(fbi->screen_size);
- if (!fbi->screen_buffer)
- return -ENOMEM;
- fbi->flags |= FBINFO_VIRTFB | FBINFO_READS_FAST;
-
- fbi->fbdefio = &drm_fbdev_defio;
- fb_deferred_io_init(fbi);
- } else {
- /* buffer is mapped for HW framebuffer */
- ret = drm_client_buffer_vmap(fb_helper->buffer, &map);
- if (ret)
- return ret;
- if (map.is_iomem) {
- fbi->screen_base = map.vaddr_iomem;
- } else {
- fbi->screen_buffer = map.vaddr;
- fbi->flags |= FBINFO_VIRTFB;
- }
-
- /*
- * Shamelessly leak the physical address to user-space. As
- * page_to_phys() is undefined for I/O memory, warn in this
- * case.
- */
-#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM)
- if (drm_leak_fbdev_smem && fbi->fix.smem_start == 0 &&
- !drm_WARN_ON_ONCE(dev, map.is_iomem))
- fbi->fix.smem_start =
- page_to_phys(virt_to_page(fbi->screen_buffer));
-#endif
- }
-
- return 0;
-}
-
-static const struct drm_fb_helper_funcs drm_fb_helper_generic_funcs = {
- .fb_probe = drm_fb_helper_generic_probe,
-};
-
-static void drm_fbdev_client_unregister(struct drm_client_dev *client)
-{
- struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
-
- if (fb_helper->fbdev)
- /* drm_fbdev_fb_destroy() takes care of cleanup */
- drm_fb_helper_unregister_fbi(fb_helper);
- else
- drm_fbdev_release(fb_helper);
-}
-
-static int drm_fbdev_client_restore(struct drm_client_dev *client)
-{
- drm_fb_helper_lastclose(client->dev);
-
- return 0;
-}
-
-static int drm_fbdev_client_hotplug(struct drm_client_dev *client)
-{
- struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
- struct drm_device *dev = client->dev;
- int ret;
-
- /* Setup is not retried if it has failed */
- if (!fb_helper->dev && fb_helper->funcs)
- return 0;
-
- if (dev->fb_helper)
- return drm_fb_helper_hotplug_event(dev->fb_helper);
-
- if (!dev->mode_config.num_connector) {
- drm_dbg_kms(dev, "No connectors found, will not create framebuffer!\n");
- return 0;
- }
-
- drm_fb_helper_prepare(dev, fb_helper, &drm_fb_helper_generic_funcs);
-
- ret = drm_fb_helper_init(dev, fb_helper);
- if (ret)
- goto err;
-
- if (!drm_drv_uses_atomic_modeset(dev))
- drm_helper_disable_unused_functions(dev);
-
- ret = drm_fb_helper_initial_config(fb_helper, fb_helper->preferred_bpp);
- if (ret)
- goto err_cleanup;
-
- return 0;
-
-err_cleanup:
- drm_fbdev_cleanup(fb_helper);
-err:
- fb_helper->dev = NULL;
- fb_helper->fbdev = NULL;
-
- drm_err(dev, "fbdev: Failed to setup generic emulation (ret=%d)\n", ret);
-
- return ret;
-}
-
-static const struct drm_client_funcs drm_fbdev_client_funcs = {
- .owner = THIS_MODULE,
- .unregister = drm_fbdev_client_unregister,
- .restore = drm_fbdev_client_restore,
- .hotplug = drm_fbdev_client_hotplug,
-};
-
-/**
- * drm_fbdev_generic_setup() - Setup generic fbdev emulation
- * @dev: DRM device
- * @preferred_bpp: Preferred bits per pixel for the device.
- * @dev->mode_config.preferred_depth is used if this is zero.
- *
- * This function sets up generic fbdev emulation for drivers that supports
- * dumb buffers with a virtual address and that can be mmap'ed.
- * drm_fbdev_generic_setup() shall be called after the DRM driver registered
- * the new DRM device with drm_dev_register().
- *
- * Restore, hotplug events and teardown are all taken care of. Drivers that do
- * suspend/resume need to call drm_fb_helper_set_suspend_unlocked() themselves.
- * Simple drivers might use drm_mode_config_helper_suspend().
- *
- * Drivers that set the dirty callback on their framebuffer will get a shadow
- * fbdev buffer that is blitted onto the real buffer. This is done in order to
- * make deferred I/O work with all kinds of buffers. A shadow buffer can be
- * requested explicitly by setting struct drm_mode_config.prefer_shadow or
- * struct drm_mode_config.prefer_shadow_fbdev to true beforehand. This is
- * required to use generic fbdev emulation with SHMEM helpers.
- *
- * This function is safe to call even when there are no connectors present.
- * Setup will be retried on the next hotplug event.
- *
- * The fbdev is destroyed by drm_dev_unregister().
- */
-void drm_fbdev_generic_setup(struct drm_device *dev,
- unsigned int preferred_bpp)
-{
- struct drm_fb_helper *fb_helper;
- int ret;
-
- drm_WARN(dev, !dev->registered, "Device has not been registered.\n");
- drm_WARN(dev, dev->fb_helper, "fb_helper is already set!\n");
-
- if (!drm_fbdev_emulation)
- return;
-
- fb_helper = kzalloc(sizeof(*fb_helper), GFP_KERNEL);
- if (!fb_helper) {
- drm_err(dev, "Failed to allocate fb_helper\n");
- return;
- }
-
- ret = drm_client_init(dev, &fb_helper->client, "fbdev", &drm_fbdev_client_funcs);
- if (ret) {
- kfree(fb_helper);
- drm_err(dev, "Failed to register client: %d\n", ret);
- return;
- }
-
- /*
- * FIXME: This mixes up depth with bpp, which results in a glorious
- * mess, resulting in some drivers picking wrong fbdev defaults and
- * others wrong preferred_depth defaults.
- */
- if (!preferred_bpp)
- preferred_bpp = dev->mode_config.preferred_depth;
- if (!preferred_bpp)
- preferred_bpp = 32;
- fb_helper->preferred_bpp = preferred_bpp;
-
- ret = drm_fbdev_client_hotplug(&fb_helper->client);
- if (ret)
- drm_dbg_kms(dev, "client hotplug ret=%d\n", ret);
-
- drm_client_register(&fb_helper->client);
-}
-EXPORT_SYMBOL(drm_fbdev_generic_setup);
diff --git a/drivers/gpu/drm/drm_fbdev_generic.c b/drivers/gpu/drm/drm_fbdev_generic.c
new file mode 100644
index 0000000..ab86956
--- /dev/null
+++ b/drivers/gpu/drm/drm_fbdev_generic.c
@@ -0,0 +1,494 @@
+// SPDX-License-Identifier: MIT
+
+#include <linux/moduleparam.h>
+#include <linux/vmalloc.h>
+
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_fb_helper.h>
+#include <drm/drm_framebuffer.h>
+#include <drm/drm_print.h>
+
+#include <drm/drm_fbdev_generic.h>
+
+static bool drm_fbdev_use_shadow_fb(struct drm_fb_helper *fb_helper)
+{
+ struct drm_device *dev = fb_helper->dev;
+ struct drm_framebuffer *fb = fb_helper->fb;
+
+ return dev->mode_config.prefer_shadow_fbdev ||
+ dev->mode_config.prefer_shadow ||
+ fb->funcs->dirty;
+}
+
+/* @user: 1=userspace, 0=fbcon */
+static int drm_fbdev_fb_open(struct fb_info *info, int user)
+{
+ struct drm_fb_helper *fb_helper = info->par;
+
+ /* No need to take a ref for fbcon because it unbinds on unregister */
+ if (user && !try_module_get(fb_helper->dev->driver->fops->owner))
+ return -ENODEV;
+
+ return 0;
+}
+
+static int drm_fbdev_fb_release(struct fb_info *info, int user)
+{
+ struct drm_fb_helper *fb_helper = info->par;
+
+ if (user)
+ module_put(fb_helper->dev->driver->fops->owner);
+
+ return 0;
+}
+
+static void drm_fbdev_cleanup(struct drm_fb_helper *fb_helper)
+{
+ struct fb_info *fbi = fb_helper->info;
+ void *shadow = NULL;
+
+ if (!fb_helper->dev)
+ return;
+
+ if (fbi) {
+ if (fbi->fbdefio)
+ fb_deferred_io_cleanup(fbi);
+ if (drm_fbdev_use_shadow_fb(fb_helper))
+ shadow = fbi->screen_buffer;
+ }
+
+ drm_fb_helper_fini(fb_helper);
+
+ if (shadow)
+ vfree(shadow);
+ else if (fb_helper->buffer)
+ drm_client_buffer_vunmap(fb_helper->buffer);
+
+ drm_client_framebuffer_delete(fb_helper->buffer);
+}
+
+static void drm_fbdev_release(struct drm_fb_helper *fb_helper)
+{
+ drm_fbdev_cleanup(fb_helper);
+ drm_client_release(&fb_helper->client);
+ kfree(fb_helper);
+}
+
+/*
+ * fb_ops.fb_destroy is called by the last put_fb_info() call at the end of
+ * unregister_framebuffer() or fb_release().
+ */
+static void drm_fbdev_fb_destroy(struct fb_info *info)
+{
+ drm_fbdev_release(info->par);
+}
+
+static int drm_fbdev_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
+{
+ struct drm_fb_helper *fb_helper = info->par;
+
+ if (drm_fbdev_use_shadow_fb(fb_helper))
+ return fb_deferred_io_mmap(info, vma);
+ else if (fb_helper->dev->driver->gem_prime_mmap)
+ return fb_helper->dev->driver->gem_prime_mmap(fb_helper->buffer->gem, vma);
+ else
+ return -ENODEV;
+}
+
+static bool drm_fbdev_use_iomem(struct fb_info *info)
+{
+ struct drm_fb_helper *fb_helper = info->par;
+ struct drm_client_buffer *buffer = fb_helper->buffer;
+
+ return !drm_fbdev_use_shadow_fb(fb_helper) && buffer->map.is_iomem;
+}
+
+static ssize_t drm_fbdev_fb_read(struct fb_info *info, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ ssize_t ret;
+
+ if (drm_fbdev_use_iomem(info))
+ ret = drm_fb_helper_cfb_read(info, buf, count, ppos);
+ else
+ ret = drm_fb_helper_sys_read(info, buf, count, ppos);
+
+ return ret;
+}
+
+static ssize_t drm_fbdev_fb_write(struct fb_info *info, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ ssize_t ret;
+
+ if (drm_fbdev_use_iomem(info))
+ ret = drm_fb_helper_cfb_write(info, buf, count, ppos);
+ else
+ ret = drm_fb_helper_sys_write(info, buf, count, ppos);
+
+ return ret;
+}
+
+static void drm_fbdev_fb_fillrect(struct fb_info *info,
+ const struct fb_fillrect *rect)
+{
+ if (drm_fbdev_use_iomem(info))
+ drm_fb_helper_cfb_fillrect(info, rect);
+ else
+ drm_fb_helper_sys_fillrect(info, rect);
+}
+
+static void drm_fbdev_fb_copyarea(struct fb_info *info,
+ const struct fb_copyarea *area)
+{
+ if (drm_fbdev_use_iomem(info))
+ drm_fb_helper_cfb_copyarea(info, area);
+ else
+ drm_fb_helper_sys_copyarea(info, area);
+}
+
+static void drm_fbdev_fb_imageblit(struct fb_info *info,
+ const struct fb_image *image)
+{
+ if (drm_fbdev_use_iomem(info))
+ drm_fb_helper_cfb_imageblit(info, image);
+ else
+ drm_fb_helper_sys_imageblit(info, image);
+}
+
+static const struct fb_ops drm_fbdev_fb_ops = {
+ .owner = THIS_MODULE,
+ DRM_FB_HELPER_DEFAULT_OPS,
+ .fb_open = drm_fbdev_fb_open,
+ .fb_release = drm_fbdev_fb_release,
+ .fb_destroy = drm_fbdev_fb_destroy,
+ .fb_mmap = drm_fbdev_fb_mmap,
+ .fb_read = drm_fbdev_fb_read,
+ .fb_write = drm_fbdev_fb_write,
+ .fb_fillrect = drm_fbdev_fb_fillrect,
+ .fb_copyarea = drm_fbdev_fb_copyarea,
+ .fb_imageblit = drm_fbdev_fb_imageblit,
+};
+
+static struct fb_deferred_io drm_fbdev_defio = {
+ .delay = HZ / 20,
+ .deferred_io = drm_fb_helper_deferred_io,
+};
+
+/*
+ * This function uses the client API to create a framebuffer backed by a dumb buffer.
+ */
+static int drm_fbdev_fb_probe(struct drm_fb_helper *fb_helper,
+ struct drm_fb_helper_surface_size *sizes)
+{
+ struct drm_client_dev *client = &fb_helper->client;
+ struct drm_device *dev = fb_helper->dev;
+ struct drm_client_buffer *buffer;
+ struct drm_framebuffer *fb;
+ struct fb_info *fbi;
+ u32 format;
+ struct iosys_map map;
+ int ret;
+
+ drm_dbg_kms(dev, "surface width(%d), height(%d) and bpp(%d)\n",
+ sizes->surface_width, sizes->surface_height,
+ sizes->surface_bpp);
+
+ format = drm_mode_legacy_fb_format(sizes->surface_bpp, sizes->surface_depth);
+ buffer = drm_client_framebuffer_create(client, sizes->surface_width,
+ sizes->surface_height, format);
+ if (IS_ERR(buffer))
+ return PTR_ERR(buffer);
+
+ fb_helper->buffer = buffer;
+ fb_helper->fb = buffer->fb;
+ fb = buffer->fb;
+
+ fbi = drm_fb_helper_alloc_info(fb_helper);
+ if (IS_ERR(fbi))
+ return PTR_ERR(fbi);
+
+ fbi->fbops = &drm_fbdev_fb_ops;
+ fbi->screen_size = sizes->surface_height * fb->pitches[0];
+ fbi->fix.smem_len = fbi->screen_size;
+ fbi->flags = FBINFO_DEFAULT;
+
+ drm_fb_helper_fill_info(fbi, fb_helper, sizes);
+
+ if (drm_fbdev_use_shadow_fb(fb_helper)) {
+ fbi->screen_buffer = vzalloc(fbi->screen_size);
+ if (!fbi->screen_buffer)
+ return -ENOMEM;
+ fbi->flags |= FBINFO_VIRTFB | FBINFO_READS_FAST;
+
+ fbi->fbdefio = &drm_fbdev_defio;
+ fb_deferred_io_init(fbi);
+ } else {
+ /* buffer is mapped for HW framebuffer */
+ ret = drm_client_buffer_vmap(fb_helper->buffer, &map);
+ if (ret)
+ return ret;
+ if (map.is_iomem) {
+ fbi->screen_base = map.vaddr_iomem;
+ } else {
+ fbi->screen_buffer = map.vaddr;
+ fbi->flags |= FBINFO_VIRTFB;
+ }
+
+ /*
+ * Shamelessly leak the physical address to user-space. As
+ * page_to_phys() is undefined for I/O memory, warn in this
+ * case.
+ */
+#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM)
+ if (fb_helper->hint_leak_smem_start && fbi->fix.smem_start == 0 &&
+ !drm_WARN_ON_ONCE(dev, map.is_iomem))
+ fbi->fix.smem_start =
+ page_to_phys(virt_to_page(fbi->screen_buffer));
+#endif
+ }
+
+ return 0;
+}
+
+static void drm_fbdev_damage_blit_real(struct drm_fb_helper *fb_helper,
+ struct drm_clip_rect *clip,
+ struct iosys_map *dst)
+{
+ struct drm_framebuffer *fb = fb_helper->fb;
+ size_t offset = clip->y1 * fb->pitches[0];
+ size_t len = clip->x2 - clip->x1;
+ unsigned int y;
+ void *src;
+
+ switch (drm_format_info_bpp(fb->format, 0)) {
+ case 1:
+ offset += clip->x1 / 8;
+ len = DIV_ROUND_UP(len + clip->x1 % 8, 8);
+ break;
+ case 2:
+ offset += clip->x1 / 4;
+ len = DIV_ROUND_UP(len + clip->x1 % 4, 4);
+ break;
+ case 4:
+ offset += clip->x1 / 2;
+ len = DIV_ROUND_UP(len + clip->x1 % 2, 2);
+ break;
+ default:
+ offset += clip->x1 * fb->format->cpp[0];
+ len *= fb->format->cpp[0];
+ break;
+ }
+
+ src = fb_helper->info->screen_buffer + offset;
+ iosys_map_incr(dst, offset); /* go to first pixel within clip rect */
+
+ for (y = clip->y1; y < clip->y2; y++) {
+ iosys_map_memcpy_to(dst, 0, src, len);
+ iosys_map_incr(dst, fb->pitches[0]);
+ src += fb->pitches[0];
+ }
+}
+
+static int drm_fbdev_damage_blit(struct drm_fb_helper *fb_helper,
+ struct drm_clip_rect *clip)
+{
+ struct drm_client_buffer *buffer = fb_helper->buffer;
+ struct iosys_map map, dst;
+ int ret;
+
+ /*
+ * We have to pin the client buffer to its current location while
+ * flushing the shadow buffer. In the general case, concurrent
+ * modesetting operations could try to move the buffer and would
+ * fail. The modeset has to be serialized by acquiring the reservation
+ * object of the underlying BO here.
+ *
+ * For fbdev emulation, we only have to protect against fbdev modeset
+ * operations. Nothing else will involve the client buffer's BO. So it
+ * is sufficient to acquire struct drm_fb_helper.lock here.
+ */
+ mutex_lock(&fb_helper->lock);
+
+ ret = drm_client_buffer_vmap(buffer, &map);
+ if (ret)
+ goto out;
+
+ dst = map;
+ drm_fbdev_damage_blit_real(fb_helper, clip, &dst);
+
+ drm_client_buffer_vunmap(buffer);
+
+out:
+ mutex_unlock(&fb_helper->lock);
+
+ return ret;
+}
+
+static int drm_fbdev_fb_dirty(struct drm_fb_helper *helper, struct drm_clip_rect *clip)
+{
+ struct drm_device *dev = helper->dev;
+ int ret;
+
+ if (!drm_fbdev_use_shadow_fb(helper))
+ return 0;
+
+ /* Call damage handlers only if necessary */
+ if (!(clip->x1 < clip->x2 && clip->y1 < clip->y2))
+ return 0;
+
+ if (helper->buffer) {
+ ret = drm_fbdev_damage_blit(helper, clip);
+ if (drm_WARN_ONCE(dev, ret, "Damage blitter failed: ret=%d\n", ret))
+ return ret;
+ }
+
+ if (helper->fb->funcs->dirty) {
+ ret = helper->fb->funcs->dirty(helper->fb, NULL, 0, 0, clip, 1);
+ if (drm_WARN_ONCE(dev, ret, "Dirty helper failed: ret=%d\n", ret))
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct drm_fb_helper_funcs drm_fb_helper_generic_funcs = {
+ .fb_probe = drm_fbdev_fb_probe,
+ .fb_dirty = drm_fbdev_fb_dirty,
+};
+
+static void drm_fbdev_client_unregister(struct drm_client_dev *client)
+{
+ struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
+
+ if (fb_helper->info)
+ /* drm_fbdev_fb_destroy() takes care of cleanup */
+ drm_fb_helper_unregister_info(fb_helper);
+ else
+ drm_fbdev_release(fb_helper);
+}
+
+static int drm_fbdev_client_restore(struct drm_client_dev *client)
+{
+ drm_fb_helper_lastclose(client->dev);
+
+ return 0;
+}
+
+static int drm_fbdev_client_hotplug(struct drm_client_dev *client)
+{
+ struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
+ struct drm_device *dev = client->dev;
+ int ret;
+
+ /* Setup is not retried if it has failed */
+ if (!fb_helper->dev && fb_helper->funcs)
+ return 0;
+
+ if (dev->fb_helper)
+ return drm_fb_helper_hotplug_event(dev->fb_helper);
+
+ if (!dev->mode_config.num_connector) {
+ drm_dbg_kms(dev, "No connectors found, will not create framebuffer!\n");
+ return 0;
+ }
+
+ drm_fb_helper_prepare(dev, fb_helper, &drm_fb_helper_generic_funcs);
+
+ ret = drm_fb_helper_init(dev, fb_helper);
+ if (ret)
+ goto err;
+
+ if (!drm_drv_uses_atomic_modeset(dev))
+ drm_helper_disable_unused_functions(dev);
+
+ ret = drm_fb_helper_initial_config(fb_helper, fb_helper->preferred_bpp);
+ if (ret)
+ goto err_cleanup;
+
+ return 0;
+
+err_cleanup:
+ drm_fbdev_cleanup(fb_helper);
+err:
+ fb_helper->dev = NULL;
+ fb_helper->info = NULL;
+
+ drm_err(dev, "fbdev: Failed to setup generic emulation (ret=%d)\n", ret);
+
+ return ret;
+}
+
+static const struct drm_client_funcs drm_fbdev_client_funcs = {
+ .owner = THIS_MODULE,
+ .unregister = drm_fbdev_client_unregister,
+ .restore = drm_fbdev_client_restore,
+ .hotplug = drm_fbdev_client_hotplug,
+};
+
+/**
+ * drm_fbdev_generic_setup() - Setup generic fbdev emulation
+ * @dev: DRM device
+ * @preferred_bpp: Preferred bits per pixel for the device.
+ * @dev->mode_config.preferred_depth is used if this is zero.
+ *
+ * This function sets up generic fbdev emulation for drivers that supports
+ * dumb buffers with a virtual address and that can be mmap'ed.
+ * drm_fbdev_generic_setup() shall be called after the DRM driver registered
+ * the new DRM device with drm_dev_register().
+ *
+ * Restore, hotplug events and teardown are all taken care of. Drivers that do
+ * suspend/resume need to call drm_fb_helper_set_suspend_unlocked() themselves.
+ * Simple drivers might use drm_mode_config_helper_suspend().
+ *
+ * Drivers that set the dirty callback on their framebuffer will get a shadow
+ * fbdev buffer that is blitted onto the real buffer. This is done in order to
+ * make deferred I/O work with all kinds of buffers. A shadow buffer can be
+ * requested explicitly by setting struct drm_mode_config.prefer_shadow or
+ * struct drm_mode_config.prefer_shadow_fbdev to true beforehand. This is
+ * required to use generic fbdev emulation with SHMEM helpers.
+ *
+ * This function is safe to call even when there are no connectors present.
+ * Setup will be retried on the next hotplug event.
+ *
+ * The fbdev is destroyed by drm_dev_unregister().
+ */
+void drm_fbdev_generic_setup(struct drm_device *dev,
+ unsigned int preferred_bpp)
+{
+ struct drm_fb_helper *fb_helper;
+ int ret;
+
+ drm_WARN(dev, !dev->registered, "Device has not been registered.\n");
+ drm_WARN(dev, dev->fb_helper, "fb_helper is already set!\n");
+
+ fb_helper = kzalloc(sizeof(*fb_helper), GFP_KERNEL);
+ if (!fb_helper)
+ return;
+
+ ret = drm_client_init(dev, &fb_helper->client, "fbdev", &drm_fbdev_client_funcs);
+ if (ret) {
+ kfree(fb_helper);
+ drm_err(dev, "Failed to register client: %d\n", ret);
+ return;
+ }
+
+ /*
+ * FIXME: This mixes up depth with bpp, which results in a glorious
+ * mess, resulting in some drivers picking wrong fbdev defaults and
+ * others wrong preferred_depth defaults.
+ */
+ if (!preferred_bpp)
+ preferred_bpp = dev->mode_config.preferred_depth;
+ if (!preferred_bpp)
+ preferred_bpp = 32;
+ fb_helper->preferred_bpp = preferred_bpp;
+
+ ret = drm_fbdev_client_hotplug(&fb_helper->client);
+ if (ret)
+ drm_dbg_kms(dev, "client hotplug ret=%d\n", ret);
+
+ drm_client_register(&fb_helper->client);
+}
+EXPORT_SYMBOL(drm_fbdev_generic_setup);
diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
index a8b4d918..64b4a3a 100644
--- a/drivers/gpu/drm/drm_file.c
+++ b/drivers/gpu/drm/drm_file.c
@@ -326,7 +326,7 @@ static int drm_cpu_valid(void)
* Creates and initializes a drm_file structure for the file private data in \p
* filp and add it into the double linked list in \p dev.
*/
-static int drm_open_helper(struct file *filp, struct drm_minor *minor)
+int drm_open_helper(struct file *filp, struct drm_minor *minor)
{
struct drm_device *dev = minor->dev;
struct drm_file *priv;
diff --git a/drivers/gpu/drm/drm_format_helper.c b/drivers/gpu/drm/drm_format_helper.c
index 3ee59ba..74ff33c 100644
--- a/drivers/gpu/drm/drm_format_helper.c
+++ b/drivers/gpu/drm/drm_format_helper.c
@@ -660,6 +660,11 @@ int drm_fb_blit(struct iosys_map *dst, const unsigned int *dst_pitch, uint32_t d
drm_fb_xrgb8888_to_rgb565(dst, dst_pitch, src, fb, clip, false);
return 0;
}
+ } else if (dst_format == (DRM_FORMAT_RGB565 | DRM_FORMAT_BIG_ENDIAN)) {
+ if (fb_format == DRM_FORMAT_RGB565) {
+ drm_fb_swab(dst, dst_pitch, src, fb, clip, false);
+ return 0;
+ }
} else if (dst_format == DRM_FORMAT_RGB888) {
if (fb_format == DRM_FORMAT_XRGB8888) {
drm_fb_xrgb8888_to_rgb888(dst, dst_pitch, src, fb, clip);
@@ -678,6 +683,11 @@ int drm_fb_blit(struct iosys_map *dst, const unsigned int *dst_pitch, uint32_t d
drm_fb_xrgb8888_to_xrgb2101010(dst, dst_pitch, src, fb, clip);
return 0;
}
+ } else if (dst_format == DRM_FORMAT_BGRX8888) {
+ if (fb_format == DRM_FORMAT_XRGB8888) {
+ drm_fb_swab(dst, dst_pitch, src, fb, clip, false);
+ return 0;
+ }
}
drm_warn_once(fb->dev, "No conversion helper from %p4cc to %p4cc found.\n",
diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
index e09331b..6242dfb 100644
--- a/drivers/gpu/drm/drm_fourcc.c
+++ b/drivers/gpu/drm/drm_fourcc.c
@@ -297,12 +297,12 @@ const struct drm_format_info *__drm_format_info(u32 format)
.vsub = 2, .is_yuv = true },
{ .format = DRM_FORMAT_Q410, .depth = 0,
.num_planes = 3, .char_per_block = { 2, 2, 2 },
- .block_w = { 1, 1, 1 }, .block_h = { 1, 1, 1 }, .hsub = 0,
- .vsub = 0, .is_yuv = true },
+ .block_w = { 1, 1, 1 }, .block_h = { 1, 1, 1 }, .hsub = 1,
+ .vsub = 1, .is_yuv = true },
{ .format = DRM_FORMAT_Q401, .depth = 0,
.num_planes = 3, .char_per_block = { 2, 2, 2 },
- .block_w = { 1, 1, 1 }, .block_h = { 1, 1, 1 }, .hsub = 0,
- .vsub = 0, .is_yuv = true },
+ .block_w = { 1, 1, 1 }, .block_h = { 1, 1, 1 }, .hsub = 1,
+ .vsub = 1, .is_yuv = true },
{ .format = DRM_FORMAT_P030, .depth = 0, .num_planes = 2,
.char_per_block = { 4, 8, 0 }, .block_w = { 3, 3, 0 }, .block_h = { 1, 1, 0 },
.hsub = 2, .vsub = 2, .is_yuv = true},
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 8b68a3c..b8db675 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1158,6 +1158,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
{
int ret;
+ dma_resv_assert_held(obj->resv);
+
if (!obj->funcs->vmap)
return -EOPNOTSUPP;
@@ -1173,6 +1175,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
{
+ dma_resv_assert_held(obj->resv);
+
if (iosys_map_is_null(map))
return;
@@ -1184,6 +1188,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
}
EXPORT_SYMBOL(drm_gem_vunmap);
+int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+{
+ int ret;
+
+ dma_resv_lock(obj->resv, NULL);
+ ret = drm_gem_vmap(obj, map);
+ dma_resv_unlock(obj->resv);
+
+ return ret;
+}
+EXPORT_SYMBOL(drm_gem_vmap_unlocked);
+
+void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+{
+ dma_resv_lock(obj->resv, NULL);
+ drm_gem_vunmap(obj, map);
+ dma_resv_unlock(obj->resv);
+}
+EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
+
/**
* drm_gem_lock_reservations - Sets up the ww context and acquires
* the lock on an array of GEM objects.
diff --git a/drivers/gpu/drm/drm_gem_atomic_helper.c b/drivers/gpu/drm/drm_gem_atomic_helper.c
index b6a0110..e428007 100644
--- a/drivers/gpu/drm/drm_gem_atomic_helper.c
+++ b/drivers/gpu/drm/drm_gem_atomic_helper.c
@@ -360,48 +360,43 @@ void drm_gem_reset_shadow_plane(struct drm_plane *plane)
EXPORT_SYMBOL(drm_gem_reset_shadow_plane);
/**
- * drm_gem_prepare_shadow_fb - prepares shadow framebuffers
+ * drm_gem_begin_shadow_fb_access - prepares shadow framebuffers for CPU access
* @plane: the plane
* @plane_state: the plane state of type struct drm_shadow_plane_state
*
- * This function implements struct &drm_plane_helper_funcs.prepare_fb. It
+ * This function implements struct &drm_plane_helper_funcs.begin_fb_access. It
* maps all buffer objects of the plane's framebuffer into kernel address
- * space and stores them in &struct drm_shadow_plane_state.map. The
- * framebuffer will be synchronized as part of the atomic commit.
+ * space and stores them in struct &drm_shadow_plane_state.map. The first data
+ * bytes are available in struct &drm_shadow_plane_state.data.
*
- * See drm_gem_cleanup_shadow_fb() for cleanup.
+ * See drm_gem_end_shadow_fb_access() for cleanup.
*
* Returns:
* 0 on success, or a negative errno code otherwise.
*/
-int drm_gem_prepare_shadow_fb(struct drm_plane *plane, struct drm_plane_state *plane_state)
+int drm_gem_begin_shadow_fb_access(struct drm_plane *plane, struct drm_plane_state *plane_state)
{
struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(plane_state);
struct drm_framebuffer *fb = plane_state->fb;
- int ret;
if (!fb)
return 0;
- ret = drm_gem_plane_helper_prepare_fb(plane, plane_state);
- if (ret)
- return ret;
-
return drm_gem_fb_vmap(fb, shadow_plane_state->map, shadow_plane_state->data);
}
-EXPORT_SYMBOL(drm_gem_prepare_shadow_fb);
+EXPORT_SYMBOL(drm_gem_begin_shadow_fb_access);
/**
- * drm_gem_cleanup_shadow_fb - releases shadow framebuffers
+ * drm_gem_end_shadow_fb_access - releases shadow framebuffers from CPU access
* @plane: the plane
* @plane_state: the plane state of type struct drm_shadow_plane_state
*
- * This function implements struct &drm_plane_helper_funcs.cleanup_fb.
- * This function unmaps all buffer objects of the plane's framebuffer.
+ * This function implements struct &drm_plane_helper_funcs.end_fb_access. It
+ * undoes all effects of drm_gem_begin_shadow_fb_access() in reverse order.
*
- * See drm_gem_prepare_shadow_fb() for more information.
+ * See drm_gem_begin_shadow_fb_access() for more information.
*/
-void drm_gem_cleanup_shadow_fb(struct drm_plane *plane, struct drm_plane_state *plane_state)
+void drm_gem_end_shadow_fb_access(struct drm_plane *plane, struct drm_plane_state *plane_state)
{
struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(plane_state);
struct drm_framebuffer *fb = plane_state->fb;
@@ -411,46 +406,45 @@ void drm_gem_cleanup_shadow_fb(struct drm_plane *plane, struct drm_plane_state *
drm_gem_fb_vunmap(fb, shadow_plane_state->map);
}
-EXPORT_SYMBOL(drm_gem_cleanup_shadow_fb);
+EXPORT_SYMBOL(drm_gem_end_shadow_fb_access);
/**
- * drm_gem_simple_kms_prepare_shadow_fb - prepares shadow framebuffers
+ * drm_gem_simple_kms_begin_shadow_fb_access - prepares shadow framebuffers for CPU access
* @pipe: the simple display pipe
* @plane_state: the plane state of type struct drm_shadow_plane_state
*
- * This function implements struct drm_simple_display_funcs.prepare_fb. It
- * maps all buffer objects of the plane's framebuffer into kernel address
- * space and stores them in struct drm_shadow_plane_state.map. The
- * framebuffer will be synchronized as part of the atomic commit.
+ * This function implements struct drm_simple_display_funcs.begin_fb_access.
*
- * See drm_gem_simple_kms_cleanup_shadow_fb() for cleanup.
+ * See drm_gem_begin_shadow_fb_access() for details and
+ * drm_gem_simple_kms_cleanup_shadow_fb() for cleanup.
*
* Returns:
* 0 on success, or a negative errno code otherwise.
*/
-int drm_gem_simple_kms_prepare_shadow_fb(struct drm_simple_display_pipe *pipe,
- struct drm_plane_state *plane_state)
+int drm_gem_simple_kms_begin_shadow_fb_access(struct drm_simple_display_pipe *pipe,
+ struct drm_plane_state *plane_state)
{
- return drm_gem_prepare_shadow_fb(&pipe->plane, plane_state);
+ return drm_gem_begin_shadow_fb_access(&pipe->plane, plane_state);
}
-EXPORT_SYMBOL(drm_gem_simple_kms_prepare_shadow_fb);
+EXPORT_SYMBOL(drm_gem_simple_kms_begin_shadow_fb_access);
/**
- * drm_gem_simple_kms_cleanup_shadow_fb - releases shadow framebuffers
+ * drm_gem_simple_kms_end_shadow_fb_access - releases shadow framebuffers from CPU access
* @pipe: the simple display pipe
* @plane_state: the plane state of type struct drm_shadow_plane_state
*
- * This function implements struct drm_simple_display_funcs.cleanup_fb.
- * This function unmaps all buffer objects of the plane's framebuffer.
+ * This function implements struct drm_simple_display_funcs.end_fb_access.
+ * It undoes all effects of drm_gem_simple_kms_begin_shadow_fb_access() in
+ * reverse order.
*
- * See drm_gem_simple_kms_prepare_shadow_fb().
+ * See drm_gem_simple_kms_begin_shadow_fb_access().
*/
-void drm_gem_simple_kms_cleanup_shadow_fb(struct drm_simple_display_pipe *pipe,
- struct drm_plane_state *plane_state)
+void drm_gem_simple_kms_end_shadow_fb_access(struct drm_simple_display_pipe *pipe,
+ struct drm_plane_state *plane_state)
{
- drm_gem_cleanup_shadow_fb(&pipe->plane, plane_state);
+ drm_gem_end_shadow_fb_access(&pipe->plane, plane_state);
}
-EXPORT_SYMBOL(drm_gem_simple_kms_cleanup_shadow_fb);
+EXPORT_SYMBOL(drm_gem_simple_kms_end_shadow_fb_access);
/**
* drm_gem_simple_kms_reset_shadow_plane - resets a shadow-buffered plane
diff --git a/drivers/gpu/drm/drm_gem_dma_helper.c b/drivers/gpu/drm/drm_gem_dma_helper.c
index f6901ff..1e658c4 100644
--- a/drivers/gpu/drm/drm_gem_dma_helper.c
+++ b/drivers/gpu/drm/drm_gem_dma_helper.c
@@ -230,7 +230,7 @@ void drm_gem_dma_free(struct drm_gem_dma_object *dma_obj)
if (gem_obj->import_attach) {
if (dma_obj->vaddr)
- dma_buf_vunmap(gem_obj->import_attach->dmabuf, &map);
+ dma_buf_vunmap_unlocked(gem_obj->import_attach->dmabuf, &map);
drm_prime_gem_destroy(gem_obj, dma_obj->sgt);
} else if (dma_obj->vaddr) {
if (dma_obj->map_noncoherent)
@@ -581,7 +581,7 @@ drm_gem_dma_prime_import_sg_table_vmap(struct drm_device *dev,
struct iosys_map map;
int ret;
- ret = dma_buf_vmap(attach->dmabuf, &map);
+ ret = dma_buf_vmap_unlocked(attach->dmabuf, &map);
if (ret) {
DRM_ERROR("Failed to vmap PRIME buffer\n");
return ERR_PTR(ret);
@@ -589,7 +589,7 @@ drm_gem_dma_prime_import_sg_table_vmap(struct drm_device *dev,
obj = drm_gem_dma_prime_import_sg_table(dev, attach, sgt);
if (IS_ERR(obj)) {
- dma_buf_vunmap(attach->dmabuf, &map);
+ dma_buf_vunmap_unlocked(attach->dmabuf, &map);
return obj;
}
diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
index 880a497..e93533b 100644
--- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
+++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
@@ -9,7 +9,6 @@
#include <linux/module.h>
#include <drm/drm_damage_helper.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem.h>
@@ -354,7 +353,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb, struct iosys_map *map,
ret = -EINVAL;
goto err_drm_gem_vunmap;
}
- ret = drm_gem_vmap(obj, &map[i]);
+ ret = drm_gem_vmap_unlocked(obj, &map[i]);
if (ret)
goto err_drm_gem_vunmap;
}
@@ -376,7 +375,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb, struct iosys_map *map,
obj = drm_gem_fb_get_obj(fb, i);
if (!obj)
continue;
- drm_gem_vunmap(obj, &map[i]);
+ drm_gem_vunmap_unlocked(obj, &map[i]);
}
return ret;
}
@@ -403,7 +402,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb, struct iosys_map *map)
continue;
if (iosys_map_is_null(&map[i]))
continue;
- drm_gem_vunmap(obj, &map[i]);
+ drm_gem_vunmap_unlocked(obj, &map[i]);
}
}
EXPORT_SYMBOL(drm_gem_fb_vunmap);
diff --git a/drivers/gpu/drm/drm_gem_ttm_helper.c b/drivers/gpu/drm/drm_gem_ttm_helper.c
index e5fc875..d5962a3 100644
--- a/drivers/gpu/drm/drm_gem_ttm_helper.c
+++ b/drivers/gpu/drm/drm_gem_ttm_helper.c
@@ -64,13 +64,8 @@ int drm_gem_ttm_vmap(struct drm_gem_object *gem,
struct iosys_map *map)
{
struct ttm_buffer_object *bo = drm_gem_ttm_of_gem(gem);
- int ret;
- dma_resv_lock(gem->resv, NULL);
- ret = ttm_bo_vmap(bo, map);
- dma_resv_unlock(gem->resv);
-
- return ret;
+ return ttm_bo_vmap(bo, map);
}
EXPORT_SYMBOL(drm_gem_ttm_vmap);
@@ -87,9 +82,7 @@ void drm_gem_ttm_vunmap(struct drm_gem_object *gem,
{
struct ttm_buffer_object *bo = drm_gem_ttm_of_gem(gem);
- dma_resv_lock(gem->resv, NULL);
ttm_bo_vunmap(bo, map);
- dma_resv_unlock(gem->resv);
}
EXPORT_SYMBOL(drm_gem_ttm_vunmap);
diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c
index 125160b..b6c7e38 100644
--- a/drivers/gpu/drm/drm_gem_vram_helper.c
+++ b/drivers/gpu/drm/drm_gem_vram_helper.c
@@ -433,25 +433,19 @@ int drm_gem_vram_vmap(struct drm_gem_vram_object *gbo, struct iosys_map *map)
{
int ret;
- ret = ttm_bo_reserve(&gbo->bo, true, false, NULL);
- if (ret)
- return ret;
+ dma_resv_assert_held(gbo->bo.base.resv);
ret = drm_gem_vram_pin_locked(gbo, 0);
if (ret)
- goto err_ttm_bo_unreserve;
+ return ret;
ret = drm_gem_vram_kmap_locked(gbo, map);
if (ret)
goto err_drm_gem_vram_unpin_locked;
- ttm_bo_unreserve(&gbo->bo);
-
return 0;
err_drm_gem_vram_unpin_locked:
drm_gem_vram_unpin_locked(gbo);
-err_ttm_bo_unreserve:
- ttm_bo_unreserve(&gbo->bo);
return ret;
}
EXPORT_SYMBOL(drm_gem_vram_vmap);
@@ -467,16 +461,10 @@ EXPORT_SYMBOL(drm_gem_vram_vmap);
void drm_gem_vram_vunmap(struct drm_gem_vram_object *gbo,
struct iosys_map *map)
{
- int ret;
-
- ret = ttm_bo_reserve(&gbo->bo, false, false, NULL);
- if (WARN_ONCE(ret, "ttm_bo_reserve_failed(): ret=%d\n", ret))
- return;
+ dma_resv_assert_held(gbo->bo.base.resv);
drm_gem_vram_kunmap_locked(gbo, map);
drm_gem_vram_unpin_locked(gbo);
-
- ttm_bo_unreserve(&gbo->bo);
}
EXPORT_SYMBOL(drm_gem_vram_vunmap);
diff --git a/drivers/gpu/drm/drm_mipi_dsi.c b/drivers/gpu/drm/drm_mipi_dsi.c
index 3ec0274..497ef4b 100644
--- a/drivers/gpu/drm/drm_mipi_dsi.c
+++ b/drivers/gpu/drm/drm_mipi_dsi.c
@@ -606,7 +606,7 @@ int mipi_dsi_turn_on_peripheral(struct mipi_dsi_device *dsi)
EXPORT_SYMBOL(mipi_dsi_turn_on_peripheral);
/*
- * mipi_dsi_set_maximum_return_packet_size() - specify the maximum size of the
+ * mipi_dsi_set_maximum_return_packet_size() - specify the maximum size of
* the payload in a long packet transmitted from the peripheral back to the
* host processor
* @dsi: DSI peripheral device
diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
index fb3a2ad..63a8b6d 100644
--- a/drivers/gpu/drm/drm_modes.c
+++ b/drivers/gpu/drm/drm_modes.c
@@ -1750,11 +1750,78 @@ static int drm_mode_parse_cmdline_options(const char *str,
return 0;
}
-static const char * const drm_named_modes_whitelist[] = {
- "NTSC",
- "PAL",
+struct drm_named_mode {
+ const char *name;
+ unsigned int pixel_clock_khz;
+ unsigned int xres;
+ unsigned int yres;
+ unsigned int flags;
};
+#define NAMED_MODE(_name, _pclk, _x, _y, _flags) \
+ { \
+ .name = _name, \
+ .pixel_clock_khz = _pclk, \
+ .xres = _x, \
+ .yres = _y, \
+ .flags = _flags, \
+ }
+
+static const struct drm_named_mode drm_named_modes[] = {
+ NAMED_MODE("NTSC", 13500, 720, 480, DRM_MODE_FLAG_INTERLACE),
+ NAMED_MODE("PAL", 13500, 720, 576, DRM_MODE_FLAG_INTERLACE),
+};
+
+static int drm_mode_parse_cmdline_named_mode(const char *name,
+ unsigned int name_end,
+ struct drm_cmdline_mode *cmdline_mode)
+{
+ unsigned int i;
+
+ if (!name_end)
+ return 0;
+
+ /* If the name starts with a digit, it's not a named mode */
+ if (isdigit(name[0]))
+ return 0;
+
+ /*
+ * If there's an equal sign in the name, the command-line
+ * contains only an option and no mode.
+ */
+ if (strnchr(name, name_end, '='))
+ return 0;
+
+ /* The connection status extras can be set without a mode. */
+ if (name_end == 1 &&
+ (name[0] == 'd' || name[0] == 'D' || name[0] == 'e'))
+ return 0;
+
+ /*
+ * We're sure we're a named mode at this point, iterate over the
+ * list of modes we're aware of.
+ */
+ for (i = 0; i < ARRAY_SIZE(drm_named_modes); i++) {
+ const struct drm_named_mode *mode = &drm_named_modes[i];
+ int ret;
+
+ ret = str_has_prefix(name, mode->name);
+ if (ret != name_end)
+ continue;
+
+ strcpy(cmdline_mode->name, mode->name);
+ cmdline_mode->pixel_clock = mode->pixel_clock_khz;
+ cmdline_mode->xres = mode->xres;
+ cmdline_mode->yres = mode->yres;
+ cmdline_mode->interlace = !!(mode->flags & DRM_MODE_FLAG_INTERLACE);
+ cmdline_mode->specified = true;
+
+ return 1;
+ }
+
+ return -EINVAL;
+}
+
/**
* drm_mode_parse_command_line_for_connector - parse command line modeline for connector
* @mode_option: optional per connector mode option
@@ -1791,7 +1858,7 @@ bool drm_mode_parse_command_line_for_connector(const char *mode_option,
const char *bpp_ptr = NULL, *refresh_ptr = NULL, *extra_ptr = NULL;
const char *options_ptr = NULL;
char *bpp_end_ptr = NULL, *refresh_end_ptr = NULL;
- int i, len, ret;
+ int len, ret;
memset(mode, 0, sizeof(*mode));
mode->panel_orientation = DRM_MODE_PANEL_ORIENTATION_UNKNOWN;
@@ -1801,19 +1868,23 @@ bool drm_mode_parse_command_line_for_connector(const char *mode_option,
name = mode_option;
- /* Try to locate the bpp and refresh specifiers, if any */
- bpp_ptr = strchr(name, '-');
- if (bpp_ptr)
- bpp_off = bpp_ptr - name;
-
- refresh_ptr = strchr(name, '@');
- if (refresh_ptr)
- refresh_off = refresh_ptr - name;
-
/* Locate the start of named options */
options_ptr = strchr(name, ',');
if (options_ptr)
options_off = options_ptr - name;
+ else
+ options_off = strlen(name);
+
+ /* Try to locate the bpp and refresh specifiers, if any */
+ bpp_ptr = strnchr(name, options_off, '-');
+ while (bpp_ptr && !isdigit(bpp_ptr[1]))
+ bpp_ptr = strnchr(bpp_ptr + 1, options_off, '-');
+ if (bpp_ptr)
+ bpp_off = bpp_ptr - name;
+
+ refresh_ptr = strnchr(name, options_off, '@');
+ if (refresh_ptr)
+ refresh_off = refresh_ptr - name;
/* Locate the end of the name / resolution, and parse it */
if (bpp_ptr) {
@@ -1828,18 +1899,19 @@ bool drm_mode_parse_command_line_for_connector(const char *mode_option,
parse_extras = true;
}
- /* First check for a named mode */
- for (i = 0; i < ARRAY_SIZE(drm_named_modes_whitelist); i++) {
- ret = str_has_prefix(name, drm_named_modes_whitelist[i]);
- if (ret == mode_end) {
- if (refresh_ptr)
- return false; /* named + refresh is invalid */
+ if (!mode_end)
+ return false;
- strcpy(mode->name, drm_named_modes_whitelist[i]);
- mode->specified = true;
- break;
- }
- }
+ ret = drm_mode_parse_cmdline_named_mode(name, mode_end, mode);
+ if (ret < 0)
+ return false;
+
+ /*
+ * Having a mode that starts by a letter (and thus is named) and
+ * an at-sign (used to specify a refresh rate) is disallowed.
+ */
+ if (ret && refresh_ptr)
+ return false;
/* No named mode? Check for a normal mode argument, e.g. 1024x768 */
if (!mode->specified && isdigit(name[0])) {
diff --git a/drivers/gpu/drm/drm_plane_helper.c b/drivers/gpu/drm/drm_plane_helper.c
index 865bd99..ba6a913 100644
--- a/drivers/gpu/drm/drm_plane_helper.c
+++ b/drivers/gpu/drm/drm_plane_helper.c
@@ -298,7 +298,9 @@ EXPORT_SYMBOL(drm_plane_helper_destroy);
* scale and positioning are not expected to change since the plane is always
* a fullscreen scanout buffer.
*
- * This is often the case for the primary plane of simple framebuffers.
+ * This is often the case for the primary plane of simple framebuffers. See
+ * also drm_crtc_helper_atomic_check() for the respective CRTC-state check
+ * helper function.
*
* RETURNS:
* Zero on success, or an errno code otherwise.
diff --git a/drivers/gpu/drm/drm_prime.c b/drivers/gpu/drm/drm_prime.c
index eb09e86..f924b8b 100644
--- a/drivers/gpu/drm/drm_prime.c
+++ b/drivers/gpu/drm/drm_prime.c
@@ -781,6 +781,8 @@ int drm_gem_dmabuf_mmap(struct dma_buf *dma_buf, struct vm_area_struct *vma)
struct drm_gem_object *obj = dma_buf->priv;
struct drm_device *dev = obj->dev;
+ dma_resv_assert_held(dma_buf->resv);
+
if (!dev->driver->gem_prime_mmap)
return -ENOSYS;
@@ -940,7 +942,7 @@ struct drm_gem_object *drm_gem_prime_import_dev(struct drm_device *dev,
get_dma_buf(dma_buf);
- sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
+ sgt = dma_buf_map_attachment_unlocked(attach, DMA_BIDIRECTIONAL);
if (IS_ERR(sgt)) {
ret = PTR_ERR(sgt);
goto fail_detach;
@@ -958,7 +960,7 @@ struct drm_gem_object *drm_gem_prime_import_dev(struct drm_device *dev,
return obj;
fail_unmap:
- dma_buf_unmap_attachment(attach, sgt, DMA_BIDIRECTIONAL);
+ dma_buf_unmap_attachment_unlocked(attach, sgt, DMA_BIDIRECTIONAL);
fail_detach:
dma_buf_detach(dma_buf, attach);
dma_buf_put(dma_buf);
@@ -1056,7 +1058,7 @@ void drm_prime_gem_destroy(struct drm_gem_object *obj, struct sg_table *sg)
attach = obj->import_attach;
if (sg)
- dma_buf_unmap_attachment(attach, sg, DMA_BIDIRECTIONAL);
+ dma_buf_unmap_attachment_unlocked(attach, sg, DMA_BIDIRECTIONAL);
dma_buf = attach->dmabuf;
dma_buf_detach(attach->dmabuf, attach);
/* remove the reference */
diff --git a/drivers/gpu/drm/drm_probe_helper.c b/drivers/gpu/drm/drm_probe_helper.c
index 69b0b2b..bcd9611 100644
--- a/drivers/gpu/drm/drm_probe_helper.c
+++ b/drivers/gpu/drm/drm_probe_helper.c
@@ -36,7 +36,6 @@
#include <drm/drm_client.h>
#include <drm/drm_crtc.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_modeset_helper_vtables.h>
#include <drm/drm_print.h>
@@ -367,7 +366,7 @@ static int drm_helper_probe_get_modes(struct drm_connector *connector)
* override/firmware EDID.
*/
if (count == 0 && connector->status == connector_status_connected)
- count = drm_add_override_edid_modes(connector);
+ count = drm_edid_override_connector_update(connector);
return count;
}
diff --git a/drivers/gpu/drm/drm_rect.c b/drivers/gpu/drm/drm_rect.c
index 0460e87..85c79a3 100644
--- a/drivers/gpu/drm/drm_rect.c
+++ b/drivers/gpu/drm/drm_rect.c
@@ -80,7 +80,7 @@ static u32 clip_scaled(int src, int dst, int *clip)
* @dst: destination window rectangle
* @clip: clip rectangle
*
- * Clip rectangle @dst by rectangle @clip. Clip rectangle @src by the
+ * Clip rectangle @dst by rectangle @clip. Clip rectangle @src by
* the corresponding amounts, retaining the vertical and horizontal scaling
* factors from @src to @dst.
*
diff --git a/drivers/gpu/drm/drm_simple_kms_helper.c b/drivers/gpu/drm/drm_simple_kms_helper.c
index e9f7821..3ef420e 100644
--- a/drivers/gpu/drm/drm_simple_kms_helper.c
+++ b/drivers/gpu/drm/drm_simple_kms_helper.c
@@ -102,10 +102,14 @@ static int drm_simple_kms_crtc_check(struct drm_crtc *crtc,
struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
int ret;
- ret = drm_atomic_helper_check_crtc_state(crtc_state, false);
+ if (!crtc_state->enable)
+ goto out;
+
+ ret = drm_atomic_helper_check_crtc_primary_plane(crtc_state);
if (ret)
return ret;
+out:
return drm_atomic_add_affected_planes(state, crtc);
}
@@ -281,6 +285,30 @@ static void drm_simple_kms_plane_cleanup_fb(struct drm_plane *plane,
pipe->funcs->cleanup_fb(pipe, state);
}
+static int drm_simple_kms_plane_begin_fb_access(struct drm_plane *plane,
+ struct drm_plane_state *new_plane_state)
+{
+ struct drm_simple_display_pipe *pipe;
+
+ pipe = container_of(plane, struct drm_simple_display_pipe, plane);
+ if (!pipe->funcs || !pipe->funcs->begin_fb_access)
+ return 0;
+
+ return pipe->funcs->begin_fb_access(pipe, new_plane_state);
+}
+
+static void drm_simple_kms_plane_end_fb_access(struct drm_plane *plane,
+ struct drm_plane_state *new_plane_state)
+{
+ struct drm_simple_display_pipe *pipe;
+
+ pipe = container_of(plane, struct drm_simple_display_pipe, plane);
+ if (!pipe->funcs || !pipe->funcs->end_fb_access)
+ return;
+
+ pipe->funcs->end_fb_access(pipe, new_plane_state);
+}
+
static bool drm_simple_kms_format_mod_supported(struct drm_plane *plane,
uint32_t format,
uint64_t modifier)
@@ -291,6 +319,8 @@ static bool drm_simple_kms_format_mod_supported(struct drm_plane *plane,
static const struct drm_plane_helper_funcs drm_simple_kms_plane_helper_funcs = {
.prepare_fb = drm_simple_kms_plane_prepare_fb,
.cleanup_fb = drm_simple_kms_plane_cleanup_fb,
+ .begin_fb_access = drm_simple_kms_plane_begin_fb_access,
+ .end_fb_access = drm_simple_kms_plane_end_fb_access,
.atomic_check = drm_simple_kms_plane_atomic_check,
.atomic_update = drm_simple_kms_plane_atomic_update,
};
diff --git a/drivers/gpu/drm/drm_sysfs.c b/drivers/gpu/drm/drm_sysfs.c
index 430e00b..b8da978 100644
--- a/drivers/gpu/drm/drm_sysfs.c
+++ b/drivers/gpu/drm/drm_sysfs.c
@@ -19,6 +19,7 @@
#include <linux/kdev_t.h>
#include <linux/slab.h>
+#include <drm/drm_accel.h>
#include <drm/drm_connector.h>
#include <drm/drm_device.h>
#include <drm/drm_file.h>
@@ -471,19 +472,26 @@ struct device *drm_sysfs_minor_alloc(struct drm_minor *minor)
struct device *kdev;
int r;
- if (minor->type == DRM_MINOR_RENDER)
- minor_str = "renderD%d";
- else
- minor_str = "card%d";
-
kdev = kzalloc(sizeof(*kdev), GFP_KERNEL);
if (!kdev)
return ERR_PTR(-ENOMEM);
device_initialize(kdev);
- kdev->devt = MKDEV(DRM_MAJOR, minor->index);
- kdev->class = drm_class;
- kdev->type = &drm_sysfs_device_minor;
+
+ if (minor->type == DRM_MINOR_ACCEL) {
+ minor_str = "accel%d";
+ accel_set_device_instance_params(kdev, minor->index);
+ } else {
+ if (minor->type == DRM_MINOR_RENDER)
+ minor_str = "renderD%d";
+ else
+ minor_str = "card%d";
+
+ kdev->devt = MKDEV(DRM_MAJOR, minor->index);
+ kdev->class = drm_class;
+ kdev->type = &drm_sysfs_device_minor;
+ }
+
kdev->parent = minor->dev->dev;
kdev->release = drm_sysfs_release;
dev_set_drvdata(kdev, minor);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.h b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
index f32f477..2bb4c25 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
@@ -6,13 +6,14 @@
#ifndef __ETNAVIV_DRV_H__
#define __ETNAVIV_DRV_H__
+#include <linux/io.h>
#include <linux/list.h>
#include <linux/mm_types.h>
#include <linux/sizes.h>
#include <linux/time64.h>
#include <linux/types.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_drv.h>
#include <drm/drm_gem.h>
#include <drm/etnaviv_drm.h>
#include <drm/gpu_scheduler.h>
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_dump.c b/drivers/gpu/drm/etnaviv/etnaviv_dump.c
index f418e0b..44b5f3c 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_dump.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_dump.c
@@ -83,10 +83,15 @@ static void etnaviv_core_dump_registers(struct core_dump_iterator *iter,
{
struct etnaviv_dump_registers *reg = iter->data;
unsigned int i;
+ u32 read_addr;
for (i = 0; i < ARRAY_SIZE(etnaviv_dump_registers); i++, reg++) {
+ read_addr = etnaviv_dump_registers[i];
+ if (read_addr >= VIVS_PM_POWER_CONTROLS &&
+ read_addr <= VIVS_PM_PULSE_EATER)
+ read_addr = gpu_fix_power_address(gpu, read_addr);
reg->reg = cpu_to_le32(etnaviv_dump_registers[i]);
- reg->value = cpu_to_le32(gpu_read(gpu, etnaviv_dump_registers[i]));
+ reg->value = cpu_to_le32(gpu_read(gpu, read_addr));
}
etnaviv_core_dump_header(iter, ETDUMP_BUF_REG, reg);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
index cc386f8..68e4446 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
@@ -130,7 +130,7 @@ static int etnaviv_gem_mmap_obj(struct etnaviv_gem_object *etnaviv_obj,
{
pgprot_t vm_page_prot;
- vma->vm_flags |= VM_IO | VM_MIXEDMAP | VM_DONTEXPAND | VM_DONTDUMP;
+ vma->vm_flags |= VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP;
vm_page_prot = vm_get_page_prot(vma->vm_flags);
@@ -165,7 +165,8 @@ static vm_fault_t etnaviv_gem_fault(struct vm_fault *vmf)
struct vm_area_struct *vma = vmf->vma;
struct drm_gem_object *obj = vma->vm_private_data;
struct etnaviv_gem_object *etnaviv_obj = to_etnaviv_bo(obj);
- struct page **pages, *page;
+ struct page **pages;
+ unsigned long pfn;
pgoff_t pgoff;
int err;
@@ -189,12 +190,12 @@ static vm_fault_t etnaviv_gem_fault(struct vm_fault *vmf)
/* We don't use vmf->pgoff since that has the fake offset: */
pgoff = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
- page = pages[pgoff];
+ pfn = page_to_pfn(pages[pgoff]);
VERB("Inserting %p pfn %lx, pa %lx", (void *)vmf->address,
- page_to_pfn(page), page_to_pfn(page) << PAGE_SHIFT);
+ pfn, pfn << PAGE_SHIFT);
- return vmf_insert_page(vma, vmf->address, page);
+ return vmf_insert_pfn(vma, vmf->address, pfn);
}
int etnaviv_gem_mmap_offset(struct drm_gem_object *obj, u64 *offset)
@@ -258,7 +259,12 @@ struct etnaviv_vram_mapping *etnaviv_gem_mapping_get(
if (mapping->use == 0) {
mutex_lock(&mmu_context->lock);
if (mapping->context == mmu_context)
- mapping->use += 1;
+ if (va && mapping->iova != va) {
+ etnaviv_iommu_reap_mapping(mapping);
+ mapping = NULL;
+ } else {
+ mapping->use += 1;
+ }
else
mapping = NULL;
mutex_unlock(&mmu_context->lock);
@@ -504,7 +510,6 @@ void etnaviv_gem_free_object(struct drm_gem_object *obj)
kfree(mapping);
}
- drm_gem_free_mmap_offset(obj);
etnaviv_obj->ops->release(etnaviv_obj);
drm_gem_object_release(obj);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.h b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
index 63688e6e..baa81cb 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
@@ -96,6 +96,7 @@ struct etnaviv_gem_submit {
int out_fence_id;
struct list_head node; /* GPU active submit list */
struct etnaviv_cmdbuf cmdbuf;
+ struct pid *pid; /* submitting process */
bool runtime_resumed;
u32 exec_state;
u32 flags;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
index 3fa2da1..7031db1 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
@@ -65,7 +65,7 @@ static void etnaviv_gem_prime_release(struct etnaviv_gem_object *etnaviv_obj)
struct iosys_map map = IOSYS_MAP_INIT_VADDR(etnaviv_obj->vaddr);
if (etnaviv_obj->vaddr)
- dma_buf_vunmap(etnaviv_obj->base.import_attach->dmabuf, &map);
+ dma_buf_vunmap_unlocked(etnaviv_obj->base.import_attach->dmabuf, &map);
/* Don't drop the pages for imported dmabuf, as they are not
* ours, just free the array we allocated:
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index 1ac916b..1491159 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -399,6 +399,9 @@ static void submit_cleanup(struct kref *kref)
mutex_unlock(&submit->gpu->fence_lock);
dma_fence_put(submit->out_fence);
}
+
+ put_pid(submit->pid);
+
kfree(submit->pmrs);
kfree(submit);
}
@@ -422,6 +425,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
struct sync_file *sync_file = NULL;
struct ww_acquire_ctx ticket;
int out_fence_fd = -1;
+ struct pid *pid = get_pid(task_pid(current));
void *stream;
int ret;
@@ -519,6 +523,8 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
goto err_submit_ww_acquire;
}
+ submit->pid = pid;
+
ret = etnaviv_cmdbuf_init(priv->cmdbuf_suballoc, &submit->cmdbuf,
ALIGN(args->stream_size, 8) + 8);
if (ret)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index 37018bc..51320ee 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -416,6 +416,12 @@ static void etnaviv_hw_identify(struct etnaviv_gpu *gpu)
if (gpu->identity.model == chipModel_GC700)
gpu->identity.features &= ~chipFeatures_FAST_CLEAR;
+ /* These models/revisions don't have the 2D pipe bit */
+ if ((gpu->identity.model == chipModel_GC500 &&
+ gpu->identity.revision <= 2) ||
+ gpu->identity.model == chipModel_GC300)
+ gpu->identity.features |= chipFeatures_PIPE_2D;
+
if ((gpu->identity.model == chipModel_GC500 &&
gpu->identity.revision < 2) ||
(gpu->identity.model == chipModel_GC300 &&
@@ -449,8 +455,9 @@ static void etnaviv_hw_identify(struct etnaviv_gpu *gpu)
gpu_read(gpu, VIVS_HI_CHIP_MINOR_FEATURE_5);
}
- /* GC600 idle register reports zero bits where modules aren't present */
- if (gpu->identity.model == chipModel_GC600)
+ /* GC600/300 idle register reports zero bits where modules aren't present */
+ if (gpu->identity.model == chipModel_GC600 ||
+ gpu->identity.model == chipModel_GC300)
gpu->idle_mask = VIVS_HI_IDLE_STATE_TX |
VIVS_HI_IDLE_STATE_RA |
VIVS_HI_IDLE_STATE_SE |
@@ -583,7 +590,7 @@ static void etnaviv_gpu_enable_mlcg(struct etnaviv_gpu *gpu)
u32 pmc, ppc;
/* enable clock gating */
- ppc = gpu_read(gpu, VIVS_PM_POWER_CONTROLS);
+ ppc = gpu_read_power(gpu, VIVS_PM_POWER_CONTROLS);
ppc |= VIVS_PM_POWER_CONTROLS_ENABLE_MODULE_CLOCK_GATING;
/* Disable stall module clock gating for 4.3.0.1 and 4.3.0.2 revs */
@@ -591,9 +598,9 @@ static void etnaviv_gpu_enable_mlcg(struct etnaviv_gpu *gpu)
gpu->identity.revision == 0x4302)
ppc |= VIVS_PM_POWER_CONTROLS_DISABLE_STALL_MODULE_CLOCK_GATING;
- gpu_write(gpu, VIVS_PM_POWER_CONTROLS, ppc);
+ gpu_write_power(gpu, VIVS_PM_POWER_CONTROLS, ppc);
- pmc = gpu_read(gpu, VIVS_PM_MODULE_CONTROLS);
+ pmc = gpu_read_power(gpu, VIVS_PM_MODULE_CONTROLS);
/* Disable PA clock gating for GC400+ without bugfix except for GC420 */
if (gpu->identity.model >= chipModel_GC400 &&
@@ -616,19 +623,20 @@ static void etnaviv_gpu_enable_mlcg(struct etnaviv_gpu *gpu)
/* Disable TX clock gating on affected core revisions. */
if (etnaviv_is_model_rev(gpu, GC4000, 0x5222) ||
- etnaviv_is_model_rev(gpu, GC2000, 0x5108))
+ etnaviv_is_model_rev(gpu, GC2000, 0x5108) ||
+ etnaviv_is_model_rev(gpu, GC2000, 0x6202) ||
+ etnaviv_is_model_rev(gpu, GC2000, 0x6203))
pmc |= VIVS_PM_MODULE_CONTROLS_DISABLE_MODULE_CLOCK_GATING_TX;
- /* Disable SE, RA and TX clock gating on affected core revisions. */
+ /* Disable SE and RA clock gating on affected core revisions. */
if (etnaviv_is_model_rev(gpu, GC7000, 0x6202))
pmc |= VIVS_PM_MODULE_CONTROLS_DISABLE_MODULE_CLOCK_GATING_SE |
- VIVS_PM_MODULE_CONTROLS_DISABLE_MODULE_CLOCK_GATING_RA |
- VIVS_PM_MODULE_CONTROLS_DISABLE_MODULE_CLOCK_GATING_TX;
+ VIVS_PM_MODULE_CONTROLS_DISABLE_MODULE_CLOCK_GATING_RA;
pmc |= VIVS_PM_MODULE_CONTROLS_DISABLE_MODULE_CLOCK_GATING_RA_HZ;
pmc |= VIVS_PM_MODULE_CONTROLS_DISABLE_MODULE_CLOCK_GATING_RA_EZ;
- gpu_write(gpu, VIVS_PM_MODULE_CONTROLS, pmc);
+ gpu_write_power(gpu, VIVS_PM_MODULE_CONTROLS, pmc);
}
void etnaviv_gpu_start_fe(struct etnaviv_gpu *gpu, u32 address, u16 prefetch)
@@ -688,11 +696,11 @@ static void etnaviv_gpu_setup_pulse_eater(struct etnaviv_gpu *gpu)
(gpu->identity.features & chipFeatures_PIPE_3D))
{
/* Performance fix: disable internal DFS */
- pulse_eater = gpu_read(gpu, VIVS_PM_PULSE_EATER);
+ pulse_eater = gpu_read_power(gpu, VIVS_PM_PULSE_EATER);
pulse_eater |= BIT(18);
}
- gpu_write(gpu, VIVS_PM_PULSE_EATER, pulse_eater);
+ gpu_write_power(gpu, VIVS_PM_PULSE_EATER, pulse_eater);
}
static void etnaviv_gpu_hw_init(struct etnaviv_gpu *gpu)
@@ -1045,12 +1053,28 @@ int etnaviv_gpu_debugfs(struct etnaviv_gpu *gpu, struct seq_file *m)
}
#endif
-void etnaviv_gpu_recover_hang(struct etnaviv_gpu *gpu)
+void etnaviv_gpu_recover_hang(struct etnaviv_gem_submit *submit)
{
+ struct etnaviv_gpu *gpu = submit->gpu;
+ char *comm = NULL, *cmd = NULL;
+ struct task_struct *task;
unsigned int i;
dev_err(gpu->dev, "recover hung GPU!\n");
+ task = get_pid_task(submit->pid, PIDTYPE_PID);
+ if (task) {
+ comm = kstrdup(task->comm, GFP_KERNEL);
+ cmd = kstrdup_quotable_cmdline(task, GFP_KERNEL);
+ put_task_struct(task);
+ }
+
+ if (comm && cmd)
+ dev_err(gpu->dev, "offending task: %s (%s)\n", comm, cmd);
+
+ kfree(cmd);
+ kfree(comm);
+
if (pm_runtime_get_sync(gpu->dev) < 0)
goto pm_put;
@@ -1294,9 +1318,9 @@ static void sync_point_perfmon_sample_pre(struct etnaviv_gpu *gpu,
u32 val;
/* disable clock gating */
- val = gpu_read(gpu, VIVS_PM_POWER_CONTROLS);
+ val = gpu_read_power(gpu, VIVS_PM_POWER_CONTROLS);
val &= ~VIVS_PM_POWER_CONTROLS_ENABLE_MODULE_CLOCK_GATING;
- gpu_write(gpu, VIVS_PM_POWER_CONTROLS, val);
+ gpu_write_power(gpu, VIVS_PM_POWER_CONTROLS, val);
/* enable debug register */
val = gpu_read(gpu, VIVS_HI_CLOCK_CONTROL);
@@ -1327,9 +1351,9 @@ static void sync_point_perfmon_sample_post(struct etnaviv_gpu *gpu,
gpu_write(gpu, VIVS_HI_CLOCK_CONTROL, val);
/* enable clock gating */
- val = gpu_read(gpu, VIVS_PM_POWER_CONTROLS);
+ val = gpu_read_power(gpu, VIVS_PM_POWER_CONTROLS);
val |= VIVS_PM_POWER_CONTROLS_ENABLE_MODULE_CLOCK_GATING;
- gpu_write(gpu, VIVS_PM_POWER_CONTROLS, val);
+ gpu_write_power(gpu, VIVS_PM_POWER_CONTROLS, val);
}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
index 85eddd4..f1204b0 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
@@ -10,6 +10,7 @@
#include "etnaviv_gem.h"
#include "etnaviv_mmu.h"
#include "etnaviv_drv.h"
+#include "common.xml.h"
struct etnaviv_gem_submit;
struct etnaviv_vram_mapping;
@@ -159,6 +160,26 @@ static inline u32 gpu_read(struct etnaviv_gpu *gpu, u32 reg)
return readl(gpu->mmio + reg);
}
+static inline u32 gpu_fix_power_address(struct etnaviv_gpu *gpu, u32 reg)
+{
+ /* Power registers in GC300 < 2.0 are offset by 0x100 */
+ if (gpu->identity.model == chipModel_GC300 &&
+ gpu->identity.revision < 0x2000)
+ reg += 0x100;
+
+ return reg;
+}
+
+static inline void gpu_write_power(struct etnaviv_gpu *gpu, u32 reg, u32 data)
+{
+ writel(data, gpu->mmio + gpu_fix_power_address(gpu, reg));
+}
+
+static inline u32 gpu_read_power(struct etnaviv_gpu *gpu, u32 reg)
+{
+ return readl(gpu->mmio + gpu_fix_power_address(gpu, reg));
+}
+
int etnaviv_gpu_get_param(struct etnaviv_gpu *gpu, u32 param, u64 *value);
int etnaviv_gpu_init(struct etnaviv_gpu *gpu);
@@ -168,7 +189,7 @@ bool etnaviv_fill_identity_from_hwdb(struct etnaviv_gpu *gpu);
int etnaviv_gpu_debugfs(struct etnaviv_gpu *gpu, struct seq_file *m);
#endif
-void etnaviv_gpu_recover_hang(struct etnaviv_gpu *gpu);
+void etnaviv_gpu_recover_hang(struct etnaviv_gem_submit *submit);
void etnaviv_gpu_retire(struct etnaviv_gpu *gpu);
int etnaviv_gpu_wait_fence_interruptible(struct etnaviv_gpu *gpu,
u32 fence, struct drm_etnaviv_timespec *timeout);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_hwdb.c b/drivers/gpu/drm/etnaviv/etnaviv_hwdb.c
index f2fc645..57f334e 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_hwdb.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_hwdb.c
@@ -70,6 +70,37 @@ static const struct etnaviv_chip_identity etnaviv_chip_identities[] = {
},
{
.model = 0x7000,
+ .revision = 0x6203,
+ .product_id = 0x70003,
+ .customer_id = 0x4,
+ .eco_id = 0,
+ .stream_count = 16,
+ .register_max = 64,
+ .thread_count = 512,
+ .shader_core_count = 2,
+ .vertex_cache_size = 16,
+ .vertex_output_buffer_size = 1024,
+ .pixel_pipes = 1,
+ .instruction_count = 512,
+ .num_constants = 320,
+ .buffer_size = 0,
+ .varyings_count = 16,
+ .features = 0xe0287c8d,
+ .minor_features0 = 0xc1589eff,
+ .minor_features1 = 0xfefbfad9,
+ .minor_features2 = 0xeb9d4fbf,
+ .minor_features3 = 0xedfffced,
+ .minor_features4 = 0xdb0dafc7,
+ .minor_features5 = 0x3b5ac333,
+ .minor_features6 = 0xfcce6000,
+ .minor_features7 = 0xfffbfa6f,
+ .minor_features8 = 0x00e10ef3,
+ .minor_features9 = 0x00c8003c,
+ .minor_features10 = 0x00004040,
+ .minor_features11 = 0x00000024,
+ },
+ {
+ .model = 0x7000,
.revision = 0x6204,
.product_id = ~0U,
.customer_id = ~0U,
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c
index dc1aa73..67bdce53 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c
@@ -80,10 +80,10 @@ static int etnaviv_iommu_map(struct etnaviv_iommu_context *context, u32 iova,
return -EINVAL;
for_each_sgtable_dma_sg(sgt, sg, i) {
- u32 pa = sg_dma_address(sg) - sg->offset;
+ phys_addr_t pa = sg_dma_address(sg) - sg->offset;
size_t bytes = sg_dma_len(sg) + sg->offset;
- VERB("map[%d]: %08x %08x(%zx)", i, iova, pa, bytes);
+ VERB("map[%d]: %08x %pap(%zx)", i, iova, &pa, bytes);
ret = etnaviv_context_map(context, da, pa, bytes, prot);
if (ret)
@@ -135,6 +135,19 @@ static void etnaviv_iommu_remove_mapping(struct etnaviv_iommu_context *context,
drm_mm_remove_node(&mapping->vram_node);
}
+void etnaviv_iommu_reap_mapping(struct etnaviv_vram_mapping *mapping)
+{
+ struct etnaviv_iommu_context *context = mapping->context;
+
+ lockdep_assert_held(&context->lock);
+ WARN_ON(mapping->use);
+
+ etnaviv_iommu_remove_mapping(context, mapping);
+ etnaviv_iommu_context_put(mapping->context);
+ mapping->context = NULL;
+ list_del_init(&mapping->mmu_node);
+}
+
static int etnaviv_iommu_find_iova(struct etnaviv_iommu_context *context,
struct drm_mm_node *node, size_t size)
{
@@ -202,10 +215,7 @@ static int etnaviv_iommu_find_iova(struct etnaviv_iommu_context *context,
* this mapping.
*/
list_for_each_entry_safe(m, n, &list, scan_node) {
- etnaviv_iommu_remove_mapping(context, m);
- etnaviv_iommu_context_put(m->context);
- m->context = NULL;
- list_del_init(&m->mmu_node);
+ etnaviv_iommu_reap_mapping(m);
list_del_init(&m->scan_node);
}
@@ -257,10 +267,7 @@ static int etnaviv_iommu_insert_exact(struct etnaviv_iommu_context *context,
}
list_for_each_entry_safe(m, n, &scan_list, scan_node) {
- etnaviv_iommu_remove_mapping(context, m);
- etnaviv_iommu_context_put(m->context);
- m->context = NULL;
- list_del_init(&m->mmu_node);
+ etnaviv_iommu_reap_mapping(m);
list_del_init(&m->scan_node);
}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.h b/drivers/gpu/drm/etnaviv/etnaviv_mmu.h
index e4a0b7d..c01a147 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.h
@@ -91,6 +91,7 @@ int etnaviv_iommu_map_gem(struct etnaviv_iommu_context *context,
struct etnaviv_vram_mapping *mapping, u64 va);
void etnaviv_iommu_unmap_gem(struct etnaviv_iommu_context *context,
struct etnaviv_vram_mapping *mapping);
+void etnaviv_iommu_reap_mapping(struct etnaviv_vram_mapping *mapping);
int etnaviv_iommu_get_suballoc_va(struct etnaviv_iommu_context *ctx,
struct etnaviv_vram_mapping *mapping,
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index 72e2553..d29f467 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -67,7 +67,7 @@ static enum drm_gpu_sched_stat etnaviv_sched_timedout_job(struct drm_sched_job
/* get the GPU back into the init state */
etnaviv_core_dump(submit);
- etnaviv_gpu_recover_hang(gpu);
+ etnaviv_gpu_recover_hang(submit);
drm_sched_resubmit_jobs(&gpu->sched);
diff --git a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
index 767afd2b..55c9237 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
@@ -49,6 +49,8 @@ static const struct fb_ops exynos_drm_fb_ops = {
.owner = THIS_MODULE,
DRM_FB_HELPER_DEFAULT_OPS,
.fb_mmap = exynos_drm_fb_mmap,
+ .fb_read = drm_fb_helper_cfb_read,
+ .fb_write = drm_fb_helper_cfb_write,
.fb_fillrect = drm_fb_helper_cfb_fillrect,
.fb_copyarea = drm_fb_helper_cfb_copyarea,
.fb_imageblit = drm_fb_helper_cfb_imageblit,
@@ -63,7 +65,7 @@ static int exynos_drm_fbdev_update(struct drm_fb_helper *helper,
unsigned int size = fb->width * fb->height * fb->format->cpp[0];
unsigned long offset;
- fbi = drm_fb_helper_alloc_fbi(helper);
+ fbi = drm_fb_helper_alloc_info(helper);
if (IS_ERR(fbi)) {
DRM_DEV_ERROR(to_dma_dev(helper->dev),
"failed to allocate fb info.\n");
@@ -201,7 +203,7 @@ static void exynos_drm_fbdev_destroy(struct drm_device *dev,
drm_framebuffer_remove(fb);
}
- drm_fb_helper_unregister_fbi(fb_helper);
+ drm_fb_helper_unregister_info(fb_helper);
drm_fb_helper_fini(fb_helper);
}
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c
index b4acc34..8579c76 100644
--- a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c
+++ b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c
@@ -20,7 +20,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_modeset_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/gma500/framebuffer.c b/drivers/gpu/drm/gma500/framebuffer.c
index aa3ecf7..8d5a37b 100644
--- a/drivers/gpu/drm/gma500/framebuffer.c
+++ b/drivers/gpu/drm/gma500/framebuffer.c
@@ -147,6 +147,8 @@ static const struct fb_ops psbfb_unaccel_ops = {
.owner = THIS_MODULE,
DRM_FB_HELPER_DEFAULT_OPS,
.fb_setcolreg = psbfb_setcolreg,
+ .fb_read = drm_fb_helper_cfb_read,
+ .fb_write = drm_fb_helper_cfb_write,
.fb_fillrect = drm_fb_helper_cfb_fillrect,
.fb_copyarea = drm_fb_helper_cfb_copyarea,
.fb_imageblit = drm_fb_helper_cfb_imageblit,
@@ -268,7 +270,7 @@ static int psbfb_create(struct drm_fb_helper *fb_helper,
memset(dev_priv->vram_addr + backing->offset, 0, size);
- info = drm_fb_helper_alloc_fbi(fb_helper);
+ info = drm_fb_helper_alloc_info(fb_helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
goto err_drm_gem_object_put;
@@ -286,7 +288,7 @@ static int psbfb_create(struct drm_fb_helper *fb_helper,
info->fbops = &psbfb_unaccel_ops;
- info->fix.smem_start = dev->mode_config.fb_base;
+ info->fix.smem_start = dev_priv->fb_base;
info->fix.smem_len = size;
info->fix.ywrapstep = 0;
info->fix.ypanstep = 0;
@@ -296,7 +298,7 @@ static int psbfb_create(struct drm_fb_helper *fb_helper,
info->screen_size = size;
if (dev_priv->gtt.stolen_size) {
- info->apertures->ranges[0].base = dev->mode_config.fb_base;
+ info->apertures->ranges[0].base = dev_priv->fb_base;
info->apertures->ranges[0].size = dev_priv->gtt.stolen_size;
}
@@ -383,7 +385,7 @@ static int psb_fbdev_destroy(struct drm_device *dev,
{
struct drm_framebuffer *fb = fb_helper->fb;
- drm_fb_helper_unregister_fbi(fb_helper);
+ drm_fb_helper_unregister_info(fb_helper);
drm_fb_helper_fini(fb_helper);
drm_framebuffer_unregister_private(fb);
@@ -527,7 +529,7 @@ void psb_modeset_init(struct drm_device *dev)
/* set memory base */
/* Oaktrail and Poulsbo should use BAR 2*/
- pci_read_config_dword(pdev, PSB_BSM, (u32 *)&(dev->mode_config.fb_base));
+ pci_read_config_dword(pdev, PSB_BSM, (u32 *)&(dev_priv->fb_base));
/* num pipes is 2 for PSB but 1 for Mrst */
for (i = 0; i < dev_priv->num_pipe; i++)
diff --git a/drivers/gpu/drm/gma500/psb_drv.h b/drivers/gpu/drm/gma500/psb_drv.h
index ae544b6..a5df6d2 100644
--- a/drivers/gpu/drm/gma500/psb_drv.h
+++ b/drivers/gpu/drm/gma500/psb_drv.h
@@ -523,6 +523,7 @@ struct drm_psb_private {
uint32_t blc_adj2;
struct drm_fb_helper *fb_helper;
+ resource_size_t fb_base;
bool dsr_enable;
u32 dsr_fb_update;
diff --git a/drivers/gpu/drm/gud/gud_connector.c b/drivers/gpu/drm/gud/gud_connector.c
index d0addd4..fa63620 100644
--- a/drivers/gpu/drm/gud/gud_connector.c
+++ b/drivers/gpu/drm/gud/gud_connector.c
@@ -355,7 +355,7 @@ static void gud_connector_reset(struct drm_connector *connector)
drm_atomic_helper_connector_reset(connector);
connector->state->tv = gconn->initial_tv_state;
/* Set margins from command line */
- drm_atomic_helper_connector_tv_reset(connector);
+ drm_atomic_helper_connector_tv_margins_reset(connector);
if (gconn->initial_brightness >= 0)
connector->state->tv.brightness = gconn->initial_brightness;
}
diff --git a/drivers/gpu/drm/gud/gud_drv.c b/drivers/gpu/drm/gud/gud_drv.c
index 8d1630b..d57dab10 100644
--- a/drivers/gpu/drm/gud/gud_drv.c
+++ b/drivers/gpu/drm/gud/gud_drv.c
@@ -18,7 +18,7 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_debugfs.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c
index fe4269c..22053c6 100644
--- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c
+++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c
@@ -17,6 +17,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_gem_vram_helper.h>
#include <drm/drm_managed.h>
@@ -105,7 +106,6 @@ static int hibmc_kms_init(struct hibmc_drm_private *priv)
dev->mode_config.max_width = 1920;
dev->mode_config.max_height = 1200;
- dev->mode_config.fb_base = priv->fb_base;
dev->mode_config.preferred_depth = 32;
dev->mode_config.prefer_shadow = 1;
@@ -212,7 +212,7 @@ static int hibmc_hw_map(struct hibmc_drm_private *priv)
{
struct drm_device *dev = &priv->dev;
struct pci_dev *pdev = to_pci_dev(dev->dev);
- resource_size_t addr, size, ioaddr, iosize;
+ resource_size_t ioaddr, iosize;
ioaddr = pci_resource_start(pdev, 1);
iosize = pci_resource_len(pdev, 1);
@@ -222,16 +222,6 @@ static int hibmc_hw_map(struct hibmc_drm_private *priv)
return -ENOMEM;
}
- addr = pci_resource_start(pdev, 0);
- size = pci_resource_len(pdev, 0);
- priv->fb_map = devm_ioremap(dev->dev, addr, size);
- if (!priv->fb_map) {
- drm_err(dev, "Cannot map framebuffer\n");
- return -ENOMEM;
- }
- priv->fb_base = addr;
- priv->fb_size = size;
-
return 0;
}
@@ -271,7 +261,8 @@ static int hibmc_load(struct drm_device *dev)
if (ret)
goto err;
- ret = drmm_vram_helper_init(dev, pci_resource_start(pdev, 0), priv->fb_size);
+ ret = drmm_vram_helper_init(dev, pci_resource_start(pdev, 0),
+ pci_resource_len(pdev, 0));
if (ret) {
drm_err(dev, "Error initializing VRAM MM; %d\n", ret);
goto err;
diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h
index 7d263f4d..f957552 100644
--- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h
+++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h
@@ -19,7 +19,6 @@
#include <linux/i2c.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_framebuffer.h>
struct hibmc_connector {
@@ -32,9 +31,6 @@ struct hibmc_connector {
struct hibmc_drm_private {
/* hw */
void __iomem *mmio;
- void __iomem *fb_map;
- resource_size_t fb_base;
- resource_size_t fb_size;
/* drm */
struct drm_device dev;
diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c
index c228091..8c6d2ea 100644
--- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c
+++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c
@@ -11,6 +11,8 @@
* Jianhua Li <lijianhua@huawei.com>
*/
+#include <linux/io.h>
+
#include <drm/drm_atomic_helper.h>
#include <drm/drm_probe_helper.h>
#include <drm/drm_print.h>
diff --git a/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c b/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
index a0d5aa7..d9978b7 100644
--- a/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
+++ b/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
@@ -658,7 +658,7 @@ static enum drm_mode_status dsi_encoder_mode_valid(struct drm_encoder *encoder,
* reset adj_mode to the mode value each time,
* so we don't adjust the mode twice
*/
- drm_mode_copy(&adj_mode, mode);
+ drm_mode_init(&adj_mode, mode);
crtc_funcs = crtc->helper_private;
if (crtc_funcs && crtc_funcs->mode_fixup)
diff --git a/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c
index 73ee7f2..9c5d49b 100644
--- a/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c
+++ b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c
@@ -19,7 +19,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/hyperv/hyperv_drm_drv.c b/drivers/gpu/drm/hyperv/hyperv_drm_drv.c
index ca127ff..427c20b 100644
--- a/drivers/gpu/drm/hyperv/hyperv_drm_drv.c
+++ b/drivers/gpu/drm/hyperv/hyperv_drm_drv.c
@@ -11,7 +11,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_simple_kms_helper.h>
diff --git a/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c b/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c
index 28e732f..6c6b572 100644
--- a/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c
+++ b/drivers/gpu/drm/hyperv/hyperv_drm_modeset.c
@@ -8,7 +8,6 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
diff --git a/drivers/gpu/drm/i915/Kconfig.profile b/drivers/gpu/drm/i915/Kconfig.profile
index 3932856..7cc38d2 100644
--- a/drivers/gpu/drm/i915/Kconfig.profile
+++ b/drivers/gpu/drm/i915/Kconfig.profile
@@ -57,10 +57,28 @@
default 640 # milliseconds
help
How long to wait (in milliseconds) for a preemption event to occur
- when submitting a new context via execlists. If the current context
- does not hit an arbitration point and yield to HW before the timer
- expires, the HW will be reset to allow the more important context
- to execute.
+ when submitting a new context. If the current context does not hit
+ an arbitration point and yield to HW before the timer expires, the
+ HW will be reset to allow the more important context to execute.
+
+ This is adjustable via
+ /sys/class/drm/card?/engine/*/preempt_timeout_ms
+
+ May be 0 to disable the timeout.
+
+ The compiled in default may get overridden at driver probe time on
+ certain platforms and certain engines which will be reflected in the
+ sysfs control.
+
+config DRM_I915_PREEMPT_TIMEOUT_COMPUTE
+ int "Preempt timeout for compute engines (ms, jiffy granularity)"
+ default 7500 # milliseconds
+ help
+ How long to wait (in milliseconds) for a preemption event to occur
+ when submitting a new context to a compute capable engine. If the
+ current context does not hit an arbitration point and yield to HW
+ before the timer expires, the HW will be reset to allow the more
+ important context to execute.
This is adjustable via
/sys/class/drm/card?/engine/*/preempt_timeout_ms
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index cea00aa..01974b8 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -127,9 +127,11 @@
gt/intel_sseu.o \
gt/intel_sseu_debugfs.o \
gt/intel_timeline.o \
+ gt/intel_wopcm.o \
gt/intel_workarounds.o \
gt/shmem_utils.o \
gt/sysfs_engines.o
+
# x86 intel-gtt module support
gt-$(CONFIG_X86) += gt/intel_ggtt_gmch.o
# autogenerated null render state
@@ -183,8 +185,7 @@
i915_trace_points.o \
i915_ttm_buddy_manager.o \
i915_vma.o \
- i915_vma_resource.o \
- intel_wopcm.o
+ i915_vma_resource.o
# general-purpose microcontroller (GuC) support
i915-y += gt/uc/intel_uc.o \
@@ -209,6 +210,9 @@
# graphics system controller (GSC) support
i915-y += gt/intel_gsc.o
+# graphics hardware monitoring (HWMON) support
+i915-$(CONFIG_HWMON) += i915_hwmon.o
+
# modesetting core code
i915-y += \
display/hsw_ips.o \
@@ -244,6 +248,7 @@
display/intel_global_state.o \
display/intel_hdcp.o \
display/intel_hotplug.o \
+ display/intel_hti.o \
display/intel_lpe_audio.o \
display/intel_modeset_verify.o \
display/intel_modeset_setup.o \
@@ -310,15 +315,18 @@
i915-y += i915_perf.o
-# Protected execution platform (PXP) support
-i915-$(CONFIG_DRM_I915_PXP) += \
+# Protected execution platform (PXP) support. Base support is required for HuC
+i915-y += \
pxp/intel_pxp.o \
+ pxp/intel_pxp_tee.o \
+ pxp/intel_pxp_huc.o
+
+i915-$(CONFIG_DRM_I915_PXP) += \
pxp/intel_pxp_cmd.o \
pxp/intel_pxp_debugfs.o \
pxp/intel_pxp_irq.o \
pxp/intel_pxp_pm.o \
- pxp/intel_pxp_session.o \
- pxp/intel_pxp_tee.o
+ pxp/intel_pxp_session.o
# Post-mortem debug and GPU hang state capture
i915-$(CONFIG_DRM_I915_CAPTURE_ERROR) += i915_gpu_error.o
diff --git a/drivers/gpu/drm/i915/display/g4x_dp.c b/drivers/gpu/drm/i915/display/g4x_dp.c
index e3e3d27..3593938 100644
--- a/drivers/gpu/drm/i915/display/g4x_dp.c
+++ b/drivers/gpu/drm/i915/display/g4x_dp.c
@@ -8,6 +8,7 @@
#include <linux/string_helpers.h>
#include "g4x_dp.h"
+#include "i915_reg.h"
#include "intel_audio.h"
#include "intel_backlight.h"
#include "intel_connector.h"
diff --git a/drivers/gpu/drm/i915/display/g4x_dp.h b/drivers/gpu/drm/i915/display/g4x_dp.h
index e1f5026..a38b3e1 100644
--- a/drivers/gpu/drm/i915/display/g4x_dp.h
+++ b/drivers/gpu/drm/i915/display/g4x_dp.h
@@ -8,7 +8,7 @@
#include <linux/types.h>
-#include "i915_reg.h"
+#include "i915_reg_defs.h"
enum pipe;
enum port;
diff --git a/drivers/gpu/drm/i915/display/g4x_hdmi.c b/drivers/gpu/drm/i915/display/g4x_hdmi.c
index 2b73f5f..121caea 100644
--- a/drivers/gpu/drm/i915/display/g4x_hdmi.c
+++ b/drivers/gpu/drm/i915/display/g4x_hdmi.c
@@ -6,6 +6,7 @@
*/
#include "g4x_hdmi.h"
+#include "i915_reg.h"
#include "intel_audio.h"
#include "intel_connector.h"
#include "intel_crtc.h"
@@ -78,6 +79,18 @@ static bool intel_hdmi_get_hw_state(struct intel_encoder *encoder,
return ret;
}
+static int g4x_hdmi_compute_config(struct intel_encoder *encoder,
+ struct intel_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+
+ if (HAS_PCH_SPLIT(i915))
+ crtc_state->has_pch_encoder = true;
+
+ return intel_hdmi_compute_config(encoder, crtc_state, conn_state);
+}
+
static void intel_hdmi_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
@@ -543,7 +556,7 @@ void g4x_hdmi_init(struct drm_i915_private *dev_priv,
"HDMI %c", port_name(port));
intel_encoder->hotplug = intel_hdmi_hotplug;
- intel_encoder->compute_config = intel_hdmi_compute_config;
+ intel_encoder->compute_config = g4x_hdmi_compute_config;
if (HAS_PCH_SPLIT(dev_priv)) {
intel_encoder->disable = pch_disable_hdmi;
intel_encoder->post_disable = pch_post_disable_hdmi;
@@ -585,7 +598,7 @@ void g4x_hdmi_init(struct drm_i915_private *dev_priv,
} else {
intel_encoder->pipe_mask = ~0;
}
- intel_encoder->cloneable = 1 << INTEL_OUTPUT_ANALOG;
+ intel_encoder->cloneable = BIT(INTEL_OUTPUT_ANALOG);
intel_encoder->hpd_pin = intel_hpd_pin_default(dev_priv, port);
/*
* BSpec is unclear about HDMI+HDMI cloning on g4x, but it seems
@@ -593,7 +606,7 @@ void g4x_hdmi_init(struct drm_i915_private *dev_priv,
* only one port anyway, nothing is lost by allowing it.
*/
if (IS_G4X(dev_priv))
- intel_encoder->cloneable |= 1 << INTEL_OUTPUT_HDMI;
+ intel_encoder->cloneable |= BIT(INTEL_OUTPUT_HDMI);
dig_port->hdmi.hdmi_reg = hdmi_reg;
dig_port->dp.output_reg = INVALID_MMIO_REG;
diff --git a/drivers/gpu/drm/i915/display/hsw_ips.c b/drivers/gpu/drm/i915/display/hsw_ips.c
index a5be4af7..83aa380 100644
--- a/drivers/gpu/drm/i915/display/hsw_ips.c
+++ b/drivers/gpu/drm/i915/display/hsw_ips.c
@@ -104,8 +104,7 @@ static bool hsw_ips_need_disable(struct intel_atomic_state *state,
* Disable IPS before we program the LUT.
*/
if (IS_HASWELL(i915) &&
- (new_crtc_state->uapi.color_mgmt_changed ||
- new_crtc_state->update_pipe) &&
+ intel_crtc_needs_color_update(new_crtc_state) &&
new_crtc_state->gamma_mode == GAMMA_MODE_MODE_SPLIT)
return true;
@@ -146,8 +145,7 @@ static bool hsw_ips_need_enable(struct intel_atomic_state *state,
* Re-enable IPS after the LUT has been programmed.
*/
if (IS_HASWELL(i915) &&
- (new_crtc_state->uapi.color_mgmt_changed ||
- new_crtc_state->update_pipe) &&
+ intel_crtc_needs_color_update(new_crtc_state) &&
new_crtc_state->gamma_mode == GAMMA_MODE_MODE_SPLIT)
return true;
@@ -155,7 +153,7 @@ static bool hsw_ips_need_enable(struct intel_atomic_state *state,
* We can't read out IPS on broadwell, assume the worst and
* forcibly enable IPS on the first fastset.
*/
- if (new_crtc_state->update_pipe && old_crtc_state->inherited)
+ if (intel_crtc_needs_fastset(new_crtc_state) && old_crtc_state->inherited)
return true;
return !old_crtc_state->ips_enabled;
diff --git a/drivers/gpu/drm/i915/display/i9xx_plane.c b/drivers/gpu/drm/i915/display/i9xx_plane.c
index 5afbe3e..ecaeb7d 100644
--- a/drivers/gpu/drm/i915/display/i9xx_plane.c
+++ b/drivers/gpu/drm/i915/display/i9xx_plane.c
@@ -8,6 +8,9 @@
#include <drm/drm_blend.h>
#include <drm/drm_fourcc.h>
+#include "i915_irq.h"
+#include "i915_reg.h"
+#include "i9xx_plane.h"
#include "intel_atomic.h"
#include "intel_atomic_plane.h"
#include "intel_de.h"
@@ -15,7 +18,6 @@
#include "intel_fb.h"
#include "intel_fbc.h"
#include "intel_sprite.h"
-#include "i9xx_plane.h"
/* Primary plane formats for gen <= 3 */
static const u32 i8xx_primary_formats[] = {
diff --git a/drivers/gpu/drm/i915/display/icl_dsi.c b/drivers/gpu/drm/i915/display/icl_dsi.c
index ed4d939..d16b30a 100644
--- a/drivers/gpu/drm/i915/display/icl_dsi.c
+++ b/drivers/gpu/drm/i915/display/icl_dsi.c
@@ -29,6 +29,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_mipi_dsi.h>
+#include "i915_reg.h"
#include "icl_dsi.h"
#include "icl_dsi_regs.h"
#include "intel_atomic.h"
@@ -1974,16 +1975,8 @@ static void icl_dsi_add_properties(struct intel_connector *connector)
{
const struct drm_display_mode *fixed_mode =
intel_panel_preferred_fixed_mode(connector);
- u32 allowed_scalers;
- allowed_scalers = BIT(DRM_MODE_SCALE_ASPECT) |
- BIT(DRM_MODE_SCALE_FULLSCREEN) |
- BIT(DRM_MODE_SCALE_CENTER);
-
- drm_connector_attach_scaling_mode_property(&connector->base,
- allowed_scalers);
-
- connector->base.state->scaling_mode = DRM_MODE_SCALE_ASPECT;
+ intel_attach_scaling_mode_property(&connector->base);
drm_connector_set_panel_orientation_with_quirk(&connector->base,
intel_dsi_get_panel_orientation(connector),
@@ -1993,7 +1986,6 @@ static void icl_dsi_add_properties(struct intel_connector *connector)
void icl_dsi_init(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_dsi *intel_dsi;
struct intel_encoder *encoder;
struct intel_connector *intel_connector;
@@ -2018,7 +2010,7 @@ void icl_dsi_init(struct drm_i915_private *dev_priv)
connector = &intel_connector->base;
/* register DSI encoder with DRM subsystem */
- drm_encoder_init(dev, &encoder->base, &gen11_dsi_encoder_funcs,
+ drm_encoder_init(&dev_priv->drm, &encoder->base, &gen11_dsi_encoder_funcs,
DRM_MODE_ENCODER_DSI, "DSI %c", port_name(port));
encoder->pre_pll_enable = gen11_dsi_pre_pll_enable;
@@ -2042,12 +2034,10 @@ void icl_dsi_init(struct drm_i915_private *dev_priv)
encoder->is_clock_enabled = gen11_dsi_is_clock_enabled;
/* register DSI connector with DRM subsystem */
- drm_connector_init(dev, connector, &gen11_dsi_connector_funcs,
+ drm_connector_init(&dev_priv->drm, connector, &gen11_dsi_connector_funcs,
DRM_MODE_CONNECTOR_DSI);
drm_connector_helper_add(connector, &gen11_dsi_connector_helper_funcs);
connector->display_info.subpixel_order = SubPixelHorizontalRGB;
- connector->interlace_allowed = false;
- connector->doublescan_allowed = false;
intel_connector->get_hw_state = intel_connector_get_hw_state;
/* attach connector to encoder */
@@ -2055,9 +2045,9 @@ void icl_dsi_init(struct drm_i915_private *dev_priv)
intel_bios_init_panel(dev_priv, &intel_connector->panel, NULL, NULL);
- mutex_lock(&dev->mode_config.mutex);
+ mutex_lock(&dev_priv->drm.mode_config.mutex);
intel_panel_add_vbt_lfp_fixed_mode(intel_connector);
- mutex_unlock(&dev->mode_config.mutex);
+ mutex_unlock(&dev_priv->drm.mode_config.mutex);
if (!intel_panel_preferred_fixed_mode(intel_connector)) {
drm_err(&dev_priv->drm, "DSI fixed mode info missing\n");
diff --git a/drivers/gpu/drm/i915/display/icl_dsi_regs.h b/drivers/gpu/drm/i915/display/icl_dsi_regs.h
index f78f28b..d4845ac 100644
--- a/drivers/gpu/drm/i915/display/icl_dsi_regs.h
+++ b/drivers/gpu/drm/i915/display/icl_dsi_regs.h
@@ -6,7 +6,7 @@
#ifndef __ICL_DSI_REGS_H__
#define __ICL_DSI_REGS_H__
-#include "i915_reg_defs.h"
+#include "intel_display_reg_defs.h"
/* Gen11 DSI */
#define _MMIO_DSI(tc, dsi0, dsi1) _MMIO_TRANS((tc) - TRANSCODER_DSI_0, \
diff --git a/drivers/gpu/drm/i915/display/intel_atomic.c b/drivers/gpu/drm/i915/display/intel_atomic.c
index 18f0a5a..6621aa2 100644
--- a/drivers/gpu/drm/i915/display/intel_atomic.c
+++ b/drivers/gpu/drm/i915/display/intel_atomic.c
@@ -252,6 +252,11 @@ intel_crtc_duplicate_state(struct drm_crtc *crtc)
if (crtc_state->hw.gamma_lut)
drm_property_blob_get(crtc_state->hw.gamma_lut);
+ if (crtc_state->pre_csc_lut)
+ drm_property_blob_get(crtc_state->pre_csc_lut);
+ if (crtc_state->post_csc_lut)
+ drm_property_blob_get(crtc_state->post_csc_lut);
+
crtc_state->update_pipe = false;
crtc_state->disable_lp_wm = false;
crtc_state->disable_cxsr = false;
@@ -274,6 +279,9 @@ static void intel_crtc_put_color_blobs(struct intel_crtc_state *crtc_state)
drm_property_blob_put(crtc_state->hw.degamma_lut);
drm_property_blob_put(crtc_state->hw.gamma_lut);
drm_property_blob_put(crtc_state->hw.ctm);
+
+ drm_property_blob_put(crtc_state->pre_csc_lut);
+ drm_property_blob_put(crtc_state->post_csc_lut);
}
void intel_crtc_free_hw_state(struct intel_crtc_state *crtc_state)
diff --git a/drivers/gpu/drm/i915/display/intel_atomic_plane.c b/drivers/gpu/drm/i915/display/intel_atomic_plane.c
index aaa6708..10e1fc9 100644
--- a/drivers/gpu/drm/i915/display/intel_atomic_plane.c
+++ b/drivers/gpu/drm/i915/display/intel_atomic_plane.c
@@ -425,6 +425,47 @@ static bool intel_plane_do_async_flip(struct intel_plane *plane,
return DISPLAY_VER(i915) < 13 || old_crtc_state->uapi.async_flip;
}
+static bool i9xx_must_disable_cxsr(const struct intel_crtc_state *new_crtc_state,
+ const struct intel_plane_state *old_plane_state,
+ const struct intel_plane_state *new_plane_state)
+{
+ struct intel_plane *plane = to_intel_plane(new_plane_state->uapi.plane);
+ bool old_visible = old_plane_state->uapi.visible;
+ bool new_visible = new_plane_state->uapi.visible;
+ u32 old_ctl = old_plane_state->ctl;
+ u32 new_ctl = new_plane_state->ctl;
+ bool modeset, turn_on, turn_off;
+
+ if (plane->id == PLANE_CURSOR)
+ return false;
+
+ modeset = intel_crtc_needs_modeset(new_crtc_state);
+ turn_off = old_visible && (!new_visible || modeset);
+ turn_on = new_visible && (!old_visible || modeset);
+
+ /* Must disable CxSR around plane enable/disable */
+ if (turn_on || turn_off)
+ return true;
+
+ if (!old_visible || !new_visible)
+ return false;
+
+ /*
+ * Most plane control register updates are blocked while in CxSR.
+ *
+ * Tiling mode is one exception where the primary plane can
+ * apparently handle it, whereas the sprites can not (the
+ * sprite issue being only relevant on VLV/CHV where CxSR
+ * is actually possible with a sprite enabled).
+ */
+ if (plane->id == PLANE_PRIMARY) {
+ old_ctl &= ~DISP_TILED;
+ new_ctl &= ~DISP_TILED;
+ }
+
+ return old_ctl != new_ctl;
+}
+
static int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_crtc_state,
struct intel_crtc_state *new_crtc_state,
const struct intel_plane_state *old_plane_state,
@@ -482,17 +523,9 @@ static int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_cr
if (turn_on) {
if (DISPLAY_VER(dev_priv) < 5 && !IS_G4X(dev_priv))
new_crtc_state->update_wm_pre = true;
-
- /* must disable cxsr around plane enable/disable */
- if (plane->id != PLANE_CURSOR)
- new_crtc_state->disable_cxsr = true;
} else if (turn_off) {
if (DISPLAY_VER(dev_priv) < 5 && !IS_G4X(dev_priv))
new_crtc_state->update_wm_post = true;
-
- /* must disable cxsr around plane enable/disable */
- if (plane->id != PLANE_CURSOR)
- new_crtc_state->disable_cxsr = true;
} else if (intel_wm_need_update(old_plane_state, new_plane_state)) {
if (DISPLAY_VER(dev_priv) < 5 && !IS_G4X(dev_priv)) {
/* FIXME bollocks */
@@ -504,6 +537,10 @@ static int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_cr
if (visible || was_visible)
new_crtc_state->fb_bits |= plane->frontbuffer_bit;
+ if (HAS_GMCH(dev_priv) &&
+ i9xx_must_disable_cxsr(new_crtc_state, old_plane_state, new_plane_state))
+ new_crtc_state->disable_cxsr = true;
+
/*
* ILK/SNB DVSACNTR/Sprite Enable
* IVB SPR_CTL/Sprite Enable
@@ -720,7 +757,7 @@ void intel_plane_update_noarm(struct intel_plane *plane,
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- trace_intel_plane_update_noarm(&plane->base, crtc);
+ trace_intel_plane_update_noarm(plane, crtc);
if (plane->update_noarm)
plane->update_noarm(plane, crtc_state, plane_state);
@@ -732,7 +769,7 @@ void intel_plane_update_arm(struct intel_plane *plane,
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- trace_intel_plane_update_arm(&plane->base, crtc);
+ trace_intel_plane_update_arm(plane, crtc);
if (crtc_state->do_async_flip && plane->async_flip)
plane->async_flip(plane, crtc_state, plane_state, true);
@@ -745,7 +782,7 @@ void intel_plane_disable_arm(struct intel_plane *plane,
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- trace_intel_plane_disable_arm(&plane->base, crtc);
+ trace_intel_plane_disable_arm(plane, crtc);
plane->disable_arm(plane, crtc_state);
}
@@ -1005,7 +1042,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
*/
if (intel_crtc_needs_modeset(crtc_state)) {
ret = i915_sw_fence_await_reservation(&state->commit_ready,
- old_obj->base.resv, NULL,
+ old_obj->base.resv,
false, 0,
GFP_KERNEL);
if (ret < 0)
@@ -1039,8 +1076,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
struct dma_fence *fence;
ret = i915_sw_fence_await_reservation(&state->commit_ready,
- obj->base.resv, NULL,
- false,
+ obj->base.resv, false,
i915_fence_timeout(dev_priv),
GFP_KERNEL);
if (ret < 0)
diff --git a/drivers/gpu/drm/i915/display/intel_audio.c b/drivers/gpu/drm/i915/display/intel_audio.c
index aacbc6d..98c3322 100644
--- a/drivers/gpu/drm/i915/display/intel_audio.c
+++ b/drivers/gpu/drm/i915/display/intel_audio.c
@@ -250,7 +250,7 @@ static const struct hdmi_aud_ncts hdmi_aud_ncts_36bpp[] = {
/* get AUD_CONFIG_PIXEL_CLOCK_HDMI_* value for mode */
static u32 audio_config_hdmi_pixel_clock(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
const struct drm_display_mode *adjusted_mode =
&crtc_state->hw.adjusted_mode;
int i;
@@ -260,17 +260,17 @@ static u32 audio_config_hdmi_pixel_clock(const struct intel_crtc_state *crtc_sta
break;
}
- if (DISPLAY_VER(dev_priv) < 12 && adjusted_mode->crtc_clock > 148500)
+ if (DISPLAY_VER(i915) < 12 && adjusted_mode->crtc_clock > 148500)
i = ARRAY_SIZE(hdmi_audio_clock);
if (i == ARRAY_SIZE(hdmi_audio_clock)) {
- drm_dbg_kms(&dev_priv->drm,
+ drm_dbg_kms(&i915->drm,
"HDMI audio pixel clock setting for %d not found, falling back to defaults\n",
adjusted_mode->crtc_clock);
i = 1;
}
- drm_dbg_kms(&dev_priv->drm,
+ drm_dbg_kms(&i915->drm,
"Configuring HDMI audio for pixel clock %d (0x%08x)\n",
hdmi_audio_clock[i].clock,
hdmi_audio_clock[i].config);
@@ -304,96 +304,67 @@ static int audio_config_hdmi_get_n(const struct intel_crtc_state *crtc_state,
return 0;
}
-static bool intel_eld_uptodate(struct drm_connector *connector,
- i915_reg_t reg_eldv, u32 bits_eldv,
- i915_reg_t reg_elda, u32 bits_elda,
- i915_reg_t reg_edid)
+/* ELD buffer size in dwords */
+static int g4x_eld_buffer_size(struct drm_i915_private *i915)
{
- struct drm_i915_private *dev_priv = to_i915(connector->dev);
- const u8 *eld = connector->eld;
u32 tmp;
- int i;
- tmp = intel_de_read(dev_priv, reg_eldv);
- tmp &= bits_eldv;
+ tmp = intel_de_read(i915, G4X_AUD_CNTL_ST);
- if (!tmp)
- return false;
-
- tmp = intel_de_read(dev_priv, reg_elda);
- tmp &= ~bits_elda;
- intel_de_write(dev_priv, reg_elda, tmp);
-
- for (i = 0; i < drm_eld_size(eld) / 4; i++)
- if (intel_de_read(dev_priv, reg_edid) != *((const u32 *)eld + i))
- return false;
-
- return true;
+ return REG_FIELD_GET(G4X_ELD_BUFFER_SIZE_MASK, tmp);
}
static void g4x_audio_codec_disable(struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- u32 eldv, tmp;
-
- tmp = intel_de_read(dev_priv, G4X_AUD_VID_DID);
- if (tmp == INTEL_AUDIO_DEVBLC || tmp == INTEL_AUDIO_DEVCL)
- eldv = G4X_ELDV_DEVCL_DEVBLC;
- else
- eldv = G4X_ELDV_DEVCTG;
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc);
/* Invalidate ELD */
- tmp = intel_de_read(dev_priv, G4X_AUD_CNTL_ST);
- tmp &= ~eldv;
- intel_de_write(dev_priv, G4X_AUD_CNTL_ST, tmp);
+ intel_de_rmw(i915, G4X_AUD_CNTL_ST,
+ G4X_ELD_VALID, 0);
+
+ intel_crtc_wait_for_next_vblank(crtc);
+ intel_crtc_wait_for_next_vblank(crtc);
}
static void g4x_audio_codec_enable(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct drm_connector *connector = conn_state->connector;
- const u8 *eld = connector->eld;
- u32 eldv;
- u32 tmp;
- int len, i;
+ const u32 *eld = (const u32 *)connector->eld;
+ int eld_buffer_size, len, i;
- tmp = intel_de_read(dev_priv, G4X_AUD_VID_DID);
- if (tmp == INTEL_AUDIO_DEVBLC || tmp == INTEL_AUDIO_DEVCL)
- eldv = G4X_ELDV_DEVCL_DEVBLC;
- else
- eldv = G4X_ELDV_DEVCTG;
+ intel_crtc_wait_for_next_vblank(crtc);
- if (intel_eld_uptodate(connector,
- G4X_AUD_CNTL_ST, eldv,
- G4X_AUD_CNTL_ST, G4X_ELD_ADDR_MASK,
- G4X_HDMIW_HDMIEDID))
- return;
+ intel_de_rmw(i915, G4X_AUD_CNTL_ST,
+ G4X_ELD_VALID | G4X_ELD_ADDRESS_MASK, 0);
- tmp = intel_de_read(dev_priv, G4X_AUD_CNTL_ST);
- tmp &= ~(eldv | G4X_ELD_ADDR_MASK);
- len = (tmp >> 9) & 0x1f; /* ELD buffer size */
- intel_de_write(dev_priv, G4X_AUD_CNTL_ST, tmp);
+ eld_buffer_size = g4x_eld_buffer_size(i915);
+ len = min(drm_eld_size(connector->eld) / 4, eld_buffer_size);
- len = min(drm_eld_size(eld) / 4, len);
for (i = 0; i < len; i++)
- intel_de_write(dev_priv, G4X_HDMIW_HDMIEDID,
- *((const u32 *)eld + i));
+ intel_de_write(i915, G4X_HDMIW_HDMIEDID, eld[i]);
+ for (; i < eld_buffer_size; i++)
+ intel_de_write(i915, G4X_HDMIW_HDMIEDID, 0);
- tmp = intel_de_read(dev_priv, G4X_AUD_CNTL_ST);
- tmp |= eldv;
- intel_de_write(dev_priv, G4X_AUD_CNTL_ST, tmp);
+ drm_WARN_ON(&i915->drm,
+ (intel_de_read(i915, G4X_AUD_CNTL_ST) & G4X_ELD_ADDRESS_MASK) != 0);
+
+ intel_de_rmw(i915, G4X_AUD_CNTL_ST,
+ 0, G4X_ELD_VALID);
}
static void
hsw_dp_audio_config_update(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->display.audio.component;
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct i915_audio_component *acomp = i915->display.audio.component;
enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
enum port port = encoder->port;
const struct dp_aud_n_m *nm;
@@ -403,12 +374,12 @@ hsw_dp_audio_config_update(struct intel_encoder *encoder,
rate = acomp ? acomp->aud_sample_rate[port] : 0;
nm = audio_config_dp_get_n_m(crtc_state, rate);
if (nm)
- drm_dbg_kms(&dev_priv->drm, "using Maud %u, Naud %u\n", nm->m,
+ drm_dbg_kms(&i915->drm, "using Maud %u, Naud %u\n", nm->m,
nm->n);
else
- drm_dbg_kms(&dev_priv->drm, "using automatic Maud, Naud\n");
+ drm_dbg_kms(&i915->drm, "using automatic Maud, Naud\n");
- tmp = intel_de_read(dev_priv, HSW_AUD_CFG(cpu_transcoder));
+ tmp = intel_de_read(i915, HSW_AUD_CFG(cpu_transcoder));
tmp &= ~AUD_CONFIG_N_VALUE_INDEX;
tmp &= ~AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK;
tmp &= ~AUD_CONFIG_N_PROG_ENABLE;
@@ -420,9 +391,9 @@ hsw_dp_audio_config_update(struct intel_encoder *encoder,
tmp |= AUD_CONFIG_N_PROG_ENABLE;
}
- intel_de_write(dev_priv, HSW_AUD_CFG(cpu_transcoder), tmp);
+ intel_de_write(i915, HSW_AUD_CFG(cpu_transcoder), tmp);
- tmp = intel_de_read(dev_priv, HSW_AUD_M_CTS_ENABLE(cpu_transcoder));
+ tmp = intel_de_read(i915, HSW_AUD_M_CTS_ENABLE(cpu_transcoder));
tmp &= ~AUD_CONFIG_M_MASK;
tmp &= ~AUD_M_CTS_M_VALUE_INDEX;
tmp &= ~AUD_M_CTS_M_PROG_ENABLE;
@@ -433,15 +404,15 @@ hsw_dp_audio_config_update(struct intel_encoder *encoder,
tmp |= AUD_M_CTS_M_PROG_ENABLE;
}
- intel_de_write(dev_priv, HSW_AUD_M_CTS_ENABLE(cpu_transcoder), tmp);
+ intel_de_write(i915, HSW_AUD_M_CTS_ENABLE(cpu_transcoder), tmp);
}
static void
hsw_hdmi_audio_config_update(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->display.audio.component;
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct i915_audio_component *acomp = i915->display.audio.component;
enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
enum port port = encoder->port;
int n, rate;
@@ -449,7 +420,7 @@ hsw_hdmi_audio_config_update(struct intel_encoder *encoder,
rate = acomp ? acomp->aud_sample_rate[port] : 0;
- tmp = intel_de_read(dev_priv, HSW_AUD_CFG(cpu_transcoder));
+ tmp = intel_de_read(i915, HSW_AUD_CFG(cpu_transcoder));
tmp &= ~AUD_CONFIG_N_VALUE_INDEX;
tmp &= ~AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK;
tmp &= ~AUD_CONFIG_N_PROG_ENABLE;
@@ -457,25 +428,25 @@ hsw_hdmi_audio_config_update(struct intel_encoder *encoder,
n = audio_config_hdmi_get_n(crtc_state, rate);
if (n != 0) {
- drm_dbg_kms(&dev_priv->drm, "using N %d\n", n);
+ drm_dbg_kms(&i915->drm, "using N %d\n", n);
tmp &= ~AUD_CONFIG_N_MASK;
tmp |= AUD_CONFIG_N(n);
tmp |= AUD_CONFIG_N_PROG_ENABLE;
} else {
- drm_dbg_kms(&dev_priv->drm, "using automatic N\n");
+ drm_dbg_kms(&i915->drm, "using automatic N\n");
}
- intel_de_write(dev_priv, HSW_AUD_CFG(cpu_transcoder), tmp);
+ intel_de_write(i915, HSW_AUD_CFG(cpu_transcoder), tmp);
/*
* Let's disable "Enable CTS or M Prog bit"
* and let HW calculate the value
*/
- tmp = intel_de_read(dev_priv, HSW_AUD_M_CTS_ENABLE(cpu_transcoder));
+ tmp = intel_de_read(i915, HSW_AUD_M_CTS_ENABLE(cpu_transcoder));
tmp &= ~AUD_M_CTS_M_PROG_ENABLE;
tmp &= ~AUD_M_CTS_M_VALUE_INDEX;
- intel_de_write(dev_priv, HSW_AUD_M_CTS_ENABLE(cpu_transcoder), tmp);
+ intel_de_write(i915, HSW_AUD_M_CTS_ENABLE(cpu_transcoder), tmp);
}
static void
@@ -488,33 +459,48 @@ hsw_audio_config_update(struct intel_encoder *encoder,
hsw_hdmi_audio_config_update(encoder, crtc_state);
}
+/* ELD buffer size in dwords */
+static int hsw_eld_buffer_size(struct drm_i915_private *i915,
+ enum transcoder cpu_transcoder)
+{
+ u32 tmp;
+
+ tmp = intel_de_read(i915, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder));
+
+ return REG_FIELD_GET(IBX_ELD_BUFFER_SIZE_MASK, tmp);
+}
+
static void hsw_audio_codec_disable(struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc);
enum transcoder cpu_transcoder = old_crtc_state->cpu_transcoder;
- u32 tmp;
- mutex_lock(&dev_priv->display.audio.mutex);
+ mutex_lock(&i915->display.audio.mutex);
/* Disable timestamps */
- tmp = intel_de_read(dev_priv, HSW_AUD_CFG(cpu_transcoder));
- tmp &= ~AUD_CONFIG_N_VALUE_INDEX;
- tmp |= AUD_CONFIG_N_PROG_ENABLE;
- tmp &= ~AUD_CONFIG_UPPER_N_MASK;
- tmp &= ~AUD_CONFIG_LOWER_N_MASK;
- if (intel_crtc_has_dp_encoder(old_crtc_state))
- tmp |= AUD_CONFIG_N_VALUE_INDEX;
- intel_de_write(dev_priv, HSW_AUD_CFG(cpu_transcoder), tmp);
+ intel_de_rmw(i915, HSW_AUD_CFG(cpu_transcoder),
+ AUD_CONFIG_N_VALUE_INDEX |
+ AUD_CONFIG_UPPER_N_MASK |
+ AUD_CONFIG_LOWER_N_MASK,
+ AUD_CONFIG_N_PROG_ENABLE |
+ (intel_crtc_has_dp_encoder(old_crtc_state) ?
+ AUD_CONFIG_N_VALUE_INDEX : 0));
/* Invalidate ELD */
- tmp = intel_de_read(dev_priv, HSW_AUD_PIN_ELD_CP_VLD);
- tmp &= ~AUDIO_ELD_VALID(cpu_transcoder);
- tmp &= ~AUDIO_OUTPUT_ENABLE(cpu_transcoder);
- intel_de_write(dev_priv, HSW_AUD_PIN_ELD_CP_VLD, tmp);
+ intel_de_rmw(i915, HSW_AUD_PIN_ELD_CP_VLD,
+ AUDIO_ELD_VALID(cpu_transcoder), 0);
- mutex_unlock(&dev_priv->display.audio.mutex);
+ intel_crtc_wait_for_next_vblank(crtc);
+ intel_crtc_wait_for_next_vblank(crtc);
+
+ /* Disable audio presence detect */
+ intel_de_rmw(i915, HSW_AUD_PIN_ELD_CP_VLD,
+ AUDIO_OUTPUT_ENABLE(cpu_transcoder), 0);
+
+ mutex_unlock(&i915->display.audio.mutex);
}
static unsigned int calc_hblank_early_prog(struct intel_encoder *encoder,
@@ -626,178 +612,190 @@ static void enable_audio_dsc_wa(struct intel_encoder *encoder,
intel_de_write(i915, AUD_CONFIG_BE, val);
}
-#undef ROUNDING_FACTOR
-
static void hsw_audio_codec_enable(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct drm_connector *connector = conn_state->connector;
enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
- const u8 *eld = connector->eld;
- u32 tmp;
- int len, i;
+ const u32 *eld = (const u32 *)connector->eld;
+ int eld_buffer_size, len, i;
- mutex_lock(&dev_priv->display.audio.mutex);
+ mutex_lock(&i915->display.audio.mutex);
/* Enable Audio WA for 4k DSC usecases */
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_DP))
enable_audio_dsc_wa(encoder, crtc_state);
- /* Enable audio presence detect, invalidate ELD */
- tmp = intel_de_read(dev_priv, HSW_AUD_PIN_ELD_CP_VLD);
- tmp |= AUDIO_OUTPUT_ENABLE(cpu_transcoder);
- tmp &= ~AUDIO_ELD_VALID(cpu_transcoder);
- intel_de_write(dev_priv, HSW_AUD_PIN_ELD_CP_VLD, tmp);
+ /* Enable audio presence detect */
+ intel_de_rmw(i915, HSW_AUD_PIN_ELD_CP_VLD,
+ 0, AUDIO_OUTPUT_ENABLE(cpu_transcoder));
- /*
- * FIXME: We're supposed to wait for vblank here, but we have vblanks
- * disabled during the mode set. The proper fix would be to push the
- * rest of the setup into a vblank work item, queued here, but the
- * infrastructure is not there yet.
- */
+ intel_crtc_wait_for_next_vblank(crtc);
- /* Reset ELD write address */
- tmp = intel_de_read(dev_priv, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder));
- tmp &= ~IBX_ELD_ADDRESS_MASK;
- intel_de_write(dev_priv, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder), tmp);
+ /* Invalidate ELD */
+ intel_de_rmw(i915, HSW_AUD_PIN_ELD_CP_VLD,
+ AUDIO_ELD_VALID(cpu_transcoder), 0);
- /* Up to 84 bytes of hw ELD buffer */
- len = min(drm_eld_size(eld), 84);
- for (i = 0; i < len / 4; i++)
- intel_de_write(dev_priv, HSW_AUD_EDID_DATA(cpu_transcoder),
- *((const u32 *)eld + i));
+ /* Reset ELD address */
+ intel_de_rmw(i915, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder),
+ IBX_ELD_ADDRESS_MASK, 0);
+
+ eld_buffer_size = hsw_eld_buffer_size(i915, cpu_transcoder);
+ len = min(drm_eld_size(connector->eld) / 4, eld_buffer_size);
+
+ for (i = 0; i < len; i++)
+ intel_de_write(i915, HSW_AUD_EDID_DATA(cpu_transcoder), eld[i]);
+ for (; i < eld_buffer_size; i++)
+ intel_de_write(i915, HSW_AUD_EDID_DATA(cpu_transcoder), 0);
+
+ drm_WARN_ON(&i915->drm,
+ (intel_de_read(i915, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder)) &
+ IBX_ELD_ADDRESS_MASK) != 0);
/* ELD valid */
- tmp = intel_de_read(dev_priv, HSW_AUD_PIN_ELD_CP_VLD);
- tmp |= AUDIO_ELD_VALID(cpu_transcoder);
- intel_de_write(dev_priv, HSW_AUD_PIN_ELD_CP_VLD, tmp);
+ intel_de_rmw(i915, HSW_AUD_PIN_ELD_CP_VLD,
+ 0, AUDIO_ELD_VALID(cpu_transcoder));
/* Enable timestamps */
hsw_audio_config_update(encoder, crtc_state);
- mutex_unlock(&dev_priv->display.audio.mutex);
+ mutex_unlock(&i915->display.audio.mutex);
+}
+
+struct ilk_audio_regs {
+ i915_reg_t hdmiw_hdmiedid, aud_config, aud_cntl_st, aud_cntrl_st2;
+};
+
+static void ilk_audio_regs_init(struct drm_i915_private *i915,
+ enum pipe pipe,
+ struct ilk_audio_regs *regs)
+{
+ if (HAS_PCH_IBX(i915)) {
+ regs->hdmiw_hdmiedid = IBX_HDMIW_HDMIEDID(pipe);
+ regs->aud_config = IBX_AUD_CFG(pipe);
+ regs->aud_cntl_st = IBX_AUD_CNTL_ST(pipe);
+ regs->aud_cntrl_st2 = IBX_AUD_CNTL_ST2;
+ } else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) {
+ regs->hdmiw_hdmiedid = VLV_HDMIW_HDMIEDID(pipe);
+ regs->aud_config = VLV_AUD_CFG(pipe);
+ regs->aud_cntl_st = VLV_AUD_CNTL_ST(pipe);
+ regs->aud_cntrl_st2 = VLV_AUD_CNTL_ST2;
+ } else {
+ regs->hdmiw_hdmiedid = CPT_HDMIW_HDMIEDID(pipe);
+ regs->aud_config = CPT_AUD_CFG(pipe);
+ regs->aud_cntl_st = CPT_AUD_CNTL_ST(pipe);
+ regs->aud_cntrl_st2 = CPT_AUD_CNTRL_ST2;
+ }
+}
+
+/* ELD buffer size in dwords */
+static int ilk_eld_buffer_size(struct drm_i915_private *i915,
+ enum pipe pipe)
+{
+ struct ilk_audio_regs regs;
+ u32 tmp;
+
+ ilk_audio_regs_init(i915, pipe, ®s);
+
+ tmp = intel_de_read(i915, regs.aud_cntl_st);
+
+ return REG_FIELD_GET(IBX_ELD_BUFFER_SIZE_MASK, tmp);
}
static void ilk_audio_codec_disable(struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc);
- enum pipe pipe = crtc->pipe;
enum port port = encoder->port;
- u32 tmp, eldv;
- i915_reg_t aud_config, aud_cntrl_st2;
+ enum pipe pipe = crtc->pipe;
+ struct ilk_audio_regs regs;
- if (drm_WARN_ON(&dev_priv->drm, port == PORT_A))
+ if (drm_WARN_ON(&i915->drm, port == PORT_A))
return;
- if (HAS_PCH_IBX(dev_priv)) {
- aud_config = IBX_AUD_CFG(pipe);
- aud_cntrl_st2 = IBX_AUD_CNTL_ST2;
- } else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- aud_config = VLV_AUD_CFG(pipe);
- aud_cntrl_st2 = VLV_AUD_CNTL_ST2;
- } else {
- aud_config = CPT_AUD_CFG(pipe);
- aud_cntrl_st2 = CPT_AUD_CNTRL_ST2;
- }
+ ilk_audio_regs_init(i915, pipe, ®s);
+
+ mutex_lock(&i915->display.audio.mutex);
/* Disable timestamps */
- tmp = intel_de_read(dev_priv, aud_config);
- tmp &= ~AUD_CONFIG_N_VALUE_INDEX;
- tmp |= AUD_CONFIG_N_PROG_ENABLE;
- tmp &= ~AUD_CONFIG_UPPER_N_MASK;
- tmp &= ~AUD_CONFIG_LOWER_N_MASK;
- if (intel_crtc_has_dp_encoder(old_crtc_state))
- tmp |= AUD_CONFIG_N_VALUE_INDEX;
- intel_de_write(dev_priv, aud_config, tmp);
-
- eldv = IBX_ELD_VALID(port);
+ intel_de_rmw(i915, regs.aud_config,
+ AUD_CONFIG_N_VALUE_INDEX |
+ AUD_CONFIG_UPPER_N_MASK |
+ AUD_CONFIG_LOWER_N_MASK,
+ AUD_CONFIG_N_PROG_ENABLE |
+ (intel_crtc_has_dp_encoder(old_crtc_state) ?
+ AUD_CONFIG_N_VALUE_INDEX : 0));
/* Invalidate ELD */
- tmp = intel_de_read(dev_priv, aud_cntrl_st2);
- tmp &= ~eldv;
- intel_de_write(dev_priv, aud_cntrl_st2, tmp);
+ intel_de_rmw(i915, regs.aud_cntrl_st2,
+ IBX_ELD_VALID(port), 0);
+
+ mutex_unlock(&i915->display.audio.mutex);
+
+ intel_crtc_wait_for_next_vblank(crtc);
+ intel_crtc_wait_for_next_vblank(crtc);
}
static void ilk_audio_codec_enable(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct drm_connector *connector = conn_state->connector;
- enum pipe pipe = crtc->pipe;
+ const u32 *eld = (const u32 *)connector->eld;
enum port port = encoder->port;
- const u8 *eld = connector->eld;
- u32 tmp, eldv;
- int len, i;
- i915_reg_t hdmiw_hdmiedid, aud_config, aud_cntl_st, aud_cntrl_st2;
+ enum pipe pipe = crtc->pipe;
+ int eld_buffer_size, len, i;
+ struct ilk_audio_regs regs;
- if (drm_WARN_ON(&dev_priv->drm, port == PORT_A))
+ if (drm_WARN_ON(&i915->drm, port == PORT_A))
return;
- /*
- * FIXME: We're supposed to wait for vblank here, but we have vblanks
- * disabled during the mode set. The proper fix would be to push the
- * rest of the setup into a vblank work item, queued here, but the
- * infrastructure is not there yet.
- */
+ intel_crtc_wait_for_next_vblank(crtc);
- if (HAS_PCH_IBX(dev_priv)) {
- hdmiw_hdmiedid = IBX_HDMIW_HDMIEDID(pipe);
- aud_config = IBX_AUD_CFG(pipe);
- aud_cntl_st = IBX_AUD_CNTL_ST(pipe);
- aud_cntrl_st2 = IBX_AUD_CNTL_ST2;
- } else if (IS_VALLEYVIEW(dev_priv) ||
- IS_CHERRYVIEW(dev_priv)) {
- hdmiw_hdmiedid = VLV_HDMIW_HDMIEDID(pipe);
- aud_config = VLV_AUD_CFG(pipe);
- aud_cntl_st = VLV_AUD_CNTL_ST(pipe);
- aud_cntrl_st2 = VLV_AUD_CNTL_ST2;
- } else {
- hdmiw_hdmiedid = CPT_HDMIW_HDMIEDID(pipe);
- aud_config = CPT_AUD_CFG(pipe);
- aud_cntl_st = CPT_AUD_CNTL_ST(pipe);
- aud_cntrl_st2 = CPT_AUD_CNTRL_ST2;
- }
+ ilk_audio_regs_init(i915, pipe, ®s);
- eldv = IBX_ELD_VALID(port);
+ mutex_lock(&i915->display.audio.mutex);
/* Invalidate ELD */
- tmp = intel_de_read(dev_priv, aud_cntrl_st2);
- tmp &= ~eldv;
- intel_de_write(dev_priv, aud_cntrl_st2, tmp);
+ intel_de_rmw(i915, regs.aud_cntrl_st2,
+ IBX_ELD_VALID(port), 0);
- /* Reset ELD write address */
- tmp = intel_de_read(dev_priv, aud_cntl_st);
- tmp &= ~IBX_ELD_ADDRESS_MASK;
- intel_de_write(dev_priv, aud_cntl_st, tmp);
+ /* Reset ELD address */
+ intel_de_rmw(i915, regs.aud_cntl_st,
+ IBX_ELD_ADDRESS_MASK, 0);
- /* Up to 84 bytes of hw ELD buffer */
- len = min(drm_eld_size(eld), 84);
- for (i = 0; i < len / 4; i++)
- intel_de_write(dev_priv, hdmiw_hdmiedid,
- *((const u32 *)eld + i));
+ eld_buffer_size = ilk_eld_buffer_size(i915, pipe);
+ len = min(drm_eld_size(connector->eld) / 4, eld_buffer_size);
+
+ for (i = 0; i < len; i++)
+ intel_de_write(i915, regs.hdmiw_hdmiedid, eld[i]);
+ for (; i < eld_buffer_size; i++)
+ intel_de_write(i915, regs.hdmiw_hdmiedid, 0);
+
+ drm_WARN_ON(&i915->drm,
+ (intel_de_read(i915, regs.aud_cntl_st) & IBX_ELD_ADDRESS_MASK) != 0);
/* ELD valid */
- tmp = intel_de_read(dev_priv, aud_cntrl_st2);
- tmp |= eldv;
- intel_de_write(dev_priv, aud_cntrl_st2, tmp);
+ intel_de_rmw(i915, regs.aud_cntrl_st2,
+ 0, IBX_ELD_VALID(port));
/* Enable timestamps */
- tmp = intel_de_read(dev_priv, aud_config);
- tmp &= ~AUD_CONFIG_N_VALUE_INDEX;
- tmp &= ~AUD_CONFIG_N_PROG_ENABLE;
- tmp &= ~AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK;
- if (intel_crtc_has_dp_encoder(crtc_state))
- tmp |= AUD_CONFIG_N_VALUE_INDEX;
- else
- tmp |= audio_config_hdmi_pixel_clock(crtc_state);
- intel_de_write(dev_priv, aud_config, tmp);
+ intel_de_rmw(i915, regs.aud_config,
+ AUD_CONFIG_N_VALUE_INDEX |
+ AUD_CONFIG_N_PROG_ENABLE |
+ AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK,
+ (intel_crtc_has_dp_encoder(crtc_state) ?
+ AUD_CONFIG_N_VALUE_INDEX :
+ audio_config_hdmi_pixel_clock(crtc_state)));
+
+ mutex_unlock(&i915->display.audio.mutex);
}
/**
@@ -813,8 +811,8 @@ void intel_audio_codec_enable(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->display.audio.component;
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct i915_audio_component *acomp = i915->display.audio.component;
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct drm_connector *connector = conn_state->connector;
const struct drm_display_mode *adjusted_mode =
@@ -825,30 +823,30 @@ void intel_audio_codec_enable(struct intel_encoder *encoder,
if (!crtc_state->has_audio)
return;
- drm_dbg_kms(&dev_priv->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Enable audio codec on pipe %c, %u bytes ELD\n",
+ drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Enable audio codec on pipe %c, %u bytes ELD\n",
connector->base.id, connector->name,
encoder->base.base.id, encoder->base.name,
pipe_name(pipe), drm_eld_size(connector->eld));
/* FIXME precompute the ELD in .compute_config() */
if (!connector->eld[0])
- drm_dbg_kms(&dev_priv->drm,
+ drm_dbg_kms(&i915->drm,
"Bogus ELD on [CONNECTOR:%d:%s]\n",
connector->base.id, connector->name);
connector->eld[6] = drm_av_sync_delay(connector, adjusted_mode) / 2;
- if (dev_priv->display.funcs.audio)
- dev_priv->display.funcs.audio->audio_codec_enable(encoder,
- crtc_state,
- conn_state);
+ if (i915->display.funcs.audio)
+ i915->display.funcs.audio->audio_codec_enable(encoder,
+ crtc_state,
+ conn_state);
- mutex_lock(&dev_priv->display.audio.mutex);
+ mutex_lock(&i915->display.audio.mutex);
encoder->audio_connector = connector;
/* referred in audio callbacks */
- dev_priv->display.audio.encoder_map[pipe] = encoder;
- mutex_unlock(&dev_priv->display.audio.mutex);
+ i915->display.audio.encoder_map[pipe] = encoder;
+ mutex_unlock(&i915->display.audio.mutex);
if (acomp && acomp->base.audio_ops &&
acomp->base.audio_ops->pin_eld_notify) {
@@ -856,10 +854,10 @@ void intel_audio_codec_enable(struct intel_encoder *encoder,
if (!intel_crtc_has_type(crtc_state, INTEL_OUTPUT_DP_MST))
pipe = -1;
acomp->base.audio_ops->pin_eld_notify(acomp->base.audio_ops->audio_ptr,
- (int) port, (int) pipe);
+ (int)port, (int)pipe);
}
- intel_lpe_audio_notify(dev_priv, pipe, port, connector->eld,
+ intel_lpe_audio_notify(i915, pipe, port, connector->eld,
crtc_state->port_clock,
intel_crtc_has_dp_encoder(crtc_state));
}
@@ -877,8 +875,8 @@ void intel_audio_codec_disable(struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct i915_audio_component *acomp = dev_priv->display.audio.component;
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct i915_audio_component *acomp = i915->display.audio.component;
struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc);
struct drm_connector *connector = old_conn_state->connector;
enum port port = encoder->port;
@@ -887,19 +885,19 @@ void intel_audio_codec_disable(struct intel_encoder *encoder,
if (!old_crtc_state->has_audio)
return;
- drm_dbg_kms(&dev_priv->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Disable audio codec on pipe %c\n",
+ drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Disable audio codec on pipe %c\n",
connector->base.id, connector->name,
encoder->base.base.id, encoder->base.name, pipe_name(pipe));
- if (dev_priv->display.funcs.audio)
- dev_priv->display.funcs.audio->audio_codec_disable(encoder,
- old_crtc_state,
- old_conn_state);
+ if (i915->display.funcs.audio)
+ i915->display.funcs.audio->audio_codec_disable(encoder,
+ old_crtc_state,
+ old_conn_state);
- mutex_lock(&dev_priv->display.audio.mutex);
+ mutex_lock(&i915->display.audio.mutex);
encoder->audio_connector = NULL;
- dev_priv->display.audio.encoder_map[pipe] = NULL;
- mutex_unlock(&dev_priv->display.audio.mutex);
+ i915->display.audio.encoder_map[pipe] = NULL;
+ mutex_unlock(&i915->display.audio.mutex);
if (acomp && acomp->base.audio_ops &&
acomp->base.audio_ops->pin_eld_notify) {
@@ -907,10 +905,10 @@ void intel_audio_codec_disable(struct intel_encoder *encoder,
if (!intel_crtc_has_type(old_crtc_state, INTEL_OUTPUT_DP_MST))
pipe = -1;
acomp->base.audio_ops->pin_eld_notify(acomp->base.audio_ops->audio_ptr,
- (int) port, (int) pipe);
+ (int)port, (int)pipe);
}
- intel_lpe_audio_notify(dev_priv, pipe, port, NULL, 0, false);
+ intel_lpe_audio_notify(i915, pipe, port, NULL, 0, false);
}
static const struct intel_audio_funcs g4x_audio_funcs = {
@@ -930,19 +928,18 @@ static const struct intel_audio_funcs hsw_audio_funcs = {
/**
* intel_audio_hooks_init - Set up chip specific audio hooks
- * @dev_priv: device private
+ * @i915: device private
*/
-void intel_audio_hooks_init(struct drm_i915_private *dev_priv)
+void intel_audio_hooks_init(struct drm_i915_private *i915)
{
- if (IS_G4X(dev_priv)) {
- dev_priv->display.funcs.audio = &g4x_audio_funcs;
- } else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- dev_priv->display.funcs.audio = &ilk_audio_funcs;
- } else if (IS_HASWELL(dev_priv) || DISPLAY_VER(dev_priv) >= 8) {
- dev_priv->display.funcs.audio = &hsw_audio_funcs;
- } else if (HAS_PCH_SPLIT(dev_priv)) {
- dev_priv->display.funcs.audio = &ilk_audio_funcs;
- }
+ if (IS_G4X(i915))
+ i915->display.funcs.audio = &g4x_audio_funcs;
+ else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915))
+ i915->display.funcs.audio = &ilk_audio_funcs;
+ else if (IS_HASWELL(i915) || DISPLAY_VER(i915) >= 8)
+ i915->display.funcs.audio = &hsw_audio_funcs;
+ else if (HAS_PCH_SPLIT(i915))
+ i915->display.funcs.audio = &ilk_audio_funcs;
}
struct aud_ts_cdclk_m_n {
@@ -1000,7 +997,7 @@ static int glk_force_audio_cdclk_commit(struct intel_atomic_state *state,
return drm_atomic_commit(&state->base);
}
-static void glk_force_audio_cdclk(struct drm_i915_private *dev_priv,
+static void glk_force_audio_cdclk(struct drm_i915_private *i915,
bool enable)
{
struct drm_modeset_acquire_ctx ctx;
@@ -1008,13 +1005,13 @@ static void glk_force_audio_cdclk(struct drm_i915_private *dev_priv,
struct intel_crtc *crtc;
int ret;
- crtc = intel_first_crtc(dev_priv);
+ crtc = intel_first_crtc(i915);
if (!crtc)
return;
drm_modeset_acquire_init(&ctx, 0);
- state = drm_atomic_state_alloc(&dev_priv->drm);
- if (drm_WARN_ON(&dev_priv->drm, !state))
+ state = drm_atomic_state_alloc(&i915->drm);
+ if (drm_WARN_ON(&i915->drm, !state))
return;
state->acquire_ctx = &ctx;
@@ -1028,7 +1025,7 @@ static void glk_force_audio_cdclk(struct drm_i915_private *dev_priv,
goto retry;
}
- drm_WARN_ON(&dev_priv->drm, ret);
+ drm_WARN_ON(&i915->drm, ret);
drm_atomic_state_put(state);
@@ -1038,30 +1035,30 @@ static void glk_force_audio_cdclk(struct drm_i915_private *dev_priv,
static unsigned long i915_audio_component_get_power(struct device *kdev)
{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
intel_wakeref_t ret;
/* Catch potential impedance mismatches before they occur! */
BUILD_BUG_ON(sizeof(intel_wakeref_t) > sizeof(unsigned long));
- ret = intel_display_power_get(dev_priv, POWER_DOMAIN_AUDIO_PLAYBACK);
+ ret = intel_display_power_get(i915, POWER_DOMAIN_AUDIO_PLAYBACK);
- if (dev_priv->display.audio.power_refcount++ == 0) {
- if (DISPLAY_VER(dev_priv) >= 9) {
- intel_de_write(dev_priv, AUD_FREQ_CNTRL,
- dev_priv->display.audio.freq_cntrl);
- drm_dbg_kms(&dev_priv->drm,
+ if (i915->display.audio.power_refcount++ == 0) {
+ if (DISPLAY_VER(i915) >= 9) {
+ intel_de_write(i915, AUD_FREQ_CNTRL,
+ i915->display.audio.freq_cntrl);
+ drm_dbg_kms(&i915->drm,
"restored AUD_FREQ_CNTRL to 0x%x\n",
- dev_priv->display.audio.freq_cntrl);
+ i915->display.audio.freq_cntrl);
}
/* Force CDCLK to 2*BCLK as long as we need audio powered. */
- if (IS_GEMINILAKE(dev_priv))
- glk_force_audio_cdclk(dev_priv, true);
+ if (IS_GEMINILAKE(i915))
+ glk_force_audio_cdclk(i915, true);
- if (DISPLAY_VER(dev_priv) >= 10)
- intel_de_write(dev_priv, AUD_PIN_BUF_CTL,
- (intel_de_read(dev_priv, AUD_PIN_BUF_CTL) | AUD_PIN_BUF_ENABLE));
+ if (DISPLAY_VER(i915) >= 10)
+ intel_de_rmw(i915, AUD_PIN_BUF_CTL,
+ 0, AUD_PIN_BUF_ENABLE);
}
return ret;
@@ -1070,24 +1067,23 @@ static unsigned long i915_audio_component_get_power(struct device *kdev)
static void i915_audio_component_put_power(struct device *kdev,
unsigned long cookie)
{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
/* Stop forcing CDCLK to 2*BCLK if no need for audio to be powered. */
- if (--dev_priv->display.audio.power_refcount == 0)
- if (IS_GEMINILAKE(dev_priv))
- glk_force_audio_cdclk(dev_priv, false);
+ if (--i915->display.audio.power_refcount == 0)
+ if (IS_GEMINILAKE(i915))
+ glk_force_audio_cdclk(i915, false);
- intel_display_power_put(dev_priv, POWER_DOMAIN_AUDIO_PLAYBACK, cookie);
+ intel_display_power_put(i915, POWER_DOMAIN_AUDIO_PLAYBACK, cookie);
}
static void i915_audio_component_codec_wake_override(struct device *kdev,
bool enable)
{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
unsigned long cookie;
- u32 tmp;
- if (DISPLAY_VER(dev_priv) < 9)
+ if (DISPLAY_VER(i915) < 9)
return;
cookie = i915_audio_component_get_power(kdev);
@@ -1096,15 +1092,13 @@ static void i915_audio_component_codec_wake_override(struct device *kdev,
* Enable/disable generating the codec wake signal, overriding the
* internal logic to generate the codec wake to controller.
*/
- tmp = intel_de_read(dev_priv, HSW_AUD_CHICKENBIT);
- tmp &= ~SKL_AUD_CODEC_WAKE_SIGNAL;
- intel_de_write(dev_priv, HSW_AUD_CHICKENBIT, tmp);
+ intel_de_rmw(i915, HSW_AUD_CHICKENBIT,
+ SKL_AUD_CODEC_WAKE_SIGNAL, 0);
usleep_range(1000, 1500);
if (enable) {
- tmp = intel_de_read(dev_priv, HSW_AUD_CHICKENBIT);
- tmp |= SKL_AUD_CODEC_WAKE_SIGNAL;
- intel_de_write(dev_priv, HSW_AUD_CHICKENBIT, tmp);
+ intel_de_rmw(i915, HSW_AUD_CHICKENBIT,
+ 0, SKL_AUD_CODEC_WAKE_SIGNAL);
usleep_range(1000, 1500);
}
@@ -1114,12 +1108,12 @@ static void i915_audio_component_codec_wake_override(struct device *kdev,
/* Get CDCLK in kHz */
static int i915_audio_component_get_cdclk_freq(struct device *kdev)
{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
- if (drm_WARN_ON_ONCE(&dev_priv->drm, !HAS_DDI(dev_priv)))
+ if (drm_WARN_ON_ONCE(&i915->drm, !HAS_DDI(i915)))
return -ENODEV;
- return dev_priv->display.cdclk.hw.cdclk;
+ return i915->display.cdclk.hw.cdclk;
}
/*
@@ -1132,24 +1126,24 @@ static int i915_audio_component_get_cdclk_freq(struct device *kdev)
* will get the right intel_encoder with port matched
* Non-MST & (pipe < 0): get the right intel_encoder with port matched
*/
-static struct intel_encoder *get_saved_enc(struct drm_i915_private *dev_priv,
- int port, int pipe)
+static struct intel_encoder *get_saved_enc(struct drm_i915_private *i915,
+ int port, int pipe)
{
- struct intel_encoder *encoder;
-
/* MST */
if (pipe >= 0) {
- if (drm_WARN_ON(&dev_priv->drm,
- pipe >= ARRAY_SIZE(dev_priv->display.audio.encoder_map)))
+ struct intel_encoder *encoder;
+
+ if (drm_WARN_ON(&i915->drm,
+ pipe >= ARRAY_SIZE(i915->display.audio.encoder_map)))
return NULL;
- encoder = dev_priv->display.audio.encoder_map[pipe];
+ encoder = i915->display.audio.encoder_map[pipe];
/*
* when bootup, audio driver may not know it is
* MST or not. So it will poll all the port & pipe
* combinations
*/
- if (encoder != NULL && encoder->port == port &&
+ if (encoder && encoder->port == port &&
encoder->type == INTEL_OUTPUT_DP_MST)
return encoder;
}
@@ -1158,15 +1152,13 @@ static struct intel_encoder *get_saved_enc(struct drm_i915_private *dev_priv,
if (pipe > 0)
return NULL;
- for_each_pipe(dev_priv, pipe) {
- encoder = dev_priv->display.audio.encoder_map[pipe];
- if (encoder == NULL)
- continue;
+ for_each_pipe(i915, pipe) {
+ struct intel_encoder *encoder;
- if (encoder->type == INTEL_OUTPUT_DP_MST)
- continue;
+ encoder = i915->display.audio.encoder_map[pipe];
- if (port == encoder->port)
+ if (encoder && encoder->port == port &&
+ encoder->type != INTEL_OUTPUT_DP_MST)
return encoder;
}
@@ -1176,23 +1168,23 @@ static struct intel_encoder *get_saved_enc(struct drm_i915_private *dev_priv,
static int i915_audio_component_sync_audio_rate(struct device *kdev, int port,
int pipe, int rate)
{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
- struct i915_audio_component *acomp = dev_priv->display.audio.component;
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
+ struct i915_audio_component *acomp = i915->display.audio.component;
struct intel_encoder *encoder;
struct intel_crtc *crtc;
unsigned long cookie;
int err = 0;
- if (!HAS_DDI(dev_priv))
+ if (!HAS_DDI(i915))
return 0;
cookie = i915_audio_component_get_power(kdev);
- mutex_lock(&dev_priv->display.audio.mutex);
+ mutex_lock(&i915->display.audio.mutex);
/* 1. get the pipe */
- encoder = get_saved_enc(dev_priv, port, pipe);
+ encoder = get_saved_enc(i915, port, pipe);
if (!encoder || !encoder->base.crtc) {
- drm_dbg_kms(&dev_priv->drm, "Not valid for port %c\n",
+ drm_dbg_kms(&i915->drm, "Not valid for port %c\n",
port_name(port));
err = -ENODEV;
goto unlock;
@@ -1206,7 +1198,7 @@ static int i915_audio_component_sync_audio_rate(struct device *kdev, int port,
hsw_audio_config_update(encoder, crtc->config);
unlock:
- mutex_unlock(&dev_priv->display.audio.mutex);
+ mutex_unlock(&i915->display.audio.mutex);
i915_audio_component_put_power(kdev, cookie);
return err;
}
@@ -1215,18 +1207,18 @@ static int i915_audio_component_get_eld(struct device *kdev, int port,
int pipe, bool *enabled,
unsigned char *buf, int max_bytes)
{
- struct drm_i915_private *dev_priv = kdev_to_i915(kdev);
+ struct drm_i915_private *i915 = kdev_to_i915(kdev);
struct intel_encoder *intel_encoder;
const u8 *eld;
int ret = -EINVAL;
- mutex_lock(&dev_priv->display.audio.mutex);
+ mutex_lock(&i915->display.audio.mutex);
- intel_encoder = get_saved_enc(dev_priv, port, pipe);
+ intel_encoder = get_saved_enc(i915, port, pipe);
if (!intel_encoder) {
- drm_dbg_kms(&dev_priv->drm, "Not valid for port %c\n",
+ drm_dbg_kms(&i915->drm, "Not valid for port %c\n",
port_name(port));
- mutex_unlock(&dev_priv->display.audio.mutex);
+ mutex_unlock(&i915->display.audio.mutex);
return ret;
}
@@ -1238,7 +1230,7 @@ static int i915_audio_component_get_eld(struct device *kdev, int port,
memcpy(buf, eld, min(max_bytes, ret));
}
- mutex_unlock(&dev_priv->display.audio.mutex);
+ mutex_unlock(&i915->display.audio.mutex);
return ret;
}
@@ -1256,25 +1248,25 @@ static int i915_audio_component_bind(struct device *i915_kdev,
struct device *hda_kdev, void *data)
{
struct i915_audio_component *acomp = data;
- struct drm_i915_private *dev_priv = kdev_to_i915(i915_kdev);
+ struct drm_i915_private *i915 = kdev_to_i915(i915_kdev);
int i;
- if (drm_WARN_ON(&dev_priv->drm, acomp->base.ops || acomp->base.dev))
+ if (drm_WARN_ON(&i915->drm, acomp->base.ops || acomp->base.dev))
return -EEXIST;
- if (drm_WARN_ON(&dev_priv->drm,
+ if (drm_WARN_ON(&i915->drm,
!device_link_add(hda_kdev, i915_kdev,
DL_FLAG_STATELESS)))
return -ENOMEM;
- drm_modeset_lock_all(&dev_priv->drm);
+ drm_modeset_lock_all(&i915->drm);
acomp->base.ops = &i915_audio_component_ops;
acomp->base.dev = i915_kdev;
BUILD_BUG_ON(MAX_PORTS != I915_MAX_PORTS);
for (i = 0; i < ARRAY_SIZE(acomp->aud_sample_rate); i++)
acomp->aud_sample_rate[i] = 0;
- dev_priv->display.audio.component = acomp;
- drm_modeset_unlock_all(&dev_priv->drm);
+ i915->display.audio.component = acomp;
+ drm_modeset_unlock_all(&i915->drm);
return 0;
}
@@ -1283,19 +1275,19 @@ static void i915_audio_component_unbind(struct device *i915_kdev,
struct device *hda_kdev, void *data)
{
struct i915_audio_component *acomp = data;
- struct drm_i915_private *dev_priv = kdev_to_i915(i915_kdev);
+ struct drm_i915_private *i915 = kdev_to_i915(i915_kdev);
- drm_modeset_lock_all(&dev_priv->drm);
+ drm_modeset_lock_all(&i915->drm);
acomp->base.ops = NULL;
acomp->base.dev = NULL;
- dev_priv->display.audio.component = NULL;
- drm_modeset_unlock_all(&dev_priv->drm);
+ i915->display.audio.component = NULL;
+ drm_modeset_unlock_all(&i915->drm);
device_link_remove(hda_kdev, i915_kdev);
- if (dev_priv->display.audio.power_refcount)
- drm_err(&dev_priv->drm, "audio power refcount %d after unbind\n",
- dev_priv->display.audio.power_refcount);
+ if (i915->display.audio.power_refcount)
+ drm_err(&i915->drm, "audio power refcount %d after unbind\n",
+ i915->display.audio.power_refcount);
}
static const struct component_ops i915_audio_component_bind_ops = {
@@ -1314,7 +1306,7 @@ static const struct component_ops i915_audio_component_bind_ops = {
/**
* i915_audio_component_init - initialize and register the audio component
- * @dev_priv: i915 device instance
+ * @i915: i915 device instance
*
* This will register with the component framework a child component which
* will bind dynamically to the snd_hda_intel driver's corresponding master
@@ -1328,83 +1320,83 @@ static const struct component_ops i915_audio_component_bind_ops = {
* We ignore any error during registration and continue with reduced
* functionality (i.e. without HDMI audio).
*/
-static void i915_audio_component_init(struct drm_i915_private *dev_priv)
+static void i915_audio_component_init(struct drm_i915_private *i915)
{
u32 aud_freq, aud_freq_init;
int ret;
- ret = component_add_typed(dev_priv->drm.dev,
+ ret = component_add_typed(i915->drm.dev,
&i915_audio_component_bind_ops,
I915_COMPONENT_AUDIO);
if (ret < 0) {
- drm_err(&dev_priv->drm,
+ drm_err(&i915->drm,
"failed to add audio component (%d)\n", ret);
/* continue with reduced functionality */
return;
}
- if (DISPLAY_VER(dev_priv) >= 9) {
- aud_freq_init = intel_de_read(dev_priv, AUD_FREQ_CNTRL);
+ if (DISPLAY_VER(i915) >= 9) {
+ aud_freq_init = intel_de_read(i915, AUD_FREQ_CNTRL);
- if (DISPLAY_VER(dev_priv) >= 12)
+ if (DISPLAY_VER(i915) >= 12)
aud_freq = AUD_FREQ_GEN12;
else
aud_freq = aud_freq_init;
/* use BIOS provided value for TGL and RKL unless it is a known bad value */
- if ((IS_TIGERLAKE(dev_priv) || IS_ROCKETLAKE(dev_priv)) &&
+ if ((IS_TIGERLAKE(i915) || IS_ROCKETLAKE(i915)) &&
aud_freq_init != AUD_FREQ_TGL_BROKEN)
aud_freq = aud_freq_init;
- drm_dbg_kms(&dev_priv->drm, "use AUD_FREQ_CNTRL of 0x%x (init value 0x%x)\n",
+ drm_dbg_kms(&i915->drm, "use AUD_FREQ_CNTRL of 0x%x (init value 0x%x)\n",
aud_freq, aud_freq_init);
- dev_priv->display.audio.freq_cntrl = aud_freq;
+ i915->display.audio.freq_cntrl = aud_freq;
}
/* init with current cdclk */
- intel_audio_cdclk_change_post(dev_priv);
+ intel_audio_cdclk_change_post(i915);
- dev_priv->display.audio.component_registered = true;
+ i915->display.audio.component_registered = true;
}
/**
* i915_audio_component_cleanup - deregister the audio component
- * @dev_priv: i915 device instance
+ * @i915: i915 device instance
*
* Deregisters the audio component, breaking any existing binding to the
* corresponding snd_hda_intel driver's master component.
*/
-static void i915_audio_component_cleanup(struct drm_i915_private *dev_priv)
+static void i915_audio_component_cleanup(struct drm_i915_private *i915)
{
- if (!dev_priv->display.audio.component_registered)
+ if (!i915->display.audio.component_registered)
return;
- component_del(dev_priv->drm.dev, &i915_audio_component_bind_ops);
- dev_priv->display.audio.component_registered = false;
+ component_del(i915->drm.dev, &i915_audio_component_bind_ops);
+ i915->display.audio.component_registered = false;
}
/**
* intel_audio_init() - Initialize the audio driver either using
* component framework or using lpe audio bridge
- * @dev_priv: the i915 drm device private data
+ * @i915: the i915 drm device private data
*
*/
-void intel_audio_init(struct drm_i915_private *dev_priv)
+void intel_audio_init(struct drm_i915_private *i915)
{
- if (intel_lpe_audio_init(dev_priv) < 0)
- i915_audio_component_init(dev_priv);
+ if (intel_lpe_audio_init(i915) < 0)
+ i915_audio_component_init(i915);
}
/**
* intel_audio_deinit() - deinitialize the audio driver
- * @dev_priv: the i915 drm device private data
+ * @i915: the i915 drm device private data
*
*/
-void intel_audio_deinit(struct drm_i915_private *dev_priv)
+void intel_audio_deinit(struct drm_i915_private *i915)
{
- if (dev_priv->display.audio.lpe.platdev != NULL)
- intel_lpe_audio_teardown(dev_priv);
+ if (i915->display.audio.lpe.platdev != NULL)
+ intel_lpe_audio_teardown(i915);
else
- i915_audio_component_cleanup(dev_priv);
+ i915_audio_component_cleanup(i915);
}
diff --git a/drivers/gpu/drm/i915/display/intel_audio_regs.h b/drivers/gpu/drm/i915/display/intel_audio_regs.h
index d1e5844..616e7b1 100644
--- a/drivers/gpu/drm/i915/display/intel_audio_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_audio_regs.h
@@ -6,18 +6,13 @@
#ifndef __INTEL_AUDIO_REGS_H__
#define __INTEL_AUDIO_REGS_H__
-#include "i915_reg_defs.h"
-
-#define G4X_AUD_VID_DID _MMIO(DISPLAY_MMIO_BASE(dev_priv) + 0x62020)
-#define INTEL_AUDIO_DEVCL 0x808629FB
-#define INTEL_AUDIO_DEVBLC 0x80862801
-#define INTEL_AUDIO_DEVCTG 0x80862802
+#include "intel_display_reg_defs.h"
#define G4X_AUD_CNTL_ST _MMIO(0x620B4)
-#define G4X_ELDV_DEVCL_DEVBLC (1 << 13)
-#define G4X_ELDV_DEVCTG (1 << 14)
-#define G4X_ELD_ADDR_MASK (0xf << 5)
-#define G4X_ELD_ACK (1 << 4)
+#define G4X_ELD_VALID REG_BIT(14)
+#define G4X_ELD_BUFFER_SIZE_MASK REG_GENMASK(13, 9)
+#define G4X_ELD_ADDRESS_MASK REG_GENMASK(8, 5)
+#define G4X_ELD_ACK REG_BIT(4)
#define G4X_HDMIW_HDMIEDID _MMIO(0x6210C)
#define _IBX_HDMIW_HDMIEDID_A 0xE2050
@@ -28,12 +23,12 @@
#define _IBX_AUD_CNTL_ST_B 0xE21B4
#define IBX_AUD_CNTL_ST(pipe) _MMIO_PIPE(pipe, _IBX_AUD_CNTL_ST_A, \
_IBX_AUD_CNTL_ST_B)
-#define IBX_ELD_BUFFER_SIZE_MASK (0x1f << 10)
-#define IBX_ELD_ADDRESS_MASK (0x1f << 5)
-#define IBX_ELD_ACK (1 << 4)
+#define IBX_ELD_BUFFER_SIZE_MASK REG_GENMASK(14, 10)
+#define IBX_ELD_ADDRESS_MASK REG_GENMASK(9, 5)
+#define IBX_ELD_ACK REG_BIT(4)
#define IBX_AUD_CNTL_ST2 _MMIO(0xE20C0)
-#define IBX_CP_READY(port) ((1 << 1) << (((port) - 1) * 4))
-#define IBX_ELD_VALID(port) ((1 << 0) << (((port) - 1) * 4))
+#define IBX_CP_READY(port) REG_BIT(((port) - 1) * 4 + 1)
+#define IBX_ELD_VALID(port) REG_BIT(((port) - 1) * 4 + 0)
#define _CPT_HDMIW_HDMIEDID_A 0xE5050
#define _CPT_HDMIW_HDMIEDID_B 0xE5150
@@ -60,34 +55,30 @@
#define _VLV_AUD_CONFIG_A (VLV_DISPLAY_BASE + 0x62000)
#define _VLV_AUD_CONFIG_B (VLV_DISPLAY_BASE + 0x62100)
#define VLV_AUD_CFG(pipe) _MMIO_PIPE(pipe, _VLV_AUD_CONFIG_A, _VLV_AUD_CONFIG_B)
-
-#define AUD_CONFIG_N_VALUE_INDEX (1 << 29)
-#define AUD_CONFIG_N_PROG_ENABLE (1 << 28)
-#define AUD_CONFIG_UPPER_N_SHIFT 20
-#define AUD_CONFIG_UPPER_N_MASK (0xff << 20)
-#define AUD_CONFIG_LOWER_N_SHIFT 4
-#define AUD_CONFIG_LOWER_N_MASK (0xfff << 4)
-#define AUD_CONFIG_N_MASK (AUD_CONFIG_UPPER_N_MASK | AUD_CONFIG_LOWER_N_MASK)
-#define AUD_CONFIG_N(n) \
- (((((n) >> 12) & 0xff) << AUD_CONFIG_UPPER_N_SHIFT) | \
- (((n) & 0xfff) << AUD_CONFIG_LOWER_N_SHIFT))
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_SHIFT 16
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK (0xf << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_25175 (0 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_25200 (1 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_27000 (2 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_27027 (3 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_54000 (4 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_54054 (5 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_74176 (6 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_74250 (7 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_148352 (8 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_148500 (9 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_296703 (10 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_297000 (11 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_593407 (12 << 16)
-#define AUD_CONFIG_PIXEL_CLOCK_HDMI_594000 (13 << 16)
-#define AUD_CONFIG_DISABLE_NCTS (1 << 3)
+#define AUD_CONFIG_N_VALUE_INDEX REG_BIT(29)
+#define AUD_CONFIG_N_PROG_ENABLE REG_BIT(28)
+#define AUD_CONFIG_UPPER_N_MASK REG_GENMASK(27, 20)
+#define AUD_CONFIG_LOWER_N_MASK REG_GENMASK(15, 4)
+#define AUD_CONFIG_N_MASK (AUD_CONFIG_UPPER_N_MASK | \
+ AUD_CONFIG_LOWER_N_MASK)
+#define AUD_CONFIG_N(n) (REG_FIELD_PREP(AUD_CONFIG_UPPER_N_MASK, (n) >> 12) | \
+ REG_FIELD_PREP(AUD_CONFIG_LOWER_N_MASK, (n) & 0xfff))
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK REG_GENMASK(19, 16)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_25175 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 0)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_25200 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 1)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_27000 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 2)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_27027 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 3)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_54000 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 4)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_54054 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 5)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_74176 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 6)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_74250 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 7)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_148352 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 8)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_148500 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 9)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_296703 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 10)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_297000 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 11)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_593407 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 12)
+#define AUD_CONFIG_PIXEL_CLOCK_HDMI_594000 REG_FIELD_PREP(AUD_CONFIG_PIXEL_CLOCK_HDMI_MASK, 13)
+#define AUD_CONFIG_DISABLE_NCTS REG_BIT(3)
#define _HSW_AUD_CONFIG_A 0x65000
#define _HSW_AUD_CONFIG_B 0x65100
@@ -100,9 +91,9 @@
#define _HSW_AUD_M_CTS_ENABLE_A 0x65028
#define _HSW_AUD_M_CTS_ENABLE_B 0x65128
#define HSW_AUD_M_CTS_ENABLE(trans) _MMIO_TRANS(trans, _HSW_AUD_M_CTS_ENABLE_A, _HSW_AUD_M_CTS_ENABLE_B)
-#define AUD_M_CTS_M_VALUE_INDEX (1 << 21)
-#define AUD_M_CTS_M_PROG_ENABLE (1 << 20)
-#define AUD_CONFIG_M_MASK 0xfffff
+#define AUD_M_CTS_M_VALUE_INDEX REG_BIT(21)
+#define AUD_M_CTS_M_PROG_ENABLE REG_BIT(20)
+#define AUD_CONFIG_M_MASK REG_GENMASK(19, 0)
#define _HSW_AUD_DIP_ELD_CTRL_ST_A 0x650b4
#define _HSW_AUD_DIP_ELD_CTRL_ST_B 0x651b4
@@ -130,11 +121,11 @@
#define AUD_DP_2DOT0_CTRL(trans) _MMIO_TRANS(trans, _AUD_TCA_DP_2DOT0_CTRL, _AUD_TCB_DP_2DOT0_CTRL)
#define AUD_ENABLE_SDP_SPLIT REG_BIT(31)
-#define HSW_AUD_CHICKENBIT _MMIO(0x65f10)
-#define SKL_AUD_CODEC_WAKE_SIGNAL (1 << 15)
+#define HSW_AUD_CHICKENBIT _MMIO(0x65f10)
+#define SKL_AUD_CODEC_WAKE_SIGNAL REG_BIT(15)
#define AUD_FREQ_CNTRL _MMIO(0x65900)
-#define AUD_PIN_BUF_CTL _MMIO(0x48414)
+#define AUD_PIN_BUF_CTL _MMIO(0x48414)
#define AUD_PIN_BUF_ENABLE REG_BIT(31)
#define AUD_TS_CDCLK_M _MMIO(0x65ea0)
diff --git a/drivers/gpu/drm/i915/display/intel_backlight.c b/drivers/gpu/drm/i915/display/intel_backlight.c
index beba39a..71af88a 100644
--- a/drivers/gpu/drm/i915/display/intel_backlight.c
+++ b/drivers/gpu/drm/i915/display/intel_backlight.c
@@ -10,6 +10,7 @@
#include <acpi/video.h>
+#include "i915_reg.h"
#include "intel_backlight.h"
#include "intel_backlight_regs.h"
#include "intel_connector.h"
diff --git a/drivers/gpu/drm/i915/display/intel_backlight_regs.h b/drivers/gpu/drm/i915/display/intel_backlight_regs.h
index 50c1210..344eb80 100644
--- a/drivers/gpu/drm/i915/display/intel_backlight_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_backlight_regs.h
@@ -6,7 +6,7 @@
#ifndef __INTEL_BACKLIGHT_REGS_H__
#define __INTEL_BACKLIGHT_REGS_H__
-#include "i915_reg_defs.h"
+#include "intel_display_reg_defs.h"
#define _VLV_BLC_PWM_CTL2_A (DISPLAY_MMIO_BASE(dev_priv) + 0x61250)
#define _VLV_BLC_PWM_CTL2_B (DISPLAY_MMIO_BASE(dev_priv) + 0x61350)
diff --git a/drivers/gpu/drm/i915/display/intel_bios.c b/drivers/gpu/drm/i915/display/intel_bios.c
index 28bdb93..572a4e3 100644
--- a/drivers/gpu/drm/i915/display/intel_bios.c
+++ b/drivers/gpu/drm/i915/display/intel_bios.c
@@ -414,7 +414,7 @@ static void *generate_lfp_data_ptrs(struct drm_i915_private *i915,
ptrs->lvds_entries++;
if (size != 0 || ptrs->lvds_entries != 3) {
- kfree(ptrs);
+ kfree(ptrs_block);
return NULL;
}
@@ -2188,7 +2188,7 @@ static u8 map_ddc_pin(struct drm_i915_private *i915, u8 vbt_pin)
const u8 *ddc_pin_map;
int n_entries;
- if (IS_ALDERLAKE_P(i915)) {
+ if (HAS_PCH_MTP(i915) || IS_ALDERLAKE_P(i915)) {
ddc_pin_map = adlp_ddc_pin_map;
n_entries = ARRAY_SIZE(adlp_ddc_pin_map);
} else if (IS_ALDERLAKE_S(i915)) {
@@ -2676,6 +2676,14 @@ static void print_ddi_port(const struct intel_bios_encoder_data *devdata,
drm_dbg_kms(&i915->drm,
"Port %c VBT DP max link rate: %d\n",
port_name(port), dp_max_link_rate);
+
+ /*
+ * FIXME need to implement support for VBT
+ * vswing/preemph tables should this ever trigger.
+ */
+ drm_WARN(&i915->drm, child->use_vbt_vswing,
+ "Port %c asks to use VBT vswing/preemph tables\n",
+ port_name(port));
}
static void parse_ddi_port(struct intel_bios_encoder_data *devdata)
diff --git a/drivers/gpu/drm/i915/display/intel_bw.c b/drivers/gpu/drm/i915/display/intel_bw.c
index 4ace026..1c236f0 100644
--- a/drivers/gpu/drm/i915/display/intel_bw.c
+++ b/drivers/gpu/drm/i915/display/intel_bw.c
@@ -439,7 +439,8 @@ static int tgl_get_bw_info(struct drm_i915_private *dev_priv, const struct intel
return ret;
}
- if (dram_info->type == INTEL_DRAM_LPDDR4 || dram_info->type == INTEL_DRAM_LPDDR5)
+ if (DISPLAY_VER(dev_priv) < 14 &&
+ (dram_info->type == INTEL_DRAM_LPDDR4 || dram_info->type == INTEL_DRAM_LPDDR5))
num_channels *= 2;
qi.deinterleave = qi.deinterleave ? : DIV_ROUND_UP(num_channels, is_y_tile ? 4 : 2);
diff --git a/drivers/gpu/drm/i915/display/intel_cdclk.c b/drivers/gpu/drm/i915/display/intel_cdclk.c
index ed05070..b74e36d 100644
--- a/drivers/gpu/drm/i915/display/intel_cdclk.c
+++ b/drivers/gpu/drm/i915/display/intel_cdclk.c
@@ -24,6 +24,7 @@
#include <linux/time.h>
#include "hsw_ips.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_atomic_plane.h"
#include "intel_audio.h"
@@ -1220,11 +1221,6 @@ static void skl_cdclk_uninit_hw(struct drm_i915_private *dev_priv)
skl_set_cdclk(dev_priv, &cdclk_config, INVALID_PIPE);
}
-static bool has_cdclk_squasher(struct drm_i915_private *i915)
-{
- return IS_DG2(i915);
-}
-
struct intel_cdclk_vals {
u32 cdclk;
u16 refclk;
@@ -1520,7 +1516,7 @@ static void bxt_get_cdclk(struct drm_i915_private *dev_priv,
return;
}
- if (has_cdclk_squasher(dev_priv))
+ if (HAS_CDCLK_SQUASH(dev_priv))
squash_ctl = intel_de_read(dev_priv, CDCLK_SQUASH_CTL);
if (squash_ctl & CDCLK_SQUASH_ENABLE) {
@@ -1689,6 +1685,38 @@ static u32 cdclk_squash_waveform(struct drm_i915_private *dev_priv,
return 0xffff;
}
+static void icl_cdclk_pll_update(struct drm_i915_private *i915, int vco)
+{
+ if (i915->display.cdclk.hw.vco != 0 &&
+ i915->display.cdclk.hw.vco != vco)
+ icl_cdclk_pll_disable(i915);
+
+ if (i915->display.cdclk.hw.vco != vco)
+ icl_cdclk_pll_enable(i915, vco);
+}
+
+static void bxt_cdclk_pll_update(struct drm_i915_private *i915, int vco)
+{
+ if (i915->display.cdclk.hw.vco != 0 &&
+ i915->display.cdclk.hw.vco != vco)
+ bxt_de_pll_disable(i915);
+
+ if (i915->display.cdclk.hw.vco != vco)
+ bxt_de_pll_enable(i915, vco);
+}
+
+static void dg2_cdclk_squash_program(struct drm_i915_private *i915,
+ u16 waveform)
+{
+ u32 squash_ctl = 0;
+
+ if (waveform)
+ squash_ctl = CDCLK_SQUASH_ENABLE |
+ CDCLK_SQUASH_WINDOW_SIZE(0xf) | waveform;
+
+ intel_de_write(i915, CDCLK_SQUASH_CTL, squash_ctl);
+}
+
static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
const struct intel_cdclk_config *cdclk_config,
enum pipe pipe)
@@ -1724,21 +1752,10 @@ static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
if (HAS_CDCLK_CRAWL(dev_priv) && dev_priv->display.cdclk.hw.vco > 0 && vco > 0) {
if (dev_priv->display.cdclk.hw.vco != vco)
adlp_cdclk_pll_crawl(dev_priv, vco);
- } else if (DISPLAY_VER(dev_priv) >= 11) {
- if (dev_priv->display.cdclk.hw.vco != 0 &&
- dev_priv->display.cdclk.hw.vco != vco)
- icl_cdclk_pll_disable(dev_priv);
-
- if (dev_priv->display.cdclk.hw.vco != vco)
- icl_cdclk_pll_enable(dev_priv, vco);
- } else {
- if (dev_priv->display.cdclk.hw.vco != 0 &&
- dev_priv->display.cdclk.hw.vco != vco)
- bxt_de_pll_disable(dev_priv);
-
- if (dev_priv->display.cdclk.hw.vco != vco)
- bxt_de_pll_enable(dev_priv, vco);
- }
+ } else if (DISPLAY_VER(dev_priv) >= 11)
+ icl_cdclk_pll_update(dev_priv, vco);
+ else
+ bxt_cdclk_pll_update(dev_priv, vco);
waveform = cdclk_squash_waveform(dev_priv, cdclk);
@@ -1747,15 +1764,8 @@ static void bxt_set_cdclk(struct drm_i915_private *dev_priv,
else
clock = cdclk;
- if (has_cdclk_squasher(dev_priv)) {
- u32 squash_ctl = 0;
-
- if (waveform)
- squash_ctl = CDCLK_SQUASH_ENABLE |
- CDCLK_SQUASH_WINDOW_SIZE(0xf) | waveform;
-
- intel_de_write(dev_priv, CDCLK_SQUASH_CTL, squash_ctl);
- }
+ if (HAS_CDCLK_SQUASH(dev_priv))
+ dg2_cdclk_squash_program(dev_priv, waveform);
val = bxt_cdclk_cd2x_div_sel(dev_priv, clock, vco) |
bxt_cdclk_cd2x_pipe(dev_priv, pipe) |
@@ -1845,7 +1855,7 @@ static void bxt_sanitize_cdclk(struct drm_i915_private *dev_priv)
expected = skl_cdclk_decimal(cdclk);
/* Figure out what CD2X divider we should be using for this cdclk */
- if (has_cdclk_squasher(dev_priv))
+ if (HAS_CDCLK_SQUASH(dev_priv))
clock = dev_priv->display.cdclk.hw.vco / 2;
else
clock = dev_priv->display.cdclk.hw.cdclk;
@@ -1976,7 +1986,7 @@ static bool intel_cdclk_can_squash(struct drm_i915_private *dev_priv,
* the moment all platforms with squasher use a fixed cd2x
* divider.
*/
- if (!has_cdclk_squasher(dev_priv))
+ if (!HAS_CDCLK_SQUASH(dev_priv))
return false;
return a->cdclk != b->cdclk &&
@@ -2028,7 +2038,7 @@ static bool intel_cdclk_can_cd2x_update(struct drm_i915_private *dev_priv,
* the moment all platforms with squasher use a fixed cd2x
* divider.
*/
- if (has_cdclk_squasher(dev_priv))
+ if (HAS_CDCLK_SQUASH(dev_priv))
return false;
return a->cdclk != b->cdclk &&
@@ -2464,10 +2474,6 @@ static int bdw_modeset_calc_cdclk(struct intel_cdclk_state *cdclk_state)
if (min_cdclk < 0)
return min_cdclk;
- /*
- * FIXME should also account for plane ratio
- * once 64bpp pixel formats are supported.
- */
cdclk = bdw_calc_cdclk(min_cdclk);
cdclk_state->logical.cdclk = cdclk;
@@ -2534,10 +2540,6 @@ static int skl_modeset_calc_cdclk(struct intel_cdclk_state *cdclk_state)
vco = skl_dpll0_vco(cdclk_state);
- /*
- * FIXME should also account for plane ratio
- * once 64bpp pixel formats are supported.
- */
cdclk = skl_calc_cdclk(min_cdclk, vco);
cdclk_state->logical.vco = vco;
@@ -2754,7 +2756,7 @@ int intel_modeset_calc_cdclk(struct intel_atomic_state *state)
if (IS_ERR(crtc_state))
return PTR_ERR(crtc_state);
- if (drm_atomic_crtc_needs_modeset(&crtc_state->uapi))
+ if (intel_crtc_needs_modeset(crtc_state))
pipe = INVALID_PIPE;
}
@@ -2762,12 +2764,12 @@ int intel_modeset_calc_cdclk(struct intel_atomic_state *state)
&old_cdclk_state->actual,
&new_cdclk_state->actual)) {
drm_dbg_kms(&dev_priv->drm,
- "Can change cdclk via squasher\n");
+ "Can change cdclk via squashing\n");
} else if (intel_cdclk_can_crawl(dev_priv,
&old_cdclk_state->actual,
&new_cdclk_state->actual)) {
drm_dbg_kms(&dev_priv->drm,
- "Can change cdclk via crawl\n");
+ "Can change cdclk via crawling\n");
} else if (pipe != INVALID_PIPE) {
new_cdclk_state->pipe = pipe;
@@ -2777,7 +2779,7 @@ int intel_modeset_calc_cdclk(struct intel_atomic_state *state)
} else if (intel_cdclk_needs_modeset(&old_cdclk_state->actual,
&new_cdclk_state->actual)) {
/* All pipes must be switched off while we change the cdclk. */
- ret = intel_modeset_all_pipes(state);
+ ret = intel_modeset_all_pipes(state, "CDCLK change");
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c
index 6bda427..250e83f 100644
--- a/drivers/gpu/drm/i915/display/intel_color.c
+++ b/drivers/gpu/drm/i915/display/intel_color.c
@@ -22,12 +22,11 @@
*
*/
+#include "i915_reg.h"
#include "intel_color.h"
#include "intel_de.h"
#include "intel_display_types.h"
-#include "intel_dpll.h"
#include "intel_dsb.h"
-#include "vlv_dsi_pll.h"
struct intel_color_funcs {
int (*color_check)(struct intel_crtc_state *crtc_state);
@@ -186,31 +185,31 @@ static void ilk_update_pipe_csc(struct intel_crtc *crtc,
const u16 coeff[9],
const u16 postoff[3])
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
enum pipe pipe = crtc->pipe;
- intel_de_write_fw(dev_priv, PIPE_CSC_PREOFF_HI(pipe), preoff[0]);
- intel_de_write_fw(dev_priv, PIPE_CSC_PREOFF_ME(pipe), preoff[1]);
- intel_de_write_fw(dev_priv, PIPE_CSC_PREOFF_LO(pipe), preoff[2]);
+ intel_de_write_fw(i915, PIPE_CSC_PREOFF_HI(pipe), preoff[0]);
+ intel_de_write_fw(i915, PIPE_CSC_PREOFF_ME(pipe), preoff[1]);
+ intel_de_write_fw(i915, PIPE_CSC_PREOFF_LO(pipe), preoff[2]);
- intel_de_write_fw(dev_priv, PIPE_CSC_COEFF_RY_GY(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_COEFF_RY_GY(pipe),
coeff[0] << 16 | coeff[1]);
- intel_de_write_fw(dev_priv, PIPE_CSC_COEFF_BY(pipe), coeff[2] << 16);
+ intel_de_write_fw(i915, PIPE_CSC_COEFF_BY(pipe), coeff[2] << 16);
- intel_de_write_fw(dev_priv, PIPE_CSC_COEFF_RU_GU(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_COEFF_RU_GU(pipe),
coeff[3] << 16 | coeff[4]);
- intel_de_write_fw(dev_priv, PIPE_CSC_COEFF_BU(pipe), coeff[5] << 16);
+ intel_de_write_fw(i915, PIPE_CSC_COEFF_BU(pipe), coeff[5] << 16);
- intel_de_write_fw(dev_priv, PIPE_CSC_COEFF_RV_GV(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_COEFF_RV_GV(pipe),
coeff[6] << 16 | coeff[7]);
- intel_de_write_fw(dev_priv, PIPE_CSC_COEFF_BV(pipe), coeff[8] << 16);
+ intel_de_write_fw(i915, PIPE_CSC_COEFF_BV(pipe), coeff[8] << 16);
- if (DISPLAY_VER(dev_priv) >= 7) {
- intel_de_write_fw(dev_priv, PIPE_CSC_POSTOFF_HI(pipe),
+ if (DISPLAY_VER(i915) >= 7) {
+ intel_de_write_fw(i915, PIPE_CSC_POSTOFF_HI(pipe),
postoff[0]);
- intel_de_write_fw(dev_priv, PIPE_CSC_POSTOFF_ME(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_POSTOFF_ME(pipe),
postoff[1]);
- intel_de_write_fw(dev_priv, PIPE_CSC_POSTOFF_LO(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_POSTOFF_LO(pipe),
postoff[2]);
}
}
@@ -220,55 +219,55 @@ static void icl_update_output_csc(struct intel_crtc *crtc,
const u16 coeff[9],
const u16 postoff[3])
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
enum pipe pipe = crtc->pipe;
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_PREOFF_HI(pipe), preoff[0]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_PREOFF_ME(pipe), preoff[1]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_PREOFF_LO(pipe), preoff[2]);
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_PREOFF_HI(pipe), preoff[0]);
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_PREOFF_ME(pipe), preoff[1]);
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_PREOFF_LO(pipe), preoff[2]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_COEFF_RY_GY(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_COEFF_RY_GY(pipe),
coeff[0] << 16 | coeff[1]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_COEFF_BY(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_COEFF_BY(pipe),
coeff[2] << 16);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_COEFF_RU_GU(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_COEFF_RU_GU(pipe),
coeff[3] << 16 | coeff[4]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_COEFF_BU(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_COEFF_BU(pipe),
coeff[5] << 16);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_COEFF_RV_GV(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_COEFF_RV_GV(pipe),
coeff[6] << 16 | coeff[7]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_COEFF_BV(pipe),
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_COEFF_BV(pipe),
coeff[8] << 16);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_POSTOFF_HI(pipe), postoff[0]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_POSTOFF_ME(pipe), postoff[1]);
- intel_de_write_fw(dev_priv, PIPE_CSC_OUTPUT_POSTOFF_LO(pipe), postoff[2]);
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_POSTOFF_HI(pipe), postoff[0]);
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_POSTOFF_ME(pipe), postoff[1]);
+ intel_de_write_fw(i915, PIPE_CSC_OUTPUT_POSTOFF_LO(pipe), postoff[2]);
}
static bool ilk_csc_limited_range(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
/*
* FIXME if there's a gamma LUT after the CSC, we should
* do the range compression using the gamma LUT instead.
*/
return crtc_state->limited_color_range &&
- (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv) ||
- IS_DISPLAY_VER(dev_priv, 9, 10));
+ (IS_HASWELL(i915) || IS_BROADWELL(i915) ||
+ IS_DISPLAY_VER(i915, 9, 10));
}
static void ilk_csc_convert_ctm(const struct intel_crtc_state *crtc_state,
- u16 coeffs[9])
+ u16 coeffs[9], bool limited_color_range)
{
const struct drm_color_ctm *ctm = crtc_state->hw.ctm->data;
const u64 *input;
u64 temp[9];
int i;
- if (ilk_csc_limited_range(crtc_state))
+ if (limited_color_range)
input = ctm_mult_by_limited(temp, ctm->matrix);
else
input = ctm->matrix;
@@ -315,13 +314,13 @@ static void ilk_csc_convert_ctm(const struct intel_crtc_state *crtc_state,
static void ilk_load_csc_matrix(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
bool limited_color_range = ilk_csc_limited_range(crtc_state);
if (crtc_state->hw.ctm) {
u16 coeff[9];
- ilk_csc_convert_ctm(crtc_state, coeff);
+ ilk_csc_convert_ctm(crtc_state, coeff, limited_color_range);
ilk_update_pipe_csc(crtc, ilk_csc_off_zero, coeff,
limited_color_range ?
ilk_csc_postoff_limited_range :
@@ -341,7 +340,7 @@ static void ilk_load_csc_matrix(const struct intel_crtc_state *crtc_state)
* LUT is needed but CSC is not we need to load an
* identity matrix.
*/
- drm_WARN_ON(&dev_priv->drm, !IS_GEMINILAKE(dev_priv));
+ drm_WARN_ON(&i915->drm, !IS_GEMINILAKE(i915));
ilk_update_pipe_csc(crtc, ilk_csc_off_zero,
ilk_csc_coeff_identity,
@@ -356,7 +355,7 @@ static void icl_load_csc_matrix(const struct intel_crtc_state *crtc_state)
if (crtc_state->hw.ctm) {
u16 coeff[9];
- ilk_csc_convert_ctm(crtc_state, coeff);
+ ilk_csc_convert_ctm(crtc_state, coeff, false);
ilk_update_pipe_csc(crtc, ilk_csc_off_zero,
coeff, ilk_csc_off_zero);
}
@@ -375,7 +374,7 @@ static void icl_load_csc_matrix(const struct intel_crtc_state *crtc_state)
static void chv_load_cgm_csc(struct intel_crtc *crtc,
const struct drm_property_blob *blob)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
const struct drm_color_ctm *ctm = blob->data;
enum pipe pipe = crtc->pipe;
u16 coeffs[9];
@@ -399,15 +398,15 @@ static void chv_load_cgm_csc(struct intel_crtc *crtc,
coeffs[i] |= (abs_coeff >> 20) & 0xfff;
}
- intel_de_write_fw(dev_priv, CGM_PIPE_CSC_COEFF01(pipe),
+ intel_de_write_fw(i915, CGM_PIPE_CSC_COEFF01(pipe),
coeffs[1] << 16 | coeffs[0]);
- intel_de_write_fw(dev_priv, CGM_PIPE_CSC_COEFF23(pipe),
+ intel_de_write_fw(i915, CGM_PIPE_CSC_COEFF23(pipe),
coeffs[3] << 16 | coeffs[2]);
- intel_de_write_fw(dev_priv, CGM_PIPE_CSC_COEFF45(pipe),
+ intel_de_write_fw(i915, CGM_PIPE_CSC_COEFF45(pipe),
coeffs[5] << 16 | coeffs[4]);
- intel_de_write_fw(dev_priv, CGM_PIPE_CSC_COEFF67(pipe),
+ intel_de_write_fw(i915, CGM_PIPE_CSC_COEFF67(pipe),
coeffs[7] << 16 | coeffs[6]);
- intel_de_write_fw(dev_priv, CGM_PIPE_CSC_COEFF8(pipe),
+ intel_de_write_fw(i915, CGM_PIPE_CSC_COEFF8(pipe),
coeffs[8]);
}
@@ -426,32 +425,32 @@ static u32 intel_color_lut_pack(u32 val, int bit_precision)
static u32 i9xx_lut_8(const struct drm_color_lut *color)
{
- return drm_color_lut_extract(color->red, 8) << 16 |
- drm_color_lut_extract(color->green, 8) << 8 |
- drm_color_lut_extract(color->blue, 8);
+ return REG_FIELD_PREP(PALETTE_RED_MASK, drm_color_lut_extract(color->red, 8)) |
+ REG_FIELD_PREP(PALETTE_GREEN_MASK, drm_color_lut_extract(color->green, 8)) |
+ REG_FIELD_PREP(PALETTE_BLUE_MASK, drm_color_lut_extract(color->blue, 8));
}
static void i9xx_lut_8_pack(struct drm_color_lut *entry, u32 val)
{
- entry->red = intel_color_lut_pack(REG_FIELD_GET(LGC_PALETTE_RED_MASK, val), 8);
- entry->green = intel_color_lut_pack(REG_FIELD_GET(LGC_PALETTE_GREEN_MASK, val), 8);
- entry->blue = intel_color_lut_pack(REG_FIELD_GET(LGC_PALETTE_BLUE_MASK, val), 8);
+ entry->red = intel_color_lut_pack(REG_FIELD_GET(PALETTE_RED_MASK, val), 8);
+ entry->green = intel_color_lut_pack(REG_FIELD_GET(PALETTE_GREEN_MASK, val), 8);
+ entry->blue = intel_color_lut_pack(REG_FIELD_GET(PALETTE_BLUE_MASK, val), 8);
}
/* i965+ "10.6" bit interpolated format "even DW" (low 8 bits) */
static u32 i965_lut_10p6_ldw(const struct drm_color_lut *color)
{
- return (color->red & 0xff) << 16 |
- (color->green & 0xff) << 8 |
- (color->blue & 0xff);
+ return REG_FIELD_PREP(PALETTE_RED_MASK, color->red & 0xff) |
+ REG_FIELD_PREP(PALETTE_GREEN_MASK, color->green & 0xff) |
+ REG_FIELD_PREP(PALETTE_BLUE_MASK, color->blue & 0xff);
}
/* i965+ "10.6" interpolated format "odd DW" (high 8 bits) */
static u32 i965_lut_10p6_udw(const struct drm_color_lut *color)
{
- return (color->red >> 8) << 16 |
- (color->green >> 8) << 8 |
- (color->blue >> 8);
+ return REG_FIELD_PREP(PALETTE_RED_MASK, color->red >> 8) |
+ REG_FIELD_PREP(PALETTE_GREEN_MASK, color->green >> 8) |
+ REG_FIELD_PREP(PALETTE_BLUE_MASK, color->blue >> 8);
}
static void i965_lut_10p6_pack(struct drm_color_lut *entry, u32 ldw, u32 udw)
@@ -472,26 +471,42 @@ static u16 i965_lut_11p6_max_pack(u32 val)
static u32 ilk_lut_10(const struct drm_color_lut *color)
{
- return drm_color_lut_extract(color->red, 10) << 20 |
- drm_color_lut_extract(color->green, 10) << 10 |
- drm_color_lut_extract(color->blue, 10);
+ return REG_FIELD_PREP(PREC_PALETTE_10_RED_MASK, drm_color_lut_extract(color->red, 10)) |
+ REG_FIELD_PREP(PREC_PALETTE_10_GREEN_MASK, drm_color_lut_extract(color->green, 10)) |
+ REG_FIELD_PREP(PREC_PALETTE_10_BLUE_MASK, drm_color_lut_extract(color->blue, 10));
}
static void ilk_lut_10_pack(struct drm_color_lut *entry, u32 val)
{
- entry->red = intel_color_lut_pack(REG_FIELD_GET(PREC_PALETTE_RED_MASK, val), 10);
- entry->green = intel_color_lut_pack(REG_FIELD_GET(PREC_PALETTE_GREEN_MASK, val), 10);
- entry->blue = intel_color_lut_pack(REG_FIELD_GET(PREC_PALETTE_BLUE_MASK, val), 10);
+ entry->red = intel_color_lut_pack(REG_FIELD_GET(PREC_PALETTE_10_RED_MASK, val), 10);
+ entry->green = intel_color_lut_pack(REG_FIELD_GET(PREC_PALETTE_10_GREEN_MASK, val), 10);
+ entry->blue = intel_color_lut_pack(REG_FIELD_GET(PREC_PALETTE_10_BLUE_MASK, val), 10);
}
-static void icl_lut_multi_seg_pack(struct drm_color_lut *entry, u32 ldw, u32 udw)
+/* ilk+ "12.4" interpolated format (low 6 bits) */
+static u32 ilk_lut_12p4_ldw(const struct drm_color_lut *color)
{
- entry->red = REG_FIELD_GET(PAL_PREC_MULTI_SEG_RED_UDW_MASK, udw) << 6 |
- REG_FIELD_GET(PAL_PREC_MULTI_SEG_RED_LDW_MASK, ldw);
- entry->green = REG_FIELD_GET(PAL_PREC_MULTI_SEG_GREEN_UDW_MASK, udw) << 6 |
- REG_FIELD_GET(PAL_PREC_MULTI_SEG_GREEN_LDW_MASK, ldw);
- entry->blue = REG_FIELD_GET(PAL_PREC_MULTI_SEG_BLUE_UDW_MASK, udw) << 6 |
- REG_FIELD_GET(PAL_PREC_MULTI_SEG_BLUE_LDW_MASK, ldw);
+ return REG_FIELD_PREP(PREC_PALETTE_12P4_RED_LDW_MASK, color->red & 0x3f) |
+ REG_FIELD_PREP(PREC_PALETTE_12P4_GREEN_LDW_MASK, color->green & 0x3f) |
+ REG_FIELD_PREP(PREC_PALETTE_12P4_BLUE_LDW_MASK, color->blue & 0x3f);
+}
+
+/* ilk+ "12.4" interpolated format (high 10 bits) */
+static u32 ilk_lut_12p4_udw(const struct drm_color_lut *color)
+{
+ return REG_FIELD_PREP(PREC_PALETTE_12P4_RED_UDW_MASK, color->red >> 6) |
+ REG_FIELD_PREP(PREC_PALETTE_12P4_GREEN_UDW_MASK, color->green >> 6) |
+ REG_FIELD_PREP(PREC_PALETTE_12P4_BLUE_UDW_MASK, color->blue >> 6);
+}
+
+static void ilk_lut_12p4_pack(struct drm_color_lut *entry, u32 ldw, u32 udw)
+{
+ entry->red = REG_FIELD_GET(PREC_PALETTE_12P4_RED_UDW_MASK, udw) << 6 |
+ REG_FIELD_GET(PREC_PALETTE_12P4_RED_LDW_MASK, ldw);
+ entry->green = REG_FIELD_GET(PREC_PALETTE_12P4_GREEN_UDW_MASK, udw) << 6 |
+ REG_FIELD_GET(PREC_PALETTE_12P4_GREEN_LDW_MASK, ldw);
+ entry->blue = REG_FIELD_GET(PREC_PALETTE_12P4_BLUE_UDW_MASK, udw) << 6 |
+ REG_FIELD_GET(PREC_PALETTE_12P4_BLUE_LDW_MASK, ldw);
}
static void icl_color_commit_noarm(const struct intel_crtc_state *crtc_state)
@@ -513,31 +528,31 @@ static void i9xx_color_commit_arm(const struct intel_crtc_state *crtc_state)
static void ilk_color_commit_arm(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
/* update PIPECONF GAMMA_MODE */
ilk_set_pipeconf(crtc_state);
- intel_de_write_fw(dev_priv, PIPE_CSC_MODE(crtc->pipe),
+ intel_de_write_fw(i915, PIPE_CSC_MODE(crtc->pipe),
crtc_state->csc_mode);
}
static void hsw_color_commit_arm(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
- intel_de_write(dev_priv, GAMMA_MODE(crtc->pipe),
+ intel_de_write(i915, GAMMA_MODE(crtc->pipe),
crtc_state->gamma_mode);
- intel_de_write_fw(dev_priv, PIPE_CSC_MODE(crtc->pipe),
+ intel_de_write_fw(i915, PIPE_CSC_MODE(crtc->pipe),
crtc_state->csc_mode);
}
static void skl_color_commit_arm(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
enum pipe pipe = crtc->pipe;
u32 val = 0;
@@ -550,15 +565,65 @@ static void skl_color_commit_arm(const struct intel_crtc_state *crtc_state)
val |= SKL_BOTTOM_COLOR_GAMMA_ENABLE;
if (crtc_state->csc_enable)
val |= SKL_BOTTOM_COLOR_CSC_ENABLE;
- intel_de_write(dev_priv, SKL_BOTTOM_COLOR(pipe), val);
+ intel_de_write(i915, SKL_BOTTOM_COLOR(pipe), val);
- intel_de_write(dev_priv, GAMMA_MODE(crtc->pipe),
+ intel_de_write(i915, GAMMA_MODE(crtc->pipe),
crtc_state->gamma_mode);
- intel_de_write_fw(dev_priv, PIPE_CSC_MODE(crtc->pipe),
+ intel_de_write_fw(i915, PIPE_CSC_MODE(crtc->pipe),
crtc_state->csc_mode);
}
+static struct drm_property_blob *
+create_linear_lut(struct drm_i915_private *i915, int lut_size)
+{
+ struct drm_property_blob *blob;
+ struct drm_color_lut *lut;
+ int i;
+
+ blob = drm_property_create_blob(&i915->drm,
+ sizeof(lut[0]) * lut_size,
+ NULL);
+ if (IS_ERR(blob))
+ return blob;
+
+ lut = blob->data;
+
+ for (i = 0; i < lut_size; i++) {
+ u16 val = 0xffff * i / (lut_size - 1);
+
+ lut[i].red = val;
+ lut[i].green = val;
+ lut[i].blue = val;
+ }
+
+ return blob;
+}
+
+static struct drm_property_blob *
+create_resized_lut(struct drm_i915_private *i915,
+ const struct drm_property_blob *blob_in, int lut_out_size)
+{
+ int i, lut_in_size = drm_color_lut_size(blob_in);
+ struct drm_property_blob *blob_out;
+ const struct drm_color_lut *lut_in;
+ struct drm_color_lut *lut_out;
+
+ blob_out = drm_property_create_blob(&i915->drm,
+ sizeof(lut_out[0]) * lut_out_size,
+ NULL);
+ if (IS_ERR(blob_out))
+ return blob_out;
+
+ lut_in = blob_in->data;
+ lut_out = blob_out->data;
+
+ for (i = 0; i < lut_out_size; i++)
+ lut_out[i] = lut_in[i * (lut_in_size - 1) / (lut_out_size - 1)];
+
+ return blob_out;
+}
+
static void i9xx_load_lut_8(struct intel_crtc *crtc,
const struct drm_property_blob *blob)
{
@@ -580,12 +645,9 @@ static void i9xx_load_lut_8(struct intel_crtc *crtc,
static void i9xx_load_luts(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
- assert_pll_enabled(dev_priv, crtc->pipe);
-
- i9xx_load_lut_8(crtc, gamma_lut);
+ i9xx_load_lut_8(crtc, post_csc_lut);
}
static void i965_load_lut_10p6(struct intel_crtc *crtc,
@@ -611,24 +673,18 @@ static void i965_load_lut_10p6(struct intel_crtc *crtc,
static void i965_load_luts(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
-
- if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_DSI))
- assert_dsi_pll_enabled(dev_priv);
- else
- assert_pll_enabled(dev_priv, crtc->pipe);
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
if (crtc_state->gamma_mode == GAMMA_MODE_MODE_8BIT)
- i9xx_load_lut_8(crtc, gamma_lut);
+ i9xx_load_lut_8(crtc, post_csc_lut);
else
- i965_load_lut_10p6(crtc, gamma_lut);
+ i965_load_lut_10p6(crtc, post_csc_lut);
}
static void ilk_load_lut_8(struct intel_crtc *crtc,
const struct drm_property_blob *blob)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
const struct drm_color_lut *lut;
enum pipe pipe = crtc->pipe;
int i;
@@ -639,34 +695,36 @@ static void ilk_load_lut_8(struct intel_crtc *crtc,
lut = blob->data;
for (i = 0; i < 256; i++)
- intel_de_write_fw(dev_priv, LGC_PALETTE(pipe, i),
+ intel_de_write_fw(i915, LGC_PALETTE(pipe, i),
i9xx_lut_8(&lut[i]));
}
static void ilk_load_lut_10(struct intel_crtc *crtc,
const struct drm_property_blob *blob)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
const struct drm_color_lut *lut = blob->data;
int i, lut_size = drm_color_lut_size(blob);
enum pipe pipe = crtc->pipe;
for (i = 0; i < lut_size; i++)
- intel_de_write_fw(dev_priv, PREC_PALETTE(pipe, i),
+ intel_de_write_fw(i915, PREC_PALETTE(pipe, i),
ilk_lut_10(&lut[i]));
}
static void ilk_load_luts(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
+ const struct drm_property_blob *pre_csc_lut = crtc_state->pre_csc_lut;
+ const struct drm_property_blob *blob = post_csc_lut ?: pre_csc_lut;
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
- ilk_load_lut_8(crtc, gamma_lut);
+ ilk_load_lut_8(crtc, blob);
break;
case GAMMA_MODE_MODE_10BIT:
- ilk_load_lut_10(crtc, gamma_lut);
+ ilk_load_lut_10(crtc, blob);
break;
default:
MISSING_CASE(crtc_state->gamma_mode);
@@ -691,27 +749,22 @@ static void ivb_load_lut_10(struct intel_crtc *crtc,
const struct drm_property_blob *blob,
u32 prec_index)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- int hw_lut_size = ivb_lut_10_size(prec_index);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
const struct drm_color_lut *lut = blob->data;
int i, lut_size = drm_color_lut_size(blob);
enum pipe pipe = crtc->pipe;
- for (i = 0; i < hw_lut_size; i++) {
- /* We discard half the user entries in split gamma mode */
- const struct drm_color_lut *entry =
- &lut[i * (lut_size - 1) / (hw_lut_size - 1)];
-
- intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), prec_index++);
- intel_de_write_fw(dev_priv, PREC_PAL_DATA(pipe),
- ilk_lut_10(entry));
+ for (i = 0; i < lut_size; i++) {
+ intel_de_write_fw(i915, PREC_PAL_INDEX(pipe), prec_index++);
+ intel_de_write_fw(i915, PREC_PAL_DATA(pipe),
+ ilk_lut_10(&lut[i]));
}
/*
* Reset the index, otherwise it prevents the legacy palette to be
* written properly.
*/
- intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), 0);
+ intel_de_write_fw(i915, PREC_PAL_INDEX(pipe), 0);
}
/* On BDW+ the index auto increment mode actually works */
@@ -719,73 +772,63 @@ static void bdw_load_lut_10(struct intel_crtc *crtc,
const struct drm_property_blob *blob,
u32 prec_index)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- int hw_lut_size = ivb_lut_10_size(prec_index);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
const struct drm_color_lut *lut = blob->data;
int i, lut_size = drm_color_lut_size(blob);
enum pipe pipe = crtc->pipe;
- intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe),
+ intel_de_write_fw(i915, PREC_PAL_INDEX(pipe),
prec_index | PAL_PREC_AUTO_INCREMENT);
- for (i = 0; i < hw_lut_size; i++) {
- /* We discard half the user entries in split gamma mode */
- const struct drm_color_lut *entry =
- &lut[i * (lut_size - 1) / (hw_lut_size - 1)];
-
- intel_de_write_fw(dev_priv, PREC_PAL_DATA(pipe),
- ilk_lut_10(entry));
- }
+ for (i = 0; i < lut_size; i++)
+ intel_de_write_fw(i915, PREC_PAL_DATA(pipe),
+ ilk_lut_10(&lut[i]));
/*
* Reset the index, otherwise it prevents the legacy palette to be
* written properly.
*/
- intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), 0);
+ intel_de_write_fw(i915, PREC_PAL_INDEX(pipe), 0);
}
static void ivb_load_lut_ext_max(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
enum pipe pipe = crtc->pipe;
/* Program the max register to clamp values > 1.0. */
intel_dsb_reg_write(crtc_state, PREC_PAL_EXT_GC_MAX(pipe, 0), 1 << 16);
intel_dsb_reg_write(crtc_state, PREC_PAL_EXT_GC_MAX(pipe, 1), 1 << 16);
intel_dsb_reg_write(crtc_state, PREC_PAL_EXT_GC_MAX(pipe, 2), 1 << 16);
+}
- /*
- * Program the gc max 2 register to clamp values > 1.0.
- * ToDo: Extend the ABI to be able to program values
- * from 3.0 to 7.0
- */
- if (DISPLAY_VER(dev_priv) >= 10) {
- intel_dsb_reg_write(crtc_state, PREC_PAL_EXT2_GC_MAX(pipe, 0),
- 1 << 16);
- intel_dsb_reg_write(crtc_state, PREC_PAL_EXT2_GC_MAX(pipe, 1),
- 1 << 16);
- intel_dsb_reg_write(crtc_state, PREC_PAL_EXT2_GC_MAX(pipe, 2),
- 1 << 16);
- }
+static void glk_load_lut_ext2_max(const struct intel_crtc_state *crtc_state)
+{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
+ enum pipe pipe = crtc->pipe;
+
+ /* Program the max register to clamp values > 1.0. */
+ intel_dsb_reg_write(crtc_state, PREC_PAL_EXT2_GC_MAX(pipe, 0), 1 << 16);
+ intel_dsb_reg_write(crtc_state, PREC_PAL_EXT2_GC_MAX(pipe, 1), 1 << 16);
+ intel_dsb_reg_write(crtc_state, PREC_PAL_EXT2_GC_MAX(pipe, 2), 1 << 16);
}
static void ivb_load_luts(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
- const struct drm_property_blob *degamma_lut = crtc_state->hw.degamma_lut;
- const struct drm_property_blob *blob = gamma_lut ?: degamma_lut;
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
+ const struct drm_property_blob *pre_csc_lut = crtc_state->pre_csc_lut;
+ const struct drm_property_blob *blob = post_csc_lut ?: pre_csc_lut;
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
ilk_load_lut_8(crtc, blob);
break;
case GAMMA_MODE_MODE_SPLIT:
- ivb_load_lut_10(crtc, degamma_lut, PAL_PREC_SPLIT_MODE |
+ ivb_load_lut_10(crtc, pre_csc_lut, PAL_PREC_SPLIT_MODE |
PAL_PREC_INDEX_VALUE(0));
ivb_load_lut_ext_max(crtc_state);
- ivb_load_lut_10(crtc, gamma_lut, PAL_PREC_SPLIT_MODE |
+ ivb_load_lut_10(crtc, post_csc_lut, PAL_PREC_SPLIT_MODE |
PAL_PREC_INDEX_VALUE(512));
break;
case GAMMA_MODE_MODE_10BIT:
@@ -802,19 +845,19 @@ static void ivb_load_luts(const struct intel_crtc_state *crtc_state)
static void bdw_load_luts(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
- const struct drm_property_blob *degamma_lut = crtc_state->hw.degamma_lut;
- const struct drm_property_blob *blob = gamma_lut ?: degamma_lut;
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
+ const struct drm_property_blob *pre_csc_lut = crtc_state->pre_csc_lut;
+ const struct drm_property_blob *blob = post_csc_lut ?: pre_csc_lut;
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
ilk_load_lut_8(crtc, blob);
break;
case GAMMA_MODE_MODE_SPLIT:
- bdw_load_lut_10(crtc, degamma_lut, PAL_PREC_SPLIT_MODE |
+ bdw_load_lut_10(crtc, pre_csc_lut, PAL_PREC_SPLIT_MODE |
PAL_PREC_INDEX_VALUE(0));
ivb_load_lut_ext_max(crtc_state);
- bdw_load_lut_10(crtc, gamma_lut, PAL_PREC_SPLIT_MODE |
+ bdw_load_lut_10(crtc, post_csc_lut, PAL_PREC_SPLIT_MODE |
PAL_PREC_INDEX_VALUE(512));
break;
case GAMMA_MODE_MODE_10BIT:
@@ -837,21 +880,22 @@ static int glk_degamma_lut_size(struct drm_i915_private *i915)
return 35;
}
-static void glk_load_degamma_lut(const struct intel_crtc_state *crtc_state)
+static void glk_load_degamma_lut(const struct intel_crtc_state *crtc_state,
+ const struct drm_property_blob *blob)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ const struct drm_color_lut *lut = blob->data;
+ int i, lut_size = drm_color_lut_size(blob);
enum pipe pipe = crtc->pipe;
- int i, lut_size = INTEL_INFO(dev_priv)->display.color.degamma_lut_size;
- const struct drm_color_lut *lut = crtc_state->hw.degamma_lut->data;
/*
* When setting the auto-increment bit, the hardware seems to
* ignore the index bits, so we need to reset it to index 0
* separately.
*/
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe),
+ intel_de_write_fw(i915, PRE_CSC_GAMC_INDEX(pipe), 0);
+ intel_de_write_fw(i915, PRE_CSC_GAMC_INDEX(pipe),
PRE_CSC_GAMC_AUTO_INCREMENT);
for (i = 0; i < lut_size; i++) {
@@ -868,71 +912,34 @@ static void glk_load_degamma_lut(const struct intel_crtc_state *crtc_state)
* ToDo: Extend to max 7.0. Enable 32 bit input value
* as compared to just 16 to achieve this.
*/
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe),
+ intel_de_write_fw(i915, PRE_CSC_GAMC_DATA(pipe),
lut[i].green);
}
/* Clamp values > 1.0. */
- while (i++ < glk_degamma_lut_size(dev_priv))
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe), 1 << 16);
+ while (i++ < glk_degamma_lut_size(i915))
+ intel_de_write_fw(i915, PRE_CSC_GAMC_DATA(pipe), 1 << 16);
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
-}
-
-static void glk_load_degamma_lut_linear(const struct intel_crtc_state *crtc_state)
-{
- struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- enum pipe pipe = crtc->pipe;
- int i, lut_size = INTEL_INFO(dev_priv)->display.color.degamma_lut_size;
-
- /*
- * When setting the auto-increment bit, the hardware seems to
- * ignore the index bits, so we need to reset it to index 0
- * separately.
- */
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe),
- PRE_CSC_GAMC_AUTO_INCREMENT);
-
- for (i = 0; i < lut_size; i++) {
- u32 v = (i << 16) / (lut_size - 1);
-
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe), v);
- }
-
- /* Clamp values > 1.0. */
- while (i++ < 35)
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_DATA(pipe), 1 << 16);
-
- intel_de_write_fw(dev_priv, PRE_CSC_GAMC_INDEX(pipe), 0);
+ intel_de_write_fw(i915, PRE_CSC_GAMC_INDEX(pipe), 0);
}
static void glk_load_luts(const struct intel_crtc_state *crtc_state)
{
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
+ const struct drm_property_blob *pre_csc_lut = crtc_state->pre_csc_lut;
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- /*
- * On GLK+ both pipe CSC and degamma LUT are controlled
- * by csc_enable. Hence for the cases where the CSC is
- * needed but degamma LUT is not we need to load a
- * linear degamma LUT. In fact we'll just always load
- * the degama LUT so that we don't have to reload
- * it every time the pipe CSC is being enabled.
- */
- if (crtc_state->hw.degamma_lut)
- glk_load_degamma_lut(crtc_state);
- else
- glk_load_degamma_lut_linear(crtc_state);
+ if (pre_csc_lut)
+ glk_load_degamma_lut(crtc_state, pre_csc_lut);
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
- ilk_load_lut_8(crtc, gamma_lut);
+ ilk_load_lut_8(crtc, post_csc_lut);
break;
case GAMMA_MODE_MODE_10BIT:
- bdw_load_lut_10(crtc, gamma_lut, PAL_PREC_INDEX_VALUE(0));
+ bdw_load_lut_10(crtc, post_csc_lut, PAL_PREC_INDEX_VALUE(0));
ivb_load_lut_ext_max(crtc_state);
+ glk_load_lut_ext2_max(crtc_state);
break;
default:
MISSING_CASE(crtc_state->gamma_mode);
@@ -940,23 +947,9 @@ static void glk_load_luts(const struct intel_crtc_state *crtc_state)
}
}
-/* ilk+ "12.4" interpolated format (high 10 bits) */
-static u32 ilk_lut_12p4_udw(const struct drm_color_lut *color)
-{
- return (color->red >> 6) << 20 | (color->green >> 6) << 10 |
- (color->blue >> 6);
-}
-
-/* ilk+ "12.4" interpolated format (low 6 bits) */
-static u32 ilk_lut_12p4_ldw(const struct drm_color_lut *color)
-{
- return (color->red & 0x3f) << 24 | (color->green & 0x3f) << 14 |
- (color->blue & 0x3f) << 4;
-}
-
static void
-icl_load_gcmax(const struct intel_crtc_state *crtc_state,
- const struct drm_color_lut *color)
+ivb_load_lut_max(const struct intel_crtc_state *crtc_state,
+ const struct drm_color_lut *color)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
enum pipe pipe = crtc->pipe;
@@ -971,7 +964,7 @@ static void
icl_program_gamma_superfine_segment(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- const struct drm_property_blob *blob = crtc_state->hw.gamma_lut;
+ const struct drm_property_blob *blob = crtc_state->post_csc_lut;
const struct drm_color_lut *lut = blob->data;
enum pipe pipe = crtc->pipe;
int i;
@@ -1000,7 +993,7 @@ static void
icl_program_gamma_multi_segment(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- const struct drm_property_blob *blob = crtc_state->hw.gamma_lut;
+ const struct drm_property_blob *blob = crtc_state->post_csc_lut;
const struct drm_color_lut *lut = blob->data;
const struct drm_color_lut *entry;
enum pipe pipe = crtc->pipe;
@@ -1048,29 +1041,32 @@ icl_program_gamma_multi_segment(const struct intel_crtc_state *crtc_state)
/* The last entry in the LUT is to be programmed in GCMAX */
entry = &lut[256 * 8 * 128];
- icl_load_gcmax(crtc_state, entry);
- ivb_load_lut_ext_max(crtc_state);
+ ivb_load_lut_max(crtc_state, entry);
}
static void icl_load_luts(const struct intel_crtc_state *crtc_state)
{
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
+ const struct drm_property_blob *pre_csc_lut = crtc_state->pre_csc_lut;
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- if (crtc_state->hw.degamma_lut)
- glk_load_degamma_lut(crtc_state);
+ if (pre_csc_lut)
+ glk_load_degamma_lut(crtc_state, pre_csc_lut);
switch (crtc_state->gamma_mode & GAMMA_MODE_MODE_MASK) {
case GAMMA_MODE_MODE_8BIT:
- ilk_load_lut_8(crtc, gamma_lut);
+ ilk_load_lut_8(crtc, post_csc_lut);
break;
case GAMMA_MODE_MODE_12BIT_MULTI_SEGMENTED:
icl_program_gamma_superfine_segment(crtc_state);
icl_program_gamma_multi_segment(crtc_state);
+ ivb_load_lut_ext_max(crtc_state);
+ glk_load_lut_ext2_max(crtc_state);
break;
case GAMMA_MODE_MODE_10BIT:
- bdw_load_lut_10(crtc, gamma_lut, PAL_PREC_INDEX_VALUE(0));
+ bdw_load_lut_10(crtc, post_csc_lut, PAL_PREC_INDEX_VALUE(0));
ivb_load_lut_ext_max(crtc_state);
+ glk_load_lut_ext2_max(crtc_state);
break;
default:
MISSING_CASE(crtc_state->gamma_mode);
@@ -1082,61 +1078,61 @@ static void icl_load_luts(const struct intel_crtc_state *crtc_state)
static u32 chv_cgm_degamma_ldw(const struct drm_color_lut *color)
{
- return drm_color_lut_extract(color->green, 14) << 16 |
- drm_color_lut_extract(color->blue, 14);
+ return REG_FIELD_PREP(CGM_PIPE_DEGAMMA_GREEN_LDW_MASK, drm_color_lut_extract(color->green, 14)) |
+ REG_FIELD_PREP(CGM_PIPE_DEGAMMA_BLUE_LDW_MASK, drm_color_lut_extract(color->blue, 14));
}
static u32 chv_cgm_degamma_udw(const struct drm_color_lut *color)
{
- return drm_color_lut_extract(color->red, 14);
+ return REG_FIELD_PREP(CGM_PIPE_DEGAMMA_RED_UDW_MASK, drm_color_lut_extract(color->red, 14));
}
static void chv_load_cgm_degamma(struct intel_crtc *crtc,
const struct drm_property_blob *blob)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
const struct drm_color_lut *lut = blob->data;
int i, lut_size = drm_color_lut_size(blob);
enum pipe pipe = crtc->pipe;
for (i = 0; i < lut_size; i++) {
- intel_de_write_fw(dev_priv, CGM_PIPE_DEGAMMA(pipe, i, 0),
+ intel_de_write_fw(i915, CGM_PIPE_DEGAMMA(pipe, i, 0),
chv_cgm_degamma_ldw(&lut[i]));
- intel_de_write_fw(dev_priv, CGM_PIPE_DEGAMMA(pipe, i, 1),
+ intel_de_write_fw(i915, CGM_PIPE_DEGAMMA(pipe, i, 1),
chv_cgm_degamma_udw(&lut[i]));
}
}
static u32 chv_cgm_gamma_ldw(const struct drm_color_lut *color)
{
- return drm_color_lut_extract(color->green, 10) << 16 |
- drm_color_lut_extract(color->blue, 10);
+ return REG_FIELD_PREP(CGM_PIPE_GAMMA_GREEN_LDW_MASK, drm_color_lut_extract(color->green, 10)) |
+ REG_FIELD_PREP(CGM_PIPE_GAMMA_BLUE_LDW_MASK, drm_color_lut_extract(color->blue, 10));
}
static u32 chv_cgm_gamma_udw(const struct drm_color_lut *color)
{
- return drm_color_lut_extract(color->red, 10);
+ return REG_FIELD_PREP(CGM_PIPE_GAMMA_RED_UDW_MASK, drm_color_lut_extract(color->red, 10));
}
static void chv_cgm_gamma_pack(struct drm_color_lut *entry, u32 ldw, u32 udw)
{
- entry->green = intel_color_lut_pack(REG_FIELD_GET(CGM_PIPE_GAMMA_GREEN_MASK, ldw), 10);
- entry->blue = intel_color_lut_pack(REG_FIELD_GET(CGM_PIPE_GAMMA_BLUE_MASK, ldw), 10);
- entry->red = intel_color_lut_pack(REG_FIELD_GET(CGM_PIPE_GAMMA_RED_MASK, udw), 10);
+ entry->green = intel_color_lut_pack(REG_FIELD_GET(CGM_PIPE_GAMMA_GREEN_LDW_MASK, ldw), 10);
+ entry->blue = intel_color_lut_pack(REG_FIELD_GET(CGM_PIPE_GAMMA_BLUE_LDW_MASK, ldw), 10);
+ entry->red = intel_color_lut_pack(REG_FIELD_GET(CGM_PIPE_GAMMA_RED_UDW_MASK, udw), 10);
}
static void chv_load_cgm_gamma(struct intel_crtc *crtc,
const struct drm_property_blob *blob)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
const struct drm_color_lut *lut = blob->data;
int i, lut_size = drm_color_lut_size(blob);
enum pipe pipe = crtc->pipe;
for (i = 0; i < lut_size; i++) {
- intel_de_write_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 0),
+ intel_de_write_fw(i915, CGM_PIPE_GAMMA(pipe, i, 0),
chv_cgm_gamma_ldw(&lut[i]));
- intel_de_write_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 1),
+ intel_de_write_fw(i915, CGM_PIPE_GAMMA(pipe, i, 1),
chv_cgm_gamma_udw(&lut[i]));
}
}
@@ -1144,46 +1140,46 @@ static void chv_load_cgm_gamma(struct intel_crtc *crtc,
static void chv_load_luts(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- const struct drm_property_blob *degamma_lut = crtc_state->hw.degamma_lut;
- const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ const struct drm_property_blob *pre_csc_lut = crtc_state->pre_csc_lut;
+ const struct drm_property_blob *post_csc_lut = crtc_state->post_csc_lut;
const struct drm_property_blob *ctm = crtc_state->hw.ctm;
if (crtc_state->cgm_mode & CGM_PIPE_MODE_CSC)
chv_load_cgm_csc(crtc, ctm);
if (crtc_state->cgm_mode & CGM_PIPE_MODE_DEGAMMA)
- chv_load_cgm_degamma(crtc, degamma_lut);
+ chv_load_cgm_degamma(crtc, pre_csc_lut);
if (crtc_state->cgm_mode & CGM_PIPE_MODE_GAMMA)
- chv_load_cgm_gamma(crtc, gamma_lut);
+ chv_load_cgm_gamma(crtc, post_csc_lut);
else
i965_load_luts(crtc_state);
- intel_de_write_fw(dev_priv, CGM_PIPE_MODE(crtc->pipe),
+ intel_de_write_fw(i915, CGM_PIPE_MODE(crtc->pipe),
crtc_state->cgm_mode);
}
void intel_color_load_luts(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
- dev_priv->display.funcs.color->load_luts(crtc_state);
+ i915->display.funcs.color->load_luts(crtc_state);
}
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
- if (dev_priv->display.funcs.color->color_commit_noarm)
- dev_priv->display.funcs.color->color_commit_noarm(crtc_state);
+ if (i915->display.funcs.color->color_commit_noarm)
+ i915->display.funcs.color->color_commit_noarm(crtc_state);
}
void intel_color_commit_arm(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
- dev_priv->display.funcs.color->color_commit_arm(crtc_state);
+ i915->display.funcs.color->color_commit_arm(crtc_state);
}
static bool intel_can_preload_luts(const struct intel_crtc_state *new_crtc_state)
@@ -1194,8 +1190,8 @@ static bool intel_can_preload_luts(const struct intel_crtc_state *new_crtc_state
const struct intel_crtc_state *old_crtc_state =
intel_atomic_get_old_crtc_state(state, crtc);
- return !old_crtc_state->hw.gamma_lut &&
- !old_crtc_state->hw.degamma_lut;
+ return !old_crtc_state->post_csc_lut &&
+ !old_crtc_state->pre_csc_lut;
}
static bool chv_can_preload_luts(const struct intel_crtc_state *new_crtc_state)
@@ -1214,46 +1210,28 @@ static bool chv_can_preload_luts(const struct intel_crtc_state *new_crtc_state)
if (old_crtc_state->cgm_mode || new_crtc_state->cgm_mode)
return false;
- return !old_crtc_state->hw.gamma_lut;
-}
-
-static bool glk_can_preload_luts(const struct intel_crtc_state *new_crtc_state)
-{
- struct intel_crtc *crtc = to_intel_crtc(new_crtc_state->uapi.crtc);
- struct intel_atomic_state *state =
- to_intel_atomic_state(new_crtc_state->uapi.state);
- const struct intel_crtc_state *old_crtc_state =
- intel_atomic_get_old_crtc_state(state, crtc);
-
- /*
- * The hardware degamma is active whenever the pipe
- * CSC is active. Thus even if the old state has no
- * software degamma we need to avoid clobbering the
- * linear hardware degamma mid scanout.
- */
- return !old_crtc_state->csc_enable &&
- !old_crtc_state->hw.gamma_lut;
+ return !old_crtc_state->post_csc_lut;
}
int intel_color_check(struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
- return dev_priv->display.funcs.color->color_check(crtc_state);
+ return i915->display.funcs.color->color_check(crtc_state);
}
void intel_color_get_config(struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
- if (dev_priv->display.funcs.color->read_luts)
- dev_priv->display.funcs.color->read_luts(crtc_state);
+ if (i915->display.funcs.color->read_luts)
+ i915->display.funcs.color->read_luts(crtc_state);
}
static bool need_plane_update(struct intel_plane *plane,
const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
+ struct drm_i915_private *i915 = to_i915(plane->base.dev);
/*
* On pre-SKL the pipe gamma enable and pipe csc enable for
@@ -1261,7 +1239,7 @@ static bool need_plane_update(struct intel_plane *plane,
* We have to reconfigure that even if the plane is inactive.
*/
return crtc_state->active_planes & BIT(plane->id) ||
- (DISPLAY_VER(dev_priv) < 9 &&
+ (DISPLAY_VER(i915) < 9 &&
plane->id == PLANE_PRIMARY);
}
@@ -1269,7 +1247,7 @@ static int
intel_color_add_affected_planes(struct intel_crtc_state *new_crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(new_crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
struct intel_atomic_state *state =
to_intel_atomic_state(new_crtc_state->uapi.state);
const struct intel_crtc_state *old_crtc_state =
@@ -1277,14 +1255,14 @@ intel_color_add_affected_planes(struct intel_crtc_state *new_crtc_state)
struct intel_plane *plane;
if (!new_crtc_state->hw.active ||
- drm_atomic_crtc_needs_modeset(&new_crtc_state->uapi))
+ intel_crtc_needs_modeset(new_crtc_state))
return 0;
if (new_crtc_state->gamma_enable == old_crtc_state->gamma_enable &&
new_crtc_state->csc_enable == old_crtc_state->csc_enable)
return 0;
- for_each_intel_plane_on_crtc(&dev_priv->drm, crtc, plane) {
+ for_each_intel_plane_on_crtc(&i915->drm, crtc, plane) {
struct intel_plane_state *plane_state;
if (!need_plane_update(plane, new_crtc_state))
@@ -1295,6 +1273,10 @@ intel_color_add_affected_planes(struct intel_crtc_state *new_crtc_state)
return PTR_ERR(plane_state);
new_crtc_state->update_planes |= BIT(plane->id);
+
+ /* plane control register changes blocked by CxSR */
+ if (HAS_GMCH(i915))
+ new_crtc_state->disable_cxsr = true;
}
return 0;
@@ -1319,7 +1301,7 @@ static int check_lut_size(const struct drm_property_blob *lut, int expected)
static int check_luts(const struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
const struct drm_property_blob *gamma_lut = crtc_state->hw.gamma_lut;
const struct drm_property_blob *degamma_lut = crtc_state->hw.degamma_lut;
int gamma_length, degamma_length;
@@ -1331,15 +1313,15 @@ static int check_luts(const struct intel_crtc_state *crtc_state)
/* C8 relies on its palette being stored in the legacy LUT */
if (crtc_state->c8_planes) {
- drm_dbg_kms(&dev_priv->drm,
+ drm_dbg_kms(&i915->drm,
"C8 pixelformat requires the legacy LUT\n");
return -EINVAL;
}
- degamma_length = INTEL_INFO(dev_priv)->display.color.degamma_lut_size;
- gamma_length = INTEL_INFO(dev_priv)->display.color.gamma_lut_size;
- degamma_tests = INTEL_INFO(dev_priv)->display.color.degamma_lut_tests;
- gamma_tests = INTEL_INFO(dev_priv)->display.color.gamma_lut_tests;
+ degamma_length = INTEL_INFO(i915)->display.color.degamma_lut_size;
+ gamma_length = INTEL_INFO(i915)->display.color.gamma_lut_size;
+ degamma_tests = INTEL_INFO(i915)->display.color.degamma_lut_tests;
+ gamma_tests = INTEL_INFO(i915)->display.color.gamma_lut_tests;
if (check_lut_size(degamma_lut, degamma_length) ||
check_lut_size(gamma_lut, gamma_length))
@@ -1361,6 +1343,40 @@ static u32 i9xx_gamma_mode(struct intel_crtc_state *crtc_state)
return GAMMA_MODE_MODE_10BIT; /* i965+ only */
}
+void intel_color_assert_luts(const struct intel_crtc_state *crtc_state)
+{
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
+
+ /* make sure {pre,post}_csc_lut were correctly assigned */
+ if (DISPLAY_VER(i915) >= 11 || HAS_GMCH(i915)) {
+ drm_WARN_ON(&i915->drm,
+ crtc_state->pre_csc_lut != crtc_state->hw.degamma_lut);
+ drm_WARN_ON(&i915->drm,
+ crtc_state->post_csc_lut != crtc_state->hw.gamma_lut);
+ } else if (DISPLAY_VER(i915) == 10) {
+ drm_WARN_ON(&i915->drm,
+ crtc_state->pre_csc_lut != crtc_state->hw.degamma_lut &&
+ crtc_state->pre_csc_lut != i915->display.color.glk_linear_degamma_lut);
+ drm_WARN_ON(&i915->drm,
+ crtc_state->post_csc_lut != crtc_state->hw.gamma_lut);
+ } else if (crtc_state->gamma_mode != GAMMA_MODE_MODE_SPLIT) {
+ drm_WARN_ON(&i915->drm,
+ crtc_state->pre_csc_lut != crtc_state->hw.degamma_lut &&
+ crtc_state->pre_csc_lut != crtc_state->hw.gamma_lut);
+ drm_WARN_ON(&i915->drm,
+ crtc_state->post_csc_lut != crtc_state->hw.degamma_lut &&
+ crtc_state->post_csc_lut != crtc_state->hw.gamma_lut);
+ }
+}
+
+static void intel_assign_luts(struct intel_crtc_state *crtc_state)
+{
+ drm_property_replace_blob(&crtc_state->pre_csc_lut,
+ crtc_state->hw.degamma_lut);
+ drm_property_replace_blob(&crtc_state->post_csc_lut,
+ crtc_state->hw.gamma_lut);
+}
+
static int i9xx_color_check(struct intel_crtc_state *crtc_state)
{
int ret;
@@ -1379,6 +1395,8 @@ static int i9xx_color_check(struct intel_crtc_state *crtc_state)
if (ret)
return ret;
+ intel_assign_luts(crtc_state);
+
crtc_state->preload_luts = intel_can_preload_luts(crtc_state);
return 0;
@@ -1433,11 +1451,27 @@ static int chv_color_check(struct intel_crtc_state *crtc_state)
if (ret)
return ret;
+ intel_assign_luts(crtc_state);
+
crtc_state->preload_luts = chv_can_preload_luts(crtc_state);
return 0;
}
+static bool ilk_gamma_enable(const struct intel_crtc_state *crtc_state)
+{
+ return (crtc_state->hw.gamma_lut ||
+ crtc_state->hw.degamma_lut) &&
+ !crtc_state->c8_planes;
+}
+
+static bool ilk_csc_enable(const struct intel_crtc_state *crtc_state)
+{
+ return crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB ||
+ ilk_csc_limited_range(crtc_state) ||
+ crtc_state->hw.ctm;
+}
+
static u32 ilk_gamma_mode(const struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable ||
@@ -1458,28 +1492,54 @@ static u32 ilk_csc_mode(const struct intel_crtc_state *crtc_state)
if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB)
return CSC_BLACK_SCREEN_OFFSET;
+ if (crtc_state->hw.degamma_lut)
+ return CSC_MODE_YUV_TO_RGB;
+
return CSC_MODE_YUV_TO_RGB |
CSC_POSITION_BEFORE_GAMMA;
}
+static void ilk_assign_luts(struct intel_crtc_state *crtc_state)
+{
+ if (crtc_state->hw.degamma_lut ||
+ crtc_state->csc_mode & CSC_POSITION_BEFORE_GAMMA) {
+ drm_property_replace_blob(&crtc_state->pre_csc_lut,
+ crtc_state->hw.degamma_lut);
+ drm_property_replace_blob(&crtc_state->post_csc_lut,
+ crtc_state->hw.gamma_lut);
+ } else {
+ drm_property_replace_blob(&crtc_state->pre_csc_lut,
+ crtc_state->hw.gamma_lut);
+ drm_property_replace_blob(&crtc_state->post_csc_lut,
+ NULL);
+ }
+}
+
static int ilk_color_check(struct intel_crtc_state *crtc_state)
{
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
int ret;
ret = check_luts(crtc_state);
if (ret)
return ret;
- crtc_state->gamma_enable =
- crtc_state->hw.gamma_lut &&
- !crtc_state->c8_planes;
+ if (crtc_state->hw.degamma_lut && crtc_state->hw.gamma_lut) {
+ drm_dbg_kms(&i915->drm,
+ "Degamma and gamma together are not possible\n");
+ return -EINVAL;
+ }
- /*
- * We don't expose the ctm on ilk/snb currently, also RGB
- * limited range output is handled by the hw automagically.
- */
- crtc_state->csc_enable =
- crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB;
+ if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB &&
+ crtc_state->hw.ctm) {
+ drm_dbg_kms(&i915->drm,
+ "YCbCr and CTM together are not possible\n");
+ return -EINVAL;
+ }
+
+ crtc_state->gamma_enable = ilk_gamma_enable(crtc_state);
+
+ crtc_state->csc_enable = ilk_csc_enable(crtc_state);
crtc_state->gamma_mode = ilk_gamma_mode(crtc_state);
@@ -1489,6 +1549,8 @@ static int ilk_color_check(struct intel_crtc_state *crtc_state)
if (ret)
return ret;
+ ilk_assign_luts(crtc_state);
+
crtc_state->preload_luts = intel_can_preload_luts(crtc_state);
return 0;
@@ -1496,14 +1558,10 @@ static int ilk_color_check(struct intel_crtc_state *crtc_state)
static u32 ivb_gamma_mode(const struct intel_crtc_state *crtc_state)
{
- if (!crtc_state->gamma_enable ||
- crtc_state_is_legacy_gamma(crtc_state))
- return GAMMA_MODE_MODE_8BIT;
- else if (crtc_state->hw.gamma_lut &&
- crtc_state->hw.degamma_lut)
+ if (crtc_state->hw.degamma_lut && crtc_state->hw.gamma_lut)
return GAMMA_MODE_MODE_SPLIT;
- else
- return GAMMA_MODE_MODE_10BIT;
+
+ return ilk_gamma_mode(crtc_state);
}
static u32 ivb_csc_mode(const struct intel_crtc_state *crtc_state)
@@ -1522,10 +1580,41 @@ static u32 ivb_csc_mode(const struct intel_crtc_state *crtc_state)
return CSC_POSITION_BEFORE_GAMMA;
}
+static int ivb_assign_luts(struct intel_crtc_state *crtc_state)
+{
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_property_blob *degamma_lut, *gamma_lut;
+
+ if (crtc_state->gamma_mode != GAMMA_MODE_MODE_SPLIT) {
+ ilk_assign_luts(crtc_state);
+ return 0;
+ }
+
+ drm_WARN_ON(&i915->drm, drm_color_lut_size(crtc_state->hw.degamma_lut) != 1024);
+ drm_WARN_ON(&i915->drm, drm_color_lut_size(crtc_state->hw.gamma_lut) != 1024);
+
+ degamma_lut = create_resized_lut(i915, crtc_state->hw.degamma_lut, 512);
+ if (IS_ERR(degamma_lut))
+ return PTR_ERR(degamma_lut);
+
+ gamma_lut = create_resized_lut(i915, crtc_state->hw.gamma_lut, 512);
+ if (IS_ERR(gamma_lut)) {
+ drm_property_blob_put(degamma_lut);
+ return PTR_ERR(gamma_lut);
+ }
+
+ drm_property_replace_blob(&crtc_state->pre_csc_lut, degamma_lut);
+ drm_property_replace_blob(&crtc_state->post_csc_lut, gamma_lut);
+
+ drm_property_blob_put(degamma_lut);
+ drm_property_blob_put(gamma_lut);
+
+ return 0;
+}
+
static int ivb_color_check(struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
- bool limited_color_range = ilk_csc_limited_range(crtc_state);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
int ret;
ret = check_luts(crtc_state);
@@ -1534,19 +1623,21 @@ static int ivb_color_check(struct intel_crtc_state *crtc_state)
if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB &&
crtc_state->hw.ctm) {
- drm_dbg_kms(&dev_priv->drm,
- "YCBCR and CTM together are not possible\n");
+ drm_dbg_kms(&i915->drm,
+ "YCbCr and CTM together are not possible\n");
return -EINVAL;
}
- crtc_state->gamma_enable =
- (crtc_state->hw.gamma_lut ||
- crtc_state->hw.degamma_lut) &&
- !crtc_state->c8_planes;
+ if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB &&
+ crtc_state->hw.degamma_lut && crtc_state->hw.gamma_lut) {
+ drm_dbg_kms(&i915->drm,
+ "YCbCr and degamma+gamma together are not possible\n");
+ return -EINVAL;
+ }
- crtc_state->csc_enable =
- crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB ||
- crtc_state->hw.ctm || limited_color_range;
+ crtc_state->gamma_enable = ilk_gamma_enable(crtc_state);
+
+ crtc_state->csc_enable = ilk_csc_enable(crtc_state);
crtc_state->gamma_mode = ivb_gamma_mode(crtc_state);
@@ -1556,6 +1647,10 @@ static int ivb_color_check(struct intel_crtc_state *crtc_state)
if (ret)
return ret;
+ ret = ivb_assign_luts(crtc_state);
+ if (ret)
+ return ret;
+
crtc_state->preload_luts = intel_can_preload_luts(crtc_state);
return 0;
@@ -1570,9 +1665,26 @@ static u32 glk_gamma_mode(const struct intel_crtc_state *crtc_state)
return GAMMA_MODE_MODE_10BIT;
}
+static void glk_assign_luts(struct intel_crtc_state *crtc_state)
+{
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
+
+ intel_assign_luts(crtc_state);
+
+ /*
+ * On GLK+ both pipe CSC and degamma LUT are controlled
+ * by csc_enable. Hence for the cases where the CSC is
+ * needed but degamma LUT is not we need to load a
+ * linear degamma LUT.
+ */
+ if (crtc_state->csc_enable && !crtc_state->pre_csc_lut)
+ drm_property_replace_blob(&crtc_state->pre_csc_lut,
+ i915->display.color.glk_linear_degamma_lut);
+}
+
static int glk_color_check(struct intel_crtc_state *crtc_state)
{
- struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
+ struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev);
int ret;
ret = check_luts(crtc_state);
@@ -1581,8 +1693,15 @@ static int glk_color_check(struct intel_crtc_state *crtc_state)
if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB &&
crtc_state->hw.ctm) {
- drm_dbg_kms(&dev_priv->drm,
- "YCBCR and CTM together are not possible\n");
+ drm_dbg_kms(&i915->drm,
+ "YCbCr and CTM together are not possible\n");
+ return -EINVAL;
+ }
+
+ if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB &&
+ crtc_state->hw.degamma_lut && crtc_state->hw.gamma_lut) {
+ drm_dbg_kms(&i915->drm,
+ "YCbCr and degamma+gamma together are not possible\n");
return -EINVAL;
}
@@ -1604,7 +1723,9 @@ static int glk_color_check(struct intel_crtc_state *crtc_state)
if (ret)
return ret;
- crtc_state->preload_luts = glk_can_preload_luts(crtc_state);
+ glk_assign_luts(crtc_state);
+
+ crtc_state->preload_luts = intel_can_preload_luts(crtc_state);
return 0;
}
@@ -1664,6 +1785,8 @@ static int icl_color_check(struct intel_crtc_state *crtc_state)
crtc_state->csc_mode = icl_csc_mode(crtc_state);
+ intel_assign_luts(crtc_state);
+
crtc_state->preload_luts = intel_can_preload_luts(crtc_state);
return 0;
@@ -1749,19 +1872,19 @@ static int icl_gamma_precision(const struct intel_crtc_state *crtc_state)
int intel_color_get_gamma_bit_precision(const struct intel_crtc_state *crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
- if (HAS_GMCH(dev_priv)) {
- if (IS_CHERRYVIEW(dev_priv))
+ if (HAS_GMCH(i915)) {
+ if (IS_CHERRYVIEW(i915))
return chv_gamma_precision(crtc_state);
else
return i9xx_gamma_precision(crtc_state);
} else {
- if (DISPLAY_VER(dev_priv) >= 11)
+ if (DISPLAY_VER(i915) >= 11)
return icl_gamma_precision(crtc_state);
- else if (DISPLAY_VER(dev_priv) == 10)
+ else if (DISPLAY_VER(i915) == 10)
return glk_gamma_precision(crtc_state);
- else if (IS_IRONLAKE(dev_priv))
+ else if (IS_IRONLAKE(i915))
return ilk_gamma_precision(crtc_state);
}
@@ -1846,7 +1969,7 @@ static struct drm_property_blob *i9xx_read_lut_8(struct intel_crtc *crtc)
int i;
blob = drm_property_create_blob(&dev_priv->drm,
- sizeof(struct drm_color_lut) * LEGACY_LUT_LENGTH,
+ sizeof(lut[0]) * LEGACY_LUT_LENGTH,
NULL);
if (IS_ERR(blob))
return NULL;
@@ -1869,7 +1992,7 @@ static void i9xx_read_luts(struct intel_crtc_state *crtc_state)
if (!crtc_state->gamma_enable)
return;
- crtc_state->hw.gamma_lut = i9xx_read_lut_8(crtc);
+ crtc_state->post_csc_lut = i9xx_read_lut_8(crtc);
}
static struct drm_property_blob *i965_read_lut_10p6(struct intel_crtc *crtc)
@@ -1881,7 +2004,7 @@ static struct drm_property_blob *i965_read_lut_10p6(struct intel_crtc *crtc)
struct drm_color_lut *lut;
blob = drm_property_create_blob(&dev_priv->drm,
- sizeof(struct drm_color_lut) * lut_size,
+ sizeof(lut[0]) * lut_size,
NULL);
if (IS_ERR(blob))
return NULL;
@@ -1910,21 +2033,21 @@ static void i965_read_luts(struct intel_crtc_state *crtc_state)
return;
if (crtc_state->gamma_mode == GAMMA_MODE_MODE_8BIT)
- crtc_state->hw.gamma_lut = i9xx_read_lut_8(crtc);
+ crtc_state->post_csc_lut = i9xx_read_lut_8(crtc);
else
- crtc_state->hw.gamma_lut = i965_read_lut_10p6(crtc);
+ crtc_state->post_csc_lut = i965_read_lut_10p6(crtc);
}
static struct drm_property_blob *chv_read_cgm_gamma(struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- int i, lut_size = INTEL_INFO(dev_priv)->display.color.gamma_lut_size;
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ int i, lut_size = INTEL_INFO(i915)->display.color.gamma_lut_size;
enum pipe pipe = crtc->pipe;
struct drm_property_blob *blob;
struct drm_color_lut *lut;
- blob = drm_property_create_blob(&dev_priv->drm,
- sizeof(struct drm_color_lut) * lut_size,
+ blob = drm_property_create_blob(&i915->drm,
+ sizeof(lut[0]) * lut_size,
NULL);
if (IS_ERR(blob))
return NULL;
@@ -1932,8 +2055,8 @@ static struct drm_property_blob *chv_read_cgm_gamma(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < lut_size; i++) {
- u32 ldw = intel_de_read_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 0));
- u32 udw = intel_de_read_fw(dev_priv, CGM_PIPE_GAMMA(pipe, i, 1));
+ u32 ldw = intel_de_read_fw(i915, CGM_PIPE_GAMMA(pipe, i, 0));
+ u32 udw = intel_de_read_fw(i915, CGM_PIPE_GAMMA(pipe, i, 1));
chv_cgm_gamma_pack(&lut[i], ldw, udw);
}
@@ -1946,21 +2069,21 @@ static void chv_read_luts(struct intel_crtc_state *crtc_state)
struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
if (crtc_state->cgm_mode & CGM_PIPE_MODE_GAMMA)
- crtc_state->hw.gamma_lut = chv_read_cgm_gamma(crtc);
+ crtc_state->post_csc_lut = chv_read_cgm_gamma(crtc);
else
i965_read_luts(crtc_state);
}
static struct drm_property_blob *ilk_read_lut_8(struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
enum pipe pipe = crtc->pipe;
struct drm_property_blob *blob;
struct drm_color_lut *lut;
int i;
- blob = drm_property_create_blob(&dev_priv->drm,
- sizeof(struct drm_color_lut) * LEGACY_LUT_LENGTH,
+ blob = drm_property_create_blob(&i915->drm,
+ sizeof(lut[0]) * LEGACY_LUT_LENGTH,
NULL);
if (IS_ERR(blob))
return NULL;
@@ -1968,7 +2091,7 @@ static struct drm_property_blob *ilk_read_lut_8(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < LEGACY_LUT_LENGTH; i++) {
- u32 val = intel_de_read_fw(dev_priv, LGC_PALETTE(pipe, i));
+ u32 val = intel_de_read_fw(i915, LGC_PALETTE(pipe, i));
i9xx_lut_8_pack(&lut[i], val);
}
@@ -1978,14 +2101,14 @@ static struct drm_property_blob *ilk_read_lut_8(struct intel_crtc *crtc)
static struct drm_property_blob *ilk_read_lut_10(struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- int i, lut_size = INTEL_INFO(dev_priv)->display.color.gamma_lut_size;
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ int i, lut_size = INTEL_INFO(i915)->display.color.gamma_lut_size;
enum pipe pipe = crtc->pipe;
struct drm_property_blob *blob;
struct drm_color_lut *lut;
- blob = drm_property_create_blob(&dev_priv->drm,
- sizeof(struct drm_color_lut) * lut_size,
+ blob = drm_property_create_blob(&i915->drm,
+ sizeof(lut[0]) * lut_size,
NULL);
if (IS_ERR(blob))
return NULL;
@@ -1993,7 +2116,7 @@ static struct drm_property_blob *ilk_read_lut_10(struct intel_crtc *crtc)
lut = blob->data;
for (i = 0; i < lut_size; i++) {
- u32 val = intel_de_read_fw(dev_priv, PREC_PALETTE(pipe, i));
+ u32 val = intel_de_read_fw(i915, PREC_PALETTE(pipe, i));
ilk_lut_10_pack(&lut[i], val);
}
@@ -2013,10 +2136,10 @@ static void ilk_read_luts(struct intel_crtc_state *crtc_state)
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
- crtc_state->hw.gamma_lut = ilk_read_lut_8(crtc);
+ crtc_state->post_csc_lut = ilk_read_lut_8(crtc);
break;
case GAMMA_MODE_MODE_10BIT:
- crtc_state->hw.gamma_lut = ilk_read_lut_10(crtc);
+ crtc_state->post_csc_lut = ilk_read_lut_10(crtc);
break;
default:
MISSING_CASE(crtc_state->gamma_mode);
@@ -2028,33 +2151,33 @@ static void ilk_read_luts(struct intel_crtc_state *crtc_state)
static struct drm_property_blob *bdw_read_lut_10(struct intel_crtc *crtc,
u32 prec_index)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
int i, hw_lut_size = ivb_lut_10_size(prec_index);
- int lut_size = INTEL_INFO(dev_priv)->display.color.gamma_lut_size;
+ int lut_size = INTEL_INFO(i915)->display.color.gamma_lut_size;
enum pipe pipe = crtc->pipe;
struct drm_property_blob *blob;
struct drm_color_lut *lut;
- drm_WARN_ON(&dev_priv->drm, lut_size != hw_lut_size);
+ drm_WARN_ON(&i915->drm, lut_size != hw_lut_size);
- blob = drm_property_create_blob(&dev_priv->drm,
- sizeof(struct drm_color_lut) * lut_size,
+ blob = drm_property_create_blob(&i915->drm,
+ sizeof(lut[0]) * lut_size,
NULL);
if (IS_ERR(blob))
return NULL;
lut = blob->data;
- intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe),
+ intel_de_write_fw(i915, PREC_PAL_INDEX(pipe),
prec_index | PAL_PREC_AUTO_INCREMENT);
for (i = 0; i < lut_size; i++) {
- u32 val = intel_de_read_fw(dev_priv, PREC_PAL_DATA(pipe));
+ u32 val = intel_de_read_fw(i915, PREC_PAL_DATA(pipe));
ilk_lut_10_pack(&lut[i], val);
}
- intel_de_write_fw(dev_priv, PREC_PAL_INDEX(pipe), 0);
+ intel_de_write_fw(i915, PREC_PAL_INDEX(pipe), 0);
return blob;
}
@@ -2068,10 +2191,10 @@ static void glk_read_luts(struct intel_crtc_state *crtc_state)
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
- crtc_state->hw.gamma_lut = ilk_read_lut_8(crtc);
+ crtc_state->post_csc_lut = ilk_read_lut_8(crtc);
break;
case GAMMA_MODE_MODE_10BIT:
- crtc_state->hw.gamma_lut = bdw_read_lut_10(crtc, PAL_PREC_INDEX_VALUE(0));
+ crtc_state->post_csc_lut = bdw_read_lut_10(crtc, PAL_PREC_INDEX_VALUE(0));
break;
default:
MISSING_CASE(crtc_state->gamma_mode);
@@ -2082,31 +2205,31 @@ static void glk_read_luts(struct intel_crtc_state *crtc_state)
static struct drm_property_blob *
icl_read_lut_multi_segment(struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- int i, lut_size = INTEL_INFO(dev_priv)->display.color.gamma_lut_size;
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ int i, lut_size = INTEL_INFO(i915)->display.color.gamma_lut_size;
enum pipe pipe = crtc->pipe;
struct drm_property_blob *blob;
struct drm_color_lut *lut;
- blob = drm_property_create_blob(&dev_priv->drm,
- sizeof(struct drm_color_lut) * lut_size,
+ blob = drm_property_create_blob(&i915->drm,
+ sizeof(lut[0]) * lut_size,
NULL);
if (IS_ERR(blob))
return NULL;
lut = blob->data;
- intel_de_write_fw(dev_priv, PREC_PAL_MULTI_SEG_INDEX(pipe),
+ intel_de_write_fw(i915, PREC_PAL_MULTI_SEG_INDEX(pipe),
PAL_PREC_AUTO_INCREMENT);
for (i = 0; i < 9; i++) {
- u32 ldw = intel_de_read_fw(dev_priv, PREC_PAL_MULTI_SEG_DATA(pipe));
- u32 udw = intel_de_read_fw(dev_priv, PREC_PAL_MULTI_SEG_DATA(pipe));
+ u32 ldw = intel_de_read_fw(i915, PREC_PAL_MULTI_SEG_DATA(pipe));
+ u32 udw = intel_de_read_fw(i915, PREC_PAL_MULTI_SEG_DATA(pipe));
- icl_lut_multi_seg_pack(&lut[i], ldw, udw);
+ ilk_lut_12p4_pack(&lut[i], ldw, udw);
}
- intel_de_write_fw(dev_priv, PREC_PAL_MULTI_SEG_INDEX(pipe), 0);
+ intel_de_write_fw(i915, PREC_PAL_MULTI_SEG_INDEX(pipe), 0);
/*
* FIXME readouts from PAL_PREC_DATA register aren't giving
@@ -2126,13 +2249,13 @@ static void icl_read_luts(struct intel_crtc_state *crtc_state)
switch (crtc_state->gamma_mode & GAMMA_MODE_MODE_MASK) {
case GAMMA_MODE_MODE_8BIT:
- crtc_state->hw.gamma_lut = ilk_read_lut_8(crtc);
+ crtc_state->post_csc_lut = ilk_read_lut_8(crtc);
break;
case GAMMA_MODE_MODE_10BIT:
- crtc_state->hw.gamma_lut = bdw_read_lut_10(crtc, PAL_PREC_INDEX_VALUE(0));
+ crtc_state->post_csc_lut = bdw_read_lut_10(crtc, PAL_PREC_INDEX_VALUE(0));
break;
case GAMMA_MODE_MODE_12BIT_MULTI_SEGMENTED:
- crtc_state->hw.gamma_lut = icl_read_lut_multi_segment(crtc);
+ crtc_state->post_csc_lut = icl_read_lut_multi_segment(crtc);
break;
default:
MISSING_CASE(crtc_state->gamma_mode);
@@ -2217,41 +2340,58 @@ static const struct intel_color_funcs ilk_color_funcs = {
.read_luts = ilk_read_luts,
};
-void intel_color_init(struct intel_crtc *crtc)
+void intel_color_crtc_init(struct intel_crtc *crtc)
{
- struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- bool has_ctm = INTEL_INFO(dev_priv)->display.color.degamma_lut_size != 0;
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ bool has_ctm = INTEL_INFO(i915)->display.color.degamma_lut_size != 0;
drm_mode_crtc_set_gamma_size(&crtc->base, 256);
- if (HAS_GMCH(dev_priv)) {
- if (IS_CHERRYVIEW(dev_priv)) {
- dev_priv->display.funcs.color = &chv_color_funcs;
- } else if (DISPLAY_VER(dev_priv) >= 4) {
- dev_priv->display.funcs.color = &i965_color_funcs;
- } else {
- dev_priv->display.funcs.color = &i9xx_color_funcs;
- }
- } else {
- if (DISPLAY_VER(dev_priv) >= 11)
- dev_priv->display.funcs.color = &icl_color_funcs;
- else if (DISPLAY_VER(dev_priv) == 10)
- dev_priv->display.funcs.color = &glk_color_funcs;
- else if (DISPLAY_VER(dev_priv) == 9)
- dev_priv->display.funcs.color = &skl_color_funcs;
- else if (DISPLAY_VER(dev_priv) == 8)
- dev_priv->display.funcs.color = &bdw_color_funcs;
- else if (DISPLAY_VER(dev_priv) == 7) {
- if (IS_HASWELL(dev_priv))
- dev_priv->display.funcs.color = &hsw_color_funcs;
- else
- dev_priv->display.funcs.color = &ivb_color_funcs;
- } else
- dev_priv->display.funcs.color = &ilk_color_funcs;
- }
-
drm_crtc_enable_color_mgmt(&crtc->base,
- INTEL_INFO(dev_priv)->display.color.degamma_lut_size,
+ INTEL_INFO(i915)->display.color.degamma_lut_size,
has_ctm,
- INTEL_INFO(dev_priv)->display.color.gamma_lut_size);
+ INTEL_INFO(i915)->display.color.gamma_lut_size);
+}
+
+int intel_color_init(struct drm_i915_private *i915)
+{
+ struct drm_property_blob *blob;
+
+ if (DISPLAY_VER(i915) != 10)
+ return 0;
+
+ blob = create_linear_lut(i915, INTEL_INFO(i915)->display.color.degamma_lut_size);
+ if (IS_ERR(blob))
+ return PTR_ERR(blob);
+
+ i915->display.color.glk_linear_degamma_lut = blob;
+
+ return 0;
+}
+
+void intel_color_init_hooks(struct drm_i915_private *i915)
+{
+ if (HAS_GMCH(i915)) {
+ if (IS_CHERRYVIEW(i915))
+ i915->display.funcs.color = &chv_color_funcs;
+ else if (DISPLAY_VER(i915) >= 4)
+ i915->display.funcs.color = &i965_color_funcs;
+ else
+ i915->display.funcs.color = &i9xx_color_funcs;
+ } else {
+ if (DISPLAY_VER(i915) >= 11)
+ i915->display.funcs.color = &icl_color_funcs;
+ else if (DISPLAY_VER(i915) == 10)
+ i915->display.funcs.color = &glk_color_funcs;
+ else if (DISPLAY_VER(i915) == 9)
+ i915->display.funcs.color = &skl_color_funcs;
+ else if (DISPLAY_VER(i915) == 8)
+ i915->display.funcs.color = &bdw_color_funcs;
+ else if (IS_HASWELL(i915))
+ i915->display.funcs.color = &hsw_color_funcs;
+ else if (DISPLAY_VER(i915) == 7)
+ i915->display.funcs.color = &ivb_color_funcs;
+ else
+ i915->display.funcs.color = &ilk_color_funcs;
+ }
}
diff --git a/drivers/gpu/drm/i915/display/intel_color.h b/drivers/gpu/drm/i915/display/intel_color.h
index fd87342..2a5ada6 100644
--- a/drivers/gpu/drm/i915/display/intel_color.h
+++ b/drivers/gpu/drm/i915/display/intel_color.h
@@ -10,9 +10,12 @@
struct intel_crtc_state;
struct intel_crtc;
+struct drm_i915_private;
struct drm_property_blob;
-void intel_color_init(struct intel_crtc *crtc);
+void intel_color_init_hooks(struct drm_i915_private *i915);
+int intel_color_init(struct drm_i915_private *i915);
+void intel_color_crtc_init(struct intel_crtc *crtc);
int intel_color_check(struct intel_crtc_state *crtc_state);
void intel_color_commit_noarm(const struct intel_crtc_state *crtc_state);
void intel_color_commit_arm(const struct intel_crtc_state *crtc_state);
@@ -22,5 +25,6 @@ int intel_color_get_gamma_bit_precision(const struct intel_crtc_state *crtc_stat
bool intel_color_lut_equal(struct drm_property_blob *blob1,
struct drm_property_blob *blob2,
u32 gamma_mode, u32 bit_precision);
+void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);
#endif /* __INTEL_COLOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_combo_phy.c b/drivers/gpu/drm/i915/display/intel_combo_phy.c
index 64890f3..8b870b2 100644
--- a/drivers/gpu/drm/i915/display/intel_combo_phy.c
+++ b/drivers/gpu/drm/i915/display/intel_combo_phy.c
@@ -3,6 +3,7 @@
* Copyright © 2018 Intel Corporation
*/
+#include "i915_reg.h"
#include "intel_combo_phy.h"
#include "intel_combo_phy_regs.h"
#include "intel_de.h"
@@ -53,7 +54,6 @@ static const struct icl_procmon {
static const struct icl_procmon *
icl_get_procmon_ref_values(struct drm_i915_private *dev_priv, enum phy phy)
{
- const struct icl_procmon *procmon;
u32 val;
val = intel_de_read(dev_priv, ICL_PORT_COMP_DW3(phy));
@@ -62,23 +62,16 @@ icl_get_procmon_ref_values(struct drm_i915_private *dev_priv, enum phy phy)
MISSING_CASE(val);
fallthrough;
case VOLTAGE_INFO_0_85V | PROCESS_INFO_DOT_0:
- procmon = &icl_procmon_values[PROCMON_0_85V_DOT_0];
- break;
+ return &icl_procmon_values[PROCMON_0_85V_DOT_0];
case VOLTAGE_INFO_0_95V | PROCESS_INFO_DOT_0:
- procmon = &icl_procmon_values[PROCMON_0_95V_DOT_0];
- break;
+ return &icl_procmon_values[PROCMON_0_95V_DOT_0];
case VOLTAGE_INFO_0_95V | PROCESS_INFO_DOT_1:
- procmon = &icl_procmon_values[PROCMON_0_95V_DOT_1];
- break;
+ return &icl_procmon_values[PROCMON_0_95V_DOT_1];
case VOLTAGE_INFO_1_05V | PROCESS_INFO_DOT_0:
- procmon = &icl_procmon_values[PROCMON_1_05V_DOT_0];
- break;
+ return &icl_procmon_values[PROCMON_1_05V_DOT_0];
case VOLTAGE_INFO_1_05V | PROCESS_INFO_DOT_1:
- procmon = &icl_procmon_values[PROCMON_1_05V_DOT_1];
- break;
+ return &icl_procmon_values[PROCMON_1_05V_DOT_1];
}
-
- return procmon;
}
static void icl_set_procmon_ref_values(struct drm_i915_private *dev_priv,
diff --git a/drivers/gpu/drm/i915/display/intel_connector.c b/drivers/gpu/drm/i915/display/intel_connector.c
index 6d5cbeb..6205ddd 100644
--- a/drivers/gpu/drm/i915/display/intel_connector.c
+++ b/drivers/gpu/drm/i915/display/intel_connector.c
@@ -293,3 +293,21 @@ intel_attach_dp_colorspace_property(struct drm_connector *connector)
if (!drm_mode_create_dp_colorspace_property(connector))
drm_connector_attach_colorspace_property(connector);
}
+
+void
+intel_attach_scaling_mode_property(struct drm_connector *connector)
+{
+ struct drm_i915_private *i915 = to_i915(connector->dev);
+ u32 scaling_modes;
+
+ scaling_modes = BIT(DRM_MODE_SCALE_ASPECT) |
+ BIT(DRM_MODE_SCALE_FULLSCREEN);
+
+ /* On GMCH platforms borders are only possible on the LVDS port */
+ if (!HAS_GMCH(i915) || connector->connector_type == DRM_MODE_CONNECTOR_LVDS)
+ scaling_modes |= BIT(DRM_MODE_SCALE_CENTER);
+
+ drm_connector_attach_scaling_mode_property(connector, scaling_modes);
+
+ connector->state->scaling_mode = DRM_MODE_SCALE_ASPECT;
+}
diff --git a/drivers/gpu/drm/i915/display/intel_connector.h b/drivers/gpu/drm/i915/display/intel_connector.h
index 661a37a..9d2bc26 100644
--- a/drivers/gpu/drm/i915/display/intel_connector.h
+++ b/drivers/gpu/drm/i915/display/intel_connector.h
@@ -6,7 +6,7 @@
#ifndef __INTEL_CONNECTOR_H__
#define __INTEL_CONNECTOR_H__
-#include "intel_display.h"
+#include <linux/types.h>
struct drm_connector;
struct edid;
@@ -32,5 +32,6 @@ void intel_attach_broadcast_rgb_property(struct drm_connector *connector);
void intel_attach_aspect_ratio_property(struct drm_connector *connector);
void intel_attach_hdmi_colorspace_property(struct drm_connector *connector);
void intel_attach_dp_colorspace_property(struct drm_connector *connector);
+void intel_attach_scaling_mode_property(struct drm_connector *connector);
#endif /* __INTEL_CONNECTOR_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_crt.c b/drivers/gpu/drm/i915/display/intel_crt.c
index 4a8ff2f..797ad94 100644
--- a/drivers/gpu/drm/i915/display/intel_crt.c
+++ b/drivers/gpu/drm/i915/display/intel_crt.c
@@ -34,6 +34,8 @@
#include <drm/drm_probe_helper.h>
#include "i915_drv.h"
+#include "i915_irq.h"
+#include "i915_reg.h"
#include "intel_connector.h"
#include "intel_crt.h"
#include "intel_crtc.h"
@@ -1044,17 +1046,14 @@ void intel_crt_init(struct drm_i915_private *dev_priv)
intel_connector_attach_encoder(intel_connector, &crt->base);
crt->base.type = INTEL_OUTPUT_ANALOG;
- crt->base.cloneable = (1 << INTEL_OUTPUT_DVO) | (1 << INTEL_OUTPUT_HDMI);
+ crt->base.cloneable = BIT(INTEL_OUTPUT_DVO) | BIT(INTEL_OUTPUT_HDMI);
if (IS_I830(dev_priv))
crt->base.pipe_mask = BIT(PIPE_A);
else
crt->base.pipe_mask = ~0;
- if (DISPLAY_VER(dev_priv) == 2)
- connector->interlace_allowed = 0;
- else
- connector->interlace_allowed = 1;
- connector->doublescan_allowed = 0;
+ if (DISPLAY_VER(dev_priv) != 2)
+ connector->interlace_allowed = true;
crt->adpa_reg = adpa_reg;
diff --git a/drivers/gpu/drm/i915/display/intel_crtc.c b/drivers/gpu/drm/i915/display/intel_crtc.c
index 6792a90..037fc14 100644
--- a/drivers/gpu/drm/i915/display/intel_crtc.c
+++ b/drivers/gpu/drm/i915/display/intel_crtc.c
@@ -365,9 +365,8 @@ int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe)
BIT(DRM_SCALING_FILTER_DEFAULT) |
BIT(DRM_SCALING_FILTER_NEAREST_NEIGHBOR));
- intel_color_init(crtc);
-
- intel_crtc_drrs_init(crtc);
+ intel_color_crtc_init(crtc);
+ intel_drrs_crtc_init(crtc);
intel_crtc_crc_init(crtc);
cpu_latency_qos_add_request(&crtc->vblank_pm_qos, PM_QOS_DEFAULT_VALUE);
@@ -387,8 +386,7 @@ static bool intel_crtc_needs_vblank_work(const struct intel_crtc_state *crtc_sta
return crtc_state->hw.active &&
!intel_crtc_needs_modeset(crtc_state) &&
!crtc_state->preload_luts &&
- (crtc_state->uapi.color_mgmt_changed ||
- crtc_state->update_pipe);
+ intel_crtc_needs_color_update(crtc_state);
}
static void intel_crtc_vblank_work(struct kthread_work *base)
diff --git a/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c b/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c
index e9212f6..e3273fe 100644
--- a/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c
+++ b/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c
@@ -298,11 +298,13 @@ void intel_crtc_state_dump(const struct intel_crtc_state *pipe_config,
pipe_config->csc_mode, pipe_config->gamma_mode,
pipe_config->gamma_enable, pipe_config->csc_enable);
- drm_dbg_kms(&i915->drm, "degamma lut: %d entries, gamma lut: %d entries\n",
- pipe_config->hw.degamma_lut ?
- drm_color_lut_size(pipe_config->hw.degamma_lut) : 0,
- pipe_config->hw.gamma_lut ?
- drm_color_lut_size(pipe_config->hw.gamma_lut) : 0);
+ drm_dbg_kms(&i915->drm, "pre csc lut: %s%d entries, post csc lut: %d entries\n",
+ pipe_config->pre_csc_lut && pipe_config->pre_csc_lut ==
+ i915->display.color.glk_linear_degamma_lut ? "(linear) " : "",
+ pipe_config->pre_csc_lut ?
+ drm_color_lut_size(pipe_config->pre_csc_lut) : 0,
+ pipe_config->post_csc_lut ?
+ drm_color_lut_size(pipe_config->post_csc_lut) : 0);
dump_planes:
if (!state)
diff --git a/drivers/gpu/drm/i915/display/intel_cursor.c b/drivers/gpu/drm/i915/display/intel_cursor.c
index 87899e89..d190fa0 100644
--- a/drivers/gpu/drm/i915/display/intel_cursor.c
+++ b/drivers/gpu/drm/i915/display/intel_cursor.c
@@ -10,12 +10,13 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_fourcc.h>
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_atomic_plane.h"
#include "intel_cursor.h"
#include "intel_de.h"
-#include "intel_display_types.h"
#include "intel_display.h"
+#include "intel_display_types.h"
#include "intel_fb.h"
#include "intel_fb_pin.h"
#include "intel_frontbuffer.h"
@@ -631,8 +632,10 @@ intel_legacy_cursor_update(struct drm_plane *_plane,
*
* FIXME bigjoiner fastpath would be good
*/
- if (!crtc_state->hw.active || intel_crtc_needs_modeset(crtc_state) ||
- crtc_state->update_pipe || crtc_state->bigjoiner_pipes)
+ if (!crtc_state->hw.active ||
+ intel_crtc_needs_modeset(crtc_state) ||
+ intel_crtc_needs_fastset(crtc_state) ||
+ crtc_state->bigjoiner_pipes)
goto slow;
/*
diff --git a/drivers/gpu/drm/i915/display/intel_ddi.c b/drivers/gpu/drm/i915/display/intel_ddi.c
index 69ecf2a..0f1ec2a 100644
--- a/drivers/gpu/drm/i915/display/intel_ddi.c
+++ b/drivers/gpu/drm/i915/display/intel_ddi.c
@@ -31,6 +31,7 @@
#include <drm/drm_privacy_screen_consumer.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_audio.h"
#include "intel_audio_regs.h"
#include "intel_backlight.h"
@@ -44,6 +45,7 @@
#include "intel_display_power.h"
#include "intel_display_types.h"
#include "intel_dkl_phy.h"
+#include "intel_dkl_phy_regs.h"
#include "intel_dp.h"
#include "intel_dp_link_training.h"
#include "intel_dp_mst.h"
@@ -55,14 +57,15 @@
#include "intel_hdcp.h"
#include "intel_hdmi.h"
#include "intel_hotplug.h"
+#include "intel_hti.h"
#include "intel_lspcon.h"
+#include "intel_mg_phy_regs.h"
#include "intel_pps.h"
#include "intel_psr.h"
#include "intel_quirks.h"
#include "intel_snps_phy.h"
#include "intel_sprite.h"
#include "intel_tc.h"
-#include "intel_tc_phy_regs.h"
#include "intel_vdsc.h"
#include "intel_vrr.h"
#include "skl_scaler.h"
@@ -845,22 +848,65 @@ bool intel_ddi_get_hw_state(struct intel_encoder *encoder,
}
static enum intel_display_power_domain
-intel_ddi_main_link_aux_domain(struct intel_digital_port *dig_port)
+intel_ddi_main_link_aux_domain(struct intel_digital_port *dig_port,
+ const struct intel_crtc_state *crtc_state)
{
- /* ICL+ HW requires corresponding AUX IOs to be powered up for PSR with
+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
+ enum phy phy = intel_port_to_phy(i915, dig_port->base.port);
+
+ /*
+ * ICL+ HW requires corresponding AUX IOs to be powered up for PSR with
* DC states enabled at the same time, while for driver initiated AUX
* transfers we need the same AUX IOs to be powered but with DC states
- * disabled. Accordingly use the AUX power domain here which leaves DC
- * states enabled.
- * However, for non-A AUX ports the corresponding non-EDP transcoders
- * would have already enabled power well 2 and DC_OFF. This means we can
- * acquire a wider POWER_DOMAIN_AUX_{B,C,D,F} reference instead of a
- * specific AUX_IO reference without powering up any extra wells.
- * Note that PSR is enabled only on Port A even though this function
- * returns the correct domain for other ports too.
+ * disabled. Accordingly use the AUX_IO_<port> power domain here which
+ * leaves DC states enabled.
+ *
+ * Before MTL TypeC PHYs (in all TypeC modes and both DP/HDMI) also require
+ * AUX IO to be enabled, but all these require DC_OFF to be enabled as
+ * well, so we can acquire a wider AUX_<port> power domain reference
+ * instead of a specific AUX_IO_<port> reference without powering up any
+ * extra wells.
*/
- return dig_port->aux_ch == AUX_CH_A ? POWER_DOMAIN_AUX_IO_A :
- intel_aux_power_domain(dig_port);
+ if (intel_encoder_can_psr(&dig_port->base))
+ return intel_display_power_aux_io_domain(i915, dig_port->aux_ch);
+ else if (DISPLAY_VER(i915) < 14 &&
+ (intel_crtc_has_dp_encoder(crtc_state) ||
+ intel_phy_is_tc(i915, phy)))
+ return intel_aux_power_domain(dig_port);
+ else
+ return POWER_DOMAIN_INVALID;
+}
+
+static void
+main_link_aux_power_domain_get(struct intel_digital_port *dig_port,
+ const struct intel_crtc_state *crtc_state)
+{
+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
+ enum intel_display_power_domain domain =
+ intel_ddi_main_link_aux_domain(dig_port, crtc_state);
+
+ drm_WARN_ON(&i915->drm, dig_port->aux_wakeref);
+
+ if (domain == POWER_DOMAIN_INVALID)
+ return;
+
+ dig_port->aux_wakeref = intel_display_power_get(i915, domain);
+}
+
+static void
+main_link_aux_power_domain_put(struct intel_digital_port *dig_port,
+ const struct intel_crtc_state *crtc_state)
+{
+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
+ enum intel_display_power_domain domain =
+ intel_ddi_main_link_aux_domain(dig_port, crtc_state);
+ intel_wakeref_t wf;
+
+ wf = fetch_and_zero(&dig_port->aux_wakeref);
+ if (!wf)
+ return;
+
+ intel_display_power_put(i915, domain, wf);
}
static void intel_ddi_get_power_domains(struct intel_encoder *encoder,
@@ -868,7 +914,6 @@ static void intel_ddi_get_power_domains(struct intel_encoder *encoder,
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_digital_port *dig_port;
- enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
/*
* TODO: Add support for MST encoders. Atm, the following should never
@@ -887,17 +932,7 @@ static void intel_ddi_get_power_domains(struct intel_encoder *encoder,
dig_port->ddi_io_power_domain);
}
- /*
- * AUX power is only needed for (e)DP mode, and for HDMI mode on TC
- * ports.
- */
- if (intel_crtc_has_dp_encoder(crtc_state) ||
- intel_phy_is_tc(dev_priv, phy)) {
- drm_WARN_ON(&dev_priv->drm, dig_port->aux_wakeref);
- dig_port->aux_wakeref =
- intel_display_power_get(dev_priv,
- intel_ddi_main_link_aux_domain(dig_port));
- }
+ main_link_aux_power_domain_get(dig_port, crtc_state);
}
void intel_ddi_enable_pipe_clock(struct intel_encoder *encoder,
@@ -1263,11 +1298,11 @@ static void tgl_dkl_phy_set_signal_levels(struct intel_encoder *encoder,
for (ln = 0; ln < 2; ln++) {
int level;
- intel_dkl_phy_write(dev_priv, DKL_TX_PMD_LANE_SUS(tc_port), ln, 0);
+ intel_dkl_phy_write(dev_priv, DKL_TX_PMD_LANE_SUS(tc_port, ln), 0);
level = intel_ddi_level(encoder, crtc_state, 2*ln+0);
- intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL0(tc_port), ln,
+ intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL0(tc_port, ln),
DKL_TX_PRESHOOT_COEFF_MASK |
DKL_TX_DE_EMPAHSIS_COEFF_MASK |
DKL_TX_VSWING_CONTROL_MASK,
@@ -1277,7 +1312,7 @@ static void tgl_dkl_phy_set_signal_levels(struct intel_encoder *encoder,
level = intel_ddi_level(encoder, crtc_state, 2*ln+1);
- intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL1(tc_port), ln,
+ intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL1(tc_port, ln),
DKL_TX_PRESHOOT_COEFF_MASK |
DKL_TX_DE_EMPAHSIS_COEFF_MASK |
DKL_TX_VSWING_CONTROL_MASK,
@@ -1285,7 +1320,7 @@ static void tgl_dkl_phy_set_signal_levels(struct intel_encoder *encoder,
DKL_TX_DE_EMPHASIS_COEFF(trans->entries[level].dkl.de_emphasis) |
DKL_TX_VSWING_CONTROL(trans->entries[level].dkl.vswing));
- intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL2(tc_port), ln,
+ intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL2(tc_port, ln),
DKL_TX_DP20BITMODE, 0);
if (IS_ALDERLAKE_P(dev_priv)) {
@@ -1304,7 +1339,7 @@ static void tgl_dkl_phy_set_signal_levels(struct intel_encoder *encoder,
val |= DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2(0);
}
- intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL2(tc_port), ln,
+ intel_dkl_phy_rmw(dev_priv, DKL_TX_DPCNTL2(tc_port, ln),
DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX1_MASK |
DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2_MASK,
val);
@@ -2017,8 +2052,8 @@ icl_program_mg_dp_mode(struct intel_digital_port *dig_port,
return;
if (DISPLAY_VER(dev_priv) >= 12) {
- ln0 = intel_dkl_phy_read(dev_priv, DKL_DP_MODE(tc_port), 0);
- ln1 = intel_dkl_phy_read(dev_priv, DKL_DP_MODE(tc_port), 1);
+ ln0 = intel_dkl_phy_read(dev_priv, DKL_DP_MODE(tc_port, 0));
+ ln1 = intel_dkl_phy_read(dev_priv, DKL_DP_MODE(tc_port, 1));
} else {
ln0 = intel_de_read(dev_priv, MG_DP_MODE(0, tc_port));
ln1 = intel_de_read(dev_priv, MG_DP_MODE(1, tc_port));
@@ -2079,8 +2114,8 @@ icl_program_mg_dp_mode(struct intel_digital_port *dig_port,
}
if (DISPLAY_VER(dev_priv) >= 12) {
- intel_dkl_phy_write(dev_priv, DKL_DP_MODE(tc_port), 0, ln0);
- intel_dkl_phy_write(dev_priv, DKL_DP_MODE(tc_port), 1, ln1);
+ intel_dkl_phy_write(dev_priv, DKL_DP_MODE(tc_port, 0), ln0);
+ intel_dkl_phy_write(dev_priv, DKL_DP_MODE(tc_port, 1), ln1);
} else {
intel_de_write(dev_priv, MG_DP_MODE(0, tc_port), ln0);
intel_de_write(dev_priv, MG_DP_MODE(1, tc_port), ln1);
@@ -2736,10 +2771,7 @@ static void intel_ddi_post_disable(struct intel_atomic_state *state,
intel_ddi_post_disable_dp(state, encoder, old_crtc_state,
old_conn_state);
- if (intel_crtc_has_dp_encoder(old_crtc_state) || is_tc_port)
- intel_display_power_put(dev_priv,
- intel_ddi_main_link_aux_domain(dig_port),
- fetch_and_zero(&dig_port->aux_wakeref));
+ main_link_aux_power_domain_put(dig_port, old_crtc_state);
if (is_tc_port)
intel_tc_port_put_link(dig_port);
@@ -3060,12 +3092,7 @@ intel_ddi_pre_pll_enable(struct intel_atomic_state *state,
if (is_tc_port)
intel_tc_port_get_link(dig_port, crtc_state->lane_count);
- if (intel_crtc_has_dp_encoder(crtc_state) || is_tc_port) {
- drm_WARN_ON(&dev_priv->drm, dig_port->aux_wakeref);
- dig_port->aux_wakeref =
- intel_display_power_get(dev_priv,
- intel_ddi_main_link_aux_domain(dig_port));
- }
+ main_link_aux_power_domain_get(dig_port, crtc_state);
if (is_tc_port && !intel_tc_port_in_tbt_alt_mode(dig_port))
/*
@@ -3085,7 +3112,7 @@ static void adlp_tbt_to_dp_alt_switch_wa(struct intel_encoder *encoder)
int ln;
for (ln = 0; ln < 2; ln++)
- intel_dkl_phy_rmw(i915, DKL_PCS_DW5(tc_port), ln, DKL_PCS_DW5_CORE_SOFTRESET, 0);
+ intel_dkl_phy_rmw(i915, DKL_PCS_DW5(tc_port, ln), DKL_PCS_DW5_CORE_SOFTRESET, 0);
}
static void intel_ddi_prepare_link_retrain(struct intel_dp *intel_dp,
@@ -3524,7 +3551,7 @@ static void icl_ddi_tc_get_clock(struct intel_encoder *encoder,
if (drm_WARN_ON(&i915->drm, !pll))
return;
- if (intel_get_shared_dpll_id(i915, pll) == DPLL_ID_ICL_TBTPLL)
+ if (pll->info->id == DPLL_ID_ICL_TBTPLL)
port_dpll_id = ICL_PORT_DPLL_DEFAULT;
else
port_dpll_id = ICL_PORT_DPLL_MG_PHY;
@@ -3537,7 +3564,7 @@ static void icl_ddi_tc_get_clock(struct intel_encoder *encoder,
icl_set_active_port_dpll(crtc_state, port_dpll_id);
- if (intel_get_shared_dpll_id(i915, crtc_state->shared_dpll) == DPLL_ID_ICL_TBTPLL)
+ if (crtc_state->shared_dpll->info->id == DPLL_ID_ICL_TBTPLL)
crtc_state->port_clock = icl_calc_tbt_pll_link(i915, encoder->port);
else
crtc_state->port_clock = intel_dpll_get_freq(i915, crtc_state->shared_dpll,
@@ -3579,7 +3606,7 @@ static void intel_ddi_sync_state(struct intel_encoder *encoder,
enum phy phy = intel_port_to_phy(i915, encoder->port);
if (intel_phy_is_tc(i915, phy))
- intel_tc_port_sanitize(enc_to_dig_port(encoder));
+ intel_tc_port_sanitize_mode(enc_to_dig_port(encoder));
if (crtc_state && intel_crtc_has_dp_encoder(crtc_state))
intel_dp_sync_state(encoder, crtc_state);
@@ -3789,11 +3816,17 @@ static void intel_ddi_encoder_destroy(struct drm_encoder *encoder)
static void intel_ddi_encoder_reset(struct drm_encoder *encoder)
{
+ struct drm_i915_private *i915 = to_i915(encoder->dev);
struct intel_dp *intel_dp = enc_to_intel_dp(to_intel_encoder(encoder));
+ struct intel_digital_port *dig_port = enc_to_dig_port(to_intel_encoder(encoder));
+ enum phy phy = intel_port_to_phy(i915, dig_port->base.port);
intel_dp->reset_link_params = true;
intel_pps_encoder_reset(intel_dp);
+
+ if (intel_phy_is_tc(i915, phy))
+ intel_tc_port_init_mode(dig_port);
}
static const struct drm_encoder_funcs intel_ddi_funcs = {
@@ -4106,12 +4139,6 @@ intel_ddi_max_lanes(struct intel_digital_port *dig_port)
return max_lanes;
}
-static bool hti_uses_phy(struct drm_i915_private *i915, enum phy phy)
-{
- return i915->hti_state & HDPORT_ENABLED &&
- i915->hti_state & HDPORT_DDI_USED(phy);
-}
-
static enum hpd_pin xelpd_hpd_pin(struct drm_i915_private *dev_priv,
enum port port)
{
@@ -4240,7 +4267,7 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port)
* driver. In that case we should skip initializing the corresponding
* outputs.
*/
- if (hti_uses_phy(dev_priv, phy)) {
+ if (intel_hti_uses_phy(dev_priv, phy)) {
drm_dbg_kms(&dev_priv->drm, "PORT %c / PHY %c reserved by HTI\n",
port_name(port), phy_name(phy));
return;
diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c
index de77054..6c2686e 100644
--- a/drivers/gpu/drm/i915/display/intel_display.c
+++ b/drivers/gpu/drm/i915/display/intel_display.c
@@ -76,6 +76,7 @@
#include "g4x_hdmi.h"
#include "hsw_ips.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "i915_utils.h"
#include "icl_dsi.h"
#include "intel_acpi.h"
@@ -90,6 +91,7 @@
#include "intel_display_types.h"
#include "intel_dmc.h"
#include "intel_dp_link_training.h"
+#include "intel_dpio_phy.h"
#include "intel_dpt.h"
#include "intel_dsb.h"
#include "intel_fbc.h"
@@ -99,6 +101,7 @@
#include "intel_frontbuffer.h"
#include "intel_hdcp.h"
#include "intel_hotplug.h"
+#include "intel_hti.h"
#include "intel_modeset_verify.h"
#include "intel_modeset_setup.h"
#include "intel_overlay.h"
@@ -831,13 +834,27 @@ intel_plane_fence_y_offset(const struct intel_plane_state *plane_state)
}
static int
+intel_display_commit_duplicated_state(struct intel_atomic_state *state,
+ struct drm_modeset_acquire_ctx *ctx)
+{
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ int ret;
+
+ ret = drm_atomic_helper_commit_duplicated_state(&state->base, ctx);
+
+ drm_WARN_ON(&i915->drm, ret == -EDEADLK);
+
+ return ret;
+}
+
+static int
__intel_display_resume(struct drm_i915_private *i915,
struct drm_atomic_state *state,
struct drm_modeset_acquire_ctx *ctx)
{
struct drm_crtc_state *crtc_state;
struct drm_crtc *crtc;
- int i, ret;
+ int i;
intel_modeset_setup_hw_state(i915, ctx);
intel_vga_redisable(i915);
@@ -863,11 +880,7 @@ __intel_display_resume(struct drm_i915_private *i915,
if (!HAS_GMCH(i915))
to_intel_atomic_state(state)->skip_intermediate_wm = true;
- ret = drm_atomic_helper_commit_duplicated_state(state, ctx);
-
- drm_WARN_ON(&i915->drm, ret == -EDEADLK);
-
- return ret;
+ return intel_display_commit_duplicated_state(to_intel_atomic_state(state), ctx);
}
static bool gpu_reset_clobbers_display(struct drm_i915_private *dev_priv)
@@ -878,8 +891,7 @@ static bool gpu_reset_clobbers_display(struct drm_i915_private *dev_priv)
void intel_display_prepare_reset(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
- struct drm_modeset_acquire_ctx *ctx = &dev_priv->reset_ctx;
+ struct drm_modeset_acquire_ctx *ctx = &dev_priv->display.restore.reset_ctx;
struct drm_atomic_state *state;
int ret;
@@ -906,10 +918,10 @@ void intel_display_prepare_reset(struct drm_i915_private *dev_priv)
* Need mode_config.mutex so that we don't
* trample ongoing ->detect() and whatnot.
*/
- mutex_lock(&dev->mode_config.mutex);
+ mutex_lock(&dev_priv->drm.mode_config.mutex);
drm_modeset_acquire_init(ctx, 0);
while (1) {
- ret = drm_modeset_lock_all_ctx(dev, ctx);
+ ret = drm_modeset_lock_all_ctx(&dev_priv->drm, ctx);
if (ret != -EDEADLK)
break;
@@ -919,7 +931,7 @@ void intel_display_prepare_reset(struct drm_i915_private *dev_priv)
* Disabling the crtcs gracefully seems nicer. Also the
* g33 docs say we should at least disable all the planes.
*/
- state = drm_atomic_helper_duplicate_state(dev, ctx);
+ state = drm_atomic_helper_duplicate_state(&dev_priv->drm, ctx);
if (IS_ERR(state)) {
ret = PTR_ERR(state);
drm_err(&dev_priv->drm, "Duplicating state failed with %i\n",
@@ -927,7 +939,7 @@ void intel_display_prepare_reset(struct drm_i915_private *dev_priv)
return;
}
- ret = drm_atomic_helper_disable_all(dev, ctx);
+ ret = drm_atomic_helper_disable_all(&dev_priv->drm, ctx);
if (ret) {
drm_err(&dev_priv->drm, "Suspending crtc's failed with %i\n",
ret);
@@ -935,13 +947,13 @@ void intel_display_prepare_reset(struct drm_i915_private *dev_priv)
return;
}
- dev_priv->modeset_restore_state = state;
+ dev_priv->display.restore.modeset_state = state;
state->acquire_ctx = ctx;
}
void intel_display_finish_reset(struct drm_i915_private *i915)
{
- struct drm_modeset_acquire_ctx *ctx = &i915->reset_ctx;
+ struct drm_modeset_acquire_ctx *ctx = &i915->display.restore.reset_ctx;
struct drm_atomic_state *state;
int ret;
@@ -952,14 +964,14 @@ void intel_display_finish_reset(struct drm_i915_private *i915)
if (!test_bit(I915_RESET_MODESET, &to_gt(i915)->reset.flags))
return;
- state = fetch_and_zero(&i915->modeset_restore_state);
+ state = fetch_and_zero(&i915->display.restore.modeset_state);
if (!state)
goto unlock;
/* reset doesn't touch the display */
if (!gpu_reset_clobbers_display(i915)) {
/* for testing only restore the display */
- ret = __intel_display_resume(i915, state, ctx);
+ ret = intel_display_commit_duplicated_state(to_intel_atomic_state(state), ctx);
if (ret)
drm_err(&i915->drm,
"Restoring old state failed with %i\n", ret);
@@ -1252,8 +1264,6 @@ static void intel_post_plane_update(struct intel_atomic_state *state,
if (needs_cursorclk_wa(old_crtc_state) &&
!needs_cursorclk_wa(new_crtc_state))
icl_wa_cursorclkgating(dev_priv, pipe, false);
-
- intel_drrs_activate(new_crtc_state);
}
static void intel_crtc_enable_flip_done(struct intel_atomic_state *state,
@@ -2434,7 +2444,7 @@ int intel_display_suspend(struct drm_device *dev)
drm_err(&dev_priv->drm, "Suspending crtc's failed with %i\n",
ret);
else
- dev_priv->modeset_restore_state = state;
+ dev_priv->display.restore.modeset_state = state;
return ret;
}
@@ -4045,20 +4055,19 @@ static bool hsw_get_pipe_config(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config)
{
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_display_power_domain_set power_domain_set = { };
bool active;
u32 tmp;
- if (!intel_display_power_get_in_set_if_enabled(dev_priv, &power_domain_set,
+ if (!intel_display_power_get_in_set_if_enabled(dev_priv, &crtc->hw_readout_power_domains,
POWER_DOMAIN_PIPE(crtc->pipe)))
return false;
pipe_config->shared_dpll = NULL;
- active = hsw_get_transcoder_state(crtc, pipe_config, &power_domain_set);
+ active = hsw_get_transcoder_state(crtc, pipe_config, &crtc->hw_readout_power_domains);
if ((IS_GEMINILAKE(dev_priv) || IS_BROXTON(dev_priv)) &&
- bxt_get_dsi_transcoder_state(crtc, pipe_config, &power_domain_set)) {
+ bxt_get_dsi_transcoder_state(crtc, pipe_config, &crtc->hw_readout_power_domains)) {
drm_WARN_ON(&dev_priv->drm, active);
active = true;
}
@@ -4117,7 +4126,7 @@ static bool hsw_get_pipe_config(struct intel_crtc *crtc,
pipe_config->ips_linetime =
REG_FIELD_GET(HSW_IPS_LINETIME_MASK, tmp);
- if (intel_display_power_get_in_set_if_enabled(dev_priv, &power_domain_set,
+ if (intel_display_power_get_in_set_if_enabled(dev_priv, &crtc->hw_readout_power_domains,
POWER_DOMAIN_PIPE_PANEL_FITTER(crtc->pipe))) {
if (DISPLAY_VER(dev_priv) >= 9)
skl_get_pfit_config(pipe_config);
@@ -4148,7 +4157,7 @@ static bool hsw_get_pipe_config(struct intel_crtc *crtc,
}
out:
- intel_display_power_put_all_in_set(dev_priv, &power_domain_set);
+ intel_display_power_put_all_in_set(dev_priv, &crtc->hw_readout_power_domains);
return active;
}
@@ -4576,8 +4585,8 @@ static bool encoders_cloneable(const struct intel_encoder *a,
const struct intel_encoder *b)
{
/* masks could be asymmetric, so check both ways */
- return a == b || (a->cloneable & (1 << b->type) &&
- b->cloneable & (1 << a->type));
+ return a == b || (a->cloneable & BIT(b->type) &&
+ b->cloneable & BIT(a->type));
}
static bool check_single_encoder_cloning(struct intel_atomic_state *state,
@@ -4828,14 +4837,14 @@ static int intel_crtc_atomic_check(struct intel_atomic_state *state,
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
struct intel_crtc_state *crtc_state =
intel_atomic_get_new_crtc_state(state, crtc);
- bool mode_changed = intel_crtc_needs_modeset(crtc_state);
int ret;
if (DISPLAY_VER(dev_priv) < 5 && !IS_G4X(dev_priv) &&
- mode_changed && !crtc_state->hw.active)
+ intel_crtc_needs_modeset(crtc_state) &&
+ !crtc_state->hw.active)
crtc_state->update_wm_post = true;
- if (mode_changed) {
+ if (intel_crtc_needs_modeset(crtc_state)) {
ret = intel_dpll_crtc_get_shared_dpll(state, crtc);
if (ret)
return ret;
@@ -4848,8 +4857,7 @@ static int intel_crtc_atomic_check(struct intel_atomic_state *state,
if (c8_planes_changed(crtc_state))
crtc_state->uapi.color_mgmt_changed = true;
- if (mode_changed || crtc_state->update_pipe ||
- crtc_state->uapi.color_mgmt_changed) {
+ if (intel_crtc_needs_color_update(crtc_state)) {
ret = intel_color_check(crtc_state);
if (ret)
return ret;
@@ -4875,7 +4883,8 @@ static int intel_crtc_atomic_check(struct intel_atomic_state *state,
}
if (DISPLAY_VER(dev_priv) >= 9) {
- if (mode_changed || crtc_state->update_pipe) {
+ if (intel_crtc_needs_modeset(crtc_state) ||
+ intel_crtc_needs_fastset(crtc_state)) {
ret = skl_update_scaler_crtc(crtc_state);
if (ret)
return ret;
@@ -5641,39 +5650,6 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
PIPE_CONF_CHECK_I(name.y2); \
} while (0)
-/* This is required for BDW+ where there is only one set of registers for
- * switching between high and low RR.
- * This macro can be used whenever a comparison has to be made between one
- * hw state and multiple sw state variables.
- */
-#define PIPE_CONF_CHECK_M_N_ALT(name, alt_name) do { \
- if (!intel_compare_link_m_n(¤t_config->name, \
- &pipe_config->name) && \
- !intel_compare_link_m_n(¤t_config->alt_name, \
- &pipe_config->name)) { \
- pipe_config_mismatch(fastset, crtc, __stringify(name), \
- "(expected tu %i data %i/%i link %i/%i, " \
- "or tu %i data %i/%i link %i/%i, " \
- "found tu %i, data %i/%i link %i/%i)", \
- current_config->name.tu, \
- current_config->name.data_m, \
- current_config->name.data_n, \
- current_config->name.link_m, \
- current_config->name.link_n, \
- current_config->alt_name.tu, \
- current_config->alt_name.data_m, \
- current_config->alt_name.data_n, \
- current_config->alt_name.link_m, \
- current_config->alt_name.link_n, \
- pipe_config->name.tu, \
- pipe_config->name.data_m, \
- pipe_config->name.data_n, \
- pipe_config->name.link_m, \
- pipe_config->name.link_n); \
- ret = false; \
- } \
-} while (0)
-
#define PIPE_CONF_CHECK_FLAGS(name, mask) do { \
if ((current_config->name ^ pipe_config->name) & (mask)) { \
pipe_config_mismatch(fastset, crtc, __stringify(name), \
@@ -5742,7 +5718,7 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
if (HAS_DOUBLE_BUFFERED_M_N(dev_priv)) {
if (!fastset || !pipe_config->seamless_m_n)
- PIPE_CONF_CHECK_M_N_ALT(dp_m_n, dp_m2_n2);
+ PIPE_CONF_CHECK_M_N(dp_m_n);
} else {
PIPE_CONF_CHECK_M_N(dp_m_n);
PIPE_CONF_CHECK_M_N(dp_m2_n2);
@@ -5819,7 +5795,7 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config,
bp_gamma = intel_color_get_gamma_bit_precision(pipe_config);
if (bp_gamma)
- PIPE_CONF_CHECK_COLOR_LUT(gamma_mode, hw.gamma_lut, bp_gamma);
+ PIPE_CONF_CHECK_COLOR_LUT(gamma_mode, post_csc_lut, bp_gamma);
if (current_config->active_planes) {
PIPE_CONF_CHECK_BOOL(has_psr);
@@ -5941,7 +5917,8 @@ intel_verify_planes(struct intel_atomic_state *state)
plane_state->uapi.visible);
}
-int intel_modeset_all_pipes(struct intel_atomic_state *state)
+int intel_modeset_all_pipes(struct intel_atomic_state *state,
+ const char *reason)
{
struct drm_i915_private *dev_priv = to_i915(state->base.dev);
struct intel_crtc *crtc;
@@ -5959,10 +5936,14 @@ int intel_modeset_all_pipes(struct intel_atomic_state *state)
return PTR_ERR(crtc_state);
if (!crtc_state->hw.active ||
- drm_atomic_crtc_needs_modeset(&crtc_state->uapi))
+ intel_crtc_needs_modeset(crtc_state))
continue;
+ drm_dbg_kms(&dev_priv->drm, "[CRTC:%d:%s] Full modeset due to %s\n",
+ crtc->base.base.id, crtc->base.name, reason);
+
crtc_state->uapi.mode_changed = true;
+ crtc_state->update_pipe = false;
ret = drm_atomic_add_affected_connectors(&state->base,
&crtc->base);
@@ -6138,7 +6119,8 @@ static void intel_crtc_check_fastset(const struct intel_crtc_state *old_crtc_sta
return;
new_crtc_state->uapi.mode_changed = false;
- new_crtc_state->update_pipe = true;
+ if (!intel_crtc_needs_modeset(new_crtc_state))
+ new_crtc_state->update_pipe = true;
}
static int intel_crtc_add_planes_to_state(struct intel_atomic_state *state,
@@ -6910,12 +6892,19 @@ static int intel_atomic_check(struct drm_device *dev,
for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
new_crtc_state, i) {
+ intel_color_assert_luts(new_crtc_state);
+
ret = intel_async_flip_check_hw(state, crtc);
if (ret)
goto fail;
+ /* Either full modeset or fastset (or neither), never both */
+ drm_WARN_ON(&dev_priv->drm,
+ intel_crtc_needs_modeset(new_crtc_state) &&
+ intel_crtc_needs_fastset(new_crtc_state));
+
if (!intel_crtc_needs_modeset(new_crtc_state) &&
- !new_crtc_state->update_pipe)
+ !intel_crtc_needs_fastset(new_crtc_state))
continue;
intel_crtc_state_dump(new_crtc_state, state,
@@ -6951,12 +6940,8 @@ static int intel_atomic_prepare_commit(struct intel_atomic_state *state)
return ret;
for_each_new_intel_crtc_in_state(state, crtc, crtc_state, i) {
- bool mode_changed = intel_crtc_needs_modeset(crtc_state);
-
- if (mode_changed || crtc_state->update_pipe ||
- crtc_state->uapi.color_mgmt_changed) {
+ if (intel_crtc_needs_color_update(crtc_state))
intel_dsb_prepare(crtc_state);
- }
}
return 0;
@@ -7037,14 +7022,13 @@ static void commit_pipe_pre_planes(struct intel_atomic_state *state,
* CRTC was enabled.
*/
if (!modeset) {
- if (new_crtc_state->uapi.color_mgmt_changed ||
- new_crtc_state->update_pipe)
+ if (intel_crtc_needs_color_update(new_crtc_state))
intel_color_commit_arm(new_crtc_state);
if (DISPLAY_VER(dev_priv) >= 9 || IS_BROADWELL(dev_priv))
bdw_set_pipemisc(new_crtc_state);
- if (new_crtc_state->update_pipe)
+ if (intel_crtc_needs_fastset(new_crtc_state))
intel_pipe_fastset(old_crtc_state, new_crtc_state);
}
@@ -7103,25 +7087,23 @@ static void intel_update_crtc(struct intel_atomic_state *state,
if (!modeset) {
if (new_crtc_state->preload_luts &&
- (new_crtc_state->uapi.color_mgmt_changed ||
- new_crtc_state->update_pipe))
+ intel_crtc_needs_color_update(new_crtc_state))
intel_color_load_luts(new_crtc_state);
intel_pre_plane_update(state, crtc);
- if (new_crtc_state->update_pipe)
+ if (intel_crtc_needs_fastset(new_crtc_state))
intel_encoders_update_pipe(state, crtc);
if (DISPLAY_VER(i915) >= 11 &&
- new_crtc_state->update_pipe)
+ intel_crtc_needs_fastset(new_crtc_state))
icl_set_pipe_chicken(new_crtc_state);
}
intel_fbc_update(state, crtc);
if (!modeset &&
- (new_crtc_state->uapi.color_mgmt_changed ||
- new_crtc_state->update_pipe))
+ intel_crtc_needs_color_update(new_crtc_state))
intel_color_commit_noarm(new_crtc_state);
intel_crtc_planes_update_noarm(state, crtc);
@@ -7143,7 +7125,7 @@ static void intel_update_crtc(struct intel_atomic_state *state,
* valid pipe configuration from the BIOS we need to take care
* of enabling them on the CRTC's first fastset.
*/
- if (new_crtc_state->update_pipe && !modeset &&
+ if (intel_crtc_needs_fastset(new_crtc_state) && !modeset &&
old_crtc_state->inherited)
intel_crtc_arm_fifo_underrun(crtc, new_crtc_state);
}
@@ -7166,9 +7148,7 @@ static void intel_old_crtc_state_disables(struct intel_atomic_state *state,
intel_fbc_disable(crtc);
intel_disable_shared_dpll(old_crtc_state);
- /* FIXME unify this for all platforms */
- if (!new_crtc_state->hw.active &&
- !HAS_GMCH(dev_priv))
+ if (!new_crtc_state->hw.active)
intel_initial_watermarks(state, crtc);
}
@@ -7503,9 +7483,8 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state)
for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
new_crtc_state, i) {
if (intel_crtc_needs_modeset(new_crtc_state) ||
- new_crtc_state->update_pipe) {
+ intel_crtc_needs_fastset(new_crtc_state))
intel_modeset_get_crtc_power_domains(new_crtc_state, &put_domains[crtc->pipe]);
- }
}
intel_commit_modeset_disables(state);
@@ -7609,6 +7588,12 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state)
intel_modeset_verify_crtc(crtc, state, old_crtc_state, new_crtc_state);
/*
+ * Activate DRRS after state readout to avoid
+ * dp_m_n vs. dp_m2_n2 confusion on BDW+.
+ */
+ intel_drrs_activate(new_crtc_state);
+
+ /*
* DSB cleanup is done in cleanup_work aligning with framebuffer
* cleanup. So copy and reset the dsb structure to sync with
* commit_done and later do dsb cleanup in cleanup_work.
@@ -8348,6 +8333,7 @@ void intel_init_display_hooks(struct drm_i915_private *dev_priv)
if (!HAS_DISPLAY(dev_priv))
return;
+ intel_color_init_hooks(dev_priv);
intel_init_cdclk_hooks(dev_priv);
intel_audio_hooks_init(dev_priv);
@@ -8589,7 +8575,7 @@ static void intel_mode_config_init(struct drm_i915_private *i915)
struct drm_mode_config *mode_config = &i915->drm.mode_config;
drm_mode_config_init(&i915->drm);
- INIT_LIST_HEAD(&i915->global_obj_list);
+ INIT_LIST_HEAD(&i915->display.global.obj_list);
mode_config->min_width = 0;
mode_config->min_height = 0;
@@ -8678,6 +8664,10 @@ int intel_modeset_init_noirq(struct drm_i915_private *i915)
if (ret)
goto cleanup_vga_client_pw_domain_dmc;
+ ret = intel_color_init(i915);
+ if (ret)
+ goto cleanup_vga_client_pw_domain_dmc;
+
ret = intel_dbuf_init(i915);
if (ret)
goto cleanup_vga_client_pw_domain_dmc;
@@ -8750,12 +8740,7 @@ int intel_modeset_init_nogem(struct drm_i915_private *i915)
if (i915->display.cdclk.max_cdclk_freq == 0)
intel_update_max_cdclk(i915);
- /*
- * If the platform has HTI, we need to find out whether it has reserved
- * any display resources before we create our display outputs.
- */
- if (INTEL_INFO(i915)->display.has_hti)
- i915->hti_state = intel_de_read(i915, HDPORT_STATE);
+ intel_hti_init(i915);
/* Just disable it once at startup */
intel_vga_disable(i915);
@@ -8918,14 +8903,14 @@ void i830_disable_pipe(struct drm_i915_private *dev_priv, enum pipe pipe)
void intel_display_resume(struct drm_device *dev)
{
struct drm_i915_private *i915 = to_i915(dev);
- struct drm_atomic_state *state = i915->modeset_restore_state;
+ struct drm_atomic_state *state = i915->display.restore.modeset_state;
struct drm_modeset_acquire_ctx ctx;
int ret;
if (!HAS_DISPLAY(i915))
return;
- i915->modeset_restore_state = NULL;
+ i915->display.restore.modeset_state = NULL;
if (state)
state->acquire_ctx = &ctx;
diff --git a/drivers/gpu/drm/i915/display/intel_display.h b/drivers/gpu/drm/i915/display/intel_display.h
index 884e8e6..7140301 100644
--- a/drivers/gpu/drm/i915/display/intel_display.h
+++ b/drivers/gpu/drm/i915/display/intel_display.h
@@ -53,6 +53,7 @@ struct intel_digital_port;
struct intel_dp;
struct intel_encoder;
struct intel_initial_plane_config;
+struct intel_link_m_n;
struct intel_load_detect_pipe;
struct intel_plane;
struct intel_plane_state;
@@ -61,24 +62,6 @@ struct intel_remapped_info;
struct intel_rotation_info;
struct pci_dev;
-enum i915_gpio {
- GPIOA,
- GPIOB,
- GPIOC,
- GPIOD,
- GPIOE,
- GPIOF,
- GPIOG,
- GPIOH,
- __GPIOI_UNUSED,
- GPIOJ,
- GPIOK,
- GPIOL,
- GPIOM,
- GPION,
- GPIOO,
-};
-
/*
* Keep the pipe enum values fixed: the code assumes that PIPE_A=0, the
* rest have consecutive values and match the enum values of transcoders
@@ -279,17 +262,6 @@ enum tc_port_mode {
TC_PORT_LEGACY,
};
-enum dpio_channel {
- DPIO_CH0,
- DPIO_CH1
-};
-
-enum dpio_phy {
- DPIO_PHY0,
- DPIO_PHY1,
- DPIO_PHY2,
-};
-
enum aux_ch {
AUX_CH_A,
AUX_CH_B,
@@ -316,15 +288,6 @@ enum aux_ch {
#define aux_ch_name(a) ((a) + 'A')
-/* Used by dp and fdi links */
-struct intel_link_m_n {
- u32 tu;
- u32 data_m;
- u32 data_n;
- u32 link_m;
- u32 link_n;
-};
-
enum phy {
PHY_NONE = -1,
@@ -469,10 +432,6 @@ enum hpd_pin {
list_for_each_entry((intel_encoder), &(dev)->mode_config.encoder_list, base.head) \
for_each_if((intel_encoder)->base.crtc == (__crtc))
-#define for_each_connector_on_encoder(dev, __encoder, intel_connector) \
- list_for_each_entry((intel_connector), &(dev)->mode_config.connector_list, base.head) \
- for_each_if((intel_connector)->base.encoder == (__encoder))
-
#define for_each_old_intel_plane_in_state(__state, plane, old_plane_state, __i) \
for ((__i) = 0; \
(__i) < (__state)->base.dev->mode_config.num_total_plane && \
@@ -683,7 +642,8 @@ void intel_modeset_driver_remove(struct drm_i915_private *i915);
void intel_modeset_driver_remove_noirq(struct drm_i915_private *i915);
void intel_modeset_driver_remove_nogem(struct drm_i915_private *i915);
void intel_display_resume(struct drm_device *dev);
-int intel_modeset_all_pipes(struct intel_atomic_state *state);
+int intel_modeset_all_pipes(struct intel_atomic_state *state,
+ const char *reason);
void intel_modeset_get_crtc_power_domains(struct intel_crtc_state *crtc_state,
struct intel_power_domain_mask *old_domains);
void intel_modeset_put_crtc_power_domains(struct intel_crtc *crtc,
diff --git a/drivers/gpu/drm/i915/display/intel_display_core.h b/drivers/gpu/drm/i915/display/intel_display_core.h
index 9b51148..57ddce3 100644
--- a/drivers/gpu/drm/i915/display/intel_display_core.h
+++ b/drivers/gpu/drm/i915/display/intel_display_core.h
@@ -14,6 +14,7 @@
#include <linux/workqueue.h>
#include <drm/drm_connector.h>
+#include <drm/drm_modeset_lock.h>
#include "intel_cdclk.h"
#include "intel_display.h"
@@ -28,6 +29,7 @@
struct drm_i915_private;
struct drm_property;
+struct drm_property_blob;
struct i915_audio_component;
struct i915_hdcp_comp_master;
struct intel_atomic_state;
@@ -309,6 +311,10 @@ struct intel_display {
} cdclk;
struct {
+ struct drm_property_blob *glk_linear_degamma_lut;
+ } color;
+
+ struct {
/* The current hardware dbuf configuration */
u8 enabled_slices;
@@ -340,6 +346,10 @@ struct intel_display {
} fdi;
struct {
+ struct list_head obj_list;
+ } global;
+
+ struct {
/*
* Base address of where the gmbus and gpio blocks are located
* (either on PCH or on SoC for platforms without PCH).
@@ -366,6 +376,16 @@ struct intel_display {
} hdcp;
struct {
+ /*
+ * HTI (aka HDPORT) state read during initial hw readout. Most
+ * platforms don't have HTI, so this will just stay 0. Those
+ * that do will use this later to figure out which PLLs and PHYs
+ * are unavailable for driver usage.
+ */
+ u32 state;
+ } hti;
+
+ struct {
struct i915_power_domains domains;
/* Shadow for DISPLAY_PHY_CONTROL which can't be safely read */
@@ -392,6 +412,12 @@ struct intel_display {
} quirks;
struct {
+ /* restore state for suspend/resume and display reset */
+ struct drm_atomic_state *modeset_state;
+ struct drm_modeset_acquire_ctx reset_ctx;
+ } restore;
+
+ struct {
enum {
I915_SAGV_UNKNOWN = 0,
I915_SAGV_DISABLED,
diff --git a/drivers/gpu/drm/i915/display/intel_display_debugfs.c b/drivers/gpu/drm/i915/display/intel_display_debugfs.c
index 7c7253a..7bcd9038 100644
--- a/drivers/gpu/drm/i915/display/intel_display_debugfs.c
+++ b/drivers/gpu/drm/i915/display/intel_display_debugfs.c
@@ -9,6 +9,8 @@
#include <drm/drm_fourcc.h>
#include "i915_debugfs.h"
+#include "i915_irq.h"
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_debugfs.h"
#include "intel_display_power.h"
@@ -22,6 +24,7 @@
#include "intel_fbdev.h"
#include "intel_hdcp.h"
#include "intel_hdmi.h"
+#include "intel_hotplug.h"
#include "intel_panel.h"
#include "intel_pm.h"
#include "intel_psr.h"
@@ -127,7 +130,6 @@ static int i915_vbt(struct seq_file *m, void *unused)
static int i915_gem_framebuffer_info(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_device *dev = &dev_priv->drm;
struct intel_framebuffer *fbdev_fb = NULL;
struct drm_framebuffer *drm_fb;
@@ -146,8 +148,8 @@ static int i915_gem_framebuffer_info(struct seq_file *m, void *data)
}
#endif
- mutex_lock(&dev->mode_config.fb_lock);
- drm_for_each_fb(drm_fb, dev) {
+ mutex_lock(&dev_priv->drm.mode_config.fb_lock);
+ drm_for_each_fb(drm_fb, &dev_priv->drm) {
struct intel_framebuffer *fb = to_intel_framebuffer(drm_fb);
if (fb == fbdev_fb)
continue;
@@ -162,7 +164,7 @@ static int i915_gem_framebuffer_info(struct seq_file *m, void *data)
i915_debugfs_describe_obj(m, intel_fb_obj(&fb->base));
seq_putc(m, '\n');
}
- mutex_unlock(&dev->mode_config.fb_lock);
+ mutex_unlock(&dev_priv->drm.mode_config.fb_lock);
return 0;
}
@@ -897,7 +899,6 @@ static void intel_crtc_info(struct seq_file *m, struct intel_crtc *crtc)
static int i915_display_info(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_device *dev = &dev_priv->drm;
struct intel_crtc *crtc;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
@@ -905,22 +906,22 @@ static int i915_display_info(struct seq_file *m, void *unused)
wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
- drm_modeset_lock_all(dev);
+ drm_modeset_lock_all(&dev_priv->drm);
seq_printf(m, "CRTC info\n");
seq_printf(m, "---------\n");
- for_each_intel_crtc(dev, crtc)
+ for_each_intel_crtc(&dev_priv->drm, crtc)
intel_crtc_info(m, crtc);
seq_printf(m, "\n");
seq_printf(m, "Connector info\n");
seq_printf(m, "--------------\n");
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter)
intel_connector_info(m, connector);
drm_connector_list_iter_end(&conn_iter);
- drm_modeset_unlock_all(dev);
+ drm_modeset_unlock_all(&dev_priv->drm);
intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
@@ -930,10 +931,9 @@ static int i915_display_info(struct seq_file *m, void *unused)
static int i915_shared_dplls_info(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_device *dev = &dev_priv->drm;
int i;
- drm_modeset_lock_all(dev);
+ drm_modeset_lock_all(&dev_priv->drm);
seq_printf(m, "PLL refclks: non-SSC: %d kHz, SSC: %d kHz\n",
dev_priv->display.dpll.ref_clks.nssc,
@@ -978,7 +978,7 @@ static int i915_shared_dplls_info(struct seq_file *m, void *unused)
seq_printf(m, " mg_pll_tdc_coldst_bias: 0x%08x\n",
pll->state.hw_state.mg_pll_tdc_coldst_bias);
}
- drm_modeset_unlock_all(dev);
+ drm_modeset_unlock_all(&dev_priv->drm);
return 0;
}
@@ -986,14 +986,13 @@ static int i915_shared_dplls_info(struct seq_file *m, void *unused)
static int i915_ddb_info(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_device *dev = &dev_priv->drm;
struct skl_ddb_entry *entry;
struct intel_crtc *crtc;
if (DISPLAY_VER(dev_priv) < 9)
return -ENODEV;
- drm_modeset_lock_all(dev);
+ drm_modeset_lock_all(&dev_priv->drm);
seq_printf(m, "%-15s%8s%8s%8s\n", "", "Start", "End", "Size");
@@ -1017,53 +1016,7 @@ static int i915_ddb_info(struct seq_file *m, void *unused)
entry->end, skl_ddb_entry_size(entry));
}
- drm_modeset_unlock_all(dev);
-
- return 0;
-}
-
-static int i915_drrs_status(struct seq_file *m, void *unused)
-{
- struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_connector_list_iter conn_iter;
- struct intel_connector *connector;
- struct intel_crtc *crtc;
-
- drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
- for_each_intel_connector_iter(connector, &conn_iter) {
- seq_printf(m, "[CONNECTOR:%d:%s] DRRS type: %s\n",
- connector->base.base.id, connector->base.name,
- intel_drrs_type_str(intel_panel_drrs_type(connector)));
- }
- drm_connector_list_iter_end(&conn_iter);
-
- seq_puts(m, "\n");
-
- for_each_intel_crtc(&dev_priv->drm, crtc) {
- const struct intel_crtc_state *crtc_state =
- to_intel_crtc_state(crtc->base.state);
-
- seq_printf(m, "[CRTC:%d:%s]:\n",
- crtc->base.base.id, crtc->base.name);
-
- mutex_lock(&crtc->drrs.mutex);
-
- /* DRRS Supported */
- seq_printf(m, "\tDRRS Enabled: %s\n",
- str_yes_no(crtc_state->has_drrs));
-
- seq_printf(m, "\tDRRS Active: %s\n",
- str_yes_no(intel_drrs_is_active(crtc)));
-
- seq_printf(m, "\tBusy_frontbuffer_bits: 0x%X\n",
- crtc->drrs.busy_frontbuffer_bits);
-
- seq_printf(m, "\tDRRS refresh rate: %s\n",
- crtc->drrs.refresh_rate == DRRS_REFRESH_RATE_LOW ?
- "low" : "high");
-
- mutex_unlock(&crtc->drrs.mutex);
- }
+ drm_modeset_unlock_all(&dev_priv->drm);
return 0;
}
@@ -1107,13 +1060,12 @@ static int i915_lpsp_status(struct seq_file *m, void *unused)
static int i915_dp_mst_info(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct drm_device *dev = &dev_priv->drm;
struct intel_encoder *intel_encoder;
struct intel_digital_port *dig_port;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
if (connector->connector_type != DRM_MODE_CONNECTOR_DisplayPort)
continue;
@@ -1200,12 +1152,11 @@ static ssize_t i915_displayport_test_active_write(struct file *file,
static int i915_displayport_test_active_show(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = m->private;
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
struct intel_encoder *encoder;
@@ -1250,12 +1201,11 @@ static const struct file_operations i915_displayport_test_active_fops = {
static int i915_displayport_test_data_show(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = m->private;
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
struct intel_encoder *encoder;
@@ -1304,12 +1254,11 @@ DEFINE_SHOW_ATTRIBUTE(i915_displayport_test_data);
static int i915_displayport_test_type_show(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = m->private;
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
drm_for_each_connector_iter(connector, &conn_iter) {
struct intel_encoder *encoder;
@@ -1336,7 +1285,6 @@ DEFINE_SHOW_ATTRIBUTE(i915_displayport_test_type);
static void wm_latency_show(struct seq_file *m, const u16 wm[8])
{
struct drm_i915_private *dev_priv = m->private;
- struct drm_device *dev = &dev_priv->drm;
int level;
int num_levels;
@@ -1349,7 +1297,7 @@ static void wm_latency_show(struct seq_file *m, const u16 wm[8])
else
num_levels = ilk_wm_max_level(dev_priv) + 1;
- drm_modeset_lock_all(dev);
+ drm_modeset_lock_all(&dev_priv->drm);
for (level = 0; level < num_levels; level++) {
unsigned int latency = wm[level];
@@ -1370,7 +1318,7 @@ static void wm_latency_show(struct seq_file *m, const u16 wm[8])
level, wm[level], latency / 10, latency % 10);
}
- drm_modeset_unlock_all(dev);
+ drm_modeset_unlock_all(&dev_priv->drm);
}
static int pri_wm_latency_show(struct seq_file *m, void *data)
@@ -1453,7 +1401,6 @@ static ssize_t wm_latency_write(struct file *file, const char __user *ubuf,
{
struct seq_file *m = file->private_data;
struct drm_i915_private *dev_priv = m->private;
- struct drm_device *dev = &dev_priv->drm;
u16 new[8] = { 0 };
int num_levels;
int level;
@@ -1483,12 +1430,12 @@ static ssize_t wm_latency_write(struct file *file, const char __user *ubuf,
if (ret != num_levels)
return -EINVAL;
- drm_modeset_lock_all(dev);
+ drm_modeset_lock_all(&dev_priv->drm);
for (level = 0; level < num_levels; level++)
wm[level] = new[level];
- drm_modeset_unlock_all(dev);
+ drm_modeset_unlock_all(&dev_priv->drm);
return len;
}
@@ -1566,209 +1513,6 @@ static const struct file_operations i915_cur_wm_latency_fops = {
.write = cur_wm_latency_write
};
-static int i915_hpd_storm_ctl_show(struct seq_file *m, void *data)
-{
- struct drm_i915_private *dev_priv = m->private;
- struct intel_hotplug *hotplug = &dev_priv->display.hotplug;
-
- /* Synchronize with everything first in case there's been an HPD
- * storm, but we haven't finished handling it in the kernel yet
- */
- intel_synchronize_irq(dev_priv);
- flush_work(&dev_priv->display.hotplug.dig_port_work);
- flush_delayed_work(&dev_priv->display.hotplug.hotplug_work);
-
- seq_printf(m, "Threshold: %d\n", hotplug->hpd_storm_threshold);
- seq_printf(m, "Detected: %s\n",
- str_yes_no(delayed_work_pending(&hotplug->reenable_work)));
-
- return 0;
-}
-
-static ssize_t i915_hpd_storm_ctl_write(struct file *file,
- const char __user *ubuf, size_t len,
- loff_t *offp)
-{
- struct seq_file *m = file->private_data;
- struct drm_i915_private *dev_priv = m->private;
- struct intel_hotplug *hotplug = &dev_priv->display.hotplug;
- unsigned int new_threshold;
- int i;
- char *newline;
- char tmp[16];
-
- if (len >= sizeof(tmp))
- return -EINVAL;
-
- if (copy_from_user(tmp, ubuf, len))
- return -EFAULT;
-
- tmp[len] = '\0';
-
- /* Strip newline, if any */
- newline = strchr(tmp, '\n');
- if (newline)
- *newline = '\0';
-
- if (strcmp(tmp, "reset") == 0)
- new_threshold = HPD_STORM_DEFAULT_THRESHOLD;
- else if (kstrtouint(tmp, 10, &new_threshold) != 0)
- return -EINVAL;
-
- if (new_threshold > 0)
- drm_dbg_kms(&dev_priv->drm,
- "Setting HPD storm detection threshold to %d\n",
- new_threshold);
- else
- drm_dbg_kms(&dev_priv->drm, "Disabling HPD storm detection\n");
-
- spin_lock_irq(&dev_priv->irq_lock);
- hotplug->hpd_storm_threshold = new_threshold;
- /* Reset the HPD storm stats so we don't accidentally trigger a storm */
- for_each_hpd_pin(i)
- hotplug->stats[i].count = 0;
- spin_unlock_irq(&dev_priv->irq_lock);
-
- /* Re-enable hpd immediately if we were in an irq storm */
- flush_delayed_work(&dev_priv->display.hotplug.reenable_work);
-
- return len;
-}
-
-static int i915_hpd_storm_ctl_open(struct inode *inode, struct file *file)
-{
- return single_open(file, i915_hpd_storm_ctl_show, inode->i_private);
-}
-
-static const struct file_operations i915_hpd_storm_ctl_fops = {
- .owner = THIS_MODULE,
- .open = i915_hpd_storm_ctl_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
- .write = i915_hpd_storm_ctl_write
-};
-
-static int i915_hpd_short_storm_ctl_show(struct seq_file *m, void *data)
-{
- struct drm_i915_private *dev_priv = m->private;
-
- seq_printf(m, "Enabled: %s\n",
- str_yes_no(dev_priv->display.hotplug.hpd_short_storm_enabled));
-
- return 0;
-}
-
-static int
-i915_hpd_short_storm_ctl_open(struct inode *inode, struct file *file)
-{
- return single_open(file, i915_hpd_short_storm_ctl_show,
- inode->i_private);
-}
-
-static ssize_t i915_hpd_short_storm_ctl_write(struct file *file,
- const char __user *ubuf,
- size_t len, loff_t *offp)
-{
- struct seq_file *m = file->private_data;
- struct drm_i915_private *dev_priv = m->private;
- struct intel_hotplug *hotplug = &dev_priv->display.hotplug;
- char *newline;
- char tmp[16];
- int i;
- bool new_state;
-
- if (len >= sizeof(tmp))
- return -EINVAL;
-
- if (copy_from_user(tmp, ubuf, len))
- return -EFAULT;
-
- tmp[len] = '\0';
-
- /* Strip newline, if any */
- newline = strchr(tmp, '\n');
- if (newline)
- *newline = '\0';
-
- /* Reset to the "default" state for this system */
- if (strcmp(tmp, "reset") == 0)
- new_state = !HAS_DP_MST(dev_priv);
- else if (kstrtobool(tmp, &new_state) != 0)
- return -EINVAL;
-
- drm_dbg_kms(&dev_priv->drm, "%sabling HPD short storm detection\n",
- new_state ? "En" : "Dis");
-
- spin_lock_irq(&dev_priv->irq_lock);
- hotplug->hpd_short_storm_enabled = new_state;
- /* Reset the HPD storm stats so we don't accidentally trigger a storm */
- for_each_hpd_pin(i)
- hotplug->stats[i].count = 0;
- spin_unlock_irq(&dev_priv->irq_lock);
-
- /* Re-enable hpd immediately if we were in an irq storm */
- flush_delayed_work(&dev_priv->display.hotplug.reenable_work);
-
- return len;
-}
-
-static const struct file_operations i915_hpd_short_storm_ctl_fops = {
- .owner = THIS_MODULE,
- .open = i915_hpd_short_storm_ctl_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
- .write = i915_hpd_short_storm_ctl_write,
-};
-
-static int i915_drrs_ctl_set(void *data, u64 val)
-{
- struct drm_i915_private *dev_priv = data;
- struct drm_device *dev = &dev_priv->drm;
- struct intel_crtc *crtc;
-
- for_each_intel_crtc(dev, crtc) {
- struct intel_crtc_state *crtc_state;
- struct drm_crtc_commit *commit;
- int ret;
-
- ret = drm_modeset_lock_single_interruptible(&crtc->base.mutex);
- if (ret)
- return ret;
-
- crtc_state = to_intel_crtc_state(crtc->base.state);
-
- if (!crtc_state->hw.active ||
- !crtc_state->has_drrs)
- goto out;
-
- commit = crtc_state->uapi.commit;
- if (commit) {
- ret = wait_for_completion_interruptible(&commit->hw_done);
- if (ret)
- goto out;
- }
-
- drm_dbg(&dev_priv->drm,
- "Manually %sactivating DRRS\n", val ? "" : "de");
-
- if (val)
- intel_drrs_activate(crtc_state);
- else
- intel_drrs_deactivate(crtc_state);
-
-out:
- drm_modeset_unlock(&crtc->base.mutex);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-
-DEFINE_SIMPLE_ATTRIBUTE(i915_drrs_ctl_fops, NULL, i915_drrs_ctl_set, "%llu\n");
-
static ssize_t
i915_fifo_underrun_reset_write(struct file *filp,
const char __user *ubuf,
@@ -1776,7 +1520,6 @@ i915_fifo_underrun_reset_write(struct file *filp,
{
struct drm_i915_private *dev_priv = filp->private_data;
struct intel_crtc *crtc;
- struct drm_device *dev = &dev_priv->drm;
int ret;
bool reset;
@@ -1787,7 +1530,7 @@ i915_fifo_underrun_reset_write(struct file *filp,
if (!reset)
return cnt;
- for_each_intel_crtc(dev, crtc) {
+ for_each_intel_crtc(&dev_priv->drm, crtc) {
struct drm_crtc_commit *commit;
struct intel_crtc_state *crtc_state;
@@ -1842,7 +1585,6 @@ static const struct drm_info_list intel_display_debugfs_list[] = {
{"i915_shared_dplls_info", i915_shared_dplls_info, 0},
{"i915_dp_mst_info", i915_dp_mst_info, 0},
{"i915_ddb_info", i915_ddb_info, 0},
- {"i915_drrs_status", i915_drrs_status, 0},
{"i915_lpsp_status", i915_lpsp_status, 0},
};
@@ -1857,9 +1599,6 @@ static const struct {
{"i915_dp_test_data", &i915_displayport_test_data_fops},
{"i915_dp_test_type", &i915_displayport_test_type_fops},
{"i915_dp_test_active", &i915_displayport_test_active_fops},
- {"i915_hpd_storm_ctl", &i915_hpd_storm_ctl_fops},
- {"i915_hpd_short_storm_ctl", &i915_hpd_short_storm_ctl_fops},
- {"i915_drrs_ctl", &i915_drrs_ctl_fops},
{"i915_edp_psr_debug", &i915_edp_psr_debug_fops},
};
@@ -1882,6 +1621,7 @@ void intel_display_debugfs_register(struct drm_i915_private *i915)
intel_dmc_debugfs_register(i915);
intel_fbc_debugfs_register(i915);
+ intel_hpd_debugfs_register(i915);
skl_watermark_ipc_debugfs_register(i915);
}
@@ -2195,6 +1935,8 @@ void intel_connector_debugfs_add(struct intel_connector *intel_connector)
if (!root)
return;
+ intel_drrs_connector_debugfs_add(intel_connector);
+
if (connector->connector_type == DRM_MODE_CONNECTOR_eDP) {
debugfs_create_file("i915_panel_timings", S_IRUGO, root,
connector, &i915_panel_fops);
@@ -2247,6 +1989,7 @@ void intel_crtc_debugfs_add(struct drm_crtc *crtc)
return;
crtc_updates_add(crtc);
+ intel_drrs_crtc_debugfs_add(to_intel_crtc(crtc));
intel_fbc_crtc_debugfs_add(to_intel_crtc(crtc));
debugfs_create_file("i915_current_bpc", 0444, crtc->debugfs_entry, crtc,
diff --git a/drivers/gpu/drm/i915/display/intel_display_power.c b/drivers/gpu/drm/i915/display/intel_display_power.c
index 1a63da2..3adba64 100644
--- a/drivers/gpu/drm/i915/display/intel_display_power.c
+++ b/drivers/gpu/drm/i915/display/intel_display_power.c
@@ -129,6 +129,18 @@ intel_display_power_domain_str(enum intel_display_power_domain domain)
return "AUDIO_MMIO";
case POWER_DOMAIN_AUDIO_PLAYBACK:
return "AUDIO_PLAYBACK";
+ case POWER_DOMAIN_AUX_IO_A:
+ return "AUX_IO_A";
+ case POWER_DOMAIN_AUX_IO_B:
+ return "AUX_IO_B";
+ case POWER_DOMAIN_AUX_IO_C:
+ return "AUX_IO_C";
+ case POWER_DOMAIN_AUX_IO_D:
+ return "AUX_IO_D";
+ case POWER_DOMAIN_AUX_IO_E:
+ return "AUX_IO_E";
+ case POWER_DOMAIN_AUX_IO_F:
+ return "AUX_IO_F";
case POWER_DOMAIN_AUX_A:
return "AUX_A";
case POWER_DOMAIN_AUX_B:
@@ -153,8 +165,6 @@ intel_display_power_domain_str(enum intel_display_power_domain domain)
return "AUX_USBC5";
case POWER_DOMAIN_AUX_USBC6:
return "AUX_USBC6";
- case POWER_DOMAIN_AUX_IO_A:
- return "AUX_IO_A";
case POWER_DOMAIN_AUX_TBT1:
return "AUX_TBT1";
case POWER_DOMAIN_AUX_TBT2:
@@ -1148,10 +1158,9 @@ static void hsw_assert_cdclk(struct drm_i915_private *dev_priv)
static void assert_can_disable_lcpll(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_crtc *crtc;
- for_each_intel_crtc(dev, crtc)
+ for_each_intel_crtc(&dev_priv->drm, crtc)
I915_STATE_WARN(crtc->active, "CRTC for pipe %c enabled\n",
pipe_name(crtc->pipe));
@@ -2290,6 +2299,7 @@ struct intel_ddi_port_domains {
enum intel_display_power_domain ddi_lanes;
enum intel_display_power_domain ddi_io;
+ enum intel_display_power_domain aux_io;
enum intel_display_power_domain aux_legacy_usbc;
enum intel_display_power_domain aux_tbt;
};
@@ -2304,6 +2314,7 @@ i9xx_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_A,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_A,
+ .aux_io = POWER_DOMAIN_AUX_IO_A,
.aux_legacy_usbc = POWER_DOMAIN_AUX_A,
.aux_tbt = POWER_DOMAIN_INVALID,
},
@@ -2319,6 +2330,7 @@ d11_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_A,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_A,
+ .aux_io = POWER_DOMAIN_AUX_IO_A,
.aux_legacy_usbc = POWER_DOMAIN_AUX_A,
.aux_tbt = POWER_DOMAIN_INVALID,
}, {
@@ -2329,6 +2341,7 @@ d11_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_C,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_C,
+ .aux_io = POWER_DOMAIN_AUX_IO_C,
.aux_legacy_usbc = POWER_DOMAIN_AUX_C,
.aux_tbt = POWER_DOMAIN_AUX_TBT1,
},
@@ -2344,6 +2357,7 @@ d12_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_A,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_A,
+ .aux_io = POWER_DOMAIN_AUX_IO_A,
.aux_legacy_usbc = POWER_DOMAIN_AUX_A,
.aux_tbt = POWER_DOMAIN_INVALID,
}, {
@@ -2354,6 +2368,7 @@ d12_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_TC1,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_TC1,
+ .aux_io = POWER_DOMAIN_INVALID,
.aux_legacy_usbc = POWER_DOMAIN_AUX_USBC1,
.aux_tbt = POWER_DOMAIN_AUX_TBT1,
},
@@ -2369,6 +2384,7 @@ d13_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_A,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_A,
+ .aux_io = POWER_DOMAIN_AUX_IO_A,
.aux_legacy_usbc = POWER_DOMAIN_AUX_A,
.aux_tbt = POWER_DOMAIN_INVALID,
}, {
@@ -2379,6 +2395,7 @@ d13_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_TC1,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_TC1,
+ .aux_io = POWER_DOMAIN_INVALID,
.aux_legacy_usbc = POWER_DOMAIN_AUX_USBC1,
.aux_tbt = POWER_DOMAIN_AUX_TBT1,
}, {
@@ -2389,6 +2406,7 @@ d13_port_domains[] = {
.ddi_lanes = POWER_DOMAIN_PORT_DDI_LANES_D,
.ddi_io = POWER_DOMAIN_PORT_DDI_IO_D,
+ .aux_io = POWER_DOMAIN_AUX_IO_D,
.aux_legacy_usbc = POWER_DOMAIN_AUX_D,
.aux_tbt = POWER_DOMAIN_INVALID,
},
@@ -2467,6 +2485,17 @@ intel_port_domains_for_aux_ch(struct drm_i915_private *i915, enum aux_ch aux_ch)
}
enum intel_display_power_domain
+intel_display_power_aux_io_domain(struct drm_i915_private *i915, enum aux_ch aux_ch)
+{
+ const struct intel_ddi_port_domains *domains = intel_port_domains_for_aux_ch(i915, aux_ch);
+
+ if (drm_WARN_ON(&i915->drm, !domains || domains->aux_io == POWER_DOMAIN_INVALID))
+ return POWER_DOMAIN_AUX_IO_A;
+
+ return domains->aux_io + (int)(aux_ch - domains->aux_ch_start);
+}
+
+enum intel_display_power_domain
intel_display_power_legacy_aux_domain(struct drm_i915_private *i915, enum aux_ch aux_ch)
{
const struct intel_ddi_port_domains *domains = intel_port_domains_for_aux_ch(i915, aux_ch);
diff --git a/drivers/gpu/drm/i915/display/intel_display_power.h b/drivers/gpu/drm/i915/display/intel_display_power.h
index 7136ea3..2154d90 100644
--- a/drivers/gpu/drm/i915/display/intel_display_power.h
+++ b/drivers/gpu/drm/i915/display/intel_display_power.h
@@ -6,11 +6,12 @@
#ifndef __INTEL_DISPLAY_POWER_H__
#define __INTEL_DISPLAY_POWER_H__
-#include "intel_runtime_pm.h"
+#include "intel_wakeref.h"
enum aux_ch;
enum dpio_channel;
enum dpio_phy;
+enum i915_drm_suspend_mode;
enum port;
struct drm_i915_private;
struct i915_power_well;
@@ -76,6 +77,14 @@ enum intel_display_power_domain {
POWER_DOMAIN_VGA,
POWER_DOMAIN_AUDIO_MMIO,
POWER_DOMAIN_AUDIO_PLAYBACK,
+
+ POWER_DOMAIN_AUX_IO_A,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_IO_C,
+ POWER_DOMAIN_AUX_IO_D,
+ POWER_DOMAIN_AUX_IO_E,
+ POWER_DOMAIN_AUX_IO_F,
+
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
@@ -90,8 +99,6 @@ enum intel_display_power_domain {
POWER_DOMAIN_AUX_USBC5,
POWER_DOMAIN_AUX_USBC6,
- POWER_DOMAIN_AUX_IO_A,
-
POWER_DOMAIN_AUX_TBT1,
POWER_DOMAIN_AUX_TBT2,
POWER_DOMAIN_AUX_TBT3,
@@ -249,6 +256,8 @@ intel_display_power_ddi_lanes_domain(struct drm_i915_private *i915, enum port po
enum intel_display_power_domain
intel_display_power_ddi_io_domain(struct drm_i915_private *i915, enum port port);
enum intel_display_power_domain
+intel_display_power_aux_io_domain(struct drm_i915_private *i915, enum aux_ch aux_ch);
+enum intel_display_power_domain
intel_display_power_legacy_aux_domain(struct drm_i915_private *i915, enum aux_ch aux_ch);
enum intel_display_power_domain
intel_display_power_tbt_aux_domain(struct drm_i915_private *i915, enum aux_ch aux_ch);
diff --git a/drivers/gpu/drm/i915/display/intel_display_power_map.c b/drivers/gpu/drm/i915/display/intel_display_power_map.c
index dc04afc..f5d66ca 100644
--- a/drivers/gpu/drm/i915/display/intel_display_power_map.c
+++ b/drivers/gpu/drm/i915/display/intel_display_power_map.c
@@ -170,6 +170,8 @@ I915_DECL_PW_DOMAINS(vlv_pwdoms_display,
POWER_DOMAIN_VGA,
POWER_DOMAIN_AUDIO_MMIO,
POWER_DOMAIN_AUDIO_PLAYBACK,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_IO_C,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_GMBUS,
@@ -179,6 +181,8 @@ I915_DECL_PW_DOMAINS(vlv_pwdoms_dpio_cmn_bc,
POWER_DOMAIN_PORT_DDI_LANES_B,
POWER_DOMAIN_PORT_DDI_LANES_C,
POWER_DOMAIN_PORT_CRT,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_IO_C,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_INIT);
@@ -186,6 +190,8 @@ I915_DECL_PW_DOMAINS(vlv_pwdoms_dpio_cmn_bc,
I915_DECL_PW_DOMAINS(vlv_pwdoms_dpio_tx_bc_lanes,
POWER_DOMAIN_PORT_DDI_LANES_B,
POWER_DOMAIN_PORT_DDI_LANES_C,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_IO_C,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_INIT);
@@ -243,6 +249,9 @@ I915_DECL_PW_DOMAINS(chv_pwdoms_display,
POWER_DOMAIN_VGA,
POWER_DOMAIN_AUDIO_MMIO,
POWER_DOMAIN_AUDIO_PLAYBACK,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_IO_C,
+ POWER_DOMAIN_AUX_IO_D,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_AUX_D,
@@ -252,12 +261,15 @@ I915_DECL_PW_DOMAINS(chv_pwdoms_display,
I915_DECL_PW_DOMAINS(chv_pwdoms_dpio_cmn_bc,
POWER_DOMAIN_PORT_DDI_LANES_B,
POWER_DOMAIN_PORT_DDI_LANES_C,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_IO_C,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(chv_pwdoms_dpio_cmn_d,
POWER_DOMAIN_PORT_DDI_LANES_D,
+ POWER_DOMAIN_AUX_IO_D,
POWER_DOMAIN_AUX_D,
POWER_DOMAIN_INIT);
@@ -305,6 +317,9 @@ static const struct i915_power_well_desc_list chv_power_wells[] = {
POWER_DOMAIN_VGA, \
POWER_DOMAIN_AUDIO_MMIO, \
POWER_DOMAIN_AUDIO_PLAYBACK, \
+ POWER_DOMAIN_AUX_IO_B, \
+ POWER_DOMAIN_AUX_IO_C, \
+ POWER_DOMAIN_AUX_IO_D, \
POWER_DOMAIN_AUX_B, \
POWER_DOMAIN_AUX_C, \
POWER_DOMAIN_AUX_D
@@ -318,6 +333,7 @@ I915_DECL_PW_DOMAINS(skl_pwdoms_dc_off,
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_MODESET,
POWER_DOMAIN_GT_IRQ,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(skl_pwdoms_ddi_io_a_e,
@@ -407,6 +423,8 @@ static const struct i915_power_well_desc_list skl_power_wells[] = {
POWER_DOMAIN_VGA, \
POWER_DOMAIN_AUDIO_MMIO, \
POWER_DOMAIN_AUDIO_PLAYBACK, \
+ POWER_DOMAIN_AUX_IO_B, \
+ POWER_DOMAIN_AUX_IO_C, \
POWER_DOMAIN_AUX_B, \
POWER_DOMAIN_AUX_C
@@ -420,16 +438,20 @@ I915_DECL_PW_DOMAINS(bxt_pwdoms_dc_off,
POWER_DOMAIN_GMBUS,
POWER_DOMAIN_MODESET,
POWER_DOMAIN_GT_IRQ,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(bxt_pwdoms_dpio_cmn_a,
POWER_DOMAIN_PORT_DDI_LANES_A,
+ POWER_DOMAIN_AUX_IO_A,
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(bxt_pwdoms_dpio_cmn_bc,
POWER_DOMAIN_PORT_DDI_LANES_B,
POWER_DOMAIN_PORT_DDI_LANES_C,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_IO_C,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_INIT);
@@ -483,6 +505,8 @@ static const struct i915_power_well_desc_list bxt_power_wells[] = {
POWER_DOMAIN_VGA, \
POWER_DOMAIN_AUDIO_MMIO, \
POWER_DOMAIN_AUDIO_PLAYBACK, \
+ POWER_DOMAIN_AUX_IO_B, \
+ POWER_DOMAIN_AUX_IO_C, \
POWER_DOMAIN_AUX_B, \
POWER_DOMAIN_AUX_C
@@ -496,6 +520,7 @@ I915_DECL_PW_DOMAINS(glk_pwdoms_dc_off,
POWER_DOMAIN_GMBUS,
POWER_DOMAIN_MODESET,
POWER_DOMAIN_GT_IRQ,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(glk_pwdoms_ddi_io_a, POWER_DOMAIN_PORT_DDI_IO_A);
@@ -504,29 +529,34 @@ I915_DECL_PW_DOMAINS(glk_pwdoms_ddi_io_c, POWER_DOMAIN_PORT_DDI_IO_C);
I915_DECL_PW_DOMAINS(glk_pwdoms_dpio_cmn_a,
POWER_DOMAIN_PORT_DDI_LANES_A,
+ POWER_DOMAIN_AUX_IO_A,
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(glk_pwdoms_dpio_cmn_b,
POWER_DOMAIN_PORT_DDI_LANES_B,
+ POWER_DOMAIN_AUX_IO_B,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(glk_pwdoms_dpio_cmn_c,
POWER_DOMAIN_PORT_DDI_LANES_C,
+ POWER_DOMAIN_AUX_IO_C,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(glk_pwdoms_aux_a,
- POWER_DOMAIN_AUX_A,
POWER_DOMAIN_AUX_IO_A,
+ POWER_DOMAIN_AUX_A,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(glk_pwdoms_aux_b,
+ POWER_DOMAIN_AUX_IO_B,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(glk_pwdoms_aux_c,
+ POWER_DOMAIN_AUX_IO_C,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_INIT);
@@ -617,6 +647,11 @@ I915_DECL_PW_DOMAINS(icl_pwdoms_pw_4,
POWER_DOMAIN_VGA, \
POWER_DOMAIN_AUDIO_MMIO, \
POWER_DOMAIN_AUDIO_PLAYBACK, \
+ POWER_DOMAIN_AUX_IO_B, \
+ POWER_DOMAIN_AUX_IO_C, \
+ POWER_DOMAIN_AUX_IO_D, \
+ POWER_DOMAIN_AUX_IO_E, \
+ POWER_DOMAIN_AUX_IO_F, \
POWER_DOMAIN_AUX_B, \
POWER_DOMAIN_AUX_C, \
POWER_DOMAIN_AUX_D, \
@@ -658,13 +693,23 @@ I915_DECL_PW_DOMAINS(icl_pwdoms_ddi_io_e, POWER_DOMAIN_PORT_DDI_IO_E);
I915_DECL_PW_DOMAINS(icl_pwdoms_ddi_io_f, POWER_DOMAIN_PORT_DDI_IO_F);
I915_DECL_PW_DOMAINS(icl_pwdoms_aux_a,
- POWER_DOMAIN_AUX_A,
- POWER_DOMAIN_AUX_IO_A);
-I915_DECL_PW_DOMAINS(icl_pwdoms_aux_b, POWER_DOMAIN_AUX_B);
-I915_DECL_PW_DOMAINS(icl_pwdoms_aux_c, POWER_DOMAIN_AUX_C);
-I915_DECL_PW_DOMAINS(icl_pwdoms_aux_d, POWER_DOMAIN_AUX_D);
-I915_DECL_PW_DOMAINS(icl_pwdoms_aux_e, POWER_DOMAIN_AUX_E);
-I915_DECL_PW_DOMAINS(icl_pwdoms_aux_f, POWER_DOMAIN_AUX_F);
+ POWER_DOMAIN_AUX_IO_A,
+ POWER_DOMAIN_AUX_A);
+I915_DECL_PW_DOMAINS(icl_pwdoms_aux_b,
+ POWER_DOMAIN_AUX_IO_B,
+ POWER_DOMAIN_AUX_B);
+I915_DECL_PW_DOMAINS(icl_pwdoms_aux_c,
+ POWER_DOMAIN_AUX_IO_C,
+ POWER_DOMAIN_AUX_C);
+I915_DECL_PW_DOMAINS(icl_pwdoms_aux_d,
+ POWER_DOMAIN_AUX_IO_D,
+ POWER_DOMAIN_AUX_D);
+I915_DECL_PW_DOMAINS(icl_pwdoms_aux_e,
+ POWER_DOMAIN_AUX_IO_E,
+ POWER_DOMAIN_AUX_E);
+I915_DECL_PW_DOMAINS(icl_pwdoms_aux_f,
+ POWER_DOMAIN_AUX_IO_F,
+ POWER_DOMAIN_AUX_F);
I915_DECL_PW_DOMAINS(icl_pwdoms_aux_tbt1, POWER_DOMAIN_AUX_TBT1);
I915_DECL_PW_DOMAINS(icl_pwdoms_aux_tbt2, POWER_DOMAIN_AUX_TBT2);
I915_DECL_PW_DOMAINS(icl_pwdoms_aux_tbt3, POWER_DOMAIN_AUX_TBT3);
@@ -816,6 +861,7 @@ I915_DECL_PW_DOMAINS(tgl_pwdoms_dc_off,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_AUX_C,
POWER_DOMAIN_MODESET,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(tgl_pwdoms_ddi_io_tc1, POWER_DOMAIN_PORT_DDI_IO_TC1);
@@ -1012,6 +1058,7 @@ I915_DECL_PW_DOMAINS(rkl_pwdoms_dc_off,
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_MODESET,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
static const struct i915_power_well_desc rkl_power_wells_main[] = {
@@ -1094,6 +1141,7 @@ I915_DECL_PW_DOMAINS(dg1_pwdoms_dc_off,
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_MODESET,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(dg1_pwdoms_pw_2,
@@ -1215,6 +1263,9 @@ I915_DECL_PW_DOMAINS(xelpd_pwdoms_pw_a,
POWER_DOMAIN_PORT_DDI_LANES_TC4, \
POWER_DOMAIN_VGA, \
POWER_DOMAIN_AUDIO_PLAYBACK, \
+ POWER_DOMAIN_AUX_IO_C, \
+ POWER_DOMAIN_AUX_IO_D, \
+ POWER_DOMAIN_AUX_IO_E, \
POWER_DOMAIN_AUX_C, \
POWER_DOMAIN_AUX_D, \
POWER_DOMAIN_AUX_E, \
@@ -1255,6 +1306,7 @@ I915_DECL_PW_DOMAINS(xelpd_pwdoms_dc_off,
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_AUX_B,
POWER_DOMAIN_MODESET,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
static const struct i915_power_well_desc xelpd_power_wells_main[] = {
@@ -1376,6 +1428,7 @@ I915_DECL_PW_DOMAINS(xelpdp_pwdoms_dc_off,
POWER_DOMAIN_MODESET,
POWER_DOMAIN_AUX_A,
POWER_DOMAIN_AUX_B,
+ POWER_DOMAIN_DC_OFF,
POWER_DOMAIN_INIT);
I915_DECL_PW_DOMAINS(xelpdp_pwdoms_aux_tc1,
diff --git a/drivers/gpu/drm/i915/display/intel_display_power_well.c b/drivers/gpu/drm/i915/display/intel_display_power_well.c
index 1d18eee..8710dd4 100644
--- a/drivers/gpu/drm/i915/display/intel_display_power_well.c
+++ b/drivers/gpu/drm/i915/display/intel_display_power_well.c
@@ -13,6 +13,7 @@
#include "intel_display_power_well.h"
#include "intel_display_types.h"
#include "intel_dkl_phy.h"
+#include "intel_dkl_phy_regs.h"
#include "intel_dmc.h"
#include "intel_dpio_phy.h"
#include "intel_dpll.h"
@@ -531,7 +532,7 @@ icl_tc_phy_aux_power_well_enable(struct drm_i915_private *dev_priv,
tc_port = TGL_AUX_PW_TO_TC_PORT(i915_power_well_instance(power_well)->hsw.idx);
- if (wait_for(intel_dkl_phy_read(dev_priv, DKL_CMN_UC_DW_27(tc_port), 2) &
+ if (wait_for(intel_dkl_phy_read(dev_priv, DKL_CMN_UC_DW_27(tc_port)) &
DKL_CMN_UC_DW27_UC_HEALTH, 1))
drm_warn(&dev_priv->drm,
"Timeout waiting TC uC health\n");
diff --git a/drivers/gpu/drm/i915/display/intel_display_power_well.h b/drivers/gpu/drm/i915/display/intel_display_power_well.h
index e13b521..ba7cb97 100644
--- a/drivers/gpu/drm/i915/display/intel_display_power_well.h
+++ b/drivers/gpu/drm/i915/display/intel_display_power_well.h
@@ -7,8 +7,8 @@
#include <linux/types.h>
-#include "intel_display.h"
#include "intel_display_power.h"
+#include "intel_dpio_phy.h"
struct drm_i915_private;
struct i915_power_well;
diff --git a/drivers/gpu/drm/i915/display/intel_display_reg_defs.h b/drivers/gpu/drm/i915/display/intel_display_reg_defs.h
new file mode 100644
index 0000000..0260541
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_display_reg_defs.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#ifndef __INTEL_DISPLAY_REG_DEFS_H__
+#define __INTEL_DISPLAY_REG_DEFS_H__
+
+#include "i915_reg_defs.h"
+
+#define DISPLAY_MMIO_BASE(dev_priv) (INTEL_INFO(dev_priv)->display.mmio_offset)
+
+#define VLV_DISPLAY_BASE 0x180000
+
+/*
+ * Named helper wrappers around _PICK_EVEN() and _PICK().
+ */
+#define _PIPE(pipe, a, b) _PICK_EVEN(pipe, a, b)
+#define _PLANE(plane, a, b) _PICK_EVEN(plane, a, b)
+#define _TRANS(tran, a, b) _PICK_EVEN(tran, a, b)
+#define _PORT(port, a, b) _PICK_EVEN(port, a, b)
+#define _PLL(pll, a, b) _PICK_EVEN(pll, a, b)
+#define _PHY(phy, a, b) _PICK_EVEN(phy, a, b)
+
+#define _MMIO_PIPE(pipe, a, b) _MMIO(_PIPE(pipe, a, b))
+#define _MMIO_PLANE(plane, a, b) _MMIO(_PLANE(plane, a, b))
+#define _MMIO_TRANS(tran, a, b) _MMIO(_TRANS(tran, a, b))
+#define _MMIO_PORT(port, a, b) _MMIO(_PORT(port, a, b))
+#define _MMIO_PLL(pll, a, b) _MMIO(_PLL(pll, a, b))
+#define _MMIO_PHY(phy, a, b) _MMIO(_PHY(phy, a, b))
+
+#define _PHY3(phy, ...) _PICK(phy, __VA_ARGS__)
+
+#define _MMIO_PIPE3(pipe, a, b, c) _MMIO(_PICK(pipe, a, b, c))
+#define _MMIO_PORT3(pipe, a, b, c) _MMIO(_PICK(pipe, a, b, c))
+#define _MMIO_PHY3(phy, a, b, c) _MMIO(_PHY3(phy, a, b, c))
+#define _MMIO_PLL3(pll, ...) _MMIO(_PICK(pll, __VA_ARGS__))
+
+/*
+ * Device info offset array based helpers for groups of registers with unevenly
+ * spaced base offsets.
+ */
+#define _MMIO_PIPE2(pipe, reg) _MMIO(INTEL_INFO(dev_priv)->display.pipe_offsets[(pipe)] - \
+ INTEL_INFO(dev_priv)->display.pipe_offsets[PIPE_A] + \
+ DISPLAY_MMIO_BASE(dev_priv) + (reg))
+#define _MMIO_TRANS2(tran, reg) _MMIO(INTEL_INFO(dev_priv)->display.trans_offsets[(tran)] - \
+ INTEL_INFO(dev_priv)->display.trans_offsets[TRANSCODER_A] + \
+ DISPLAY_MMIO_BASE(dev_priv) + (reg))
+#define _MMIO_CURSOR2(pipe, reg) _MMIO(INTEL_INFO(dev_priv)->display.cursor_offsets[(pipe)] - \
+ INTEL_INFO(dev_priv)->display.cursor_offsets[PIPE_A] + \
+ DISPLAY_MMIO_BASE(dev_priv) + (reg))
+
+#endif /* __INTEL_DISPLAY_REG_DEFS_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_display_trace.h b/drivers/gpu/drm/i915/display/intel_display_trace.h
index 2dd5a4b..725aba3f 100644
--- a/drivers/gpu/drm/i915/display/intel_display_trace.h
+++ b/drivers/gpu/drm/i915/display/intel_display_trace.h
@@ -18,11 +18,15 @@
#include "intel_crtc.h"
#include "intel_display_types.h"
+#define __dev_name_i915(i915) dev_name((i915)->drm.dev)
+#define __dev_name_kms(obj) dev_name((obj)->base.dev->dev)
+
TRACE_EVENT(intel_pipe_enable,
TP_PROTO(struct intel_crtc *crtc),
TP_ARGS(crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__array(u32, frame, 3)
__array(u32, scanline, 3)
__field(enum pipe, pipe)
@@ -30,6 +34,7 @@ TRACE_EVENT(intel_pipe_enable,
TP_fast_assign(
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
struct intel_crtc *it__;
+ __assign_str(dev, __dev_name_kms(crtc));
for_each_intel_crtc(&dev_priv->drm, it__) {
__entry->frame[it__->pipe] = intel_crtc_get_vblank_counter(it__);
__entry->scanline[it__->pipe] = intel_get_crtc_scanline(it__);
@@ -37,8 +42,8 @@ TRACE_EVENT(intel_pipe_enable,
__entry->pipe = crtc->pipe;
),
- TP_printk("pipe %c enable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
+ TP_printk("dev %s, pipe %c enable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
__entry->frame[PIPE_A], __entry->scanline[PIPE_A],
__entry->frame[PIPE_B], __entry->scanline[PIPE_B],
__entry->frame[PIPE_C], __entry->scanline[PIPE_C])
@@ -49,6 +54,7 @@ TRACE_EVENT(intel_pipe_disable,
TP_ARGS(crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__array(u32, frame, 3)
__array(u32, scanline, 3)
__field(enum pipe, pipe)
@@ -57,6 +63,7 @@ TRACE_EVENT(intel_pipe_disable,
TP_fast_assign(
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
struct intel_crtc *it__;
+ __assign_str(dev, __dev_name_kms(crtc));
for_each_intel_crtc(&dev_priv->drm, it__) {
__entry->frame[it__->pipe] = intel_crtc_get_vblank_counter(it__);
__entry->scanline[it__->pipe] = intel_get_crtc_scanline(it__);
@@ -64,8 +71,8 @@ TRACE_EVENT(intel_pipe_disable,
__entry->pipe = crtc->pipe;
),
- TP_printk("pipe %c disable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
+ TP_printk("dev %s, pipe %c disable, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
__entry->frame[PIPE_A], __entry->scanline[PIPE_A],
__entry->frame[PIPE_B], __entry->scanline[PIPE_B],
__entry->frame[PIPE_C], __entry->scanline[PIPE_C])
@@ -76,6 +83,7 @@ TRACE_EVENT(intel_pipe_crc,
TP_ARGS(crtc, crcs),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -83,16 +91,19 @@ TRACE_EVENT(intel_pipe_crc,
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
memcpy(__entry->crcs, crcs, sizeof(__entry->crcs));
),
- TP_printk("pipe %c, frame=%u, scanline=%u crc=%08x %08x %08x %08x %08x",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline,
- __entry->crcs[0], __entry->crcs[1], __entry->crcs[2],
- __entry->crcs[3], __entry->crcs[4])
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u crc=%08x %08x %08x %08x %08x",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline,
+ __entry->crcs[0], __entry->crcs[1],
+ __entry->crcs[2], __entry->crcs[3],
+ __entry->crcs[4])
);
TRACE_EVENT(intel_cpu_fifo_underrun,
@@ -100,6 +111,7 @@ TRACE_EVENT(intel_cpu_fifo_underrun,
TP_ARGS(dev_priv, pipe),
TP_STRUCT__entry(
+ __string(dev, __dev_name_i915(dev_priv))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -107,13 +119,14 @@ TRACE_EVENT(intel_cpu_fifo_underrun,
TP_fast_assign(
struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
__entry->frame, __entry->scanline)
);
@@ -122,6 +135,7 @@ TRACE_EVENT(intel_pch_fifo_underrun,
TP_ARGS(dev_priv, pch_transcoder),
TP_STRUCT__entry(
+ __string(dev, __dev_name_i915(dev_priv))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -130,13 +144,14 @@ TRACE_EVENT(intel_pch_fifo_underrun,
TP_fast_assign(
enum pipe pipe = pch_transcoder;
struct intel_crtc *crtc = intel_crtc_for_pipe(dev_priv, pipe);
+ __assign_str(dev, __dev_name_i915(dev_priv));
__entry->pipe = pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pch transcoder %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe),
+ TP_printk("dev %s, pch transcoder %c, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
__entry->frame, __entry->scanline)
);
@@ -145,6 +160,7 @@ TRACE_EVENT(intel_memory_cxsr,
TP_ARGS(dev_priv, old, new),
TP_STRUCT__entry(
+ __string(dev, __dev_name_i915(dev_priv))
__array(u32, frame, 3)
__array(u32, scanline, 3)
__field(bool, old)
@@ -153,6 +169,7 @@ TRACE_EVENT(intel_memory_cxsr,
TP_fast_assign(
struct intel_crtc *crtc;
+ __assign_str(dev, __dev_name_i915(dev_priv));
for_each_intel_crtc(&dev_priv->drm, crtc) {
__entry->frame[crtc->pipe] = intel_crtc_get_vblank_counter(crtc);
__entry->scanline[crtc->pipe] = intel_get_crtc_scanline(crtc);
@@ -161,8 +178,8 @@ TRACE_EVENT(intel_memory_cxsr,
__entry->new = new;
),
- TP_printk("%s->%s, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
- str_on_off(__entry->old), str_on_off(__entry->new),
+ TP_printk("dev %s, cxsr %s->%s, pipe A: frame=%u, scanline=%u, pipe B: frame=%u, scanline=%u, pipe C: frame=%u, scanline=%u",
+ __get_str(dev), str_on_off(__entry->old), str_on_off(__entry->new),
__entry->frame[PIPE_A], __entry->scanline[PIPE_A],
__entry->frame[PIPE_B], __entry->scanline[PIPE_B],
__entry->frame[PIPE_C], __entry->scanline[PIPE_C])
@@ -173,6 +190,7 @@ TRACE_EVENT(g4x_wm,
TP_ARGS(crtc, wm),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -191,6 +209,7 @@ TRACE_EVENT(g4x_wm,
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
@@ -208,8 +227,9 @@ TRACE_EVENT(g4x_wm,
__entry->fbc = wm->fbc_en;
),
- TP_printk("pipe %c, frame=%u, scanline=%u, wm %d/%d/%d, sr %s/%d/%d/%d, hpll %s/%d/%d/%d, fbc %s",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline,
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u, wm %d/%d/%d, sr %s/%d/%d/%d, hpll %s/%d/%d/%d, fbc %s",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline,
__entry->primary, __entry->sprite, __entry->cursor,
str_yes_no(__entry->cxsr), __entry->sr_plane, __entry->sr_cursor, __entry->sr_fbc,
str_yes_no(__entry->hpll), __entry->hpll_plane, __entry->hpll_cursor, __entry->hpll_fbc,
@@ -221,6 +241,7 @@ TRACE_EVENT(vlv_wm,
TP_ARGS(crtc, wm),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -235,6 +256,7 @@ TRACE_EVENT(vlv_wm,
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
@@ -248,9 +270,10 @@ TRACE_EVENT(vlv_wm,
__entry->sr_cursor = wm->sr.cursor;
),
- TP_printk("pipe %c, frame=%u, scanline=%u, level=%d, cxsr=%d, wm %d/%d/%d/%d, sr %d/%d",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->level, __entry->cxsr,
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u, level=%d, cxsr=%d, wm %d/%d/%d/%d, sr %d/%d",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline,
+ __entry->level, __entry->cxsr,
__entry->primary, __entry->sprite0, __entry->sprite1, __entry->cursor,
__entry->sr_plane, __entry->sr_cursor)
);
@@ -260,6 +283,7 @@ TRACE_EVENT(vlv_fifo_size,
TP_ARGS(crtc, sprite0_start, sprite1_start, fifo_size),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -269,6 +293,7 @@ TRACE_EVENT(vlv_fifo_size,
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
@@ -277,90 +302,96 @@ TRACE_EVENT(vlv_fifo_size,
__entry->fifo_size = fifo_size;
),
- TP_printk("pipe %c, frame=%u, scanline=%u, %d/%d/%d",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->sprite0_start,
- __entry->sprite1_start, __entry->fifo_size)
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u, %d/%d/%d",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline,
+ __entry->sprite0_start, __entry->sprite1_start, __entry->fifo_size)
);
TRACE_EVENT(intel_plane_update_noarm,
- TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_PROTO(struct intel_plane *plane, struct intel_crtc *crtc),
TP_ARGS(plane, crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(plane))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
__array(int, src, 4)
__array(int, dst, 4)
- __string(name, plane->name)
+ __string(name, plane->base.name)
),
TP_fast_assign(
- __assign_str(name, plane->name);
+ __assign_str(dev, __dev_name_kms(plane));
+ __assign_str(name, plane->base.name);
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
- memcpy(__entry->src, &plane->state->src, sizeof(__entry->src));
- memcpy(__entry->dst, &plane->state->dst, sizeof(__entry->dst));
+ memcpy(__entry->src, &plane->base.state->src, sizeof(__entry->src));
+ memcpy(__entry->dst, &plane->base.state->dst, sizeof(__entry->dst));
),
- TP_printk("pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
- pipe_name(__entry->pipe), __get_str(name),
+ TP_printk("dev %s, pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
+ __get_str(dev), pipe_name(__entry->pipe), __get_str(name),
__entry->frame, __entry->scanline,
DRM_RECT_FP_ARG((const struct drm_rect *)__entry->src),
DRM_RECT_ARG((const struct drm_rect *)__entry->dst))
);
TRACE_EVENT(intel_plane_update_arm,
- TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_PROTO(struct intel_plane *plane, struct intel_crtc *crtc),
TP_ARGS(plane, crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(plane))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
__array(int, src, 4)
__array(int, dst, 4)
- __string(name, plane->name)
+ __string(name, plane->base.name)
),
TP_fast_assign(
- __assign_str(name, plane->name);
+ __assign_str(dev, __dev_name_kms(plane));
+ __assign_str(name, plane->base.name);
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
- memcpy(__entry->src, &plane->state->src, sizeof(__entry->src));
- memcpy(__entry->dst, &plane->state->dst, sizeof(__entry->dst));
+ memcpy(__entry->src, &plane->base.state->src, sizeof(__entry->src));
+ memcpy(__entry->dst, &plane->base.state->dst, sizeof(__entry->dst));
),
- TP_printk("pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
- pipe_name(__entry->pipe), __get_str(name),
+ TP_printk("dev %s, pipe %c, plane %s, frame=%u, scanline=%u, " DRM_RECT_FP_FMT " -> " DRM_RECT_FMT,
+ __get_str(dev), pipe_name(__entry->pipe), __get_str(name),
__entry->frame, __entry->scanline,
DRM_RECT_FP_ARG((const struct drm_rect *)__entry->src),
DRM_RECT_ARG((const struct drm_rect *)__entry->dst))
);
TRACE_EVENT(intel_plane_disable_arm,
- TP_PROTO(struct drm_plane *plane, struct intel_crtc *crtc),
+ TP_PROTO(struct intel_plane *plane, struct intel_crtc *crtc),
TP_ARGS(plane, crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(plane))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
- __string(name, plane->name)
+ __string(name, plane->base.name)
),
TP_fast_assign(
- __assign_str(name, plane->name);
+ __assign_str(dev, __dev_name_kms(plane));
+ __assign_str(name, plane->base.name);
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pipe %c, plane %s, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __get_str(name),
+ TP_printk("dev %s, pipe %c, plane %s, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe), __get_str(name),
__entry->frame, __entry->scanline)
);
@@ -369,6 +400,8 @@ TRACE_EVENT(intel_fbc_activate,
TP_ARGS(plane),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(plane))
+ __string(name, plane->base.name)
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -377,13 +410,16 @@ TRACE_EVENT(intel_fbc_activate,
TP_fast_assign(
struct intel_crtc *crtc = intel_crtc_for_pipe(to_i915(plane->base.dev),
plane->pipe);
+ __assign_str(dev, __dev_name_kms(plane));
+ __assign_str(name, plane->base.name)
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
+ TP_printk("dev %s, pipe %c, plane %s, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe), __get_str(name),
+ __entry->frame, __entry->scanline)
);
TRACE_EVENT(intel_fbc_deactivate,
@@ -391,6 +427,8 @@ TRACE_EVENT(intel_fbc_deactivate,
TP_ARGS(plane),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(plane))
+ __string(name, plane->base.name)
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -399,13 +437,16 @@ TRACE_EVENT(intel_fbc_deactivate,
TP_fast_assign(
struct intel_crtc *crtc = intel_crtc_for_pipe(to_i915(plane->base.dev),
plane->pipe);
+ __assign_str(dev, __dev_name_kms(plane));
+ __assign_str(name, plane->base.name)
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
+ TP_printk("dev %s, pipe %c, plane %s, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe), __get_str(name),
+ __entry->frame, __entry->scanline)
);
TRACE_EVENT(intel_fbc_nuke,
@@ -413,6 +454,8 @@ TRACE_EVENT(intel_fbc_nuke,
TP_ARGS(plane),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(plane))
+ __string(name, plane->base.name)
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -421,13 +464,16 @@ TRACE_EVENT(intel_fbc_nuke,
TP_fast_assign(
struct intel_crtc *crtc = intel_crtc_for_pipe(to_i915(plane->base.dev),
plane->pipe);
+ __assign_str(dev, __dev_name_kms(plane));
+ __assign_str(name, plane->base.name)
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame, __entry->scanline)
+ TP_printk("dev %s, pipe %c, plane %s, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe), __get_str(name),
+ __entry->frame, __entry->scanline)
);
TRACE_EVENT(intel_crtc_vblank_work_start,
@@ -435,20 +481,22 @@ TRACE_EVENT(intel_crtc_vblank_work_start,
TP_ARGS(crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline)
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline)
);
TRACE_EVENT(intel_crtc_vblank_work_end,
@@ -456,20 +504,22 @@ TRACE_EVENT(intel_crtc_vblank_work_end,
TP_ARGS(crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
),
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline)
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline)
);
TRACE_EVENT(intel_pipe_update_start,
@@ -477,6 +527,7 @@ TRACE_EVENT(intel_pipe_update_start,
TP_ARGS(crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -485,6 +536,7 @@ TRACE_EVENT(intel_pipe_update_start,
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = intel_crtc_get_vblank_counter(crtc);
__entry->scanline = intel_get_crtc_scanline(crtc);
@@ -492,9 +544,10 @@ TRACE_EVENT(intel_pipe_update_start,
__entry->max = crtc->debug.max_vbl;
),
- TP_printk("pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->min, __entry->max)
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline,
+ __entry->min, __entry->max)
);
TRACE_EVENT(intel_pipe_update_vblank_evaded,
@@ -502,6 +555,7 @@ TRACE_EVENT(intel_pipe_update_vblank_evaded,
TP_ARGS(crtc),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
@@ -510,6 +564,7 @@ TRACE_EVENT(intel_pipe_update_vblank_evaded,
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = crtc->debug.start_vbl_count;
__entry->scanline = crtc->debug.scanline_start;
@@ -517,9 +572,10 @@ TRACE_EVENT(intel_pipe_update_vblank_evaded,
__entry->max = crtc->debug.max_vbl;
),
- TP_printk("pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline, __entry->min, __entry->max)
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u, min=%u, max=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline,
+ __entry->min, __entry->max)
);
TRACE_EVENT(intel_pipe_update_end,
@@ -527,56 +583,64 @@ TRACE_EVENT(intel_pipe_update_end,
TP_ARGS(crtc, frame, scanline_end),
TP_STRUCT__entry(
+ __string(dev, __dev_name_kms(crtc))
__field(enum pipe, pipe)
__field(u32, frame)
__field(u32, scanline)
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_kms(crtc));
__entry->pipe = crtc->pipe;
__entry->frame = frame;
__entry->scanline = scanline_end;
),
- TP_printk("pipe %c, frame=%u, scanline=%u",
- pipe_name(__entry->pipe), __entry->frame,
- __entry->scanline)
+ TP_printk("dev %s, pipe %c, frame=%u, scanline=%u",
+ __get_str(dev), pipe_name(__entry->pipe),
+ __entry->frame, __entry->scanline)
);
TRACE_EVENT(intel_frontbuffer_invalidate,
- TP_PROTO(unsigned int frontbuffer_bits, unsigned int origin),
- TP_ARGS(frontbuffer_bits, origin),
+ TP_PROTO(struct drm_i915_private *i915,
+ unsigned int frontbuffer_bits, unsigned int origin),
+ TP_ARGS(i915, frontbuffer_bits, origin),
TP_STRUCT__entry(
+ __string(dev, __dev_name_i915(i915))
__field(unsigned int, frontbuffer_bits)
__field(unsigned int, origin)
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_i915(i915));
__entry->frontbuffer_bits = frontbuffer_bits;
__entry->origin = origin;
),
- TP_printk("frontbuffer_bits=0x%08x, origin=%u",
- __entry->frontbuffer_bits, __entry->origin)
+ TP_printk("dev %s, frontbuffer_bits=0x%08x, origin=%u",
+ __get_str(dev), __entry->frontbuffer_bits, __entry->origin)
);
TRACE_EVENT(intel_frontbuffer_flush,
- TP_PROTO(unsigned int frontbuffer_bits, unsigned int origin),
- TP_ARGS(frontbuffer_bits, origin),
+ TP_PROTO(struct drm_i915_private *i915,
+ unsigned int frontbuffer_bits, unsigned int origin),
+ TP_ARGS(i915, frontbuffer_bits, origin),
TP_STRUCT__entry(
+ __string(dev, __dev_name_i915(i915))
__field(unsigned int, frontbuffer_bits)
__field(unsigned int, origin)
),
TP_fast_assign(
+ __assign_str(dev, __dev_name_i915(i915));
__entry->frontbuffer_bits = frontbuffer_bits;
__entry->origin = origin;
),
- TP_printk("frontbuffer_bits=0x%08x, origin=%u",
- __entry->frontbuffer_bits, __entry->origin)
+ TP_printk("dev %s, frontbuffer_bits=0x%08x, origin=%u",
+ __get_str(dev), __entry->frontbuffer_bits, __entry->origin)
);
#endif /* __INTEL_DISPLAY_TRACE_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_display_types.h b/drivers/gpu/drm/i915/display/intel_display_types.h
index 298d00a..f073950 100644
--- a/drivers/gpu/drm/i915/display/intel_display_types.h
+++ b/drivers/gpu/drm/i915/display/intel_display_types.h
@@ -969,6 +969,15 @@ struct intel_mpllb_state {
u32 mpllb_sscstep;
};
+/* Used by dp and fdi links */
+struct intel_link_m_n {
+ u32 tu;
+ u32 data_m;
+ u32 data_n;
+ u32 link_m;
+ u32 link_n;
+};
+
struct intel_crtc_state {
/*
* uapi (drm) state. This is the software state shown to userspace.
@@ -1001,11 +1010,15 @@ struct intel_crtc_state {
*/
struct {
bool active, enable;
+ /* logical state of LUTs */
struct drm_property_blob *degamma_lut, *gamma_lut, *ctm;
struct drm_display_mode mode, pipe_mode, adjusted_mode;
enum drm_scaling_filter scaling_filter;
} hw;
+ /* actual state of LUTs */
+ struct drm_property_blob *pre_csc_lut, *post_csc_lut;
+
/**
* quirks - bitfield with hw state readout quirks
*
@@ -1362,6 +1375,7 @@ struct intel_crtc {
u16 vmax_vblank_start;
struct intel_display_power_domain_set enabled_power_domains;
+ struct intel_display_power_domain_set hw_readout_power_domains;
struct intel_overlay *overlay;
struct intel_crtc_state *config;
@@ -1799,51 +1813,6 @@ struct intel_dp_mst_encoder {
struct intel_connector *connector;
};
-static inline enum dpio_channel
-vlv_dig_port_to_channel(struct intel_digital_port *dig_port)
-{
- switch (dig_port->base.port) {
- default:
- MISSING_CASE(dig_port->base.port);
- fallthrough;
- case PORT_B:
- case PORT_D:
- return DPIO_CH0;
- case PORT_C:
- return DPIO_CH1;
- }
-}
-
-static inline enum dpio_phy
-vlv_dig_port_to_phy(struct intel_digital_port *dig_port)
-{
- switch (dig_port->base.port) {
- default:
- MISSING_CASE(dig_port->base.port);
- fallthrough;
- case PORT_B:
- case PORT_C:
- return DPIO_PHY0;
- case PORT_D:
- return DPIO_PHY1;
- }
-}
-
-static inline enum dpio_channel
-vlv_pipe_to_channel(enum pipe pipe)
-{
- switch (pipe) {
- default:
- MISSING_CASE(pipe);
- fallthrough;
- case PIPE_A:
- case PIPE_C:
- return DPIO_CH0;
- case PIPE_B:
- return DPIO_CH1;
- }
-}
-
struct intel_load_detect_pipe {
struct drm_atomic_state *restore_state;
};
@@ -2040,15 +2009,16 @@ static inline bool
intel_crtc_has_type(const struct intel_crtc_state *crtc_state,
enum intel_output_type type)
{
- return crtc_state->output_types & (1 << type);
+ return crtc_state->output_types & BIT(type);
}
+
static inline bool
intel_crtc_has_dp_encoder(const struct intel_crtc_state *crtc_state)
{
return crtc_state->output_types &
- ((1 << INTEL_OUTPUT_DP) |
- (1 << INTEL_OUTPUT_DP_MST) |
- (1 << INTEL_OUTPUT_EDP));
+ (BIT(INTEL_OUTPUT_DP) |
+ BIT(INTEL_OUTPUT_DP_MST) |
+ BIT(INTEL_OUTPUT_EDP));
}
static inline bool
@@ -2057,6 +2027,20 @@ intel_crtc_needs_modeset(const struct intel_crtc_state *crtc_state)
return drm_atomic_crtc_needs_modeset(&crtc_state->uapi);
}
+static inline bool
+intel_crtc_needs_fastset(const struct intel_crtc_state *crtc_state)
+{
+ return crtc_state->update_pipe;
+}
+
+static inline bool
+intel_crtc_needs_color_update(const struct intel_crtc_state *crtc_state)
+{
+ return crtc_state->uapi.color_mgmt_changed ||
+ intel_crtc_needs_fastset(crtc_state) ||
+ intel_crtc_needs_modeset(crtc_state);
+}
+
static inline u32 intel_plane_ggtt_offset(const struct intel_plane_state *plane_state)
{
return i915_ggtt_offset(plane_state->ggtt_vma);
diff --git a/drivers/gpu/drm/i915/display/intel_dkl_phy.c b/drivers/gpu/drm/i915/display/intel_dkl_phy.c
index 710b030..57cc3ed 100644
--- a/drivers/gpu/drm/i915/display/intel_dkl_phy.c
+++ b/drivers/gpu/drm/i915/display/intel_dkl_phy.c
@@ -9,9 +9,10 @@
#include "intel_de.h"
#include "intel_display.h"
#include "intel_dkl_phy.h"
+#include "intel_dkl_phy_regs.h"
static void
-dkl_phy_set_hip_idx(struct drm_i915_private *i915, i915_reg_t reg, int idx)
+dkl_phy_set_hip_idx(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg)
{
enum tc_port tc_port = DKL_REG_TC_PORT(reg);
@@ -19,28 +20,27 @@ dkl_phy_set_hip_idx(struct drm_i915_private *i915, i915_reg_t reg, int idx)
intel_de_write(i915,
HIP_INDEX_REG(tc_port),
- HIP_INDEX_VAL(tc_port, idx));
+ HIP_INDEX_VAL(tc_port, reg.bank_idx));
}
/**
* intel_dkl_phy_read - read a Dekel PHY register
* @i915: i915 device instance
* @reg: Dekel PHY register
- * @ln: lane instance of @reg
*
* Read the @reg Dekel PHY register.
*
* Returns the read value.
*/
u32
-intel_dkl_phy_read(struct drm_i915_private *i915, i915_reg_t reg, int ln)
+intel_dkl_phy_read(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg)
{
u32 val;
spin_lock(&i915->display.dkl.phy_lock);
- dkl_phy_set_hip_idx(i915, reg, ln);
- val = intel_de_read(i915, reg);
+ dkl_phy_set_hip_idx(i915, reg);
+ val = intel_de_read(i915, DKL_REG_MMIO(reg));
spin_unlock(&i915->display.dkl.phy_lock);
@@ -51,18 +51,17 @@ intel_dkl_phy_read(struct drm_i915_private *i915, i915_reg_t reg, int ln)
* intel_dkl_phy_write - write a Dekel PHY register
* @i915: i915 device instance
* @reg: Dekel PHY register
- * @ln: lane instance of @reg
* @val: value to write
*
* Write @val to the @reg Dekel PHY register.
*/
void
-intel_dkl_phy_write(struct drm_i915_private *i915, i915_reg_t reg, int ln, u32 val)
+intel_dkl_phy_write(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg, u32 val)
{
spin_lock(&i915->display.dkl.phy_lock);
- dkl_phy_set_hip_idx(i915, reg, ln);
- intel_de_write(i915, reg, val);
+ dkl_phy_set_hip_idx(i915, reg);
+ intel_de_write(i915, DKL_REG_MMIO(reg), val);
spin_unlock(&i915->display.dkl.phy_lock);
}
@@ -71,7 +70,6 @@ intel_dkl_phy_write(struct drm_i915_private *i915, i915_reg_t reg, int ln, u32 v
* intel_dkl_phy_rmw - read-modify-write a Dekel PHY register
* @i915: i915 device instance
* @reg: Dekel PHY register
- * @ln: lane instance of @reg
* @clear: mask to clear
* @set: mask to set
*
@@ -79,12 +77,12 @@ intel_dkl_phy_write(struct drm_i915_private *i915, i915_reg_t reg, int ln, u32 v
* this value back to the register if the value differs from the read one.
*/
void
-intel_dkl_phy_rmw(struct drm_i915_private *i915, i915_reg_t reg, int ln, u32 clear, u32 set)
+intel_dkl_phy_rmw(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg, u32 clear, u32 set)
{
spin_lock(&i915->display.dkl.phy_lock);
- dkl_phy_set_hip_idx(i915, reg, ln);
- intel_de_rmw(i915, reg, clear, set);
+ dkl_phy_set_hip_idx(i915, reg);
+ intel_de_rmw(i915, DKL_REG_MMIO(reg), clear, set);
spin_unlock(&i915->display.dkl.phy_lock);
}
@@ -93,17 +91,16 @@ intel_dkl_phy_rmw(struct drm_i915_private *i915, i915_reg_t reg, int ln, u32 cle
* intel_dkl_phy_posting_read - do a posting read from a Dekel PHY register
* @i915: i915 device instance
* @reg: Dekel PHY register
- * @ln: lane instance of @reg
*
* Read the @reg Dekel PHY register without returning the read value.
*/
void
-intel_dkl_phy_posting_read(struct drm_i915_private *i915, i915_reg_t reg, int ln)
+intel_dkl_phy_posting_read(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg)
{
spin_lock(&i915->display.dkl.phy_lock);
- dkl_phy_set_hip_idx(i915, reg, ln);
- intel_de_posting_read(i915, reg);
+ dkl_phy_set_hip_idx(i915, reg);
+ intel_de_posting_read(i915, DKL_REG_MMIO(reg));
spin_unlock(&i915->display.dkl.phy_lock);
}
diff --git a/drivers/gpu/drm/i915/display/intel_dkl_phy.h b/drivers/gpu/drm/i915/display/intel_dkl_phy.h
index 260ad12..570ee36 100644
--- a/drivers/gpu/drm/i915/display/intel_dkl_phy.h
+++ b/drivers/gpu/drm/i915/display/intel_dkl_phy.h
@@ -8,17 +8,17 @@
#include <linux/types.h>
-#include "i915_reg_defs.h"
+#include "intel_dkl_phy_regs.h"
struct drm_i915_private;
u32
-intel_dkl_phy_read(struct drm_i915_private *i915, i915_reg_t reg, int ln);
+intel_dkl_phy_read(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg);
void
-intel_dkl_phy_write(struct drm_i915_private *i915, i915_reg_t reg, int ln, u32 val);
+intel_dkl_phy_write(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg, u32 val);
void
-intel_dkl_phy_rmw(struct drm_i915_private *i915, i915_reg_t reg, int ln, u32 clear, u32 set);
+intel_dkl_phy_rmw(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg, u32 clear, u32 set);
void
-intel_dkl_phy_posting_read(struct drm_i915_private *i915, i915_reg_t reg, int ln);
+intel_dkl_phy_posting_read(struct drm_i915_private *i915, struct intel_dkl_phy_reg reg);
#endif /* __INTEL_DKL_PHY_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_dkl_phy_regs.h b/drivers/gpu/drm/i915/display/intel_dkl_phy_regs.h
new file mode 100644
index 0000000..56085b3
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_dkl_phy_regs.h
@@ -0,0 +1,204 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#ifndef __INTEL_DKL_PHY_REGS__
+#define __INTEL_DKL_PHY_REGS__
+
+#include <linux/types.h>
+
+struct intel_dkl_phy_reg {
+ u32 reg:24;
+ u32 bank_idx:4;
+};
+
+#define _DKL_PHY1_BASE 0x168000
+#define _DKL_PHY2_BASE 0x169000
+#define _DKL_PHY3_BASE 0x16A000
+#define _DKL_PHY4_BASE 0x16B000
+#define _DKL_PHY5_BASE 0x16C000
+#define _DKL_PHY6_BASE 0x16D000
+
+#define DKL_REG_TC_PORT(__reg) \
+ (TC_PORT_1 + ((__reg).reg - _DKL_PHY1_BASE) / (_DKL_PHY2_BASE - _DKL_PHY1_BASE))
+
+/* DEKEL PHY MMIO Address = Phy base + (internal address & ~index_mask) */
+#define DKL_REG_MMIO(__reg) _MMIO((__reg).reg)
+
+#define _DKL_REG_PHY_BASE(tc_port) _PORT(tc_port, \
+ _DKL_PHY1_BASE, \
+ _DKL_PHY2_BASE)
+
+#define _DKL_BANK_SHIFT 12
+#define _DKL_REG_BANK_OFFSET(phy_offset) \
+ ((phy_offset) & ((1 << _DKL_BANK_SHIFT) - 1))
+#define _DKL_REG_BANK_IDX(phy_offset) \
+ (((phy_offset) >> _DKL_BANK_SHIFT) & 0xf)
+
+#define _DKL_REG(tc_port, phy_offset) \
+ ((const struct intel_dkl_phy_reg) { \
+ .reg = _DKL_REG_PHY_BASE(tc_port) + \
+ _DKL_REG_BANK_OFFSET(phy_offset), \
+ .bank_idx = _DKL_REG_BANK_IDX(phy_offset), \
+ })
+
+#define _DKL_REG_LN(tc_port, ln_idx, ln0_offs, ln1_offs) \
+ _DKL_REG(tc_port, (ln0_offs) + (ln_idx) * ((ln1_offs) - (ln0_offs)))
+
+#define _DKL_PCS_DW5_LN0 0x0014
+#define _DKL_PCS_DW5_LN1 0x1014
+#define DKL_PCS_DW5(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_PCS_DW5_LN0, \
+ _DKL_PCS_DW5_LN1)
+#define DKL_PCS_DW5_CORE_SOFTRESET REG_BIT(11)
+
+#define _DKL_PLL_DIV0 0x2200
+#define DKL_PLL_DIV0(tc_port) _DKL_REG(tc_port, \
+ _DKL_PLL_DIV0)
+#define DKL_PLL_DIV0_AFC_STARTUP_MASK REG_GENMASK(27, 25)
+#define DKL_PLL_DIV0_AFC_STARTUP(val) REG_FIELD_PREP(DKL_PLL_DIV0_AFC_STARTUP_MASK, (val))
+#define DKL_PLL_DIV0_INTEG_COEFF(x) ((x) << 16)
+#define DKL_PLL_DIV0_INTEG_COEFF_MASK (0x1F << 16)
+#define DKL_PLL_DIV0_PROP_COEFF(x) ((x) << 12)
+#define DKL_PLL_DIV0_PROP_COEFF_MASK (0xF << 12)
+#define DKL_PLL_DIV0_FBPREDIV_SHIFT (8)
+#define DKL_PLL_DIV0_FBPREDIV(x) ((x) << DKL_PLL_DIV0_FBPREDIV_SHIFT)
+#define DKL_PLL_DIV0_FBPREDIV_MASK (0xF << DKL_PLL_DIV0_FBPREDIV_SHIFT)
+#define DKL_PLL_DIV0_FBDIV_INT(x) ((x) << 0)
+#define DKL_PLL_DIV0_FBDIV_INT_MASK (0xFF << 0)
+#define DKL_PLL_DIV0_MASK (DKL_PLL_DIV0_INTEG_COEFF_MASK | \
+ DKL_PLL_DIV0_PROP_COEFF_MASK | \
+ DKL_PLL_DIV0_FBPREDIV_MASK | \
+ DKL_PLL_DIV0_FBDIV_INT_MASK)
+
+#define _DKL_PLL_DIV1 0x2204
+#define DKL_PLL_DIV1(tc_port) _DKL_REG(tc_port, \
+ _DKL_PLL_DIV1)
+#define DKL_PLL_DIV1_IREF_TRIM(x) ((x) << 16)
+#define DKL_PLL_DIV1_IREF_TRIM_MASK (0x1F << 16)
+#define DKL_PLL_DIV1_TDC_TARGET_CNT(x) ((x) << 0)
+#define DKL_PLL_DIV1_TDC_TARGET_CNT_MASK (0xFF << 0)
+
+#define _DKL_PLL_SSC 0x2210
+#define DKL_PLL_SSC(tc_port) _DKL_REG(tc_port, \
+ _DKL_PLL_SSC)
+#define DKL_PLL_SSC_IREF_NDIV_RATIO(x) ((x) << 29)
+#define DKL_PLL_SSC_IREF_NDIV_RATIO_MASK (0x7 << 29)
+#define DKL_PLL_SSC_STEP_LEN(x) ((x) << 16)
+#define DKL_PLL_SSC_STEP_LEN_MASK (0xFF << 16)
+#define DKL_PLL_SSC_STEP_NUM(x) ((x) << 11)
+#define DKL_PLL_SSC_STEP_NUM_MASK (0x7 << 11)
+#define DKL_PLL_SSC_EN (1 << 9)
+
+#define _DKL_PLL_BIAS 0x2214
+#define DKL_PLL_BIAS(tc_port) _DKL_REG(tc_port, \
+ _DKL_PLL_BIAS)
+#define DKL_PLL_BIAS_FRAC_EN_H (1 << 30)
+#define DKL_PLL_BIAS_FBDIV_SHIFT (8)
+#define DKL_PLL_BIAS_FBDIV_FRAC(x) ((x) << DKL_PLL_BIAS_FBDIV_SHIFT)
+#define DKL_PLL_BIAS_FBDIV_FRAC_MASK (0x3FFFFF << DKL_PLL_BIAS_FBDIV_SHIFT)
+
+#define _DKL_PLL_TDC_COLDST_BIAS 0x2218
+#define DKL_PLL_TDC_COLDST_BIAS(tc_port) _DKL_REG(tc_port, \
+ _DKL_PLL_TDC_COLDST_BIAS)
+#define DKL_PLL_TDC_SSC_STEP_SIZE(x) ((x) << 8)
+#define DKL_PLL_TDC_SSC_STEP_SIZE_MASK (0xFF << 8)
+#define DKL_PLL_TDC_FEED_FWD_GAIN(x) ((x) << 0)
+#define DKL_PLL_TDC_FEED_FWD_GAIN_MASK (0xFF << 0)
+
+#define _DKL_REFCLKIN_CTL 0x212C
+#define DKL_REFCLKIN_CTL(tc_port) _DKL_REG(tc_port, \
+ _DKL_REFCLKIN_CTL)
+/* Bits are the same as MG_REFCLKIN_CTL */
+
+#define _DKL_CLKTOP2_HSCLKCTL 0x20D4
+#define DKL_CLKTOP2_HSCLKCTL(rc_port) _DKL_REG(tc_port, \
+ _DKL_CLKTOP2_HSCLKCTL)
+/* Bits are the same as MG_CLKTOP2_HSCLKCTL */
+
+#define _DKL_CLKTOP2_CORECLKCTL1 0x20D8
+#define DKL_CLKTOP2_CORECLKCTL1(tc_port) _DKL_REG(tc_port, \
+ _DKL_CLKTOP2_CORECLKCTL1)
+/* Bits are the same as MG_CLKTOP2_CORECLKCTL1 */
+
+#define _DKL_TX_DPCNTL0_LN0 0x02C0
+#define _DKL_TX_DPCNTL0_LN1 0x12C0
+#define DKL_TX_DPCNTL0(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_TX_DPCNTL0_LN0, \
+ _DKL_TX_DPCNTL0_LN1)
+#define DKL_TX_PRESHOOT_COEFF(x) ((x) << 13)
+#define DKL_TX_PRESHOOT_COEFF_MASK (0x1f << 13)
+#define DKL_TX_DE_EMPHASIS_COEFF(x) ((x) << 8)
+#define DKL_TX_DE_EMPAHSIS_COEFF_MASK (0x1f << 8)
+#define DKL_TX_VSWING_CONTROL(x) ((x) << 0)
+#define DKL_TX_VSWING_CONTROL_MASK (0x7 << 0)
+
+#define _DKL_TX_DPCNTL1_LN0 0x02C4
+#define _DKL_TX_DPCNTL1_LN1 0x12C4
+#define DKL_TX_DPCNTL1(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_TX_DPCNTL1_LN0, \
+ _DKL_TX_DPCNTL1_LN1)
+/* Bits are the same as DKL_TX_DPCNTRL0 */
+
+#define _DKL_TX_DPCNTL2_LN0 0x02C8
+#define _DKL_TX_DPCNTL2_LN1 0x12C8
+#define DKL_TX_DPCNTL2(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_TX_DPCNTL2_LN0, \
+ _DKL_TX_DPCNTL2_LN1)
+#define DKL_TX_DP20BITMODE REG_BIT(2)
+#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX1_MASK REG_GENMASK(4, 3)
+#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX1(val) REG_FIELD_PREP(DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX1_MASK, (val))
+#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2_MASK REG_GENMASK(6, 5)
+#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2(val) REG_FIELD_PREP(DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2_MASK, (val))
+
+#define _DKL_TX_FW_CALIB_LN0 0x02F8
+#define _DKL_TX_FW_CALIB_LN1 0x12F8
+#define DKL_TX_FW_CALIB(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_TX_FW_CALIB_LN0, \
+ _DKL_TX_FW_CALIB_LN1)
+#define DKL_TX_CFG_DISABLE_WAIT_INIT (1 << 7)
+
+#define _DKL_TX_PMD_LANE_SUS_LN0 0x0D00
+#define _DKL_TX_PMD_LANE_SUS_LN1 0x1D00
+#define DKL_TX_PMD_LANE_SUS(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_TX_PMD_LANE_SUS_LN0, \
+ _DKL_TX_PMD_LANE_SUS_LN1)
+
+#define _DKL_TX_DW17_LN0 0x0DC4
+#define _DKL_TX_DW17_LN1 0x1DC4
+#define DKL_TX_DW17(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_TX_DW17_LN0, \
+ _DKL_TX_DW17_LN1)
+
+#define _DKL_TX_DW18_LN0 0x0DC8
+#define _DKL_TX_DW18_LN1 0x1DC8
+#define DKL_TX_DW18(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_TX_DW18_LN0, \
+ _DKL_TX_DW18_LN1)
+
+#define _DKL_DP_MODE_LN0 0x00A0
+#define _DKL_DP_MODE_LN1 0x10A0
+#define DKL_DP_MODE(tc_port, ln) _DKL_REG_LN(tc_port, ln, \
+ _DKL_DP_MODE_LN0, \
+ _DKL_DP_MODE_LN1)
+
+#define _DKL_CMN_UC_DW27 0x236C
+#define DKL_CMN_UC_DW_27(tc_port) _DKL_REG(tc_port, \
+ _DKL_CMN_UC_DW27)
+#define DKL_CMN_UC_DW27_UC_HEALTH (0x1 << 15)
+
+/*
+ * Each Dekel PHY is addressed through a 4KB aperture. Each PHY has more than
+ * 4KB of register space, so a separate index is programmed in HIP_INDEX_REG0
+ * or HIP_INDEX_REG1, based on the port number, to set the upper 2 address
+ * bits that point the 4KB window into the full PHY register space.
+ */
+#define _HIP_INDEX_REG0 0x1010A0
+#define _HIP_INDEX_REG1 0x1010A4
+#define HIP_INDEX_REG(tc_port) _MMIO((tc_port) < 4 ? _HIP_INDEX_REG0 \
+ : _HIP_INDEX_REG1)
+#define _HIP_INDEX_SHIFT(tc_port) (8 * ((tc_port) % 4))
+#define HIP_INDEX_VAL(tc_port, val) ((val) << _HIP_INDEX_SHIFT(tc_port))
+
+#endif /* __INTEL_DKL_PHY_REGS__ */
diff --git a/drivers/gpu/drm/i915/display/intel_dmc.c b/drivers/gpu/drm/i915/display/intel_dmc.c
index e52ecc0..eff3add 100644
--- a/drivers/gpu/drm/i915/display/intel_dmc.c
+++ b/drivers/gpu/drm/i915/display/intel_dmc.c
@@ -52,8 +52,8 @@
#define DISPLAY_VER12_DMC_MAX_FW_SIZE ICL_DMC_MAX_FW_SIZE
-#define DG2_DMC_PATH DMC_PATH(dg2, 2, 07)
-#define DG2_DMC_VERSION_REQUIRED DMC_VERSION(2, 07)
+#define DG2_DMC_PATH DMC_PATH(dg2, 2, 08)
+#define DG2_DMC_VERSION_REQUIRED DMC_VERSION(2, 8)
MODULE_FIRMWARE(DG2_DMC_PATH);
#define ADLP_DMC_PATH DMC_PATH(adlp, 2, 16)
@@ -1065,12 +1065,13 @@ static int intel_dmc_debugfs_status_show(struct seq_file *m, void *unused)
seq_printf(m, "fw loaded: %s\n",
str_yes_no(intel_dmc_has_payload(i915)));
seq_printf(m, "path: %s\n", dmc->fw_path);
- seq_printf(m, "Pipe A fw support: %s\n",
+ seq_printf(m, "Pipe A fw needed: %s\n",
str_yes_no(GRAPHICS_VER(i915) >= 12));
seq_printf(m, "Pipe A fw loaded: %s\n",
str_yes_no(dmc->dmc_info[DMC_FW_PIPEA].payload));
- seq_printf(m, "Pipe B fw support: %s\n",
- str_yes_no(IS_ALDERLAKE_P(i915)));
+ seq_printf(m, "Pipe B fw needed: %s\n",
+ str_yes_no(IS_ALDERLAKE_P(i915) ||
+ DISPLAY_VER(i915) >= 14));
seq_printf(m, "Pipe B fw loaded: %s\n",
str_yes_no(dmc->dmc_info[DMC_FW_PIPEB].payload));
@@ -1081,22 +1082,19 @@ static int intel_dmc_debugfs_status_show(struct seq_file *m, void *unused)
DMC_VERSION_MINOR(dmc->version));
if (DISPLAY_VER(i915) >= 12) {
- if (IS_DGFX(i915)) {
+ i915_reg_t dc3co_reg;
+
+ if (IS_DGFX(i915) || DISPLAY_VER(i915) >= 14) {
+ dc3co_reg = DG1_DMC_DEBUG3;
dc5_reg = DG1_DMC_DEBUG_DC5_COUNT;
} else {
+ dc3co_reg = TGL_DMC_DEBUG3;
dc5_reg = TGL_DMC_DEBUG_DC5_COUNT;
dc6_reg = TGL_DMC_DEBUG_DC6_COUNT;
}
- /*
- * NOTE: DMC_DEBUG3 is a general purpose reg.
- * According to B.Specs:49196 DMC f/w reuses DC5/6 counter
- * reg for DC3CO debugging and validation,
- * but TGL DMC f/w is using DMC_DEBUG3 reg for DC3CO counter.
- */
seq_printf(m, "DC3CO count: %d\n",
- intel_de_read(i915, IS_DGFX(i915) ?
- DG1_DMC_DEBUG3 : TGL_DMC_DEBUG3));
+ intel_de_read(i915, dc3co_reg));
} else {
dc5_reg = IS_BROXTON(i915) ? BXT_DMC_DC3_DC5_COUNT :
SKL_DMC_DC3_DC5_COUNT;
diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c
index 2b5bc95..6708971 100644
--- a/drivers/gpu/drm/i915/display/intel_dp.c
+++ b/drivers/gpu/drm/i915/display/intel_dp.c
@@ -46,6 +46,7 @@
#include "g4x_dp.h"
#include "i915_debugfs.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_audio.h"
#include "intel_backlight.h"
@@ -2306,6 +2307,7 @@ bool intel_dp_initial_fastset_check(struct intel_encoder *encoder,
{
struct drm_i915_private *i915 = to_i915(encoder->base.dev);
struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
+ bool fastset = true;
/*
* If BIOS has set an unsupported or non-standard link rate for some
@@ -2313,9 +2315,10 @@ bool intel_dp_initial_fastset_check(struct intel_encoder *encoder,
*/
if (intel_dp_rate_index(intel_dp->source_rates, intel_dp->num_source_rates,
crtc_state->port_clock) < 0) {
- drm_dbg_kms(&i915->drm, "Forcing full modeset due to unsupported link rate\n");
+ drm_dbg_kms(&i915->drm, "[ENCODER:%d:%s] Forcing full modeset due to unsupported link rate\n",
+ encoder->base.base.id, encoder->base.name);
crtc_state->uapi.connectors_changed = true;
- return false;
+ fastset = false;
}
/*
@@ -2326,18 +2329,20 @@ bool intel_dp_initial_fastset_check(struct intel_encoder *encoder,
* Remove once we have readout for DSC.
*/
if (crtc_state->dsc.compression_enable) {
- drm_dbg_kms(&i915->drm, "Forcing full modeset due to DSC being enabled\n");
+ drm_dbg_kms(&i915->drm, "[ENCODER:%d:%s] Forcing full modeset due to DSC being enabled\n",
+ encoder->base.base.id, encoder->base.name);
crtc_state->uapi.mode_changed = true;
- return false;
+ fastset = false;
}
if (CAN_PSR(intel_dp)) {
- drm_dbg_kms(&i915->drm, "Forcing full modeset to compute PSR state\n");
+ drm_dbg_kms(&i915->drm, "[ENCODER:%d:%s] Forcing full modeset to compute PSR state\n",
+ encoder->base.base.id, encoder->base.name);
crtc_state->uapi.mode_changed = true;
- return false;
+ fastset = false;
}
- return true;
+ return fastset;
}
static void intel_dp_get_pcon_dsc_cap(struct intel_dp *intel_dp)
@@ -2686,7 +2691,6 @@ void intel_dp_configure_protocol_converter(struct intel_dp *intel_dp,
str_enable_disable(tmp));
}
-
bool intel_dp_get_colorimetry_status(struct intel_dp *intel_dp)
{
u8 dprx = 0;
@@ -4873,6 +4877,12 @@ void intel_dp_encoder_flush_work(struct drm_encoder *encoder)
intel_pps_vdd_off_sync(intel_dp);
+ /*
+ * Ensure power off delay is respected on module remove, so that we can
+ * reduce delays at driver probe. See pps_init_timestamps().
+ */
+ intel_pps_wait_power_cycle(intel_dp);
+
intel_dp_aux_fini(intel_dp);
}
@@ -5174,19 +5184,6 @@ intel_dp_add_properties(struct intel_dp *intel_dp, struct drm_connector *connect
if (has_gamut_metadata_dip(dev_priv, port))
drm_connector_attach_hdr_output_metadata_property(connector);
- if (intel_dp_is_edp(intel_dp)) {
- u32 allowed_scalers;
-
- allowed_scalers = BIT(DRM_MODE_SCALE_ASPECT) | BIT(DRM_MODE_SCALE_FULLSCREEN);
- if (!HAS_GMCH(dev_priv))
- allowed_scalers |= BIT(DRM_MODE_SCALE_CENTER);
-
- drm_connector_attach_scaling_mode_property(connector, allowed_scalers);
-
- connector->state->scaling_mode = DRM_MODE_SCALE_ASPECT;
-
- }
-
if (HAS_VRR(dev_priv))
drm_connector_attach_vrr_capable_property(connector);
}
@@ -5199,8 +5196,7 @@ intel_edp_add_properties(struct intel_dp *intel_dp)
const struct drm_display_mode *fixed_mode =
intel_panel_preferred_fixed_mode(connector);
- if (!fixed_mode)
- return;
+ intel_attach_scaling_mode_property(&connector->base);
drm_connector_set_panel_orientation_with_quirk(&connector->base,
i915->display.vbt.orientation,
@@ -5208,16 +5204,43 @@ intel_edp_add_properties(struct intel_dp *intel_dp)
fixed_mode->vdisplay);
}
+static void intel_edp_backlight_setup(struct intel_dp *intel_dp,
+ struct intel_connector *connector)
+{
+ struct drm_i915_private *i915 = dp_to_i915(intel_dp);
+ enum pipe pipe = INVALID_PIPE;
+
+ if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) {
+ /*
+ * Figure out the current pipe for the initial backlight setup.
+ * If the current pipe isn't valid, try the PPS pipe, and if that
+ * fails just assume pipe A.
+ */
+ pipe = vlv_active_pipe(intel_dp);
+
+ if (pipe != PIPE_A && pipe != PIPE_B)
+ pipe = intel_dp->pps.pps_pipe;
+
+ if (pipe != PIPE_A && pipe != PIPE_B)
+ pipe = PIPE_A;
+
+ drm_dbg_kms(&i915->drm,
+ "[CONNECTOR:%d:%s] using pipe %c for initial backlight setup\n",
+ connector->base.base.id, connector->base.name,
+ pipe_name(pipe));
+ }
+
+ intel_backlight_setup(connector, pipe);
+}
+
static bool intel_edp_init_connector(struct intel_dp *intel_dp,
struct intel_connector *intel_connector)
{
struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector = &intel_connector->base;
struct drm_display_mode *fixed_mode;
struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;
bool has_dpcd;
- enum pipe pipe = INVALID_PIPE;
struct edid *edid;
if (!intel_dp_is_edp(intel_dp))
@@ -5230,7 +5253,7 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp,
* with an already powered-on LVDS power sequencer.
*/
if (intel_get_lvds_encoder(dev_priv)) {
- drm_WARN_ON(dev,
+ drm_WARN_ON(&dev_priv->drm,
!(HAS_PCH_IBX(dev_priv) || HAS_PCH_CPT(dev_priv)));
drm_info(&dev_priv->drm,
"LVDS was detected, not registering eDP\n");
@@ -5246,11 +5269,12 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp,
if (!has_dpcd) {
/* if this fails, presume the device is a ghost */
drm_info(&dev_priv->drm,
- "failed to retrieve link info, disabling eDP\n");
+ "[ENCODER:%d:%s] failed to retrieve link info, disabling eDP\n",
+ encoder->base.base.id, encoder->base.name);
goto out_vdd_off;
}
- mutex_lock(&dev->mode_config.mutex);
+ mutex_lock(&dev_priv->drm.mode_config.mutex);
edid = drm_get_edid(connector, &intel_dp->aux.ddc);
if (!edid) {
/* Fallback to EDID from ACPI OpRegion, if any */
@@ -5275,9 +5299,7 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp,
intel_bios_init_panel(dev_priv, &intel_connector->panel,
encoder->devdata, IS_ERR(edid) ? NULL : edid);
- intel_panel_add_edid_fixed_modes(intel_connector,
- intel_connector->panel.vbt.drrs_type != DRRS_TYPE_NONE ||
- intel_vrr_is_capable(intel_connector));
+ intel_panel_add_edid_fixed_modes(intel_connector, true);
/* MSO requires information from the EDID */
intel_edp_mso_init(intel_dp);
@@ -5290,30 +5312,18 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp,
if (!intel_panel_preferred_fixed_mode(intel_connector))
intel_panel_add_vbt_lfp_fixed_mode(intel_connector);
- mutex_unlock(&dev->mode_config.mutex);
+ mutex_unlock(&dev_priv->drm.mode_config.mutex);
- if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- /*
- * Figure out the current pipe for the initial backlight setup.
- * If the current pipe isn't valid, try the PPS pipe, and if that
- * fails just assume pipe A.
- */
- pipe = vlv_active_pipe(intel_dp);
-
- if (pipe != PIPE_A && pipe != PIPE_B)
- pipe = intel_dp->pps.pps_pipe;
-
- if (pipe != PIPE_A && pipe != PIPE_B)
- pipe = PIPE_A;
-
- drm_dbg_kms(&dev_priv->drm,
- "using pipe %c for initial backlight setup\n",
- pipe_name(pipe));
+ if (!intel_panel_preferred_fixed_mode(intel_connector)) {
+ drm_info(&dev_priv->drm,
+ "[ENCODER:%d:%s] failed to find fixed mode for the panel, disabling eDP\n",
+ encoder->base.base.id, encoder->base.name);
+ goto out_vdd_off;
}
intel_panel_init(intel_connector);
- intel_backlight_setup(intel_connector, pipe);
+ intel_edp_backlight_setup(intel_dp, intel_connector);
intel_edp_add_properties(intel_dp);
@@ -5415,7 +5425,6 @@ intel_dp_init_connector(struct intel_digital_port *dig_port,
if (!HAS_GMCH(dev_priv))
connector->interlace_allowed = true;
- connector->doublescan_allowed = 0;
intel_connector->polled = DRM_CONNECTOR_POLL_HPD;
diff --git a/drivers/gpu/drm/i915/display/intel_dp_aux.c b/drivers/gpu/drm/i915/display/intel_dp_aux.c
index 48c375c..664bebd 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_aux.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_aux.c
@@ -4,6 +4,7 @@
*/
#include "i915_drv.h"
+#include "i915_reg.h"
#include "i915_trace.h"
#include "intel_display_types.h"
#include "intel_dp_aux.h"
diff --git a/drivers/gpu/drm/i915/display/intel_dp_hdcp.c b/drivers/gpu/drm/i915/display/intel_dp_hdcp.c
index 8868912..e0c1771 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_hdcp.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_hdcp.c
@@ -11,6 +11,7 @@
#include <drm/display/drm_hdcp_helper.h>
#include <drm/drm_print.h>
+#include "i915_reg.h"
#include "intel_ddi.h"
#include "intel_de.h"
#include "intel_display_types.h"
@@ -19,28 +20,20 @@
#include "intel_hdcp.h"
#include "intel_hdcp_regs.h"
-static unsigned int transcoder_to_stream_enc_status(enum transcoder cpu_transcoder)
+static u32 transcoder_to_stream_enc_status(enum transcoder cpu_transcoder)
{
- u32 stream_enc_mask;
-
switch (cpu_transcoder) {
case TRANSCODER_A:
- stream_enc_mask = HDCP_STATUS_STREAM_A_ENC;
- break;
+ return HDCP_STATUS_STREAM_A_ENC;
case TRANSCODER_B:
- stream_enc_mask = HDCP_STATUS_STREAM_B_ENC;
- break;
+ return HDCP_STATUS_STREAM_B_ENC;
case TRANSCODER_C:
- stream_enc_mask = HDCP_STATUS_STREAM_C_ENC;
- break;
+ return HDCP_STATUS_STREAM_C_ENC;
case TRANSCODER_D:
- stream_enc_mask = HDCP_STATUS_STREAM_D_ENC;
- break;
+ return HDCP_STATUS_STREAM_D_ENC;
default:
- stream_enc_mask = 0;
+ return 0;
}
-
- return stream_enc_mask;
}
static void intel_dp_hdcp_wait_for_cp_irq(struct intel_hdcp *hdcp, int timeout)
diff --git a/drivers/gpu/drm/i915/display/intel_dp_mst.c b/drivers/gpu/drm/i915/display/intel_dp_mst.c
index 03604a3..4077a97 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_mst.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_mst.c
@@ -29,6 +29,7 @@
#include <drm/drm_probe_helper.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_audio.h"
#include "intel_connector.h"
@@ -793,7 +794,35 @@ static bool intel_dp_mst_get_hw_state(struct intel_connector *connector)
return false;
}
-static struct drm_connector *intel_dp_add_mst_connector(struct drm_dp_mst_topology_mgr *mgr, struct drm_dp_mst_port *port, const char *pathprop)
+static int intel_dp_mst_add_properties(struct intel_dp *intel_dp,
+ struct drm_connector *connector,
+ const char *pathprop)
+{
+ struct drm_i915_private *i915 = to_i915(connector->dev);
+
+ drm_object_attach_property(&connector->base,
+ i915->drm.mode_config.path_property, 0);
+ drm_object_attach_property(&connector->base,
+ i915->drm.mode_config.tile_property, 0);
+
+ intel_attach_force_audio_property(connector);
+ intel_attach_broadcast_rgb_property(connector);
+
+ /*
+ * Reuse the prop from the SST connector because we're
+ * not allowed to create new props after device registration.
+ */
+ connector->max_bpc_property =
+ intel_dp->attached_connector->base.max_bpc_property;
+ if (connector->max_bpc_property)
+ drm_connector_attach_max_bpc_property(connector, 6, 12);
+
+ return drm_connector_set_path_property(connector, pathprop);
+}
+
+static struct drm_connector *intel_dp_add_mst_connector(struct drm_dp_mst_topology_mgr *mgr,
+ struct drm_dp_mst_port *port,
+ const char *pathprop)
{
struct intel_dp *intel_dp = container_of(mgr, struct intel_dp, mst_mgr);
struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
@@ -833,28 +862,14 @@ static struct drm_connector *intel_dp_add_mst_connector(struct drm_dp_mst_topolo
goto err;
}
- drm_object_attach_property(&connector->base, dev->mode_config.path_property, 0);
- drm_object_attach_property(&connector->base, dev->mode_config.tile_property, 0);
-
- ret = drm_connector_set_path_property(connector, pathprop);
+ ret = intel_dp_mst_add_properties(intel_dp, connector, pathprop);
if (ret)
goto err;
- intel_attach_force_audio_property(connector);
- intel_attach_broadcast_rgb_property(connector);
-
ret = intel_dp_hdcp_init(dig_port, intel_connector);
if (ret)
drm_dbg_kms(&dev_priv->drm, "[%s:%d] HDCP MST init failed, skipping.\n",
connector->name, connector->base.id);
- /*
- * Reuse the prop from the SST connector because we're
- * not allowed to create new props after device registration.
- */
- connector->max_bpc_property =
- intel_dp->attached_connector->base.max_bpc_property;
- if (connector->max_bpc_property)
- drm_connector_attach_max_bpc_property(connector, 6, 12);
return connector;
diff --git a/drivers/gpu/drm/i915/display/intel_dpio_phy.c b/drivers/gpu/drm/i915/display/intel_dpio_phy.c
index 8732b87..7eb7440 100644
--- a/drivers/gpu/drm/i915/display/intel_dpio_phy.c
+++ b/drivers/gpu/drm/i915/display/intel_dpio_phy.c
@@ -21,6 +21,7 @@
* DEALINGS IN THE SOFTWARE.
*/
+#include "i915_reg.h"
#include "intel_ddi.h"
#include "intel_ddi_buf_trans.h"
#include "intel_de.h"
@@ -655,6 +656,48 @@ bxt_ddi_phy_get_lane_lat_optim_mask(struct intel_encoder *encoder)
return mask;
}
+enum dpio_channel vlv_dig_port_to_channel(struct intel_digital_port *dig_port)
+{
+ switch (dig_port->base.port) {
+ default:
+ MISSING_CASE(dig_port->base.port);
+ fallthrough;
+ case PORT_B:
+ case PORT_D:
+ return DPIO_CH0;
+ case PORT_C:
+ return DPIO_CH1;
+ }
+}
+
+enum dpio_phy vlv_dig_port_to_phy(struct intel_digital_port *dig_port)
+{
+ switch (dig_port->base.port) {
+ default:
+ MISSING_CASE(dig_port->base.port);
+ fallthrough;
+ case PORT_B:
+ case PORT_C:
+ return DPIO_PHY0;
+ case PORT_D:
+ return DPIO_PHY1;
+ }
+}
+
+enum dpio_channel vlv_pipe_to_channel(enum pipe pipe)
+{
+ switch (pipe) {
+ default:
+ MISSING_CASE(pipe);
+ fallthrough;
+ case PIPE_A:
+ case PIPE_C:
+ return DPIO_CH0;
+ case PIPE_B:
+ return DPIO_CH1;
+ }
+}
+
void chv_set_phy_signal_level(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
u32 deemph_reg_value, u32 margin_reg_value,
diff --git a/drivers/gpu/drm/i915/display/intel_dpio_phy.h b/drivers/gpu/drm/i915/display/intel_dpio_phy.h
index 9c3d008..9c7725d 100644
--- a/drivers/gpu/drm/i915/display/intel_dpio_phy.h
+++ b/drivers/gpu/drm/i915/display/intel_dpio_phy.h
@@ -8,13 +8,24 @@
#include <linux/types.h>
-enum dpio_channel;
-enum dpio_phy;
+enum pipe;
enum port;
struct drm_i915_private;
struct intel_crtc_state;
+struct intel_digital_port;
struct intel_encoder;
+enum dpio_channel {
+ DPIO_CH0,
+ DPIO_CH1,
+};
+
+enum dpio_phy {
+ DPIO_PHY0,
+ DPIO_PHY1,
+ DPIO_PHY2,
+};
+
void bxt_port_to_phy_channel(struct drm_i915_private *dev_priv, enum port port,
enum dpio_phy *phy, enum dpio_channel *ch);
void bxt_ddi_phy_set_signal_levels(struct intel_encoder *encoder,
@@ -30,6 +41,10 @@ void bxt_ddi_phy_set_lane_optim_mask(struct intel_encoder *encoder,
u8 lane_lat_optim_mask);
u8 bxt_ddi_phy_get_lane_lat_optim_mask(struct intel_encoder *encoder);
+enum dpio_channel vlv_dig_port_to_channel(struct intel_digital_port *dig_port);
+enum dpio_phy vlv_dig_port_to_phy(struct intel_digital_port *dig_port);
+enum dpio_channel vlv_pipe_to_channel(enum pipe pipe);
+
void chv_set_phy_signal_level(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
u32 deemph_reg_value, u32 margin_reg_value,
diff --git a/drivers/gpu/drm/i915/display/intel_dpll.c b/drivers/gpu/drm/i915/display/intel_dpll.c
index b15ba78..c236aaf 100644
--- a/drivers/gpu/drm/i915/display/intel_dpll.c
+++ b/drivers/gpu/drm/i915/display/intel_dpll.c
@@ -6,10 +6,12 @@
#include <linux/kernel.h>
#include <linux/string_helpers.h>
+#include "i915_reg.h"
#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display.h"
#include "intel_display_types.h"
+#include "intel_dpio_phy.h"
#include "intel_dpll.h"
#include "intel_lvds.h"
#include "intel_panel.h"
diff --git a/drivers/gpu/drm/i915/display/intel_dpll_mgr.c b/drivers/gpu/drm/i915/display/intel_dpll_mgr.c
index 64dd603..1974eb5 100644
--- a/drivers/gpu/drm/i915/display/intel_dpll_mgr.c
+++ b/drivers/gpu/drm/i915/display/intel_dpll_mgr.c
@@ -23,15 +23,18 @@
#include <linux/string_helpers.h>
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dkl_phy.h"
+#include "intel_dkl_phy_regs.h"
#include "intel_dpio_phy.h"
#include "intel_dpll.h"
#include "intel_dpll_mgr.h"
+#include "intel_hti.h"
+#include "intel_mg_phy_regs.h"
#include "intel_pch_refclk.h"
#include "intel_tc.h"
-#include "intel_tc_phy_regs.h"
/**
* DOC: Display PLLs
@@ -153,28 +156,6 @@ intel_get_shared_dpll_by_id(struct drm_i915_private *dev_priv,
return &dev_priv->display.dpll.shared_dplls[id];
}
-/**
- * intel_get_shared_dpll_id - get the id of a DPLL
- * @dev_priv: i915 device instance
- * @pll: the DPLL
- *
- * Returns:
- * The id of @pll
- */
-enum intel_dpll_id
-intel_get_shared_dpll_id(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- long pll_idx = pll - dev_priv->display.dpll.shared_dplls;
-
- if (drm_WARN_ON(&dev_priv->drm,
- pll_idx < 0 ||
- pll_idx >= dev_priv->display.dpll.num_shared_dpll))
- return -1;
-
- return pll_idx;
-}
-
/* For ILK+ */
void assert_shared_dpll(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll,
@@ -385,20 +366,30 @@ intel_reference_shared_dpll(struct intel_atomic_state *state,
if (shared_dpll[id].pipe_mask == 0)
shared_dpll[id].hw_state = *pll_state;
- drm_dbg(&i915->drm, "using %s for pipe %c\n", pll->info->name,
- pipe_name(crtc->pipe));
+ drm_WARN_ON(&i915->drm, (shared_dpll[id].pipe_mask & BIT(crtc->pipe)) != 0);
shared_dpll[id].pipe_mask |= BIT(crtc->pipe);
+
+ drm_dbg_kms(&i915->drm, "[CRTC:%d:%s] reserving %s\n",
+ crtc->base.base.id, crtc->base.name, pll->info->name);
}
static void intel_unreference_shared_dpll(struct intel_atomic_state *state,
const struct intel_crtc *crtc,
const struct intel_shared_dpll *pll)
{
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
struct intel_shared_dpll_state *shared_dpll;
+ const enum intel_dpll_id id = pll->info->id;
shared_dpll = intel_atomic_get_shared_dpll_state(&state->base);
- shared_dpll[pll->info->id].pipe_mask &= ~BIT(crtc->pipe);
+
+ drm_WARN_ON(&i915->drm, (shared_dpll[id].pipe_mask & BIT(crtc->pipe)) == 0);
+
+ shared_dpll[id].pipe_mask &= ~BIT(crtc->pipe);
+
+ drm_dbg_kms(&i915->drm, "[CRTC:%d:%s] releasing %s\n",
+ crtc->base.base.id, crtc->base.name, pll->info->name);
}
static void intel_put_dpll(struct intel_atomic_state *state,
@@ -709,8 +700,6 @@ struct hsw_wrpll_rnp {
static unsigned hsw_wrpll_get_budget_for_freq(int clock)
{
- unsigned budget;
-
switch (clock) {
case 25175000:
case 25200000:
@@ -743,21 +732,18 @@ static unsigned hsw_wrpll_get_budget_for_freq(int clock)
case 222750000:
case 296703000:
case 297000000:
- budget = 0;
- break;
+ return 0;
case 233500000:
case 245250000:
case 247750000:
case 253250000:
case 298000000:
- budget = 1500;
- break;
+ return 1500;
case 169128000:
case 169500000:
case 179500000:
case 202000000:
- budget = 2000;
- break;
+ return 2000;
case 256250000:
case 262500000:
case 270000000:
@@ -767,18 +753,13 @@ static unsigned hsw_wrpll_get_budget_for_freq(int clock)
case 281250000:
case 286000000:
case 291750000:
- budget = 4000;
- break;
+ return 4000;
case 267250000:
case 268500000:
- budget = 5000;
- break;
+ return 5000;
default:
- budget = 1000;
- break;
+ return 1000;
}
-
- return budget;
}
static void hsw_wrpll_update_rnp(u64 freq2k, unsigned int budget,
@@ -3184,14 +3165,6 @@ static void icl_update_active_dpll(struct intel_atomic_state *state,
icl_set_active_port_dpll(crtc_state, port_dpll_id);
}
-static u32 intel_get_hti_plls(struct drm_i915_private *i915)
-{
- if (!(i915->hti_state & HDPORT_ENABLED))
- return 0;
-
- return REG_FIELD_GET(HDPORT_DPLL_USED_MASK, i915->hti_state);
-}
-
static int icl_compute_combo_phy_dpll(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
@@ -3266,7 +3239,7 @@ static int icl_get_combo_phy_dpll(struct intel_atomic_state *state,
}
/* Eliminate DPLLs from consideration if reserved by HTI */
- dpll_mask &= ~intel_get_hti_plls(dev_priv);
+ dpll_mask &= ~intel_hti_dpll_mask(dev_priv);
port_dpll->pll = intel_find_shared_dpll(state, crtc,
&port_dpll->hw_state,
@@ -3510,11 +3483,11 @@ static bool dkl_pll_get_hw_state(struct drm_i915_private *dev_priv,
* they are on different building blocks
*/
hw_state->mg_refclkin_ctl = intel_dkl_phy_read(dev_priv,
- DKL_REFCLKIN_CTL(tc_port), 2);
+ DKL_REFCLKIN_CTL(tc_port));
hw_state->mg_refclkin_ctl &= MG_REFCLKIN_CTL_OD_2_MUX_MASK;
hw_state->mg_clktop2_hsclkctl =
- intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_HSCLKCTL(tc_port), 2);
+ intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_HSCLKCTL(tc_port));
hw_state->mg_clktop2_hsclkctl &=
MG_CLKTOP2_HSCLKCTL_TLINEDRV_CLKSEL_MASK |
MG_CLKTOP2_HSCLKCTL_CORE_INPUTSEL_MASK |
@@ -3522,32 +3495,32 @@ static bool dkl_pll_get_hw_state(struct drm_i915_private *dev_priv,
MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO_MASK;
hw_state->mg_clktop2_coreclkctl1 =
- intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_CORECLKCTL1(tc_port), 2);
+ intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_CORECLKCTL1(tc_port));
hw_state->mg_clktop2_coreclkctl1 &=
MG_CLKTOP2_CORECLKCTL1_A_DIVRATIO_MASK;
- hw_state->mg_pll_div0 = intel_dkl_phy_read(dev_priv, DKL_PLL_DIV0(tc_port), 2);
+ hw_state->mg_pll_div0 = intel_dkl_phy_read(dev_priv, DKL_PLL_DIV0(tc_port));
val = DKL_PLL_DIV0_MASK;
if (dev_priv->display.vbt.override_afc_startup)
val |= DKL_PLL_DIV0_AFC_STARTUP_MASK;
hw_state->mg_pll_div0 &= val;
- hw_state->mg_pll_div1 = intel_dkl_phy_read(dev_priv, DKL_PLL_DIV1(tc_port), 2);
+ hw_state->mg_pll_div1 = intel_dkl_phy_read(dev_priv, DKL_PLL_DIV1(tc_port));
hw_state->mg_pll_div1 &= (DKL_PLL_DIV1_IREF_TRIM_MASK |
DKL_PLL_DIV1_TDC_TARGET_CNT_MASK);
- hw_state->mg_pll_ssc = intel_dkl_phy_read(dev_priv, DKL_PLL_SSC(tc_port), 2);
+ hw_state->mg_pll_ssc = intel_dkl_phy_read(dev_priv, DKL_PLL_SSC(tc_port));
hw_state->mg_pll_ssc &= (DKL_PLL_SSC_IREF_NDIV_RATIO_MASK |
DKL_PLL_SSC_STEP_LEN_MASK |
DKL_PLL_SSC_STEP_NUM_MASK |
DKL_PLL_SSC_EN);
- hw_state->mg_pll_bias = intel_dkl_phy_read(dev_priv, DKL_PLL_BIAS(tc_port), 2);
+ hw_state->mg_pll_bias = intel_dkl_phy_read(dev_priv, DKL_PLL_BIAS(tc_port));
hw_state->mg_pll_bias &= (DKL_PLL_BIAS_FRAC_EN_H |
DKL_PLL_BIAS_FBDIV_FRAC_MASK);
hw_state->mg_pll_tdc_coldst_bias =
- intel_dkl_phy_read(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port), 2);
+ intel_dkl_phy_read(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port));
hw_state->mg_pll_tdc_coldst_bias &= (DKL_PLL_TDC_SSC_STEP_SIZE_MASK |
DKL_PLL_TDC_FEED_FWD_GAIN_MASK);
@@ -3736,57 +3709,57 @@ static void dkl_pll_write(struct drm_i915_private *dev_priv,
* though on different building block
*/
/* All the registers are RMW */
- val = intel_dkl_phy_read(dev_priv, DKL_REFCLKIN_CTL(tc_port), 2);
+ val = intel_dkl_phy_read(dev_priv, DKL_REFCLKIN_CTL(tc_port));
val &= ~MG_REFCLKIN_CTL_OD_2_MUX_MASK;
val |= hw_state->mg_refclkin_ctl;
- intel_dkl_phy_write(dev_priv, DKL_REFCLKIN_CTL(tc_port), 2, val);
+ intel_dkl_phy_write(dev_priv, DKL_REFCLKIN_CTL(tc_port), val);
- val = intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_CORECLKCTL1(tc_port), 2);
+ val = intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_CORECLKCTL1(tc_port));
val &= ~MG_CLKTOP2_CORECLKCTL1_A_DIVRATIO_MASK;
val |= hw_state->mg_clktop2_coreclkctl1;
- intel_dkl_phy_write(dev_priv, DKL_CLKTOP2_CORECLKCTL1(tc_port), 2, val);
+ intel_dkl_phy_write(dev_priv, DKL_CLKTOP2_CORECLKCTL1(tc_port), val);
- val = intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_HSCLKCTL(tc_port), 2);
+ val = intel_dkl_phy_read(dev_priv, DKL_CLKTOP2_HSCLKCTL(tc_port));
val &= ~(MG_CLKTOP2_HSCLKCTL_TLINEDRV_CLKSEL_MASK |
MG_CLKTOP2_HSCLKCTL_CORE_INPUTSEL_MASK |
MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_MASK |
MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO_MASK);
val |= hw_state->mg_clktop2_hsclkctl;
- intel_dkl_phy_write(dev_priv, DKL_CLKTOP2_HSCLKCTL(tc_port), 2, val);
+ intel_dkl_phy_write(dev_priv, DKL_CLKTOP2_HSCLKCTL(tc_port), val);
val = DKL_PLL_DIV0_MASK;
if (dev_priv->display.vbt.override_afc_startup)
val |= DKL_PLL_DIV0_AFC_STARTUP_MASK;
- intel_dkl_phy_rmw(dev_priv, DKL_PLL_DIV0(tc_port), 2, val,
+ intel_dkl_phy_rmw(dev_priv, DKL_PLL_DIV0(tc_port), val,
hw_state->mg_pll_div0);
- val = intel_dkl_phy_read(dev_priv, DKL_PLL_DIV1(tc_port), 2);
+ val = intel_dkl_phy_read(dev_priv, DKL_PLL_DIV1(tc_port));
val &= ~(DKL_PLL_DIV1_IREF_TRIM_MASK |
DKL_PLL_DIV1_TDC_TARGET_CNT_MASK);
val |= hw_state->mg_pll_div1;
- intel_dkl_phy_write(dev_priv, DKL_PLL_DIV1(tc_port), 2, val);
+ intel_dkl_phy_write(dev_priv, DKL_PLL_DIV1(tc_port), val);
- val = intel_dkl_phy_read(dev_priv, DKL_PLL_SSC(tc_port), 2);
+ val = intel_dkl_phy_read(dev_priv, DKL_PLL_SSC(tc_port));
val &= ~(DKL_PLL_SSC_IREF_NDIV_RATIO_MASK |
DKL_PLL_SSC_STEP_LEN_MASK |
DKL_PLL_SSC_STEP_NUM_MASK |
DKL_PLL_SSC_EN);
val |= hw_state->mg_pll_ssc;
- intel_dkl_phy_write(dev_priv, DKL_PLL_SSC(tc_port), 2, val);
+ intel_dkl_phy_write(dev_priv, DKL_PLL_SSC(tc_port), val);
- val = intel_dkl_phy_read(dev_priv, DKL_PLL_BIAS(tc_port), 2);
+ val = intel_dkl_phy_read(dev_priv, DKL_PLL_BIAS(tc_port));
val &= ~(DKL_PLL_BIAS_FRAC_EN_H |
DKL_PLL_BIAS_FBDIV_FRAC_MASK);
val |= hw_state->mg_pll_bias;
- intel_dkl_phy_write(dev_priv, DKL_PLL_BIAS(tc_port), 2, val);
+ intel_dkl_phy_write(dev_priv, DKL_PLL_BIAS(tc_port), val);
- val = intel_dkl_phy_read(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port), 2);
+ val = intel_dkl_phy_read(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port));
val &= ~(DKL_PLL_TDC_SSC_STEP_SIZE_MASK |
DKL_PLL_TDC_FEED_FWD_GAIN_MASK);
val |= hw_state->mg_pll_tdc_coldst_bias;
- intel_dkl_phy_write(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port), 2, val);
+ intel_dkl_phy_write(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port), val);
- intel_dkl_phy_posting_read(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port), 2);
+ intel_dkl_phy_posting_read(dev_priv, DKL_PLL_TDC_COLDST_BIAS(tc_port));
}
static void icl_pll_power_enable(struct drm_i915_private *dev_priv,
@@ -4188,6 +4161,8 @@ void intel_shared_dpll_init(struct drm_i915_private *dev_priv)
const struct dpll_info *dpll_info;
int i;
+ mutex_init(&dev_priv->display.dpll.lock);
+
if (IS_DG2(dev_priv))
/* No shared DPLLs on DG2; port PLLs are part of the PHY */
dpll_mgr = NULL;
@@ -4232,7 +4207,6 @@ void intel_shared_dpll_init(struct drm_i915_private *dev_priv)
dev_priv->display.dpll.mgr = dpll_mgr;
dev_priv->display.dpll.num_shared_dpll = i;
- mutex_init(&dev_priv->display.dpll.lock);
}
/**
diff --git a/drivers/gpu/drm/i915/display/intel_dpll_mgr.h b/drivers/gpu/drm/i915/display/intel_dpll_mgr.h
index 3247dc3..3854f1b 100644
--- a/drivers/gpu/drm/i915/display/intel_dpll_mgr.h
+++ b/drivers/gpu/drm/i915/display/intel_dpll_mgr.h
@@ -328,9 +328,6 @@ struct intel_shared_dpll {
struct intel_shared_dpll *
intel_get_shared_dpll_by_id(struct drm_i915_private *dev_priv,
enum intel_dpll_id id);
-enum intel_dpll_id
-intel_get_shared_dpll_id(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll);
void assert_shared_dpll(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll,
bool state);
diff --git a/drivers/gpu/drm/i915/display/intel_dpt.c b/drivers/gpu/drm/i915/display/intel_dpt.c
index ac58764..ad1a37b 100644
--- a/drivers/gpu/drm/i915/display/intel_dpt.c
+++ b/drivers/gpu/drm/i915/display/intel_dpt.c
@@ -5,6 +5,7 @@
#include "gem/i915_gem_domain.h"
#include "gem/i915_gem_internal.h"
+#include "gem/i915_gem_lmem.h"
#include "gt/gen8_ppgtt.h"
#include "i915_drv.h"
diff --git a/drivers/gpu/drm/i915/display/intel_drrs.c b/drivers/gpu/drm/i915/display/intel_drrs.c
index 7da4a9c..5b9e444 100644
--- a/drivers/gpu/drm/i915/display/intel_drrs.c
+++ b/drivers/gpu/drm/i915/display/intel_drrs.c
@@ -4,6 +4,7 @@
*/
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_de.h"
#include "intel_display_types.h"
@@ -284,16 +285,124 @@ void intel_drrs_flush(struct drm_i915_private *dev_priv,
}
/**
- * intel_crtc_drrs_init - Init DRRS for CRTC
+ * intel_drrs_crtc_init - Init DRRS for CRTC
* @crtc: crtc
*
* This function is called only once at driver load to initialize basic
* DRRS stuff.
*
*/
-void intel_crtc_drrs_init(struct intel_crtc *crtc)
+void intel_drrs_crtc_init(struct intel_crtc *crtc)
{
INIT_DELAYED_WORK(&crtc->drrs.work, intel_drrs_downclock_work);
mutex_init(&crtc->drrs.mutex);
crtc->drrs.cpu_transcoder = INVALID_TRANSCODER;
}
+
+static int intel_drrs_debugfs_status_show(struct seq_file *m, void *unused)
+{
+ struct intel_crtc *crtc = m->private;
+ const struct intel_crtc_state *crtc_state;
+ int ret;
+
+ ret = drm_modeset_lock_single_interruptible(&crtc->base.mutex);
+ if (ret)
+ return ret;
+
+ crtc_state = to_intel_crtc_state(crtc->base.state);
+
+ mutex_lock(&crtc->drrs.mutex);
+
+ seq_printf(m, "DRRS enabled: %s\n",
+ str_yes_no(crtc_state->has_drrs));
+
+ seq_printf(m, "DRRS active: %s\n",
+ str_yes_no(intel_drrs_is_active(crtc)));
+
+ seq_printf(m, "DRRS refresh rate: %s\n",
+ crtc->drrs.refresh_rate == DRRS_REFRESH_RATE_LOW ?
+ "low" : "high");
+
+ seq_printf(m, "DRRS busy frontbuffer bits: 0x%x\n",
+ crtc->drrs.busy_frontbuffer_bits);
+
+ mutex_unlock(&crtc->drrs.mutex);
+
+ drm_modeset_unlock(&crtc->base.mutex);
+
+ return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(intel_drrs_debugfs_status);
+
+static int intel_drrs_debugfs_ctl_set(void *data, u64 val)
+{
+ struct intel_crtc *crtc = data;
+ struct drm_i915_private *i915 = to_i915(crtc->base.dev);
+ struct intel_crtc_state *crtc_state;
+ struct drm_crtc_commit *commit;
+ int ret;
+
+ ret = drm_modeset_lock_single_interruptible(&crtc->base.mutex);
+ if (ret)
+ return ret;
+
+ crtc_state = to_intel_crtc_state(crtc->base.state);
+
+ if (!crtc_state->hw.active ||
+ !crtc_state->has_drrs)
+ goto out;
+
+ commit = crtc_state->uapi.commit;
+ if (commit) {
+ ret = wait_for_completion_interruptible(&commit->hw_done);
+ if (ret)
+ goto out;
+ }
+
+ drm_dbg(&i915->drm,
+ "Manually %sactivating DRRS\n", val ? "" : "de");
+
+ if (val)
+ intel_drrs_activate(crtc_state);
+ else
+ intel_drrs_deactivate(crtc_state);
+
+out:
+ drm_modeset_unlock(&crtc->base.mutex);
+
+ return ret;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(intel_drrs_debugfs_ctl_fops,
+ NULL, intel_drrs_debugfs_ctl_set, "%llu\n");
+
+void intel_drrs_crtc_debugfs_add(struct intel_crtc *crtc)
+{
+ debugfs_create_file("i915_drrs_status", 0444, crtc->base.debugfs_entry,
+ crtc, &intel_drrs_debugfs_status_fops);
+
+ debugfs_create_file("i915_drrs_ctl", 0644, crtc->base.debugfs_entry,
+ crtc, &intel_drrs_debugfs_ctl_fops);
+}
+
+static int intel_drrs_debugfs_type_show(struct seq_file *m, void *unused)
+{
+ struct intel_connector *connector = m->private;
+
+ seq_printf(m, "DRRS type: %s\n",
+ intel_drrs_type_str(intel_panel_drrs_type(connector)));
+
+ return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(intel_drrs_debugfs_type);
+
+void intel_drrs_connector_debugfs_add(struct intel_connector *connector)
+{
+ if (intel_panel_drrs_type(connector) == DRRS_TYPE_NONE)
+ return;
+
+ debugfs_create_file("i915_drrs_type", 0444, connector->base.debugfs_entry,
+ connector, &intel_drrs_debugfs_type_fops);
+}
diff --git a/drivers/gpu/drm/i915/display/intel_drrs.h b/drivers/gpu/drm/i915/display/intel_drrs.h
index 3ad1be1..8ef5f93a 100644
--- a/drivers/gpu/drm/i915/display/intel_drrs.h
+++ b/drivers/gpu/drm/i915/display/intel_drrs.h
@@ -23,6 +23,8 @@ void intel_drrs_invalidate(struct drm_i915_private *dev_priv,
unsigned int frontbuffer_bits);
void intel_drrs_flush(struct drm_i915_private *dev_priv,
unsigned int frontbuffer_bits);
-void intel_crtc_drrs_init(struct intel_crtc *crtc);
+void intel_drrs_crtc_init(struct intel_crtc *crtc);
+void intel_drrs_crtc_debugfs_add(struct intel_crtc *crtc);
+void intel_drrs_connector_debugfs_add(struct intel_connector *connector);
#endif /* __INTEL_DRRS_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_dsb.c b/drivers/gpu/drm/i915/display/intel_dsb.c
index fc9c3e4..1e1c610 100644
--- a/drivers/gpu/drm/i915/display/intel_dsb.c
+++ b/drivers/gpu/drm/i915/display/intel_dsb.c
@@ -7,6 +7,7 @@
#include "gem/i915_gem_internal.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dsb.h"
diff --git a/drivers/gpu/drm/i915/display/intel_dsi_vbt.c b/drivers/gpu/drm/i915/display/intel_dsi_vbt.c
index 75e8cc43..fce69fa 100644
--- a/drivers/gpu/drm/i915/display/intel_dsi_vbt.c
+++ b/drivers/gpu/drm/i915/display/intel_dsi_vbt.c
@@ -137,9 +137,9 @@ static enum port intel_dsi_seq_port_to_port(struct intel_dsi *intel_dsi,
return ffs(intel_dsi->ports) - 1;
if (seq_port) {
- if (intel_dsi->ports & PORT_B)
+ if (intel_dsi->ports & BIT(PORT_B))
return PORT_B;
- else if (intel_dsi->ports & PORT_C)
+ else if (intel_dsi->ports & BIT(PORT_C))
return PORT_C;
}
diff --git a/drivers/gpu/drm/i915/display/intel_dvo.c b/drivers/gpu/drm/i915/display/intel_dvo.c
index 5572e43..c86f989 100644
--- a/drivers/gpu/drm/i915/display/intel_dvo.c
+++ b/drivers/gpu/drm/i915/display/intel_dvo.c
@@ -32,6 +32,7 @@
#include <drm/drm_crtc.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_connector.h"
#include "intel_de.h"
#include "intel_display_types.h"
@@ -491,8 +492,8 @@ void intel_dvo_init(struct drm_i915_private *dev_priv)
intel_encoder->pipe_mask = ~0;
if (dvo->type != INTEL_DVO_CHIP_LVDS)
- intel_encoder->cloneable = (1 << INTEL_OUTPUT_ANALOG) |
- (1 << INTEL_OUTPUT_DVO);
+ intel_encoder->cloneable = BIT(INTEL_OUTPUT_ANALOG) |
+ BIT(INTEL_OUTPUT_DVO);
switch (dvo->type) {
case INTEL_DVO_CHIP_TMDS:
@@ -515,8 +516,6 @@ void intel_dvo_init(struct drm_i915_private *dev_priv)
drm_connector_helper_add(connector,
&intel_dvo_connector_helper_funcs);
connector->display_info.subpixel_order = SubPixelHorizontalRGB;
- connector->interlace_allowed = false;
- connector->doublescan_allowed = false;
intel_connector_attach_encoder(intel_connector, intel_encoder);
if (dvo->type == INTEL_DVO_CHIP_LVDS) {
diff --git a/drivers/gpu/drm/i915/display/intel_dvo_dev.h b/drivers/gpu/drm/i915/display/intel_dvo_dev.h
index 50205f0..ecff7b1 100644
--- a/drivers/gpu/drm/i915/display/intel_dvo_dev.h
+++ b/drivers/gpu/drm/i915/display/intel_dvo_dev.h
@@ -23,12 +23,12 @@
#ifndef __INTEL_DVO_DEV_H__
#define __INTEL_DVO_DEV_H__
-#include <linux/i2c.h>
-
-#include <drm/drm_crtc.h>
-
#include "i915_reg_defs.h"
+enum drm_connector_status;
+struct drm_display_mode;
+struct i2c_adapter;
+
struct intel_dvo_device {
const char *name;
int type;
diff --git a/drivers/gpu/drm/i915/display/intel_fb.c b/drivers/gpu/drm/i915/display/intel_fb.c
index eefa33c5..63137ae 100644
--- a/drivers/gpu/drm/i915/display/intel_fb.c
+++ b/drivers/gpu/drm/i915/display/intel_fb.c
@@ -301,6 +301,19 @@ static bool plane_caps_contain_all(u8 caps, u8 mask)
}
/**
+ * intel_fb_is_tiled_modifier: Check if a modifier is a tiled modifier type
+ * @modifier: Modifier to check
+ *
+ * Returns:
+ * Returns %true if @modifier is a tiled modifier.
+ */
+bool intel_fb_is_tiled_modifier(u64 modifier)
+{
+ return plane_caps_contain_any(lookup_modifier(modifier)->plane_caps,
+ INTEL_PLANE_CAP_TILING_MASK);
+}
+
+/**
* intel_fb_is_ccs_modifier: Check if a modifier is a CCS modifier type
* @modifier: Modifier to check
*
diff --git a/drivers/gpu/drm/i915/display/intel_fb.h b/drivers/gpu/drm/i915/display/intel_fb.h
index 12386f1..4662b81 100644
--- a/drivers/gpu/drm/i915/display/intel_fb.h
+++ b/drivers/gpu/drm/i915/display/intel_fb.h
@@ -29,6 +29,7 @@ struct intel_plane_state;
#define INTEL_PLANE_CAP_TILING_Yf BIT(5)
#define INTEL_PLANE_CAP_TILING_4 BIT(6)
+bool intel_fb_is_tiled_modifier(u64 modifier);
bool intel_fb_is_ccs_modifier(u64 modifier);
bool intel_fb_is_rc_ccs_cc_modifier(u64 modifier);
bool intel_fb_is_mc_ccs_modifier(u64 modifier);
diff --git a/drivers/gpu/drm/i915/display/intel_fb_pin.c b/drivers/gpu/drm/i915/display/intel_fb_pin.c
index 1dddd6a..6900acb 100644
--- a/drivers/gpu/drm/i915/display/intel_fb_pin.c
+++ b/drivers/gpu/drm/i915/display/intel_fb_pin.c
@@ -167,7 +167,6 @@ intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
ret = i915_gem_object_attach_phys(obj, alignment);
else if (!ret && HAS_LMEM(dev_priv))
ret = i915_gem_object_migrate(obj, &ww, INTEL_REGION_LMEM_0);
- /* TODO: Do we need to sync when migration becomes async? */
if (!ret)
ret = i915_gem_object_pin_pages(obj);
if (ret)
diff --git a/drivers/gpu/drm/i915/display/intel_fbc.c b/drivers/gpu/drm/i915/display/intel_fbc.c
index f381753..b5ee5ea 100644
--- a/drivers/gpu/drm/i915/display/intel_fbc.c
+++ b/drivers/gpu/drm/i915/display/intel_fbc.c
@@ -670,6 +670,7 @@ static void intel_fbc_nuke(struct intel_fbc *fbc)
{
struct drm_i915_private *i915 = fbc->i915;
+ lockdep_assert_held(&fbc->lock);
drm_WARN_ON(&i915->drm, fbc->flip_pending);
trace_intel_fbc_nuke(fbc->state.plane);
@@ -679,6 +680,8 @@ static void intel_fbc_nuke(struct intel_fbc *fbc)
static void intel_fbc_activate(struct intel_fbc *fbc)
{
+ lockdep_assert_held(&fbc->lock);
+
intel_fbc_hw_activate(fbc);
intel_fbc_nuke(fbc);
@@ -687,9 +690,7 @@ static void intel_fbc_activate(struct intel_fbc *fbc)
static void intel_fbc_deactivate(struct intel_fbc *fbc, const char *reason)
{
- struct drm_i915_private *i915 = fbc->i915;
-
- drm_WARN_ON(&i915->drm, !mutex_is_locked(&fbc->lock));
+ lockdep_assert_held(&fbc->lock);
if (fbc->active)
intel_fbc_hw_deactivate(fbc);
@@ -1009,7 +1010,8 @@ static bool intel_fbc_is_fence_ok(const struct intel_plane_state *plane_state)
{
struct drm_i915_private *i915 = to_i915(plane_state->uapi.plane->dev);
- /* The use of a CPU fence is one of two ways to detect writes by the
+ /*
+ * The use of a CPU fence is one of two ways to detect writes by the
* CPU to the scanout and trigger updates to the FBC.
*
* The other method is by software tracking (see
@@ -1019,12 +1021,6 @@ static bool intel_fbc_is_fence_ok(const struct intel_plane_state *plane_state)
* Note that is possible for a tiled surface to be unmappable (and
* so have no fence associated with it) due to aperture constraints
* at the time of pinning.
- *
- * FIXME with 90/270 degree rotation we should use the fence on
- * the normal GTT view (the rotated view doesn't even have a
- * fence). Would need changes to the FBC fence Y offset as well.
- * For now this will effectively disable FBC with 90/270 degree
- * rotation.
*/
return DISPLAY_VER(i915) >= 9 ||
(plane_state->flags & PLANE_HAS_FENCE &&
@@ -1187,7 +1183,7 @@ static bool intel_fbc_can_flip_nuke(struct intel_atomic_state *state,
const struct drm_framebuffer *old_fb = old_plane_state->hw.fb;
const struct drm_framebuffer *new_fb = new_plane_state->hw.fb;
- if (drm_atomic_crtc_needs_modeset(&new_crtc_state->uapi))
+ if (intel_crtc_needs_modeset(new_crtc_state))
return false;
if (!intel_fbc_is_ok(old_plane_state) ||
@@ -1227,6 +1223,8 @@ static bool __intel_fbc_pre_update(struct intel_atomic_state *state,
struct intel_fbc *fbc = plane->fbc;
bool need_vblank_wait = false;
+ lockdep_assert_held(&fbc->lock);
+
fbc->flip_pending = true;
if (intel_fbc_can_flip_nuke(state, crtc, plane))
@@ -1284,7 +1282,7 @@ static void __intel_fbc_disable(struct intel_fbc *fbc)
struct drm_i915_private *i915 = fbc->i915;
struct intel_plane *plane = fbc->state.plane;
- drm_WARN_ON(&i915->drm, !mutex_is_locked(&fbc->lock));
+ lockdep_assert_held(&fbc->lock);
drm_WARN_ON(&i915->drm, fbc->active);
drm_dbg_kms(&i915->drm, "Disabling FBC on [PLANE:%d:%s]\n",
@@ -1299,9 +1297,9 @@ static void __intel_fbc_disable(struct intel_fbc *fbc)
static void __intel_fbc_post_update(struct intel_fbc *fbc)
{
- struct drm_i915_private *i915 = fbc->i915;
+ lockdep_assert_held(&fbc->lock);
- drm_WARN_ON(&i915->drm, !mutex_is_locked(&fbc->lock));
+ fbc->flip_pending = false;
if (!fbc->busy_bits)
intel_fbc_activate(fbc);
@@ -1324,10 +1322,8 @@ void intel_fbc_post_update(struct intel_atomic_state *state,
mutex_lock(&fbc->lock);
- if (fbc->state.plane == plane) {
- fbc->flip_pending = false;
+ if (fbc->state.plane == plane)
__intel_fbc_post_update(fbc);
- }
mutex_unlock(&fbc->lock);
}
@@ -1437,6 +1433,8 @@ static void __intel_fbc_enable(struct intel_atomic_state *state,
intel_atomic_get_new_plane_state(state, plane);
struct intel_fbc *fbc = plane->fbc;
+ lockdep_assert_held(&fbc->lock);
+
if (fbc->state.plane) {
if (fbc->state.plane != plane)
return;
@@ -1522,7 +1520,8 @@ void intel_fbc_update(struct intel_atomic_state *state,
mutex_lock(&fbc->lock);
- if (crtc_state->update_pipe && plane_state->no_fbc_reason) {
+ if (intel_crtc_needs_fastset(crtc_state) &&
+ plane_state->no_fbc_reason) {
if (fbc->state.plane == plane)
__intel_fbc_disable(fbc);
} else {
diff --git a/drivers/gpu/drm/i915/display/intel_fbdev.c b/drivers/gpu/drm/i915/display/intel_fbdev.c
index 112aa04..5575d7a 100644
--- a/drivers/gpu/drm/i915/display/intel_fbdev.c
+++ b/drivers/gpu/drm/i915/display/intel_fbdev.c
@@ -124,6 +124,8 @@ static const struct fb_ops intelfb_ops = {
.owner = THIS_MODULE,
DRM_FB_HELPER_DEFAULT_OPS,
.fb_set_par = intel_fbdev_set_par,
+ .fb_read = drm_fb_helper_cfb_read,
+ .fb_write = drm_fb_helper_cfb_write,
.fb_fillrect = drm_fb_helper_cfb_fillrect,
.fb_copyarea = drm_fb_helper_cfb_copyarea,
.fb_imageblit = drm_fb_helper_cfb_imageblit,
@@ -175,7 +177,7 @@ static int intelfb_alloc(struct drm_fb_helper *helper,
}
if (IS_ERR(obj)) {
- drm_err(&dev_priv->drm, "failed to allocate framebuffer\n");
+ drm_err(&dev_priv->drm, "failed to allocate framebuffer (%pe)\n", obj);
return PTR_ERR(obj);
}
@@ -254,9 +256,9 @@ static int intelfb_create(struct drm_fb_helper *helper,
goto out_unlock;
}
- info = drm_fb_helper_alloc_fbi(helper);
+ info = drm_fb_helper_alloc_info(helper);
if (IS_ERR(info)) {
- drm_err(&dev_priv->drm, "Failed to allocate fb_info\n");
+ drm_err(&dev_priv->drm, "Failed to allocate fb_info (%pe)\n", info);
ret = PTR_ERR(info);
goto out_unpin;
}
@@ -291,7 +293,7 @@ static int intelfb_create(struct drm_fb_helper *helper,
vaddr = i915_vma_pin_iomap(vma);
if (IS_ERR(vaddr)) {
drm_err(&dev_priv->drm,
- "Failed to remap framebuffer into virtual memory\n");
+ "Failed to remap framebuffer into virtual memory (%pe)\n", vaddr);
ret = PTR_ERR(vaddr);
goto out_unpin;
}
@@ -584,7 +586,7 @@ void intel_fbdev_unregister(struct drm_i915_private *dev_priv)
if (!current_is_async())
intel_fbdev_sync(ifbdev);
- drm_fb_helper_unregister_fbi(&ifbdev->helper);
+ drm_fb_helper_unregister_info(&ifbdev->helper);
}
void intel_fbdev_fini(struct drm_i915_private *dev_priv)
@@ -627,7 +629,7 @@ void intel_fbdev_set_suspend(struct drm_device *dev, int state, bool synchronous
if (!ifbdev || !ifbdev->vma)
goto set_suspend;
- info = ifbdev->helper.fbdev;
+ info = ifbdev->helper.info;
if (synchronous) {
/* Flush any pending work to turn the console on, and then
diff --git a/drivers/gpu/drm/i915/display/intel_fdi.c b/drivers/gpu/drm/i915/display/intel_fdi.c
index 7f47e5c..063f1da 100644
--- a/drivers/gpu/drm/i915/display/intel_fdi.c
+++ b/drivers/gpu/drm/i915/display/intel_fdi.c
@@ -5,6 +5,7 @@
#include <linux/string_helpers.h>
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_crtc.h"
#include "intel_ddi.h"
diff --git a/drivers/gpu/drm/i915/display/intel_fifo_underrun.h b/drivers/gpu/drm/i915/display/intel_fifo_underrun.h
index e04f22a..2e47d7d 100644
--- a/drivers/gpu/drm/i915/display/intel_fifo_underrun.h
+++ b/drivers/gpu/drm/i915/display/intel_fifo_underrun.h
@@ -8,9 +8,8 @@
#include <linux/types.h>
-#include "intel_display.h"
-
struct drm_i915_private;
+enum pipe;
bool intel_set_cpu_fifo_underrun_reporting(struct drm_i915_private *dev_priv,
enum pipe pipe, bool enable);
diff --git a/drivers/gpu/drm/i915/display/intel_frontbuffer.c b/drivers/gpu/drm/i915/display/intel_frontbuffer.c
index d80e3e8..17a7aa8 100644
--- a/drivers/gpu/drm/i915/display/intel_frontbuffer.c
+++ b/drivers/gpu/drm/i915/display/intel_frontbuffer.c
@@ -88,7 +88,7 @@ static void frontbuffer_flush(struct drm_i915_private *i915,
if (!frontbuffer_bits)
return;
- trace_intel_frontbuffer_flush(frontbuffer_bits, origin);
+ trace_intel_frontbuffer_flush(i915, frontbuffer_bits, origin);
might_sleep();
intel_drrs_flush(i915, frontbuffer_bits);
@@ -176,7 +176,7 @@ void __intel_fb_invalidate(struct intel_frontbuffer *front,
spin_unlock(&i915->display.fb_tracking.lock);
}
- trace_intel_frontbuffer_invalidate(frontbuffer_bits, origin);
+ trace_intel_frontbuffer_invalidate(i915, frontbuffer_bits, origin);
might_sleep();
intel_psr_invalidate(i915, frontbuffer_bits, origin);
diff --git a/drivers/gpu/drm/i915/display/intel_global_state.c b/drivers/gpu/drm/i915/display/intel_global_state.c
index 7a19215..02b593b 100644
--- a/drivers/gpu/drm/i915/display/intel_global_state.c
+++ b/drivers/gpu/drm/i915/display/intel_global_state.c
@@ -45,14 +45,14 @@ void intel_atomic_global_obj_init(struct drm_i915_private *dev_priv,
obj->state = state;
obj->funcs = funcs;
- list_add_tail(&obj->head, &dev_priv->global_obj_list);
+ list_add_tail(&obj->head, &dev_priv->display.global.obj_list);
}
void intel_atomic_global_obj_cleanup(struct drm_i915_private *dev_priv)
{
struct intel_global_obj *obj, *next;
- list_for_each_entry_safe(obj, next, &dev_priv->global_obj_list, head) {
+ list_for_each_entry_safe(obj, next, &dev_priv->display.global.obj_list, head) {
list_del(&obj->head);
drm_WARN_ON(&dev_priv->drm, kref_read(&obj->state->ref) != 1);
diff --git a/drivers/gpu/drm/i915/display/intel_global_state.h b/drivers/gpu/drm/i915/display/intel_global_state.h
index 1f16fa3..f01ee0b 100644
--- a/drivers/gpu/drm/i915/display/intel_global_state.h
+++ b/drivers/gpu/drm/i915/display/intel_global_state.h
@@ -27,7 +27,7 @@ struct intel_global_obj {
};
#define intel_for_each_global_obj(obj, dev_priv) \
- list_for_each_entry(obj, &(dev_priv)->global_obj_list, head)
+ list_for_each_entry(obj, &(dev_priv)->display.global.obj_list, head)
#define for_each_new_global_obj_in_state(__state, obj, new_obj_state, __i) \
for ((__i) = 0; \
diff --git a/drivers/gpu/drm/i915/display/intel_gmbus.c b/drivers/gpu/drm/i915/display/intel_gmbus.c
index 74443f5..a5840a2 100644
--- a/drivers/gpu/drm/i915/display/intel_gmbus.c
+++ b/drivers/gpu/drm/i915/display/intel_gmbus.c
@@ -34,6 +34,8 @@
#include <drm/display/drm_hdcp_helper.h>
#include "i915_drv.h"
+#include "i915_irq.h"
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_gmbus.h"
@@ -49,9 +51,27 @@ struct intel_gmbus {
struct drm_i915_private *i915;
};
+enum gmbus_gpio {
+ GPIOA,
+ GPIOB,
+ GPIOC,
+ GPIOD,
+ GPIOE,
+ GPIOF,
+ GPIOG,
+ GPIOH,
+ __GPIOI_UNUSED,
+ GPIOJ,
+ GPIOK,
+ GPIOL,
+ GPIOM,
+ GPION,
+ GPIOO,
+};
+
struct gmbus_pin {
const char *name;
- enum i915_gpio gpio;
+ enum gmbus_gpio gpio;
};
/* Map gmbus pin pairs to names and registers. */
diff --git a/drivers/gpu/drm/i915/display/intel_hdcp_regs.h b/drivers/gpu/drm/i915/display/intel_hdcp_regs.h
index 2a3733e..8023c85 100644
--- a/drivers/gpu/drm/i915/display/intel_hdcp_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_hdcp_regs.h
@@ -6,7 +6,7 @@
#ifndef __INTEL_HDCP_REGS_H__
#define __INTEL_HDCP_REGS_H__
-#include "i915_reg_defs.h"
+#include "intel_display_reg_defs.h"
/* HDCP Key Registers */
#define HDCP_KEY_CONF _MMIO(0x66c00)
diff --git a/drivers/gpu/drm/i915/display/intel_hdmi.c b/drivers/gpu/drm/i915/display/intel_hdmi.c
index 7816b2a..bac85d8 100644
--- a/drivers/gpu/drm/i915/display/intel_hdmi.c
+++ b/drivers/gpu/drm/i915/display/intel_hdmi.c
@@ -42,6 +42,7 @@
#include "i915_debugfs.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_connector.h"
#include "intel_ddi.h"
@@ -2057,13 +2058,6 @@ static bool hdmi_bpc_possible(const struct intel_crtc_state *crtc_state, int bpc
if (!intel_hdmi_source_bpc_possible(dev_priv, bpc))
return false;
- /*
- * HDMI deep color affects the clocks, so it's only possible
- * when not cloning with other encoder types.
- */
- if (bpc > 8 && crtc_state->output_types != BIT(INTEL_OUTPUT_HDMI))
- return false;
-
/* Display Wa_1405510057:icl,ehl */
if (intel_hdmi_is_ycbcr420(crtc_state) &&
bpc == 10 && DISPLAY_VER(dev_priv) == 11 &&
@@ -2190,9 +2184,13 @@ static bool intel_hdmi_has_audio(struct intel_encoder *encoder,
}
static enum intel_output_format
-intel_hdmi_output_format(struct intel_connector *connector,
+intel_hdmi_output_format(const struct intel_crtc_state *crtc_state,
+ struct intel_connector *connector,
bool ycbcr_420_output)
{
+ if (!crtc_state->has_hdmi_sink)
+ return INTEL_OUTPUT_FORMAT_RGB;
+
if (connector->base.ycbcr_420_allowed && ycbcr_420_output)
return INTEL_OUTPUT_FORMAT_YCBCR420;
else
@@ -2211,7 +2209,8 @@ static int intel_hdmi_compute_output_format(struct intel_encoder *encoder,
bool ycbcr_420_only = drm_mode_is_420_only(info, adjusted_mode);
int ret;
- crtc_state->output_format = intel_hdmi_output_format(connector, ycbcr_420_only);
+ crtc_state->output_format =
+ intel_hdmi_output_format(crtc_state, connector, ycbcr_420_only);
if (ycbcr_420_only && !intel_hdmi_is_ycbcr420(crtc_state)) {
drm_dbg_kms(&i915->drm,
@@ -2226,13 +2225,19 @@ static int intel_hdmi_compute_output_format(struct intel_encoder *encoder,
!drm_mode_is_420_also(info, adjusted_mode))
return ret;
- crtc_state->output_format = intel_hdmi_output_format(connector, true);
+ crtc_state->output_format = intel_hdmi_output_format(crtc_state, connector, true);
ret = intel_hdmi_compute_clock(encoder, crtc_state, respect_downstream_limits);
}
return ret;
}
+static bool intel_hdmi_is_cloned(const struct intel_crtc_state *crtc_state)
+{
+ return crtc_state->uapi.encoder_mask &&
+ !is_power_of_2(crtc_state->uapi.encoder_mask);
+}
+
int intel_hdmi_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state)
@@ -2248,8 +2253,9 @@ int intel_hdmi_compute_config(struct intel_encoder *encoder,
return -EINVAL;
pipe_config->output_format = INTEL_OUTPUT_FORMAT_RGB;
- pipe_config->has_hdmi_sink = intel_has_hdmi_sink(intel_hdmi,
- conn_state);
+ pipe_config->has_hdmi_sink =
+ intel_has_hdmi_sink(intel_hdmi, conn_state) &&
+ !intel_hdmi_is_cloned(pipe_config);
if (pipe_config->has_hdmi_sink)
pipe_config->has_infoframe = true;
@@ -2257,9 +2263,6 @@ int intel_hdmi_compute_config(struct intel_encoder *encoder,
if (adjusted_mode->flags & DRM_MODE_FLAG_DBLCLK)
pipe_config->pixel_multiplier = 2;
- if (HAS_PCH_SPLIT(dev_priv) && !HAS_DDI(dev_priv))
- pipe_config->has_pch_encoder = true;
-
pipe_config->has_audio =
intel_hdmi_has_audio(encoder, pipe_config, conn_state);
@@ -2355,7 +2358,7 @@ intel_hdmi_unset_edid(struct drm_connector *connector)
}
static void
-intel_hdmi_dp_dual_mode_detect(struct drm_connector *connector, bool has_edid)
+intel_hdmi_dp_dual_mode_detect(struct drm_connector *connector)
{
struct drm_i915_private *dev_priv = to_i915(connector->dev);
struct intel_hdmi *hdmi = intel_attached_hdmi(to_intel_connector(connector));
@@ -2371,16 +2374,10 @@ intel_hdmi_dp_dual_mode_detect(struct drm_connector *connector, bool has_edid)
* CONFIG1 pin, but no such luck on our hardware.
*
* The only method left to us is to check the VBT to see
- * if the port is a dual mode capable DP port. But let's
- * only do that when we sucesfully read the EDID, to avoid
- * confusing log messages about DP dual mode adaptors when
- * there's nothing connected to the port.
+ * if the port is a dual mode capable DP port.
*/
if (type == DRM_DP_DUAL_MODE_UNKNOWN) {
- /* An overridden EDID imply that we want this port for testing.
- * Make sure not to set limits for that port.
- */
- if (has_edid && !connector->override_edid &&
+ if (!connector->force &&
intel_bios_is_port_dp_dual_mode(dev_priv, port)) {
drm_dbg_kms(&dev_priv->drm,
"Assuming DP dual mode adaptor presence based on VBT\n");
@@ -2435,18 +2432,18 @@ intel_hdmi_set_edid(struct drm_connector *connector)
intel_gmbus_force_bit(i2c, false);
}
- intel_hdmi_dp_dual_mode_detect(connector, edid != NULL);
-
- intel_display_power_put(dev_priv, POWER_DOMAIN_GMBUS, wakeref);
-
to_intel_connector(connector)->detect_edid = edid;
if (edid && edid->input & DRM_EDID_INPUT_DIGITAL) {
intel_hdmi->has_audio = drm_detect_monitor_audio(edid);
intel_hdmi->has_hdmi_sink = drm_detect_hdmi_monitor(edid);
+ intel_hdmi_dp_dual_mode_detect(connector);
+
connected = true;
}
+ intel_display_power_put(dev_priv, POWER_DOMAIN_GMBUS, wakeref);
+
cec_notifier_set_phys_addr_from_edid(intel_hdmi->cec_notifier, edid);
return connected;
@@ -2956,9 +2953,8 @@ void intel_hdmi_init_connector(struct intel_digital_port *dig_port,
ddc);
drm_connector_helper_add(connector, &intel_hdmi_connector_helper_funcs);
- connector->interlace_allowed = 1;
- connector->doublescan_allowed = 0;
- connector->stereo_allowed = 1;
+ connector->interlace_allowed = true;
+ connector->stereo_allowed = true;
if (DISPLAY_VER(dev_priv) >= 10)
connector->ycbcr_420_allowed = true;
diff --git a/drivers/gpu/drm/i915/display/intel_hdmi.h b/drivers/gpu/drm/i915/display/intel_hdmi.h
index 93f65a9..774dda2 100644
--- a/drivers/gpu/drm/i915/display/intel_hdmi.h
+++ b/drivers/gpu/drm/i915/display/intel_hdmi.h
@@ -6,20 +6,20 @@
#ifndef __INTEL_HDMI_H__
#define __INTEL_HDMI_H__
-#include <linux/hdmi.h>
#include <linux/types.h>
+enum hdmi_infoframe_type;
+enum port;
struct drm_connector;
+struct drm_connector_state;
struct drm_encoder;
struct drm_i915_private;
struct intel_connector;
+struct intel_crtc_state;
struct intel_digital_port;
struct intel_encoder;
-struct intel_crtc_state;
struct intel_hdmi;
-struct drm_connector_state;
union hdmi_infoframe;
-enum port;
void intel_hdmi_init_connector(struct intel_digital_port *dig_port,
struct intel_connector *intel_connector);
diff --git a/drivers/gpu/drm/i915/display/intel_hotplug.c b/drivers/gpu/drm/i915/display/intel_hotplug.c
index f7a2f48..907ab75 100644
--- a/drivers/gpu/drm/i915/display/intel_hotplug.c
+++ b/drivers/gpu/drm/i915/display/intel_hotplug.c
@@ -90,6 +90,9 @@ enum hpd_pin intel_hpd_pin_default(struct drm_i915_private *dev_priv,
return HPD_PORT_A + port - PORT_A;
}
+/* Threshold == 5 for long IRQs, 50 for short */
+#define HPD_STORM_DEFAULT_THRESHOLD 50
+
#define HPD_STORM_DETECT_PERIOD 1000
#define HPD_STORM_REENABLE_DELAY (2 * 60 * 1000)
#define HPD_RETRY_DELAY 1000
@@ -175,14 +178,13 @@ static bool intel_hpd_irq_storm_detect(struct drm_i915_private *dev_priv,
static void
intel_hpd_irq_storm_switch_to_polling(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector_list_iter conn_iter;
struct intel_connector *connector;
bool hpd_disabled = false;
lockdep_assert_held(&dev_priv->irq_lock);
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
for_each_intel_connector_iter(connector, &conn_iter) {
enum hpd_pin pin;
@@ -208,7 +210,7 @@ intel_hpd_irq_storm_switch_to_polling(struct drm_i915_private *dev_priv)
/* Enable polling and queue hotplug re-enabling. */
if (hpd_disabled) {
- drm_kms_helper_poll_enable(dev);
+ drm_kms_helper_poll_enable(&dev_priv->drm);
mod_delayed_work(system_wq, &dev_priv->display.hotplug.reenable_work,
msecs_to_jiffies(HPD_STORM_REENABLE_DELAY));
}
@@ -219,7 +221,6 @@ static void intel_hpd_irq_storm_reenable_work(struct work_struct *work)
struct drm_i915_private *dev_priv =
container_of(work, typeof(*dev_priv),
display.hotplug.reenable_work.work);
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector_list_iter conn_iter;
struct intel_connector *connector;
intel_wakeref_t wakeref;
@@ -229,7 +230,7 @@ static void intel_hpd_irq_storm_reenable_work(struct work_struct *work)
spin_lock_irq(&dev_priv->irq_lock);
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
for_each_intel_connector_iter(connector, &conn_iter) {
pin = intel_connector_hpd_pin(connector);
if (pin == HPD_NONE ||
@@ -367,14 +368,13 @@ static void i915_hotplug_work_func(struct work_struct *work)
struct drm_i915_private *dev_priv =
container_of(work, struct drm_i915_private,
display.hotplug.hotplug_work.work);
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector_list_iter conn_iter;
struct intel_connector *connector;
u32 changed = 0, retry = 0;
u32 hpd_event_bits;
u32 hpd_retry_bits;
- mutex_lock(&dev->mode_config.mutex);
+ mutex_lock(&dev_priv->drm.mode_config.mutex);
drm_dbg_kms(&dev_priv->drm, "running encoder hotplug functions\n");
spin_lock_irq(&dev_priv->irq_lock);
@@ -389,7 +389,7 @@ static void i915_hotplug_work_func(struct work_struct *work)
spin_unlock_irq(&dev_priv->irq_lock);
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
for_each_intel_connector_iter(connector, &conn_iter) {
enum hpd_pin pin;
u32 hpd_bit;
@@ -426,10 +426,10 @@ static void i915_hotplug_work_func(struct work_struct *work)
}
}
drm_connector_list_iter_end(&conn_iter);
- mutex_unlock(&dev->mode_config.mutex);
+ mutex_unlock(&dev_priv->drm.mode_config.mutex);
if (changed)
- drm_kms_helper_hotplug_event(dev);
+ drm_kms_helper_hotplug_event(&dev_priv->drm);
/* Remove shared HPD pins that have changed */
retry &= ~changed;
@@ -612,16 +612,15 @@ static void i915_hpd_poll_init_work(struct work_struct *work)
struct drm_i915_private *dev_priv =
container_of(work, struct drm_i915_private,
display.hotplug.poll_init_work);
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector_list_iter conn_iter;
struct intel_connector *connector;
bool enabled;
- mutex_lock(&dev->mode_config.mutex);
+ mutex_lock(&dev_priv->drm.mode_config.mutex);
enabled = READ_ONCE(dev_priv->display.hotplug.poll_enabled);
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
for_each_intel_connector_iter(connector, &conn_iter) {
enum hpd_pin pin;
@@ -638,16 +637,16 @@ static void i915_hpd_poll_init_work(struct work_struct *work)
drm_connector_list_iter_end(&conn_iter);
if (enabled)
- drm_kms_helper_poll_enable(dev);
+ drm_kms_helper_poll_enable(&dev_priv->drm);
- mutex_unlock(&dev->mode_config.mutex);
+ mutex_unlock(&dev_priv->drm.mode_config.mutex);
/*
* We might have missed any hotplugs that happened while we were
* in the middle of disabling polling
*/
if (!enabled)
- drm_helper_hpd_irq_event(dev);
+ drm_helper_hpd_irq_event(&dev_priv->drm);
}
/**
@@ -711,14 +710,23 @@ void intel_hpd_poll_disable(struct drm_i915_private *dev_priv)
schedule_work(&dev_priv->display.hotplug.poll_init_work);
}
-void intel_hpd_init_work(struct drm_i915_private *dev_priv)
+void intel_hpd_init_early(struct drm_i915_private *i915)
{
- INIT_DELAYED_WORK(&dev_priv->display.hotplug.hotplug_work,
+ INIT_DELAYED_WORK(&i915->display.hotplug.hotplug_work,
i915_hotplug_work_func);
- INIT_WORK(&dev_priv->display.hotplug.dig_port_work, i915_digport_work_func);
- INIT_WORK(&dev_priv->display.hotplug.poll_init_work, i915_hpd_poll_init_work);
- INIT_DELAYED_WORK(&dev_priv->display.hotplug.reenable_work,
+ INIT_WORK(&i915->display.hotplug.dig_port_work, i915_digport_work_func);
+ INIT_WORK(&i915->display.hotplug.poll_init_work, i915_hpd_poll_init_work);
+ INIT_DELAYED_WORK(&i915->display.hotplug.reenable_work,
intel_hpd_irq_storm_reenable_work);
+
+ i915->display.hotplug.hpd_storm_threshold = HPD_STORM_DEFAULT_THRESHOLD;
+ /* If we have MST support, we want to avoid doing short HPD IRQ storm
+ * detection, as short HPD storms will occur as a natural part of
+ * sideband messaging with MST.
+ * On older platforms however, IRQ storms can occur with both long and
+ * short pulses, as seen on some G4x systems.
+ */
+ i915->display.hotplug.hpd_short_storm_enabled = !HAS_DP_MST(i915);
}
void intel_hpd_cancel_work(struct drm_i915_private *dev_priv)
@@ -767,3 +775,169 @@ void intel_hpd_enable(struct drm_i915_private *dev_priv, enum hpd_pin pin)
dev_priv->display.hotplug.stats[pin].state = HPD_ENABLED;
spin_unlock_irq(&dev_priv->irq_lock);
}
+
+static int i915_hpd_storm_ctl_show(struct seq_file *m, void *data)
+{
+ struct drm_i915_private *dev_priv = m->private;
+ struct intel_hotplug *hotplug = &dev_priv->display.hotplug;
+
+ /* Synchronize with everything first in case there's been an HPD
+ * storm, but we haven't finished handling it in the kernel yet
+ */
+ intel_synchronize_irq(dev_priv);
+ flush_work(&dev_priv->display.hotplug.dig_port_work);
+ flush_delayed_work(&dev_priv->display.hotplug.hotplug_work);
+
+ seq_printf(m, "Threshold: %d\n", hotplug->hpd_storm_threshold);
+ seq_printf(m, "Detected: %s\n",
+ str_yes_no(delayed_work_pending(&hotplug->reenable_work)));
+
+ return 0;
+}
+
+static ssize_t i915_hpd_storm_ctl_write(struct file *file,
+ const char __user *ubuf, size_t len,
+ loff_t *offp)
+{
+ struct seq_file *m = file->private_data;
+ struct drm_i915_private *dev_priv = m->private;
+ struct intel_hotplug *hotplug = &dev_priv->display.hotplug;
+ unsigned int new_threshold;
+ int i;
+ char *newline;
+ char tmp[16];
+
+ if (len >= sizeof(tmp))
+ return -EINVAL;
+
+ if (copy_from_user(tmp, ubuf, len))
+ return -EFAULT;
+
+ tmp[len] = '\0';
+
+ /* Strip newline, if any */
+ newline = strchr(tmp, '\n');
+ if (newline)
+ *newline = '\0';
+
+ if (strcmp(tmp, "reset") == 0)
+ new_threshold = HPD_STORM_DEFAULT_THRESHOLD;
+ else if (kstrtouint(tmp, 10, &new_threshold) != 0)
+ return -EINVAL;
+
+ if (new_threshold > 0)
+ drm_dbg_kms(&dev_priv->drm,
+ "Setting HPD storm detection threshold to %d\n",
+ new_threshold);
+ else
+ drm_dbg_kms(&dev_priv->drm, "Disabling HPD storm detection\n");
+
+ spin_lock_irq(&dev_priv->irq_lock);
+ hotplug->hpd_storm_threshold = new_threshold;
+ /* Reset the HPD storm stats so we don't accidentally trigger a storm */
+ for_each_hpd_pin(i)
+ hotplug->stats[i].count = 0;
+ spin_unlock_irq(&dev_priv->irq_lock);
+
+ /* Re-enable hpd immediately if we were in an irq storm */
+ flush_delayed_work(&dev_priv->display.hotplug.reenable_work);
+
+ return len;
+}
+
+static int i915_hpd_storm_ctl_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, i915_hpd_storm_ctl_show, inode->i_private);
+}
+
+static const struct file_operations i915_hpd_storm_ctl_fops = {
+ .owner = THIS_MODULE,
+ .open = i915_hpd_storm_ctl_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+ .write = i915_hpd_storm_ctl_write
+};
+
+static int i915_hpd_short_storm_ctl_show(struct seq_file *m, void *data)
+{
+ struct drm_i915_private *dev_priv = m->private;
+
+ seq_printf(m, "Enabled: %s\n",
+ str_yes_no(dev_priv->display.hotplug.hpd_short_storm_enabled));
+
+ return 0;
+}
+
+static int
+i915_hpd_short_storm_ctl_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, i915_hpd_short_storm_ctl_show,
+ inode->i_private);
+}
+
+static ssize_t i915_hpd_short_storm_ctl_write(struct file *file,
+ const char __user *ubuf,
+ size_t len, loff_t *offp)
+{
+ struct seq_file *m = file->private_data;
+ struct drm_i915_private *dev_priv = m->private;
+ struct intel_hotplug *hotplug = &dev_priv->display.hotplug;
+ char *newline;
+ char tmp[16];
+ int i;
+ bool new_state;
+
+ if (len >= sizeof(tmp))
+ return -EINVAL;
+
+ if (copy_from_user(tmp, ubuf, len))
+ return -EFAULT;
+
+ tmp[len] = '\0';
+
+ /* Strip newline, if any */
+ newline = strchr(tmp, '\n');
+ if (newline)
+ *newline = '\0';
+
+ /* Reset to the "default" state for this system */
+ if (strcmp(tmp, "reset") == 0)
+ new_state = !HAS_DP_MST(dev_priv);
+ else if (kstrtobool(tmp, &new_state) != 0)
+ return -EINVAL;
+
+ drm_dbg_kms(&dev_priv->drm, "%sabling HPD short storm detection\n",
+ new_state ? "En" : "Dis");
+
+ spin_lock_irq(&dev_priv->irq_lock);
+ hotplug->hpd_short_storm_enabled = new_state;
+ /* Reset the HPD storm stats so we don't accidentally trigger a storm */
+ for_each_hpd_pin(i)
+ hotplug->stats[i].count = 0;
+ spin_unlock_irq(&dev_priv->irq_lock);
+
+ /* Re-enable hpd immediately if we were in an irq storm */
+ flush_delayed_work(&dev_priv->display.hotplug.reenable_work);
+
+ return len;
+}
+
+static const struct file_operations i915_hpd_short_storm_ctl_fops = {
+ .owner = THIS_MODULE,
+ .open = i915_hpd_short_storm_ctl_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+ .write = i915_hpd_short_storm_ctl_write,
+};
+
+void intel_hpd_debugfs_register(struct drm_i915_private *i915)
+{
+ struct drm_minor *minor = i915->drm.primary;
+
+ debugfs_create_file("i915_hpd_storm_ctl", 0644, minor->debugfs_root,
+ i915, &i915_hpd_storm_ctl_fops);
+ debugfs_create_file("i915_hpd_short_storm_ctl", 0644, minor->debugfs_root,
+ i915, &i915_hpd_short_storm_ctl_fops);
+}
diff --git a/drivers/gpu/drm/i915/display/intel_hotplug.h b/drivers/gpu/drm/i915/display/intel_hotplug.h
index b87e95d..424ae5d 100644
--- a/drivers/gpu/drm/i915/display/intel_hotplug.h
+++ b/drivers/gpu/drm/i915/display/intel_hotplug.h
@@ -22,11 +22,12 @@ void intel_hpd_irq_handler(struct drm_i915_private *dev_priv,
u32 pin_mask, u32 long_mask);
void intel_hpd_trigger_irq(struct intel_digital_port *dig_port);
void intel_hpd_init(struct drm_i915_private *dev_priv);
-void intel_hpd_init_work(struct drm_i915_private *dev_priv);
+void intel_hpd_init_early(struct drm_i915_private *i915);
void intel_hpd_cancel_work(struct drm_i915_private *dev_priv);
enum hpd_pin intel_hpd_pin_default(struct drm_i915_private *dev_priv,
enum port port);
bool intel_hpd_disable(struct drm_i915_private *dev_priv, enum hpd_pin pin);
void intel_hpd_enable(struct drm_i915_private *dev_priv, enum hpd_pin pin);
+void intel_hpd_debugfs_register(struct drm_i915_private *i915);
#endif /* __INTEL_HOTPLUG_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_hti.c b/drivers/gpu/drm/i915/display/intel_hti.c
new file mode 100644
index 0000000..12a1f4c
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_hti.c
@@ -0,0 +1,38 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#include "i915_drv.h"
+#include "intel_de.h"
+#include "intel_display.h"
+#include "intel_hti.h"
+#include "intel_hti_regs.h"
+
+void intel_hti_init(struct drm_i915_private *i915)
+{
+ /*
+ * If the platform has HTI, we need to find out whether it has reserved
+ * any display resources before we create our display outputs.
+ */
+ if (INTEL_INFO(i915)->display.has_hti)
+ i915->display.hti.state = intel_de_read(i915, HDPORT_STATE);
+}
+
+bool intel_hti_uses_phy(struct drm_i915_private *i915, enum phy phy)
+{
+ return i915->display.hti.state & HDPORT_ENABLED &&
+ i915->display.hti.state & HDPORT_DDI_USED(phy);
+}
+
+u32 intel_hti_dpll_mask(struct drm_i915_private *i915)
+{
+ if (!(i915->display.hti.state & HDPORT_ENABLED))
+ return 0;
+
+ /*
+ * Note: This is subtle. The values must coincide with what's defined
+ * for the platform.
+ */
+ return REG_FIELD_GET(HDPORT_DPLL_USED_MASK, i915->display.hti.state);
+}
diff --git a/drivers/gpu/drm/i915/display/intel_hti.h b/drivers/gpu/drm/i915/display/intel_hti.h
new file mode 100644
index 0000000..2893d66
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_hti.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#ifndef __INTEL_HTI_H__
+#define __INTEL_HTI_H__
+
+#include <linux/types.h>
+
+struct drm_i915_private;
+enum phy;
+
+void intel_hti_init(struct drm_i915_private *i915);
+bool intel_hti_uses_phy(struct drm_i915_private *i915, enum phy phy);
+u32 intel_hti_dpll_mask(struct drm_i915_private *i915);
+
+#endif /* __INTEL_HTI_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_hti_regs.h b/drivers/gpu/drm/i915/display/intel_hti_regs.h
new file mode 100644
index 0000000..e206f28
--- /dev/null
+++ b/drivers/gpu/drm/i915/display/intel_hti_regs.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#ifndef __INTEL_HTI_REGS_H__
+#define __INTEL_HTI_REGS_H__
+
+#include "i915_reg_defs.h"
+
+#define HDPORT_STATE _MMIO(0x45050)
+#define HDPORT_DPLL_USED_MASK REG_GENMASK(15, 12)
+#define HDPORT_DDI_USED(phy) REG_BIT(2 * (phy) + 1)
+#define HDPORT_ENABLED REG_BIT(0)
+
+#endif /* __INTEL_HTI_REGS_H__ */
diff --git a/drivers/gpu/drm/i915/display/intel_lpe_audio.c b/drivers/gpu/drm/i915/display/intel_lpe_audio.c
index dca6003..8aaaef4 100644
--- a/drivers/gpu/drm/i915/display/intel_lpe_audio.c
+++ b/drivers/gpu/drm/i915/display/intel_lpe_audio.c
@@ -71,6 +71,8 @@
#include <drm/intel_lpe_audio.h>
#include "i915_drv.h"
+#include "i915_irq.h"
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_lpe_audio.h"
#include "intel_pci_config.h"
@@ -80,8 +82,7 @@
static struct platform_device *
lpe_audio_platdev_create(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
- struct pci_dev *pdev = to_pci_dev(dev->dev);
+ struct pci_dev *pdev = to_pci_dev(dev_priv->drm.dev);
struct platform_device_info pinfo = {};
struct resource *rsc;
struct platform_device *platdev;
@@ -101,14 +102,14 @@ lpe_audio_platdev_create(struct drm_i915_private *dev_priv)
rsc[0].flags = IORESOURCE_IRQ;
rsc[0].name = "hdmi-lpe-audio-irq";
- rsc[1].start = pci_resource_start(pdev, GTTMMADR_BAR) +
+ rsc[1].start = pci_resource_start(pdev, GEN4_GTTMMADR_BAR) +
I915_HDMI_LPE_AUDIO_BASE;
- rsc[1].end = pci_resource_start(pdev, GTTMMADR_BAR) +
+ rsc[1].end = pci_resource_start(pdev, GEN4_GTTMMADR_BAR) +
I915_HDMI_LPE_AUDIO_BASE + I915_HDMI_LPE_AUDIO_SIZE - 1;
rsc[1].flags = IORESOURCE_MEM;
rsc[1].name = "hdmi-lpe-audio-mmio";
- pinfo.parent = dev->dev;
+ pinfo.parent = dev_priv->drm.dev;
pinfo.name = "hdmi-lpe-audio";
pinfo.id = -1;
pinfo.res = rsc;
diff --git a/drivers/gpu/drm/i915/display/intel_lspcon.c b/drivers/gpu/drm/i915/display/intel_lspcon.c
index 15d59de..9ff1c0b 100644
--- a/drivers/gpu/drm/i915/display/intel_lspcon.c
+++ b/drivers/gpu/drm/i915/display/intel_lspcon.c
@@ -28,6 +28,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_edid.h>
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_dp.h"
diff --git a/drivers/gpu/drm/i915/display/intel_lvds.c b/drivers/gpu/drm/i915/display/intel_lvds.c
index e535223..7bf1bdf 100644
--- a/drivers/gpu/drm/i915/display/intel_lvds.c
+++ b/drivers/gpu/drm/i915/display/intel_lvds.c
@@ -39,6 +39,7 @@
#include <drm/drm_edid.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_backlight.h"
#include "intel_connector.h"
@@ -78,9 +79,9 @@ struct intel_lvds_encoder {
struct intel_connector *attached_connector;
};
-static struct intel_lvds_encoder *to_lvds_encoder(struct drm_encoder *encoder)
+static struct intel_lvds_encoder *to_lvds_encoder(struct intel_encoder *encoder)
{
- return container_of(encoder, struct intel_lvds_encoder, base.base);
+ return container_of(encoder, struct intel_lvds_encoder, base);
}
bool intel_lvds_port_enabled(struct drm_i915_private *dev_priv,
@@ -103,7 +104,7 @@ static bool intel_lvds_get_hw_state(struct intel_encoder *encoder,
enum pipe *pipe)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(&encoder->base);
+ struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(encoder);
intel_wakeref_t wakeref;
bool ret;
@@ -123,7 +124,7 @@ static void intel_lvds_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
- struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(&encoder->base);
+ struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(encoder);
u32 tmp, flags = 0;
pipe_config->output_types |= BIT(INTEL_OUTPUT_LVDS);
@@ -229,7 +230,7 @@ static void intel_pre_enable_lvds(struct intel_atomic_state *state,
const struct intel_crtc_state *pipe_config,
const struct drm_connector_state *conn_state)
{
- struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(&encoder->base);
+ struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(encoder);
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_crtc *crtc = to_intel_crtc(pipe_config->uapi.crtc);
const struct drm_display_mode *adjusted_mode = &pipe_config->hw.adjusted_mode;
@@ -312,7 +313,7 @@ static void intel_enable_lvds(struct intel_atomic_state *state,
const struct drm_connector_state *conn_state)
{
struct drm_device *dev = encoder->base.dev;
- struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(&encoder->base);
+ struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(encoder);
struct drm_i915_private *dev_priv = to_i915(dev);
intel_de_write(dev_priv, lvds_encoder->reg,
@@ -334,7 +335,7 @@ static void intel_disable_lvds(struct intel_atomic_state *state,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state)
{
- struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(&encoder->base);
+ struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(encoder);
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
intel_de_write(dev_priv, PP_CONTROL(0),
@@ -413,7 +414,7 @@ static int intel_lvds_compute_config(struct intel_encoder *intel_encoder,
{
struct drm_i915_private *dev_priv = to_i915(intel_encoder->base.dev);
struct intel_lvds_encoder *lvds_encoder =
- to_lvds_encoder(&intel_encoder->base);
+ to_lvds_encoder(intel_encoder);
struct intel_connector *intel_connector =
lvds_encoder->attached_connector;
struct drm_display_mode *adjusted_mode = &pipe_config->hw.adjusted_mode;
@@ -775,7 +776,7 @@ bool intel_is_dual_link_lvds(struct drm_i915_private *dev_priv)
{
struct intel_encoder *encoder = intel_get_lvds_encoder(dev_priv);
- return encoder && to_lvds_encoder(&encoder->base)->is_dual_link;
+ return encoder && to_lvds_encoder(encoder)->is_dual_link;
}
static bool compute_is_dual_link_lvds(struct intel_lvds_encoder *lvds_encoder)
@@ -814,6 +815,11 @@ static bool compute_is_dual_link_lvds(struct intel_lvds_encoder *lvds_encoder)
return (val & LVDS_CLKB_POWER_MASK) == LVDS_CLKB_POWER_UP;
}
+static void intel_lvds_add_properties(struct drm_connector *connector)
+{
+ intel_attach_scaling_mode_property(connector);
+}
+
/**
* intel_lvds_init - setup LVDS connectors on this device
* @dev_priv: i915 device
@@ -823,7 +829,6 @@ static bool compute_is_dual_link_lvds(struct intel_lvds_encoder *lvds_encoder)
*/
void intel_lvds_init(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_lvds_encoder *lvds_encoder;
struct intel_encoder *intel_encoder;
struct intel_connector *intel_connector;
@@ -833,11 +838,10 @@ void intel_lvds_init(struct drm_i915_private *dev_priv)
i915_reg_t lvds_reg;
u32 lvds;
u8 pin;
- u32 allowed_scalers;
/* Skip init on machines we know falsely report LVDS */
if (dmi_check_system(intel_no_lvds)) {
- drm_WARN(dev, !dev_priv->display.vbt.int_lvds_support,
+ drm_WARN(&dev_priv->drm, !dev_priv->display.vbt.int_lvds_support,
"Useless DMI match. Internal LVDS support disabled by VBT\n");
return;
}
@@ -886,10 +890,10 @@ void intel_lvds_init(struct drm_i915_private *dev_priv)
intel_encoder = &lvds_encoder->base;
encoder = &intel_encoder->base;
connector = &intel_connector->base;
- drm_connector_init(dev, &intel_connector->base, &intel_lvds_connector_funcs,
+ drm_connector_init(&dev_priv->drm, &intel_connector->base, &intel_lvds_connector_funcs,
DRM_MODE_CONNECTOR_LVDS);
- drm_encoder_init(dev, &intel_encoder->base, &intel_lvds_enc_funcs,
+ drm_encoder_init(&dev_priv->drm, &intel_encoder->base, &intel_lvds_enc_funcs,
DRM_MODE_ENCODER_LVDS, "LVDS");
intel_encoder->enable = intel_enable_lvds;
@@ -920,17 +924,10 @@ void intel_lvds_init(struct drm_i915_private *dev_priv)
drm_connector_helper_add(connector, &intel_lvds_connector_helper_funcs);
connector->display_info.subpixel_order = SubPixelHorizontalRGB;
- connector->interlace_allowed = false;
- connector->doublescan_allowed = false;
lvds_encoder->reg = lvds_reg;
- /* create the scaling mode property */
- allowed_scalers = BIT(DRM_MODE_SCALE_ASPECT);
- allowed_scalers |= BIT(DRM_MODE_SCALE_FULLSCREEN);
- allowed_scalers |= BIT(DRM_MODE_SCALE_CENTER);
- drm_connector_attach_scaling_mode_property(connector, allowed_scalers);
- connector->state->scaling_mode = DRM_MODE_SCALE_ASPECT;
+ intel_lvds_add_properties(connector);
intel_lvds_pps_get_hw_state(dev_priv, &lvds_encoder->init_pps);
lvds_encoder->init_lvds_val = lvds;
@@ -947,7 +944,7 @@ void intel_lvds_init(struct drm_i915_private *dev_priv)
* Attempt to get the fixed panel mode from DDC. Assume that the
* preferred mode is the right one.
*/
- mutex_lock(&dev->mode_config.mutex);
+ mutex_lock(&dev_priv->drm.mode_config.mutex);
if (vga_switcheroo_handler_flags() & VGA_SWITCHEROO_CAN_SWITCH_DDC)
edid = drm_get_edid_switcheroo(connector,
intel_gmbus_get_adapter(dev_priv, pin));
@@ -971,8 +968,7 @@ void intel_lvds_init(struct drm_i915_private *dev_priv)
IS_ERR(edid) ? NULL : edid);
/* Try EDID first */
- intel_panel_add_edid_fixed_modes(intel_connector,
- intel_connector->panel.vbt.drrs_type != DRRS_TYPE_NONE);
+ intel_panel_add_edid_fixed_modes(intel_connector, true);
/* Failed to get EDID, what about VBT? */
if (!intel_panel_preferred_fixed_mode(intel_connector))
@@ -986,7 +982,7 @@ void intel_lvds_init(struct drm_i915_private *dev_priv)
if (!intel_panel_preferred_fixed_mode(intel_connector))
intel_panel_add_encoder_fixed_mode(intel_connector, intel_encoder);
- mutex_unlock(&dev->mode_config.mutex);
+ mutex_unlock(&dev_priv->drm.mode_config.mutex);
/* If we still don't have a mode after all that, give up. */
if (!intel_panel_preferred_fixed_mode(intel_connector))
diff --git a/drivers/gpu/drm/i915/display/intel_tc_phy_regs.h b/drivers/gpu/drm/i915/display/intel_mg_phy_regs.h
similarity index 98%
rename from drivers/gpu/drm/i915/display/intel_tc_phy_regs.h
rename to drivers/gpu/drm/i915/display/intel_mg_phy_regs.h
index 5a54508..0e8248b 100644
--- a/drivers/gpu/drm/i915/display/intel_tc_phy_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_mg_phy_regs.h
@@ -3,10 +3,10 @@
* Copyright © 2022 Intel Corporation
*/
-#ifndef __INTEL_TC_PHY_REGS__
-#define __INTEL_TC_PHY_REGS__
+#ifndef __INTEL_MG_PHY_REGS__
+#define __INTEL_MG_PHY_REGS__
-#include "i915_reg_defs.h"
+#include "intel_display_reg_defs.h"
#define MG_PHY_PORT_LN(ln, tc_port, ln0p1, ln0p2, ln1p1) \
_MMIO(_PORT(tc_port, ln0p1, ln0p2) + (ln) * ((ln1p1) - (ln0p1)))
@@ -277,4 +277,4 @@
_MG_PLL_TDC_COLDST_BIAS_PORT1, \
_MG_PLL_TDC_COLDST_BIAS_PORT2)
-#endif /* __INTEL_TC_PHY_REGS__ */
+#endif /* __INTEL_MG_PHY_REGS__ */
diff --git a/drivers/gpu/drm/i915/display/intel_modeset_setup.c b/drivers/gpu/drm/i915/display/intel_modeset_setup.c
index cbfabd5..96395bf 100644
--- a/drivers/gpu/drm/i915/display/intel_modeset_setup.c
+++ b/drivers/gpu/drm/i915/display/intel_modeset_setup.c
@@ -10,6 +10,7 @@
#include <drm/drm_atomic_state_helper.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_bw.h"
#include "intel_color.h"
@@ -155,6 +156,12 @@ static void intel_crtc_copy_hw_to_uapi_state(struct intel_crtc_state *crtc_state
crtc_state->uapi.adjusted_mode = crtc_state->hw.adjusted_mode;
crtc_state->uapi.scaling_filter = crtc_state->hw.scaling_filter;
+ /* assume 1:1 mapping */
+ drm_property_replace_blob(&crtc_state->hw.degamma_lut,
+ crtc_state->pre_csc_lut);
+ drm_property_replace_blob(&crtc_state->hw.gamma_lut,
+ crtc_state->post_csc_lut);
+
drm_property_replace_blob(&crtc_state->uapi.degamma_lut,
crtc_state->hw.degamma_lut);
drm_property_replace_blob(&crtc_state->uapi.gamma_lut,
@@ -205,13 +212,21 @@ static bool intel_crtc_has_encoders(struct intel_crtc *crtc)
static struct intel_connector *intel_encoder_find_connector(struct intel_encoder *encoder)
{
- struct drm_device *dev = encoder->base.dev;
+ struct drm_i915_private *i915 = to_i915(encoder->base.dev);
+ struct drm_connector_list_iter conn_iter;
struct intel_connector *connector;
+ struct intel_connector *found_connector = NULL;
- for_each_connector_on_encoder(dev, &encoder->base, connector)
- return connector;
+ drm_connector_list_iter_begin(&i915->drm, &conn_iter);
+ for_each_intel_connector_iter(connector, &conn_iter) {
+ if (&encoder->base == connector->base.encoder) {
+ found_connector = connector;
+ break;
+ }
+ }
+ drm_connector_list_iter_end(&conn_iter);
- return NULL;
+ return found_connector;
}
static void intel_sanitize_fifo_underrun_reporting(const struct intel_crtc_state *crtc_state)
diff --git a/drivers/gpu/drm/i915/display/intel_modeset_verify.c b/drivers/gpu/drm/i915/display/intel_modeset_verify.c
index 0fdcf2e..842d70f 100644
--- a/drivers/gpu/drm/i915/display/intel_modeset_verify.c
+++ b/drivers/gpu/drm/i915/display/intel_modeset_verify.c
@@ -227,7 +227,8 @@ void intel_modeset_verify_crtc(struct intel_crtc *crtc,
struct intel_crtc_state *old_crtc_state,
struct intel_crtc_state *new_crtc_state)
{
- if (!intel_crtc_needs_modeset(new_crtc_state) && !new_crtc_state->update_pipe)
+ if (!intel_crtc_needs_modeset(new_crtc_state) &&
+ !intel_crtc_needs_fastset(new_crtc_state))
return;
intel_wm_state_verify(crtc, new_crtc_state);
diff --git a/drivers/gpu/drm/i915/display/intel_opregion.c b/drivers/gpu/drm/i915/display/intel_opregion.c
index caa07ef..e018474 100644
--- a/drivers/gpu/drm/i915/display/intel_opregion.c
+++ b/drivers/gpu/drm/i915/display/intel_opregion.c
@@ -463,7 +463,6 @@ static u32 asle_set_backlight(struct drm_i915_private *dev_priv, u32 bclp)
struct intel_connector *connector;
struct drm_connector_list_iter conn_iter;
struct opregion_asle *asle = dev_priv->display.opregion.asle;
- struct drm_device *dev = &dev_priv->drm;
drm_dbg(&dev_priv->drm, "bclp = 0x%08x\n", bclp);
@@ -480,7 +479,7 @@ static u32 asle_set_backlight(struct drm_i915_private *dev_priv, u32 bclp)
if (bclp > 255)
return ASLC_BACKLIGHT_FAILED;
- drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+ drm_modeset_lock(&dev_priv->drm.mode_config.connection_mutex, NULL);
/*
* Update backlight on all connectors that support backlight (usually
@@ -488,13 +487,13 @@ static u32 asle_set_backlight(struct drm_i915_private *dev_priv, u32 bclp)
*/
drm_dbg_kms(&dev_priv->drm, "updating opregion backlight %d/255\n",
bclp);
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
for_each_intel_connector_iter(connector, &conn_iter)
intel_backlight_set_acpi(connector->base.state, bclp, 255);
drm_connector_list_iter_end(&conn_iter);
asle->cblv = DIV_ROUND_UP(bclp * 100, 255) | ASLE_CBLV_VALID;
- drm_modeset_unlock(&dev->mode_config.connection_mutex);
+ drm_modeset_unlock(&dev_priv->drm.mode_config.connection_mutex);
return 0;
diff --git a/drivers/gpu/drm/i915/display/intel_panel.c b/drivers/gpu/drm/i915/display/intel_panel.c
index 41cec9d..1640726 100644
--- a/drivers/gpu/drm/i915/display/intel_panel.c
+++ b/drivers/gpu/drm/i915/display/intel_panel.c
@@ -31,6 +31,7 @@
#include <linux/kernel.h>
#include <linux/pwm.h>
+#include "i915_reg.h"
#include "intel_backlight.h"
#include "intel_connector.h"
#include "intel_de.h"
@@ -85,9 +86,10 @@ static bool is_alt_drrs_mode(const struct drm_display_mode *mode,
static bool is_alt_fixed_mode(const struct drm_display_mode *mode,
const struct drm_display_mode *preferred_mode)
{
- return drm_mode_match(mode, preferred_mode,
- DRM_MODE_MATCH_FLAGS |
- DRM_MODE_MATCH_3D_FLAGS) &&
+ u32 sync_flags = DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_NHSYNC |
+ DRM_MODE_FLAG_PVSYNC | DRM_MODE_FLAG_NVSYNC;
+
+ return (mode->flags & ~sync_flags) == (preferred_mode->flags & ~sync_flags) &&
mode->hdisplay == preferred_mode->hdisplay &&
mode->vdisplay == preferred_mode->vdisplay;
}
@@ -147,12 +149,24 @@ int intel_panel_get_modes(struct intel_connector *connector)
return num_modes;
}
+static bool has_drrs_modes(struct intel_connector *connector)
+{
+ const struct drm_display_mode *mode1;
+
+ list_for_each_entry(mode1, &connector->panel.fixed_modes, head) {
+ const struct drm_display_mode *mode2 = mode1;
+
+ list_for_each_entry_continue(mode2, &connector->panel.fixed_modes, head) {
+ if (is_alt_drrs_mode(mode1, mode2))
+ return true;
+ }
+ }
+
+ return false;
+}
+
enum drrs_type intel_panel_drrs_type(struct intel_connector *connector)
{
- if (list_empty(&connector->panel.fixed_modes) ||
- list_is_singular(&connector->panel.fixed_modes))
- return DRRS_TYPE_NONE;
-
return connector->panel.vbt.drrs_type;
}
@@ -653,6 +667,9 @@ int intel_panel_init(struct intel_connector *connector)
intel_backlight_init_funcs(panel);
+ if (!has_drrs_modes(connector))
+ connector->panel.vbt.drrs_type = DRRS_TYPE_NONE;
+
drm_dbg_kms(connector->base.dev,
"[CONNECTOR:%d:%s] DRRS type: %s\n",
connector->base.base.id, connector->base.name,
diff --git a/drivers/gpu/drm/i915/display/intel_pch_display.c b/drivers/gpu/drm/i915/display/intel_pch_display.c
index 837152dc..cecc0d0 100644
--- a/drivers/gpu/drm/i915/display/intel_pch_display.c
+++ b/drivers/gpu/drm/i915/display/intel_pch_display.c
@@ -4,6 +4,7 @@
*/
#include "g4x_dp.h"
+#include "i915_reg.h"
#include "intel_crt.h"
#include "intel_de.h"
#include "intel_display_types.h"
diff --git a/drivers/gpu/drm/i915/display/intel_pch_refclk.c b/drivers/gpu/drm/i915/display/intel_pch_refclk.c
index a66097c..08a9436 100644
--- a/drivers/gpu/drm/i915/display/intel_pch_refclk.c
+++ b/drivers/gpu/drm/i915/display/intel_pch_refclk.c
@@ -3,6 +3,7 @@
* Copyright © 2021 Intel Corporation
*/
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_panel.h"
diff --git a/drivers/gpu/drm/i915/display/intel_pipe_crc.c b/drivers/gpu/drm/i915/display/intel_pipe_crc.c
index 8ac263f..e977467 100644
--- a/drivers/gpu/drm/i915/display/intel_pipe_crc.c
+++ b/drivers/gpu/drm/i915/display/intel_pipe_crc.c
@@ -24,11 +24,12 @@
*
*/
-#include <linux/circ_buf.h>
#include <linux/ctype.h>
#include <linux/debugfs.h>
#include <linux/seq_file.h>
+#include "i915_irq.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_de.h"
#include "intel_display_types.h"
@@ -75,7 +76,6 @@ static int i9xx_pipe_crc_auto_source(struct drm_i915_private *dev_priv,
enum pipe pipe,
enum intel_pipe_crc_source *source)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_encoder *encoder;
struct intel_crtc *crtc;
struct intel_digital_port *dig_port;
@@ -83,8 +83,8 @@ static int i9xx_pipe_crc_auto_source(struct drm_i915_private *dev_priv,
*source = INTEL_PIPE_CRC_SOURCE_PIPE;
- drm_modeset_lock_all(dev);
- for_each_intel_encoder(dev, encoder) {
+ drm_modeset_lock_all(&dev_priv->drm);
+ for_each_intel_encoder(&dev_priv->drm, encoder) {
if (!encoder->base.crtc)
continue;
@@ -111,7 +111,7 @@ static int i9xx_pipe_crc_auto_source(struct drm_i915_private *dev_priv,
*source = INTEL_PIPE_CRC_SOURCE_DP_D;
break;
default:
- drm_WARN(dev, 1, "nonexisting DP port %c\n",
+ drm_WARN(&dev_priv->drm, 1, "nonexisting DP port %c\n",
port_name(dig_port->base.port));
break;
}
@@ -120,7 +120,7 @@ static int i9xx_pipe_crc_auto_source(struct drm_i915_private *dev_priv,
break;
}
}
- drm_modeset_unlock_all(dev);
+ drm_modeset_unlock_all(&dev_priv->drm);
return ret;
}
diff --git a/drivers/gpu/drm/i915/display/intel_pps.c b/drivers/gpu/drm/i915/display/intel_pps.c
index 21944f5..9bbf41a 100644
--- a/drivers/gpu/drm/i915/display/intel_pps.c
+++ b/drivers/gpu/drm/i915/display/intel_pps.c
@@ -5,10 +5,12 @@
#include "g4x_dp.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_power_well.h"
#include "intel_display_types.h"
#include "intel_dp.h"
+#include "intel_dpio_phy.h"
#include "intel_dpll.h"
#include "intel_lvds.h"
#include "intel_pps.h"
@@ -1098,7 +1100,13 @@ bool intel_pps_have_panel_power_or_vdd(struct intel_dp *intel_dp)
static void pps_init_timestamps(struct intel_dp *intel_dp)
{
- intel_dp->pps.panel_power_off_time = ktime_get_boottime();
+ /*
+ * Initialize panel power off time to 0, assuming panel power could have
+ * been toggled between kernel boot and now only by a previously loaded
+ * and removed i915, which has already ensured sufficient power off
+ * delay at module remove.
+ */
+ intel_dp->pps.panel_power_off_time = 0;
intel_dp->pps.last_power_on = jiffies;
intel_dp->pps.last_backlight_off = jiffies;
}
diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c
index 15c3e44..5b67891 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.c
+++ b/drivers/gpu/drm/i915/display/intel_psr.c
@@ -27,6 +27,7 @@
#include "display/intel_dp.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_crtc.h"
#include "intel_de.h"
@@ -533,7 +534,7 @@ static void hsw_activate_psr2(struct intel_dp *intel_dp)
val |= psr_compute_idle_frames(intel_dp) << EDP_PSR2_IDLE_FRAME_SHIFT;
- if (!IS_ALDERLAKE_P(dev_priv))
+ if (DISPLAY_VER(dev_priv) <= 13 && !IS_ALDERLAKE_P(dev_priv))
val |= EDP_SU_TRACK_ENABLE;
if (DISPLAY_VER(dev_priv) >= 10 && DISPLAY_VER(dev_priv) <= 12)
@@ -616,7 +617,7 @@ static void hsw_activate_psr2(struct intel_dp *intel_dp)
static bool
transcoder_has_psr2(struct drm_i915_private *dev_priv, enum transcoder trans)
{
- if (IS_ALDERLAKE_P(dev_priv))
+ if (IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14)
return trans == TRANSCODER_A || trans == TRANSCODER_B;
else if (DISPLAY_VER(dev_priv) >= 12)
return trans == TRANSCODER_A;
@@ -696,7 +697,7 @@ dc3co_is_pipe_port_compatible(struct intel_dp *intel_dp,
struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
enum port port = dig_port->base.port;
- if (IS_ALDERLAKE_P(dev_priv))
+ if (IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14)
return pipe <= PIPE_B && port <= PORT_B;
else
return pipe == PIPE_A && port == PORT_A;
@@ -779,6 +780,7 @@ static bool psr2_granularity_check(struct intel_dp *intel_dp,
struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
+ const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config;
const int crtc_hdisplay = crtc_state->hw.adjusted_mode.crtc_hdisplay;
const int crtc_vdisplay = crtc_state->hw.adjusted_mode.crtc_vdisplay;
u16 y_granularity = 0;
@@ -795,11 +797,11 @@ static bool psr2_granularity_check(struct intel_dp *intel_dp,
return intel_dp->psr.su_y_granularity == 4;
/*
- * adl_p has 1 line granularity. For other platforms with SW tracking we
- * can adjust the y coordinates to match sink requirement if multiple of
- * 4.
+ * adl_p and display 14+ platforms has 1 line granularity.
+ * For other platforms with SW tracking we can adjust the y coordinates
+ * to match sink requirement if multiple of 4.
*/
- if (IS_ALDERLAKE_P(dev_priv))
+ if (IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14)
y_granularity = intel_dp->psr.su_y_granularity;
else if (intel_dp->psr.su_y_granularity <= 2)
y_granularity = 4;
@@ -809,6 +811,10 @@ static bool psr2_granularity_check(struct intel_dp *intel_dp,
if (y_granularity == 0 || crtc_vdisplay % y_granularity)
return false;
+ if (crtc_state->dsc.compression_enable &&
+ vdsc_cfg->slice_height % y_granularity)
+ return false;
+
crtc_state->su_y_granularity = y_granularity;
return true;
}
@@ -883,7 +889,8 @@ static bool intel_psr2_config_valid(struct intel_dp *intel_dp,
* resolution requires DSC to be enabled, priority is given to DSC
* over PSR2.
*/
- if (crtc_state->dsc.compression_enable) {
+ if (crtc_state->dsc.compression_enable &&
+ (DISPLAY_VER(dev_priv) <= 13 && !IS_ALDERLAKE_P(dev_priv))) {
drm_dbg_kms(&dev_priv->drm,
"PSR2 cannot be enabled since DSC is enabled\n");
return false;
@@ -1469,26 +1476,27 @@ void intel_psr_resume(struct intel_dp *intel_dp)
static u32 man_trk_ctl_enable_bit_get(struct drm_i915_private *dev_priv)
{
- return IS_ALDERLAKE_P(dev_priv) ? 0 : PSR2_MAN_TRK_CTL_ENABLE;
+ return IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14 ? 0 :
+ PSR2_MAN_TRK_CTL_ENABLE;
}
static u32 man_trk_ctl_single_full_frame_bit_get(struct drm_i915_private *dev_priv)
{
- return IS_ALDERLAKE_P(dev_priv) ?
+ return IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14 ?
ADLP_PSR2_MAN_TRK_CTL_SF_SINGLE_FULL_FRAME :
PSR2_MAN_TRK_CTL_SF_SINGLE_FULL_FRAME;
}
static u32 man_trk_ctl_partial_frame_bit_get(struct drm_i915_private *dev_priv)
{
- return IS_ALDERLAKE_P(dev_priv) ?
+ return IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14 ?
ADLP_PSR2_MAN_TRK_CTL_SF_PARTIAL_FRAME_UPDATE :
PSR2_MAN_TRK_CTL_SF_PARTIAL_FRAME_UPDATE;
}
static u32 man_trk_ctl_continuos_full_frame(struct drm_i915_private *dev_priv)
{
- return IS_ALDERLAKE_P(dev_priv) ?
+ return IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14 ?
ADLP_PSR2_MAN_TRK_CTL_SF_CONTINUOS_FULL_FRAME :
PSR2_MAN_TRK_CTL_SF_CONTINUOS_FULL_FRAME;
}
@@ -1627,7 +1635,7 @@ static void psr2_man_trk_ctl_calc(struct intel_crtc_state *crtc_state,
if (clip->y1 == -1)
goto exit;
- if (IS_ALDERLAKE_P(dev_priv)) {
+ if (IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14) {
val |= ADLP_PSR2_MAN_TRK_CTL_SU_REGION_START_ADDR(clip->y1);
val |= ADLP_PSR2_MAN_TRK_CTL_SU_REGION_END_ADDR(clip->y2 - 1);
} else {
@@ -1664,14 +1672,19 @@ static void intel_psr2_sel_fetch_pipe_alignment(const struct intel_crtc_state *c
struct drm_rect *pipe_clip)
{
struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
- const u16 y_alignment = crtc_state->su_y_granularity;
+ const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config;
+ u16 y_alignment;
+
+ /* ADLP aligns the SU region to vdsc slice height in case dsc is enabled */
+ if (crtc_state->dsc.compression_enable &&
+ (IS_ALDERLAKE_P(dev_priv) || DISPLAY_VER(dev_priv) >= 14))
+ y_alignment = vdsc_cfg->slice_height;
+ else
+ y_alignment = crtc_state->su_y_granularity;
pipe_clip->y1 -= pipe_clip->y1 % y_alignment;
if (pipe_clip->y2 % y_alignment)
pipe_clip->y2 = ((pipe_clip->y2 / y_alignment) + 1) * y_alignment;
-
- if (IS_ALDERLAKE_P(dev_priv) && crtc_state->dsc.compression_enable)
- drm_warn(&dev_priv->drm, "Missing PSR2 sel fetch alignment with DSC\n");
}
/*
@@ -2054,13 +2067,12 @@ static bool __psr_wait_for_idle_locked(struct intel_dp *intel_dp)
static int intel_psr_fastset_force(struct drm_i915_private *dev_priv)
{
struct drm_connector_list_iter conn_iter;
- struct drm_device *dev = &dev_priv->drm;
struct drm_modeset_acquire_ctx ctx;
struct drm_atomic_state *state;
struct drm_connector *conn;
int err = 0;
- state = drm_atomic_state_alloc(dev);
+ state = drm_atomic_state_alloc(&dev_priv->drm);
if (!state)
return -ENOMEM;
@@ -2069,7 +2081,7 @@ static int intel_psr_fastset_force(struct drm_i915_private *dev_priv)
retry:
- drm_connector_list_iter_begin(dev, &conn_iter);
+ drm_connector_list_iter_begin(&dev_priv->drm, &conn_iter);
drm_for_each_connector_iter(conn, &conn_iter) {
struct drm_connector_state *conn_state;
struct drm_crtc_state *crtc_state;
diff --git a/drivers/gpu/drm/i915/display/intel_sdvo.c b/drivers/gpu/drm/i915/display/intel_sdvo.c
index 774c1dc..329b9d9 100644
--- a/drivers/gpu/drm/i915/display/intel_sdvo.c
+++ b/drivers/gpu/drm/i915/display/intel_sdvo.c
@@ -37,6 +37,7 @@
#include <drm/drm_edid.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_connector.h"
#include "intel_crtc.h"
@@ -199,7 +200,7 @@ to_intel_sdvo_connector(struct drm_connector *connector)
container_of((conn_state), struct intel_sdvo_connector_state, base.base)
static bool
-intel_sdvo_output_setup(struct intel_sdvo *intel_sdvo, u16 flags);
+intel_sdvo_output_setup(struct intel_sdvo *intel_sdvo);
static bool
intel_sdvo_tv_create_property(struct intel_sdvo *intel_sdvo,
struct intel_sdvo_connector *intel_sdvo_connector,
@@ -1297,13 +1298,28 @@ static bool intel_sdvo_limited_color_range(struct intel_encoder *encoder,
return intel_hdmi_limited_color_range(crtc_state, conn_state);
}
+static bool intel_sdvo_has_audio(struct intel_encoder *encoder,
+ const struct intel_crtc_state *crtc_state,
+ const struct drm_connector_state *conn_state)
+{
+ struct intel_sdvo *intel_sdvo = to_sdvo(encoder);
+ const struct intel_digital_connector_state *intel_conn_state =
+ to_intel_digital_connector_state(conn_state);
+
+ if (!crtc_state->has_hdmi_sink)
+ return false;
+
+ if (intel_conn_state->force_audio == HDMI_AUDIO_AUTO)
+ return intel_sdvo->has_hdmi_audio;
+ else
+ return intel_conn_state->force_audio == HDMI_AUDIO_ON;
+}
+
static int intel_sdvo_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state)
{
struct intel_sdvo *intel_sdvo = to_sdvo(encoder);
- struct intel_sdvo_connector_state *intel_sdvo_state =
- to_intel_sdvo_connector_state(conn_state);
struct intel_sdvo_connector *intel_sdvo_connector =
to_intel_sdvo_connector(conn_state->connector);
struct drm_display_mode *adjusted_mode = &pipe_config->hw.adjusted_mode;
@@ -1362,13 +1378,7 @@ static int intel_sdvo_compute_config(struct intel_encoder *encoder,
pipe_config->has_hdmi_sink = intel_has_hdmi_sink(intel_sdvo, conn_state);
- if (pipe_config->has_hdmi_sink) {
- if (intel_sdvo_state->base.force_audio == HDMI_AUDIO_AUTO)
- pipe_config->has_audio = intel_sdvo->has_hdmi_audio;
- else
- pipe_config->has_audio =
- intel_sdvo_state->base.force_audio == HDMI_AUDIO_ON;
- }
+ pipe_config->has_audio = intel_sdvo_has_audio(encoder, pipe_config, conn_state);
pipe_config->limited_color_range =
intel_sdvo_limited_color_range(encoder, pipe_config,
@@ -2290,17 +2300,12 @@ static int intel_sdvo_get_tv_modes(struct drm_connector *connector)
static int intel_sdvo_get_lvds_modes(struct drm_connector *connector)
{
- struct intel_sdvo *intel_sdvo = intel_attached_sdvo(to_intel_connector(connector));
struct drm_i915_private *dev_priv = to_i915(connector->dev);
- int num_modes = 0;
drm_dbg_kms(&dev_priv->drm, "[CONNECTOR:%d:%s]\n",
connector->base.id, connector->name);
- num_modes += intel_panel_get_modes(to_intel_connector(connector));
- num_modes += intel_ddc_get_modes(connector, &intel_sdvo->ddc);
-
- return num_modes;
+ return intel_panel_get_modes(to_intel_connector(connector));
}
static int intel_sdvo_get_modes(struct drm_connector *connector)
@@ -2627,7 +2632,7 @@ intel_sdvo_unselect_i2c_bus(struct intel_sdvo *sdvo)
}
static bool
-intel_sdvo_is_hdmi_connector(struct intel_sdvo *intel_sdvo, int device)
+intel_sdvo_is_hdmi_connector(struct intel_sdvo *intel_sdvo)
{
return intel_sdvo_check_supp_encode(intel_sdvo);
}
@@ -2689,9 +2694,8 @@ intel_sdvo_connector_init(struct intel_sdvo_connector *connector,
drm_connector_helper_add(drm_connector,
&intel_sdvo_connector_helper_funcs);
- connector->base.base.interlace_allowed = 1;
- connector->base.base.doublescan_allowed = 0;
connector->base.base.display_info.subpixel_order = SubPixelHorizontalRGB;
+ connector->base.base.interlace_allowed = true;
connector->base.get_hw_state = intel_sdvo_connector_get_hw_state;
intel_connector_attach_encoder(&connector->base, &encoder->base);
@@ -2733,7 +2737,7 @@ static struct intel_sdvo_connector *intel_sdvo_connector_alloc(void)
}
static bool
-intel_sdvo_dvi_init(struct intel_sdvo *intel_sdvo, int device)
+intel_sdvo_dvi_init(struct intel_sdvo *intel_sdvo, u16 type)
{
struct drm_encoder *encoder = &intel_sdvo->base.base;
struct drm_connector *connector;
@@ -2741,16 +2745,13 @@ intel_sdvo_dvi_init(struct intel_sdvo *intel_sdvo, int device)
struct intel_connector *intel_connector;
struct intel_sdvo_connector *intel_sdvo_connector;
- DRM_DEBUG_KMS("initialising DVI device %d\n", device);
+ DRM_DEBUG_KMS("initialising DVI type 0x%x\n", type);
intel_sdvo_connector = intel_sdvo_connector_alloc();
if (!intel_sdvo_connector)
return false;
- if (device == 0)
- intel_sdvo_connector->output_flag = SDVO_OUTPUT_TMDS0;
- else if (device == 1)
- intel_sdvo_connector->output_flag = SDVO_OUTPUT_TMDS1;
+ intel_sdvo_connector->output_flag = type;
intel_connector = &intel_sdvo_connector->base;
connector = &intel_connector->base;
@@ -2770,7 +2771,7 @@ intel_sdvo_dvi_init(struct intel_sdvo *intel_sdvo, int device)
encoder->encoder_type = DRM_MODE_ENCODER_TMDS;
connector->connector_type = DRM_MODE_CONNECTOR_DVID;
- if (intel_sdvo_is_hdmi_connector(intel_sdvo, device)) {
+ if (intel_sdvo_is_hdmi_connector(intel_sdvo)) {
connector->connector_type = DRM_MODE_CONNECTOR_HDMIA;
intel_sdvo_connector->is_hdmi = true;
}
@@ -2787,14 +2788,14 @@ intel_sdvo_dvi_init(struct intel_sdvo *intel_sdvo, int device)
}
static bool
-intel_sdvo_tv_init(struct intel_sdvo *intel_sdvo, int type)
+intel_sdvo_tv_init(struct intel_sdvo *intel_sdvo, u16 type)
{
struct drm_encoder *encoder = &intel_sdvo->base.base;
struct drm_connector *connector;
struct intel_connector *intel_connector;
struct intel_sdvo_connector *intel_sdvo_connector;
- DRM_DEBUG_KMS("initialising TV type %d\n", type);
+ DRM_DEBUG_KMS("initialising TV type 0x%x\n", type);
intel_sdvo_connector = intel_sdvo_connector_alloc();
if (!intel_sdvo_connector)
@@ -2826,14 +2827,14 @@ intel_sdvo_tv_init(struct intel_sdvo *intel_sdvo, int type)
}
static bool
-intel_sdvo_analog_init(struct intel_sdvo *intel_sdvo, int device)
+intel_sdvo_analog_init(struct intel_sdvo *intel_sdvo, u16 type)
{
struct drm_encoder *encoder = &intel_sdvo->base.base;
struct drm_connector *connector;
struct intel_connector *intel_connector;
struct intel_sdvo_connector *intel_sdvo_connector;
- DRM_DEBUG_KMS("initialising analog device %d\n", device);
+ DRM_DEBUG_KMS("initialising analog type 0x%x\n", type);
intel_sdvo_connector = intel_sdvo_connector_alloc();
if (!intel_sdvo_connector)
@@ -2845,10 +2846,7 @@ intel_sdvo_analog_init(struct intel_sdvo *intel_sdvo, int device)
encoder->encoder_type = DRM_MODE_ENCODER_DAC;
connector->connector_type = DRM_MODE_CONNECTOR_VGA;
- if (device == 0)
- intel_sdvo_connector->output_flag = SDVO_OUTPUT_RGB0;
- else if (device == 1)
- intel_sdvo_connector->output_flag = SDVO_OUTPUT_RGB1;
+ intel_sdvo_connector->output_flag = type;
if (intel_sdvo_connector_init(intel_sdvo_connector, intel_sdvo) < 0) {
kfree(intel_sdvo_connector);
@@ -2859,7 +2857,7 @@ intel_sdvo_analog_init(struct intel_sdvo *intel_sdvo, int device)
}
static bool
-intel_sdvo_lvds_init(struct intel_sdvo *intel_sdvo, int device)
+intel_sdvo_lvds_init(struct intel_sdvo *intel_sdvo, u16 type)
{
struct drm_encoder *encoder = &intel_sdvo->base.base;
struct drm_i915_private *i915 = to_i915(encoder->dev);
@@ -2867,7 +2865,7 @@ intel_sdvo_lvds_init(struct intel_sdvo *intel_sdvo, int device)
struct intel_connector *intel_connector;
struct intel_sdvo_connector *intel_sdvo_connector;
- DRM_DEBUG_KMS("initialising LVDS device %d\n", device);
+ DRM_DEBUG_KMS("initialising LVDS type 0x%x\n", type);
intel_sdvo_connector = intel_sdvo_connector_alloc();
if (!intel_sdvo_connector)
@@ -2878,10 +2876,7 @@ intel_sdvo_lvds_init(struct intel_sdvo *intel_sdvo, int device)
encoder->encoder_type = DRM_MODE_ENCODER_LVDS;
connector->connector_type = DRM_MODE_CONNECTOR_LVDS;
- if (device == 0)
- intel_sdvo_connector->output_flag = SDVO_OUTPUT_LVDS0;
- else if (device == 1)
- intel_sdvo_connector->output_flag = SDVO_OUTPUT_LVDS1;
+ intel_sdvo_connector->output_flag = type;
if (intel_sdvo_connector_init(intel_sdvo_connector, intel_sdvo) < 0) {
kfree(intel_sdvo_connector);
@@ -2937,63 +2932,61 @@ static u16 intel_sdvo_filter_output_flags(u16 flags)
return flags;
}
-static bool
-intel_sdvo_output_setup(struct intel_sdvo *intel_sdvo, u16 flags)
+static bool intel_sdvo_output_init(struct intel_sdvo *sdvo, u16 type)
{
- struct drm_i915_private *i915 = to_i915(intel_sdvo->base.base.dev);
+ if (type & SDVO_TMDS_MASK)
+ return intel_sdvo_dvi_init(sdvo, type);
+ else if (type & SDVO_TV_MASK)
+ return intel_sdvo_tv_init(sdvo, type);
+ else if (type & SDVO_RGB_MASK)
+ return intel_sdvo_analog_init(sdvo, type);
+ else if (type & SDVO_LVDS_MASK)
+ return intel_sdvo_lvds_init(sdvo, type);
+ else
+ return false;
+}
- flags = intel_sdvo_filter_output_flags(flags);
+static bool
+intel_sdvo_output_setup(struct intel_sdvo *intel_sdvo)
+{
+ static const u16 probe_order[] = {
+ SDVO_OUTPUT_TMDS0,
+ SDVO_OUTPUT_TMDS1,
+ /* TV has no XXX1 function block */
+ SDVO_OUTPUT_SVID0,
+ SDVO_OUTPUT_CVBS0,
+ SDVO_OUTPUT_YPRPB0,
+ SDVO_OUTPUT_RGB0,
+ SDVO_OUTPUT_RGB1,
+ SDVO_OUTPUT_LVDS0,
+ SDVO_OUTPUT_LVDS1,
+ };
+ struct drm_i915_private *i915 = to_i915(intel_sdvo->base.base.dev);
+ u16 flags;
+ int i;
+
+ flags = intel_sdvo_filter_output_flags(intel_sdvo->caps.output_flags);
+
+ if (flags == 0) {
+ DRM_DEBUG_KMS("%s: Unknown SDVO output type (0x%04x)\n",
+ SDVO_NAME(intel_sdvo), intel_sdvo->caps.output_flags);
+ return false;
+ }
intel_sdvo->controlled_output = flags;
intel_sdvo_select_ddc_bus(i915, intel_sdvo);
- if (flags & SDVO_OUTPUT_TMDS0)
- if (!intel_sdvo_dvi_init(intel_sdvo, 0))
+ for (i = 0; i < ARRAY_SIZE(probe_order); i++) {
+ u16 type = flags & probe_order[i];
+
+ if (!type)
+ continue;
+
+ if (!intel_sdvo_output_init(intel_sdvo, type))
return false;
-
- if (flags & SDVO_OUTPUT_TMDS1)
- if (!intel_sdvo_dvi_init(intel_sdvo, 1))
- return false;
-
- /* TV has no XXX1 function block */
- if (flags & SDVO_OUTPUT_SVID0)
- if (!intel_sdvo_tv_init(intel_sdvo, SDVO_OUTPUT_SVID0))
- return false;
-
- if (flags & SDVO_OUTPUT_CVBS0)
- if (!intel_sdvo_tv_init(intel_sdvo, SDVO_OUTPUT_CVBS0))
- return false;
-
- if (flags & SDVO_OUTPUT_YPRPB0)
- if (!intel_sdvo_tv_init(intel_sdvo, SDVO_OUTPUT_YPRPB0))
- return false;
-
- if (flags & SDVO_OUTPUT_RGB0)
- if (!intel_sdvo_analog_init(intel_sdvo, 0))
- return false;
-
- if (flags & SDVO_OUTPUT_RGB1)
- if (!intel_sdvo_analog_init(intel_sdvo, 1))
- return false;
-
- if (flags & SDVO_OUTPUT_LVDS0)
- if (!intel_sdvo_lvds_init(intel_sdvo, 0))
- return false;
-
- if (flags & SDVO_OUTPUT_LVDS1)
- if (!intel_sdvo_lvds_init(intel_sdvo, 1))
- return false;
-
- if (flags == 0) {
- unsigned char bytes[2];
-
- memcpy(bytes, &intel_sdvo->caps.output_flags, 2);
- DRM_DEBUG_KMS("%s: Unknown SDVO output type (0x%02x%02x)\n",
- SDVO_NAME(intel_sdvo),
- bytes[0], bytes[1]);
- return false;
}
+
intel_sdvo->base.pipe_mask = ~0;
return true;
@@ -3369,8 +3362,7 @@ bool intel_sdvo_init(struct drm_i915_private *dev_priv,
intel_sdvo->colorimetry_cap =
intel_sdvo_get_colorimetry_cap(intel_sdvo);
- if (intel_sdvo_output_setup(intel_sdvo,
- intel_sdvo->caps.output_flags) != true) {
+ if (!intel_sdvo_output_setup(intel_sdvo)) {
drm_dbg_kms(&dev_priv->drm,
"SDVO output failed to setup on %s\n",
SDVO_NAME(intel_sdvo));
@@ -3421,9 +3413,12 @@ bool intel_sdvo_init(struct drm_i915_private *dev_priv,
(intel_sdvo->caps.sdvo_inputs_mask & 0x2) ? 'Y' : 'N',
/* check currently supported outputs */
intel_sdvo->caps.output_flags &
- (SDVO_OUTPUT_TMDS0 | SDVO_OUTPUT_RGB0) ? 'Y' : 'N',
+ (SDVO_OUTPUT_TMDS0 | SDVO_OUTPUT_RGB0 |
+ SDVO_OUTPUT_LVDS0 | SDVO_OUTPUT_SVID0 |
+ SDVO_OUTPUT_CVBS0 | SDVO_OUTPUT_YPRPB0) ? 'Y' : 'N',
intel_sdvo->caps.output_flags &
- (SDVO_OUTPUT_TMDS1 | SDVO_OUTPUT_RGB1) ? 'Y' : 'N');
+ (SDVO_OUTPUT_TMDS1 | SDVO_OUTPUT_RGB1 |
+ SDVO_OUTPUT_LVDS1) ? 'Y' : 'N');
return true;
err_output:
diff --git a/drivers/gpu/drm/i915/display/intel_snps_phy.c b/drivers/gpu/drm/i915/display/intel_snps_phy.c
index 937cefd6..c799e89 100644
--- a/drivers/gpu/drm/i915/display/intel_snps_phy.c
+++ b/drivers/gpu/drm/i915/display/intel_snps_phy.c
@@ -5,6 +5,7 @@
#include <linux/util_macros.h>
+#include "i915_reg.h"
#include "intel_ddi.h"
#include "intel_ddi_buf_trans.h"
#include "intel_de.h"
diff --git a/drivers/gpu/drm/i915/display/intel_snps_phy_regs.h b/drivers/gpu/drm/i915/display/intel_snps_phy_regs.h
index 0543465..a04d692 100644
--- a/drivers/gpu/drm/i915/display/intel_snps_phy_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_snps_phy_regs.h
@@ -6,7 +6,7 @@
#ifndef __INTEL_SNPS_PHY_REGS__
#define __INTEL_SNPS_PHY_REGS__
-#include "i915_reg_defs.h"
+#include "intel_display_reg_defs.h"
#define _SNPS_PHY_A_BASE 0x168000
#define _SNPS_PHY_B_BASE 0x169000
diff --git a/drivers/gpu/drm/i915/display/intel_sprite.c b/drivers/gpu/drm/i915/display/intel_sprite.c
index 7649c50..e6b4d24 100644
--- a/drivers/gpu/drm/i915/display/intel_sprite.c
+++ b/drivers/gpu/drm/i915/display/intel_sprite.c
@@ -42,6 +42,7 @@
#include <drm/drm_rect.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "i915_vgpu.h"
#include "i9xx_plane.h"
#include "intel_atomic_plane.h"
diff --git a/drivers/gpu/drm/i915/display/intel_sprite.h b/drivers/gpu/drm/i915/display/intel_sprite.h
index 4f63e49..4635c7a 100644
--- a/drivers/gpu/drm/i915/display/intel_sprite.h
+++ b/drivers/gpu/drm/i915/display/intel_sprite.h
@@ -8,14 +8,13 @@
#include <linux/types.h>
-#include "intel_display.h"
-
struct drm_device;
struct drm_display_mode;
struct drm_file;
struct drm_i915_private;
struct intel_crtc_state;
struct intel_plane_state;
+enum pipe;
/*
* FIXME: We should instead only take spinlocks once for the entire update
@@ -34,12 +33,6 @@ int intel_sprite_set_colorkey_ioctl(struct drm_device *dev, void *data,
int intel_plane_check_src_coordinates(struct intel_plane_state *plane_state);
int chv_plane_check_rotation(const struct intel_plane_state *plane_state);
-static inline u8 icl_hdr_plane_mask(void)
-{
- return BIT(PLANE_PRIMARY) |
- BIT(PLANE_SPRITE0) | BIT(PLANE_SPRITE1);
-}
-
int ivb_plane_min_cdclk(const struct intel_crtc_state *crtc_state,
const struct intel_plane_state *plane_state);
int hsw_plane_min_cdclk(const struct intel_crtc_state *crtc_state,
diff --git a/drivers/gpu/drm/i915/display/intel_tc.c b/drivers/gpu/drm/i915/display/intel_tc.c
index e5af955..70624b4 100644
--- a/drivers/gpu/drm/i915/display/intel_tc.c
+++ b/drivers/gpu/drm/i915/display/intel_tc.c
@@ -8,9 +8,10 @@
#include "intel_display.h"
#include "intel_display_power_map.h"
#include "intel_display_types.h"
+#include "intel_dkl_phy_regs.h"
#include "intel_dp_mst.h"
+#include "intel_mg_phy_regs.h"
#include "intel_tc.h"
-#include "intel_tc_phy_regs.h"
static const char *tc_port_mode_name(enum tc_port_mode mode)
{
@@ -408,14 +409,9 @@ static bool adl_tc_phy_take_ownership(struct intel_digital_port *dig_port,
struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
struct intel_uncore *uncore = &i915->uncore;
enum port port = dig_port->base.port;
- u32 val;
- val = intel_uncore_read(uncore, DDI_BUF_CTL(port));
- if (take)
- val |= DDI_BUF_CTL_TC_PHY_OWNERSHIP;
- else
- val &= ~DDI_BUF_CTL_TC_PHY_OWNERSHIP;
- intel_uncore_write(uncore, DDI_BUF_CTL(port), val);
+ intel_uncore_rmw(uncore, DDI_BUF_CTL(port), DDI_BUF_CTL_TC_PHY_OWNERSHIP,
+ take ? DDI_BUF_CTL_TC_PHY_OWNERSHIP : 0);
return true;
}
@@ -687,18 +683,58 @@ static void
intel_tc_port_link_init_refcount(struct intel_digital_port *dig_port,
int refcount)
{
- struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
-
- drm_WARN_ON(&i915->drm, dig_port->tc_link_refcount);
dig_port->tc_link_refcount = refcount;
}
-void intel_tc_port_sanitize(struct intel_digital_port *dig_port)
+/**
+ * intel_tc_port_init_mode: Read out HW state and init the given port's TypeC mode
+ * @dig_port: digital port
+ *
+ * Read out the HW state and initialize the TypeC mode of @dig_port. The mode
+ * will be locked until intel_tc_port_sanitize_mode() is called.
+ */
+void intel_tc_port_init_mode(struct intel_digital_port *dig_port)
+{
+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
+ intel_wakeref_t tc_cold_wref;
+ enum intel_display_power_domain domain;
+
+ mutex_lock(&dig_port->tc_lock);
+
+ drm_WARN_ON(&i915->drm, dig_port->tc_mode != TC_PORT_DISCONNECTED);
+ drm_WARN_ON(&i915->drm, dig_port->tc_lock_wakeref);
+ drm_WARN_ON(&i915->drm, dig_port->tc_link_refcount);
+
+ tc_cold_wref = tc_cold_block(dig_port, &domain);
+
+ dig_port->tc_mode = intel_tc_port_get_current_mode(dig_port);
+ /* Prevent changing dig_port->tc_mode until intel_tc_port_sanitize_mode() is called. */
+ intel_tc_port_link_init_refcount(dig_port, 1);
+ dig_port->tc_lock_wakeref = tc_cold_block(dig_port, &dig_port->tc_lock_power_domain);
+
+ tc_cold_unblock(dig_port, domain, tc_cold_wref);
+
+ drm_dbg_kms(&i915->drm, "Port %s: init mode (%s)\n",
+ dig_port->tc_port_name,
+ tc_port_mode_name(dig_port->tc_mode));
+
+ mutex_unlock(&dig_port->tc_lock);
+}
+
+/**
+ * intel_tc_port_sanitize_mode: Sanitize the given port's TypeC mode
+ * @dig_port: digital port
+ *
+ * Sanitize @dig_port's TypeC mode wrt. the encoder's state right after driver
+ * loading and system resume:
+ * If the encoder is enabled keep the TypeC mode/PHY connected state locked until
+ * the encoder is disabled.
+ * If the encoder is disabled make sure the PHY is disconnected.
+ */
+void intel_tc_port_sanitize_mode(struct intel_digital_port *dig_port)
{
struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
struct intel_encoder *encoder = &dig_port->base;
- intel_wakeref_t tc_cold_wref;
- enum intel_display_power_domain domain;
int active_links = 0;
mutex_lock(&dig_port->tc_lock);
@@ -708,21 +744,14 @@ void intel_tc_port_sanitize(struct intel_digital_port *dig_port)
else if (encoder->base.crtc)
active_links = to_intel_crtc(encoder->base.crtc)->active;
- drm_WARN_ON(&i915->drm, dig_port->tc_mode != TC_PORT_DISCONNECTED);
- drm_WARN_ON(&i915->drm, dig_port->tc_lock_wakeref);
+ drm_WARN_ON(&i915->drm, dig_port->tc_link_refcount != 1);
+ intel_tc_port_link_init_refcount(dig_port, active_links);
- tc_cold_wref = tc_cold_block(dig_port, &domain);
-
- dig_port->tc_mode = intel_tc_port_get_current_mode(dig_port);
if (active_links) {
if (!icl_tc_phy_is_connected(dig_port))
drm_dbg_kms(&i915->drm,
"Port %s: PHY disconnected with %d active link(s)\n",
dig_port->tc_port_name, active_links);
- intel_tc_port_link_init_refcount(dig_port, active_links);
-
- dig_port->tc_lock_wakeref = tc_cold_block(dig_port,
- &dig_port->tc_lock_power_domain);
} else {
/*
* TBT-alt is the default mode in any case the PHY ownership is not
@@ -736,9 +765,10 @@ void intel_tc_port_sanitize(struct intel_digital_port *dig_port)
dig_port->tc_port_name,
tc_port_mode_name(dig_port->tc_mode));
icl_tc_phy_disconnect(dig_port);
- }
- tc_cold_unblock(dig_port, domain, tc_cold_wref);
+ tc_cold_unblock(dig_port, dig_port->tc_lock_power_domain,
+ fetch_and_zero(&dig_port->tc_lock_wakeref));
+ }
drm_dbg_kms(&i915->drm, "Port %s: sanitize mode (%s)\n",
dig_port->tc_port_name,
@@ -923,4 +953,6 @@ void intel_tc_port_init(struct intel_digital_port *dig_port, bool is_legacy)
dig_port->tc_mode = TC_PORT_DISCONNECTED;
dig_port->tc_link_refcount = 0;
tc_port_load_fia_params(i915, dig_port);
+
+ intel_tc_port_init_mode(dig_port);
}
diff --git a/drivers/gpu/drm/i915/display/intel_tc.h b/drivers/gpu/drm/i915/display/intel_tc.h
index 6b47b29..d54082e 100644
--- a/drivers/gpu/drm/i915/display/intel_tc.h
+++ b/drivers/gpu/drm/i915/display/intel_tc.h
@@ -24,7 +24,8 @@ int intel_tc_port_fia_max_lane_count(struct intel_digital_port *dig_port);
void intel_tc_port_set_fia_lane_count(struct intel_digital_port *dig_port,
int required_lanes);
-void intel_tc_port_sanitize(struct intel_digital_port *dig_port);
+void intel_tc_port_init_mode(struct intel_digital_port *dig_port);
+void intel_tc_port_sanitize_mode(struct intel_digital_port *dig_port);
void intel_tc_port_lock(struct intel_digital_port *dig_port);
void intel_tc_port_unlock(struct intel_digital_port *dig_port);
void intel_tc_port_flush_work(struct intel_digital_port *dig_port);
diff --git a/drivers/gpu/drm/i915/display/intel_tv.c b/drivers/gpu/drm/i915/display/intel_tv.c
index dcf89d7..4d2101c 100644
--- a/drivers/gpu/drm/i915/display/intel_tv.c
+++ b/drivers/gpu/drm/i915/display/intel_tv.c
@@ -35,6 +35,8 @@
#include <drm/drm_edid.h>
#include "i915_drv.h"
+#include "i915_reg.h"
+#include "i915_irq.h"
#include "intel_connector.h"
#include "intel_crtc.h"
#include "intel_de.h"
@@ -1880,18 +1882,56 @@ static const struct drm_encoder_funcs intel_tv_enc_funcs = {
.destroy = intel_encoder_destroy,
};
+static void intel_tv_add_properties(struct drm_connector *connector)
+{
+ struct drm_i915_private *i915 = to_i915(connector->dev);
+ struct drm_connector_state *conn_state = connector->state;
+ const char *tv_format_names[ARRAY_SIZE(tv_modes)];
+ int i;
+
+ /* BIOS margin values */
+ conn_state->tv.margins.left = 54;
+ conn_state->tv.margins.top = 36;
+ conn_state->tv.margins.right = 46;
+ conn_state->tv.margins.bottom = 37;
+
+ conn_state->tv.mode = 0;
+
+ /* Create TV properties then attach current values */
+ for (i = 0; i < ARRAY_SIZE(tv_modes); i++) {
+ /* 1080p50/1080p60 not supported on gen3 */
+ if (DISPLAY_VER(i915) == 3 && tv_modes[i].oversample == 1)
+ break;
+
+ tv_format_names[i] = tv_modes[i].name;
+ }
+ drm_mode_create_tv_properties(&i915->drm, i, tv_format_names);
+
+ drm_object_attach_property(&connector->base,
+ i915->drm.mode_config.tv_mode_property,
+ conn_state->tv.mode);
+ drm_object_attach_property(&connector->base,
+ i915->drm.mode_config.tv_left_margin_property,
+ conn_state->tv.margins.left);
+ drm_object_attach_property(&connector->base,
+ i915->drm.mode_config.tv_top_margin_property,
+ conn_state->tv.margins.top);
+ drm_object_attach_property(&connector->base,
+ i915->drm.mode_config.tv_right_margin_property,
+ conn_state->tv.margins.right);
+ drm_object_attach_property(&connector->base,
+ i915->drm.mode_config.tv_bottom_margin_property,
+ conn_state->tv.margins.bottom);
+}
+
void
intel_tv_init(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector;
struct intel_tv *intel_tv;
struct intel_encoder *intel_encoder;
struct intel_connector *intel_connector;
u32 tv_dac_on, tv_dac_off, save_tv_dac;
- const char *tv_format_names[ARRAY_SIZE(tv_modes)];
- int i, initial_mode = 0;
- struct drm_connector_state *state;
if ((intel_de_read(dev_priv, TV_CTL) & TV_FUSE_STATE_MASK) == TV_FUSE_STATE_DISABLED)
return;
@@ -1937,7 +1977,6 @@ intel_tv_init(struct drm_i915_private *dev_priv)
intel_encoder = &intel_tv->base;
connector = &intel_connector->base;
- state = connector->state;
/*
* The documentation, for the older chipsets at least, recommend
@@ -1951,10 +1990,10 @@ intel_tv_init(struct drm_i915_private *dev_priv)
*/
intel_connector->polled = DRM_CONNECTOR_POLL_CONNECT;
- drm_connector_init(dev, connector, &intel_tv_connector_funcs,
+ drm_connector_init(&dev_priv->drm, connector, &intel_tv_connector_funcs,
DRM_MODE_CONNECTOR_SVIDEO);
- drm_encoder_init(dev, &intel_encoder->base, &intel_tv_enc_funcs,
+ drm_encoder_init(&dev_priv->drm, &intel_encoder->base, &intel_tv_enc_funcs,
DRM_MODE_ENCODER_TVDAC, "TV");
intel_encoder->compute_config = intel_tv_compute_config;
@@ -1974,41 +2013,7 @@ intel_tv_init(struct drm_i915_private *dev_priv)
intel_encoder->cloneable = 0;
intel_tv->type = DRM_MODE_CONNECTOR_Unknown;
- /* BIOS margin values */
- state->tv.margins.left = 54;
- state->tv.margins.top = 36;
- state->tv.margins.right = 46;
- state->tv.margins.bottom = 37;
-
- state->tv.mode = initial_mode;
-
drm_connector_helper_add(connector, &intel_tv_connector_helper_funcs);
- connector->interlace_allowed = false;
- connector->doublescan_allowed = false;
- /* Create TV properties then attach current values */
- for (i = 0; i < ARRAY_SIZE(tv_modes); i++) {
- /* 1080p50/1080p60 not supported on gen3 */
- if (DISPLAY_VER(dev_priv) == 3 &&
- tv_modes[i].oversample == 1)
- break;
-
- tv_format_names[i] = tv_modes[i].name;
- }
- drm_mode_create_tv_properties(dev, i, tv_format_names);
-
- drm_object_attach_property(&connector->base, dev->mode_config.tv_mode_property,
- state->tv.mode);
- drm_object_attach_property(&connector->base,
- dev->mode_config.tv_left_margin_property,
- state->tv.margins.left);
- drm_object_attach_property(&connector->base,
- dev->mode_config.tv_top_margin_property,
- state->tv.margins.top);
- drm_object_attach_property(&connector->base,
- dev->mode_config.tv_right_margin_property,
- state->tv.margins.right);
- drm_object_attach_property(&connector->base,
- dev->mode_config.tv_bottom_margin_property,
- state->tv.margins.bottom);
+ intel_tv_add_properties(connector);
}
diff --git a/drivers/gpu/drm/i915/display/intel_vdsc.c b/drivers/gpu/drm/i915/display/intel_vdsc.c
index 269f9792..9d3b77b 100644
--- a/drivers/gpu/drm/i915/display/intel_vdsc.c
+++ b/drivers/gpu/drm/i915/display/intel_vdsc.c
@@ -10,6 +10,7 @@
#include <drm/display/drm_dsc_helper.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_crtc.h"
#include "intel_de.h"
#include "intel_display_types.h"
diff --git a/drivers/gpu/drm/i915/display/intel_vga.c b/drivers/gpu/drm/i915/display/intel_vga.c
index b5d0584..a69bfca 100644
--- a/drivers/gpu/drm/i915/display/intel_vga.c
+++ b/drivers/gpu/drm/i915/display/intel_vga.c
@@ -10,6 +10,7 @@
#include <video/vga.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_vga.h"
diff --git a/drivers/gpu/drm/i915/display/intel_vrr.c b/drivers/gpu/drm/i915/display/intel_vrr.c
index 5eac990..7b1357e 100644
--- a/drivers/gpu/drm/i915/display/intel_vrr.c
+++ b/drivers/gpu/drm/i915/display/intel_vrr.c
@@ -5,6 +5,7 @@
*/
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_vrr.h"
diff --git a/drivers/gpu/drm/i915/display/skl_scaler.c b/drivers/gpu/drm/i915/display/skl_scaler.c
index 4092679..d739006 100644
--- a/drivers/gpu/drm/i915/display/skl_scaler.c
+++ b/drivers/gpu/drm/i915/display/skl_scaler.c
@@ -2,6 +2,8 @@
/*
* Copyright © 2020 Intel Corporation
*/
+
+#include "i915_reg.h"
#include "intel_de.h"
#include "intel_display_types.h"
#include "intel_fb.h"
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index 7cb7130..76490cc 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -9,6 +9,8 @@
#include <drm/drm_fourcc.h>
#include "i915_drv.h"
+#include "i915_irq.h"
+#include "i915_reg.h"
#include "intel_atomic_plane.h"
#include "intel_de.h"
#include "intel_display_types.h"
@@ -246,6 +248,11 @@ bool icl_is_nv12_y_plane(struct drm_i915_private *dev_priv,
icl_nv12_y_plane_mask(dev_priv) & BIT(plane_id);
}
+u8 icl_hdr_plane_mask(void)
+{
+ return BIT(PLANE_PRIMARY) | BIT(PLANE_SPRITE0) | BIT(PLANE_SPRITE1);
+}
+
bool icl_is_hdr_plane(struct drm_i915_private *dev_priv, enum plane_id plane_id)
{
return DISPLAY_VER(dev_priv) >= 11 &&
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.h b/drivers/gpu/drm/i915/display/skl_universal_plane.h
index 351040b..be64c20 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.h
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.h
@@ -30,6 +30,7 @@ int skl_calc_main_surface_offset(const struct intel_plane_state *plane_state,
bool icl_is_nv12_y_plane(struct drm_i915_private *dev_priv,
enum plane_id plane_id);
+u8 icl_hdr_plane_mask(void);
bool icl_is_hdr_plane(struct drm_i915_private *dev_priv, enum plane_id plane_id);
#endif
diff --git a/drivers/gpu/drm/i915/display/skl_watermark.c b/drivers/gpu/drm/i915/display/skl_watermark.c
index 18178b0..e0766d1 100644
--- a/drivers/gpu/drm/i915/display/skl_watermark.c
+++ b/drivers/gpu/drm/i915/display/skl_watermark.c
@@ -1706,26 +1706,10 @@ skl_compute_wm_params(const struct intel_crtc_state *crtc_state,
return -EINVAL;
}
- wp->y_tiled = modifier == I915_FORMAT_MOD_Y_TILED ||
- modifier == I915_FORMAT_MOD_4_TILED ||
- modifier == I915_FORMAT_MOD_Yf_TILED ||
- modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
- modifier == I915_FORMAT_MOD_Yf_TILED_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC ||
- modifier == I915_FORMAT_MOD_4_TILED_DG2_RC_CCS ||
- modifier == I915_FORMAT_MOD_4_TILED_DG2_MC_CCS ||
- modifier == I915_FORMAT_MOD_4_TILED_DG2_RC_CCS_CC;
wp->x_tiled = modifier == I915_FORMAT_MOD_X_TILED;
- wp->rc_surface = modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
- modifier == I915_FORMAT_MOD_Yf_TILED_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS ||
- modifier == I915_FORMAT_MOD_Y_TILED_GEN12_RC_CCS_CC ||
- modifier == I915_FORMAT_MOD_4_TILED_DG2_RC_CCS ||
- modifier == I915_FORMAT_MOD_4_TILED_DG2_MC_CCS ||
- modifier == I915_FORMAT_MOD_4_TILED_DG2_RC_CCS_CC;
+ wp->y_tiled = modifier != I915_FORMAT_MOD_X_TILED &&
+ intel_fb_is_tiled_modifier(modifier);
+ wp->rc_surface = intel_fb_is_ccs_modifier(modifier);
wp->is_planar = intel_format_info_is_yuv_semiplanar(format, modifier);
wp->width = width;
@@ -2498,7 +2482,7 @@ skl_compute_ddb(struct intel_atomic_state *state)
if (old_dbuf_state->joined_mbus != new_dbuf_state->joined_mbus) {
/* TODO: Implement vblank synchronized MBUS joining changes */
- ret = intel_modeset_all_pipes(state);
+ ret = intel_modeset_all_pipes(state, "MBUS joining change");
if (ret)
return ret;
}
@@ -2760,7 +2744,7 @@ static int skl_wm_add_affected_planes(struct intel_atomic_state *state,
* power well the hardware state will go out of sync
* with the software state.
*/
- if (!drm_atomic_crtc_needs_modeset(&new_crtc_state->uapi) &&
+ if (!intel_crtc_needs_modeset(new_crtc_state) &&
skl_plane_selected_wm_equals(plane,
&old_crtc_state->wm.skl.optimal,
&new_crtc_state->wm.skl.optimal))
diff --git a/drivers/gpu/drm/i915/display/vlv_dsi.c b/drivers/gpu/drm/i915/display/vlv_dsi.c
index b3f5ca2..84481030 100644
--- a/drivers/gpu/drm/i915/display/vlv_dsi.c
+++ b/drivers/gpu/drm/i915/display/vlv_dsi.c
@@ -31,6 +31,7 @@
#include <drm/drm_mipi_dsi.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_atomic.h"
#include "intel_backlight.h"
#include "intel_connector.h"
@@ -1659,19 +1660,10 @@ static const struct drm_connector_funcs intel_dsi_connector_funcs = {
static void vlv_dsi_add_properties(struct intel_connector *connector)
{
- struct drm_i915_private *dev_priv = to_i915(connector->base.dev);
const struct drm_display_mode *fixed_mode =
intel_panel_preferred_fixed_mode(connector);
- u32 allowed_scalers;
- allowed_scalers = BIT(DRM_MODE_SCALE_ASPECT) | BIT(DRM_MODE_SCALE_FULLSCREEN);
- if (!HAS_GMCH(dev_priv))
- allowed_scalers |= BIT(DRM_MODE_SCALE_CENTER);
-
- drm_connector_attach_scaling_mode_property(&connector->base,
- allowed_scalers);
-
- connector->base.state->scaling_mode = DRM_MODE_SCALE_ASPECT;
+ intel_attach_scaling_mode_property(&connector->base);
drm_connector_set_panel_orientation_with_quirk(&connector->base,
intel_dsi_get_panel_orientation(connector),
@@ -1854,7 +1846,6 @@ static void vlv_dphy_param_init(struct intel_dsi *intel_dsi)
void vlv_dsi_init(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_dsi *intel_dsi;
struct intel_encoder *intel_encoder;
struct drm_encoder *encoder;
@@ -1891,7 +1882,7 @@ void vlv_dsi_init(struct drm_i915_private *dev_priv)
connector = &intel_connector->base;
- drm_encoder_init(dev, encoder, &intel_dsi_funcs, DRM_MODE_ENCODER_DSI,
+ drm_encoder_init(&dev_priv->drm, encoder, &intel_dsi_funcs, DRM_MODE_ENCODER_DSI,
"DSI %c", port_name(port));
intel_encoder->compute_config = intel_dsi_compute_config;
@@ -1974,20 +1965,18 @@ void vlv_dsi_init(struct drm_i915_private *dev_priv)
intel_dsi_vbt_gpio_init(intel_dsi,
intel_dsi_get_hw_state(intel_encoder, &pipe));
- drm_connector_init(dev, connector, &intel_dsi_connector_funcs,
+ drm_connector_init(&dev_priv->drm, connector, &intel_dsi_connector_funcs,
DRM_MODE_CONNECTOR_DSI);
drm_connector_helper_add(connector, &intel_dsi_connector_helper_funcs);
connector->display_info.subpixel_order = SubPixelHorizontalRGB; /*XXX*/
- connector->interlace_allowed = false;
- connector->doublescan_allowed = false;
intel_connector_attach_encoder(intel_connector, intel_encoder);
- mutex_lock(&dev->mode_config.mutex);
+ mutex_lock(&dev_priv->drm.mode_config.mutex);
intel_panel_add_vbt_lfp_fixed_mode(intel_connector);
- mutex_unlock(&dev->mode_config.mutex);
+ mutex_unlock(&dev_priv->drm.mode_config.mutex);
if (!intel_panel_preferred_fixed_mode(intel_connector)) {
drm_dbg_kms(&dev_priv->drm, "no fixed mode\n");
diff --git a/drivers/gpu/drm/i915/display/vlv_dsi_regs.h b/drivers/gpu/drm/i915/display/vlv_dsi_regs.h
index e065b8f..abbe427 100644
--- a/drivers/gpu/drm/i915/display/vlv_dsi_regs.h
+++ b/drivers/gpu/drm/i915/display/vlv_dsi_regs.h
@@ -6,7 +6,7 @@
#ifndef __VLV_DSI_REGS_H__
#define __VLV_DSI_REGS_H__
-#include "i915_reg_defs.h"
+#include "intel_display_reg_defs.h"
#define VLV_MIPI_BASE VLV_DISPLAY_BASE
#define BXT_MIPI_BASE 0x60000
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
index 0512afdd..b3b398f 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
@@ -113,7 +113,7 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj,
clflush = clflush_work_create(obj);
if (clflush) {
i915_sw_fence_await_reservation(&clflush->base.chain,
- obj->base.resv, NULL, true,
+ obj->base.resv, true,
i915_fence_timeout(i915),
I915_FENCE_GFP);
dma_resv_add_fence(obj->base.resv, &clflush->base.dma,
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index 1e29b1e..7f2831e 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -546,7 +546,7 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)
}
if (intel_engine_uses_guc(master)) {
- DRM_DEBUG("bonding extension not supported with GuC submission");
+ drm_dbg(&i915->drm, "bonding extension not supported with GuC submission");
return -ENODEV;
}
@@ -1452,7 +1452,7 @@ static void engines_idle_release(struct i915_gem_context *ctx,
int err;
/* serialises with execbuf */
- set_bit(CONTEXT_CLOSED_BIT, &ce->flags);
+ intel_context_close(ce);
if (!intel_context_pin_if_active(ce))
continue;
@@ -2298,7 +2298,6 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
}
args->ctx_id = id;
- drm_dbg(&i915->drm, "HW context %d created\n", args->ctx_id);
return 0;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
index 824971a1..fd556a0 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
@@ -25,43 +25,44 @@ static struct drm_i915_gem_object *dma_buf_to_obj(struct dma_buf *buf)
return to_intel_bo(buf->priv);
}
-static struct sg_table *i915_gem_map_dma_buf(struct dma_buf_attachment *attachment,
+static struct sg_table *i915_gem_map_dma_buf(struct dma_buf_attachment *attach,
enum dma_data_direction dir)
{
- struct drm_i915_gem_object *obj = dma_buf_to_obj(attachment->dmabuf);
- struct sg_table *st;
+ struct drm_i915_gem_object *obj = dma_buf_to_obj(attach->dmabuf);
+ struct sg_table *sgt;
struct scatterlist *src, *dst;
int ret, i;
- /* Copy sg so that we make an independent mapping */
- st = kmalloc(sizeof(struct sg_table), GFP_KERNEL);
- if (st == NULL) {
+ /*
+ * Make a copy of the object's sgt, so that we can make an independent
+ * mapping
+ */
+ sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
+ if (!sgt) {
ret = -ENOMEM;
goto err;
}
- ret = sg_alloc_table(st, obj->mm.pages->orig_nents, GFP_KERNEL);
+ ret = sg_alloc_table(sgt, obj->mm.pages->orig_nents, GFP_KERNEL);
if (ret)
goto err_free;
- src = obj->mm.pages->sgl;
- dst = st->sgl;
- for (i = 0; i < obj->mm.pages->orig_nents; i++) {
+ dst = sgt->sgl;
+ for_each_sg(obj->mm.pages->sgl, src, obj->mm.pages->orig_nents, i) {
sg_set_page(dst, sg_page(src), src->length, 0);
dst = sg_next(dst);
- src = sg_next(src);
}
- ret = dma_map_sgtable(attachment->dev, st, dir, DMA_ATTR_SKIP_CPU_SYNC);
+ ret = dma_map_sgtable(attach->dev, sgt, dir, DMA_ATTR_SKIP_CPU_SYNC);
if (ret)
goto err_free_sg;
- return st;
+ return sgt;
err_free_sg:
- sg_free_table(st);
+ sg_free_table(sgt);
err_free:
- kfree(st);
+ kfree(sgt);
err:
return ERR_PTR(ret);
}
@@ -72,7 +73,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
void *vaddr;
- vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
+ vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
if (IS_ERR(vaddr))
return PTR_ERR(vaddr);
@@ -96,6 +97,8 @@ static int i915_gem_dmabuf_mmap(struct dma_buf *dma_buf, struct vm_area_struct *
struct drm_i915_private *i915 = to_i915(obj->base.dev);
int ret;
+ dma_resv_assert_held(dma_buf->resv);
+
if (obj->base.size < vma->vm_end - vma->vm_start)
return -EINVAL;
@@ -236,15 +239,14 @@ struct dma_buf *i915_gem_prime_export(struct drm_gem_object *gem_obj, int flags)
static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
{
struct drm_i915_private *i915 = to_i915(obj->base.dev);
- struct sg_table *pages;
- unsigned int sg_page_sizes;
+ struct sg_table *sgt;
assert_object_held(obj);
- pages = dma_buf_map_attachment(obj->base.import_attach,
- DMA_BIDIRECTIONAL);
- if (IS_ERR(pages))
- return PTR_ERR(pages);
+ sgt = dma_buf_map_attachment(obj->base.import_attach,
+ DMA_BIDIRECTIONAL);
+ if (IS_ERR(sgt))
+ return PTR_ERR(sgt);
/*
* DG1 is special here since it still snoops transactions even with
@@ -261,16 +263,15 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
(!HAS_LLC(i915) && !IS_DG1(i915)))
wbinvd_on_all_cpus();
- sg_page_sizes = i915_sg_dma_sizes(pages->sgl);
- __i915_gem_object_set_pages(obj, pages, sg_page_sizes);
+ __i915_gem_object_set_pages(obj, sgt);
return 0;
}
static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
- struct sg_table *pages)
+ struct sg_table *sgt)
{
- dma_buf_unmap_attachment(obj->base.import_attach, pages,
+ dma_buf_unmap_attachment(obj->base.import_attach, sgt,
DMA_BIDIRECTIONAL);
}
@@ -313,7 +314,7 @@ struct drm_gem_object *i915_gem_prime_import(struct drm_device *dev,
get_dma_buf(dma_buf);
obj = i915_gem_object_alloc();
- if (obj == NULL) {
+ if (!obj) {
ret = -ENOMEM;
goto fail_detach;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 29d2459..da09767 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -30,6 +30,7 @@
#include "i915_gem_context.h"
#include "i915_gem_evict.h"
#include "i915_gem_ioctls.h"
+#include "i915_reg.h"
#include "i915_trace.h"
#include "i915_user_extensions.h"
@@ -53,13 +54,13 @@ enum {
#define DBG_FORCE_RELOC 0 /* choose one of the above! */
};
-/* __EXEC_OBJECT_NO_RESERVE is BIT(31), defined in i915_vma.h */
-#define __EXEC_OBJECT_HAS_PIN BIT(30)
-#define __EXEC_OBJECT_HAS_FENCE BIT(29)
-#define __EXEC_OBJECT_USERPTR_INIT BIT(28)
-#define __EXEC_OBJECT_NEEDS_MAP BIT(27)
-#define __EXEC_OBJECT_NEEDS_BIAS BIT(26)
-#define __EXEC_OBJECT_INTERNAL_FLAGS (~0u << 26) /* all of the above + */
+/* __EXEC_OBJECT_ flags > BIT(29) defined in i915_vma.h */
+#define __EXEC_OBJECT_HAS_PIN BIT(29)
+#define __EXEC_OBJECT_HAS_FENCE BIT(28)
+#define __EXEC_OBJECT_USERPTR_INIT BIT(27)
+#define __EXEC_OBJECT_NEEDS_MAP BIT(26)
+#define __EXEC_OBJECT_NEEDS_BIAS BIT(25)
+#define __EXEC_OBJECT_INTERNAL_FLAGS (~0u << 25) /* all of the above + */
#define __EXEC_OBJECT_RESERVED (__EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_FENCE)
#define __EXEC_HAS_RELOC BIT(31)
@@ -2101,7 +2102,8 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
eb->composite_fence ?
eb->composite_fence :
&eb->requests[j]->fence,
- flags | __EXEC_OBJECT_NO_RESERVE);
+ flags | __EXEC_OBJECT_NO_RESERVE |
+ __EXEC_OBJECT_NO_REQUEST_AWAIT);
}
}
@@ -2148,7 +2150,8 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
return err;
}
-static int i915_gem_check_execbuffer(struct drm_i915_gem_execbuffer2 *exec)
+static int i915_gem_check_execbuffer(struct drm_i915_private *i915,
+ struct drm_i915_gem_execbuffer2 *exec)
{
if (exec->flags & __I915_EXEC_ILLEGAL_FLAGS)
return -EINVAL;
@@ -2161,7 +2164,7 @@ static int i915_gem_check_execbuffer(struct drm_i915_gem_execbuffer2 *exec)
}
if (exec->DR4 == 0xffffffff) {
- DRM_DEBUG("UXA submitting garbage DR4, fixing up\n");
+ drm_dbg(&i915->drm, "UXA submitting garbage DR4, fixing up\n");
exec->DR4 = 0;
}
if (exec->DR1 || exec->DR4)
@@ -2799,7 +2802,8 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
syncobj = drm_syncobj_find(eb->file, user_fence.handle);
if (!syncobj) {
- DRM_DEBUG("Invalid syncobj handle provided\n");
+ drm_dbg(&eb->i915->drm,
+ "Invalid syncobj handle provided\n");
return -ENOENT;
}
@@ -2807,7 +2811,8 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
if (!fence && user_fence.flags &&
!(user_fence.flags & I915_EXEC_FENCE_SIGNAL)) {
- DRM_DEBUG("Syncobj handle has no fence\n");
+ drm_dbg(&eb->i915->drm,
+ "Syncobj handle has no fence\n");
drm_syncobj_put(syncobj);
return -EINVAL;
}
@@ -2816,7 +2821,9 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
err = dma_fence_chain_find_seqno(&fence, point);
if (err && !(user_fence.flags & I915_EXEC_FENCE_SIGNAL)) {
- DRM_DEBUG("Syncobj handle missing requested point %llu\n", point);
+ drm_dbg(&eb->i915->drm,
+ "Syncobj handle missing requested point %llu\n",
+ point);
dma_fence_put(fence);
drm_syncobj_put(syncobj);
return err;
@@ -2842,7 +2849,8 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
* 0) would break the timeline.
*/
if (user_fence.flags & I915_EXEC_FENCE_WAIT) {
- DRM_DEBUG("Trying to wait & signal the same timeline point.\n");
+ drm_dbg(&eb->i915->drm,
+ "Trying to wait & signal the same timeline point.\n");
dma_fence_put(fence);
drm_syncobj_put(syncobj);
return -EINVAL;
@@ -2913,14 +2921,16 @@ static int add_fence_array(struct i915_execbuffer *eb)
syncobj = drm_syncobj_find(eb->file, user_fence.handle);
if (!syncobj) {
- DRM_DEBUG("Invalid syncobj handle provided\n");
+ drm_dbg(&eb->i915->drm,
+ "Invalid syncobj handle provided\n");
return -ENOENT;
}
if (user_fence.flags & I915_EXEC_FENCE_WAIT) {
fence = drm_syncobj_fence_get(syncobj);
if (!fence) {
- DRM_DEBUG("Syncobj handle has no fence\n");
+ drm_dbg(&eb->i915->drm,
+ "Syncobj handle has no fence\n");
drm_syncobj_put(syncobj);
return -EINVAL;
}
@@ -2954,11 +2964,6 @@ await_fence_array(struct i915_execbuffer *eb,
int err;
for (n = 0; n < eb->num_fences; n++) {
- struct drm_syncobj *syncobj;
- unsigned int flags;
-
- syncobj = ptr_unpack_bits(eb->fences[n].syncobj, &flags, 2);
-
if (!eb->fences[n].dma_fence)
continue;
@@ -3520,7 +3525,7 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,
return -EINVAL;
}
- err = i915_gem_check_execbuffer(args);
+ err = i915_gem_check_execbuffer(i915, args);
if (err)
return err;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_internal.c b/drivers/gpu/drm/i915/gem/i915_gem_internal.c
index 629acb4..f66bcef 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_internal.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_internal.c
@@ -35,7 +35,6 @@ static int i915_gem_object_get_pages_internal(struct drm_i915_gem_object *obj)
struct drm_i915_private *i915 = to_i915(obj->base.dev);
struct sg_table *st;
struct scatterlist *sg;
- unsigned int sg_page_sizes;
unsigned int npages;
int max_order = MAX_ORDER;
unsigned int max_segment;
@@ -64,7 +63,6 @@ static int i915_gem_object_get_pages_internal(struct drm_i915_gem_object *obj)
sg = st->sgl;
st->nents = 0;
- sg_page_sizes = 0;
do {
int order = min(fls(npages) - 1, max_order);
@@ -83,7 +81,6 @@ static int i915_gem_object_get_pages_internal(struct drm_i915_gem_object *obj)
} while (1);
sg_set_page(sg, page, PAGE_SIZE << order, 0);
- sg_page_sizes |= PAGE_SIZE << order;
st->nents++;
npages -= 1 << order;
@@ -105,7 +102,7 @@ static int i915_gem_object_get_pages_internal(struct drm_i915_gem_object *obj)
goto err;
}
- __i915_gem_object_set_pages(obj, st, sg_page_sizes);
+ __i915_gem_object_set_pages(obj, st);
return 0;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c
index 73d9eda..c29efde 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c
@@ -330,7 +330,7 @@ static vm_fault_t vm_fault_gtt(struct vm_fault *vmf)
if (ret)
goto err_rpm;
- ret = intel_gt_reset_trylock(ggtt->vm.gt, &srcu);
+ ret = intel_gt_reset_lock_interruptible(ggtt->vm.gt, &srcu);
if (ret)
goto err_pages;
@@ -413,7 +413,7 @@ static vm_fault_t vm_fault_gtt(struct vm_fault *vmf)
vma->mmo = mmo;
if (CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND)
- intel_wakeref_auto(&to_gt(i915)->userfault_wakeref,
+ intel_wakeref_auto(&i915->runtime_pm.userfault_wakeref,
msecs_to_jiffies_timeout(CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND));
if (write) {
@@ -557,11 +557,13 @@ void i915_gem_object_runtime_pm_release_mmap_offset(struct drm_i915_gem_object *
drm_vma_node_unmap(&bo->base.vma_node, bdev->dev_mapping);
- if (obj->userfault_count) {
- /* rpm wakeref provide exclusive access */
- list_del(&obj->userfault_link);
- obj->userfault_count = 0;
- }
+ /*
+ * We have exclusive access here via runtime suspend. All other callers
+ * must first grab the rpm wakeref.
+ */
+ GEM_BUG_ON(!obj->userfault_count);
+ list_del(&obj->userfault_link);
+ obj->userfault_count = 0;
}
void i915_gem_object_release_mmap_offset(struct drm_i915_gem_object *obj)
@@ -587,13 +589,6 @@ void i915_gem_object_release_mmap_offset(struct drm_i915_gem_object *obj)
spin_lock(&obj->mmo.lock);
}
spin_unlock(&obj->mmo.lock);
-
- if (obj->userfault_count) {
- mutex_lock(&to_gt(to_i915(obj->base.dev))->lmem_userfault_lock);
- list_del(&obj->userfault_link);
- mutex_unlock(&to_gt(to_i915(obj->base.dev))->lmem_userfault_lock);
- obj->userfault_count = 0;
- }
}
static struct i915_mmap_offset *
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c
index 369006c..7336960 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c
@@ -290,7 +290,21 @@ void __i915_gem_object_pages_fini(struct drm_i915_gem_object *obj)
__i915_gem_object_free_mmaps(obj);
atomic_set(&obj->mm.pages_pin_count, 0);
+
+ /*
+ * dma_buf_unmap_attachment() requires reservation to be
+ * locked. The imported GEM shouldn't share reservation lock
+ * and ttm_bo_cleanup_memtype_use() shouldn't be invoked for
+ * dma-buf, so it's safe to take the lock.
+ */
+ if (obj->base.import_attach)
+ i915_gem_object_lock(obj, NULL);
+
__i915_gem_object_put_pages(obj);
+
+ if (obj->base.import_attach)
+ i915_gem_object_unlock(obj);
+
GEM_BUG_ON(i915_gem_object_has_pages(obj));
}
@@ -444,6 +458,16 @@ i915_gem_object_read_from_page_iomap(struct drm_i915_gem_object *obj, u64 offset
io_mapping_unmap(src_map);
}
+static bool object_has_mappable_iomem(struct drm_i915_gem_object *obj)
+{
+ GEM_BUG_ON(!i915_gem_object_has_iomem(obj));
+
+ if (IS_DGFX(to_i915(obj->base.dev)))
+ return i915_ttm_resource_mappable(i915_gem_to_ttm(obj)->resource);
+
+ return true;
+}
+
/**
* i915_gem_object_read_from_page - read data from the page of a GEM object
* @obj: GEM object to read from
@@ -466,7 +490,7 @@ int i915_gem_object_read_from_page(struct drm_i915_gem_object *obj, u64 offset,
if (i915_gem_object_has_struct_page(obj))
i915_gem_object_read_from_page_kmap(obj, offset, dst, size);
- else if (i915_gem_object_has_iomem(obj))
+ else if (i915_gem_object_has_iomem(obj) && object_has_mappable_iomem(obj))
i915_gem_object_read_from_page_iomap(obj, offset, dst, size);
else
return -ENODEV;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h
index 1723af9..3db5376 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h
@@ -403,8 +403,7 @@ i915_gem_object_get_dma_address(struct drm_i915_gem_object *obj,
unsigned long n);
void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
- struct sg_table *pages,
- unsigned int sg_page_sizes);
+ struct sg_table *pages);
int ____i915_gem_object_get_pages(struct drm_i915_gem_object *obj);
int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj);
@@ -482,6 +481,10 @@ void *__must_check i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
void *__must_check i915_gem_object_pin_map_unlocked(struct drm_i915_gem_object *obj,
enum i915_map_type type);
+enum i915_map_type i915_coherent_map_type(struct drm_i915_private *i915,
+ struct drm_i915_gem_object *obj,
+ bool always_coherent);
+
void __i915_gem_object_flush_map(struct drm_i915_gem_object *obj,
unsigned long offset,
unsigned long size);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i915/gem/i915_gem_pages.c
index 4df50b0..05a2772 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c
@@ -16,8 +16,7 @@
#include "i915_gem_mman.h"
void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
- struct sg_table *pages,
- unsigned int sg_page_sizes)
+ struct sg_table *pages)
{
struct drm_i915_private *i915 = to_i915(obj->base.dev);
unsigned long supported = RUNTIME_INFO(i915)->page_sizes;
@@ -45,8 +44,8 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
obj->mm.pages = pages;
- GEM_BUG_ON(!sg_page_sizes);
- obj->mm.page_sizes.phys = sg_page_sizes;
+ obj->mm.page_sizes.phys = i915_sg_dma_sizes(pages->sgl);
+ GEM_BUG_ON(!obj->mm.page_sizes.phys);
/*
* Calculate the supported page-sizes which fit into the given
@@ -466,6 +465,18 @@ void *i915_gem_object_pin_map_unlocked(struct drm_i915_gem_object *obj,
return ret;
}
+enum i915_map_type i915_coherent_map_type(struct drm_i915_private *i915,
+ struct drm_i915_gem_object *obj,
+ bool always_coherent)
+{
+ if (i915_gem_object_is_lmem(obj))
+ return I915_MAP_WC;
+ if (HAS_LLC(i915) || always_coherent)
+ return I915_MAP_WB;
+ else
+ return I915_MAP_WC;
+}
+
void __i915_gem_object_flush_map(struct drm_i915_gem_object *obj,
unsigned long offset,
unsigned long size)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_phys.c b/drivers/gpu/drm/i915/gem/i915_gem_phys.c
index 0d0e46da..6845357 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_phys.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_phys.c
@@ -79,7 +79,7 @@ static int i915_gem_object_get_pages_phys(struct drm_i915_gem_object *obj)
/* We're no longer struct page backed */
obj->mem_flags &= ~I915_BO_FLAG_STRUCT_PAGE;
- __i915_gem_object_set_pages(obj, st, sg->length);
+ __i915_gem_object_set_pages(obj, st);
return 0;
@@ -209,11 +209,8 @@ static int i915_gem_object_shmem_to_phys(struct drm_i915_gem_object *obj)
return 0;
err_xfer:
- if (!IS_ERR_OR_NULL(pages)) {
- unsigned int sg_page_sizes = i915_sg_dma_sizes(pages->sgl);
-
- __i915_gem_object_set_pages(obj, pages, sg_page_sizes);
- }
+ if (!IS_ERR_OR_NULL(pages))
+ __i915_gem_object_set_pages(obj, pages);
return err;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c
index 3428f73..0d812f4 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c
@@ -22,9 +22,12 @@
void i915_gem_suspend(struct drm_i915_private *i915)
{
+ struct intel_gt *gt;
+ unsigned int i;
+
GEM_TRACE("%s\n", dev_name(i915->drm.dev));
- intel_wakeref_auto(&to_gt(i915)->userfault_wakeref, 0);
+ intel_wakeref_auto(&i915->runtime_pm.userfault_wakeref, 0);
flush_workqueue(i915->wq);
/*
@@ -36,7 +39,8 @@ void i915_gem_suspend(struct drm_i915_private *i915)
* state. Fortunately, the kernel_context is disposable and we do
* not rely on its state.
*/
- intel_gt_suspend_prepare(to_gt(i915));
+ for_each_gt(gt, i915, i)
+ intel_gt_suspend_prepare(gt);
i915_gem_drain_freed_objects(i915);
}
@@ -131,7 +135,9 @@ void i915_gem_suspend_late(struct drm_i915_private *i915)
&i915->mm.purge_list,
NULL
}, **phase;
+ struct intel_gt *gt;
unsigned long flags;
+ unsigned int i;
bool flush = false;
/*
@@ -154,7 +160,8 @@ void i915_gem_suspend_late(struct drm_i915_private *i915)
* machine in an unusable condition.
*/
- intel_gt_suspend_late(to_gt(i915));
+ for_each_gt(gt, i915, i)
+ intel_gt_suspend_late(gt);
spin_lock_irqsave(&i915->mm.obj_lock, flags);
for (phase = phases; *phase; phase++) {
@@ -212,7 +219,8 @@ int i915_gem_freeze_late(struct drm_i915_private *i915)
void i915_gem_resume(struct drm_i915_private *i915)
{
- int ret;
+ struct intel_gt *gt;
+ int ret, i, j;
GEM_TRACE("%s\n", dev_name(i915->drm.dev));
@@ -224,8 +232,25 @@ void i915_gem_resume(struct drm_i915_private *i915)
* guarantee that the context image is complete. So let's just reset
* it and start again.
*/
- intel_gt_resume(to_gt(i915));
+ for_each_gt(gt, i915, i)
+ if (intel_gt_resume(gt))
+ goto err_wedged;
ret = lmem_restore(i915, I915_TTM_BACKUP_ALLOW_GPU);
GEM_WARN_ON(ret);
+
+ return;
+
+err_wedged:
+ for_each_gt(gt, i915, j) {
+ if (!intel_gt_is_wedged(gt)) {
+ dev_err(i915->drm.dev,
+ "Failed to re-initialize GPU[%u], declaring it wedged!\n",
+ j);
+ intel_gt_set_wedged(gt);
+ }
+
+ if (j == i)
+ break;
+ }
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
index 2f78044..9c759df 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
@@ -247,7 +247,7 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
if (i915_gem_object_can_bypass_llc(obj))
obj->cache_dirty = true;
- __i915_gem_object_set_pages(obj, st, i915_sg_dma_sizes(st->sgl));
+ __i915_gem_object_set_pages(obj, st);
return 0;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
index acc561c..bc95210 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
@@ -77,22 +77,26 @@ void i915_gem_stolen_remove_node(struct drm_i915_private *i915,
mutex_unlock(&i915->mm.stolen_lock);
}
-static int i915_adjust_stolen(struct drm_i915_private *i915,
- struct resource *dsm)
+static bool valid_stolen_size(struct drm_i915_private *i915, struct resource *dsm)
+{
+ return (dsm->start != 0 || HAS_LMEMBAR_SMEM_STOLEN(i915)) && dsm->end > dsm->start;
+}
+
+static int adjust_stolen(struct drm_i915_private *i915,
+ struct resource *dsm)
{
struct i915_ggtt *ggtt = to_gt(i915)->ggtt;
struct intel_uncore *uncore = ggtt->vm.gt->uncore;
- struct resource *r;
- if (dsm->start == 0 || dsm->end <= dsm->start)
+ if (!valid_stolen_size(i915, dsm))
return -EINVAL;
/*
+ * Make sure we don't clobber the GTT if it's within stolen memory
+ *
* TODO: We have yet too encounter the case where the GTT wasn't at the
* end of stolen. With that assumption we could simplify this.
*/
-
- /* Make sure we don't clobber the GTT if it's within stolen memory */
if (GRAPHICS_VER(i915) <= 4 &&
!IS_G33(i915) && !IS_PINEVIEW(i915) && !IS_G4X(i915)) {
struct resource stolen[2] = {*dsm, *dsm};
@@ -131,12 +135,25 @@ static int i915_adjust_stolen(struct drm_i915_private *i915,
}
}
+ if (!valid_stolen_size(i915, dsm))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int request_smem_stolen(struct drm_i915_private *i915,
+ struct resource *dsm)
+{
+ struct resource *r;
+
/*
- * With stolen lmem, we don't need to check if the address range
- * overlaps with the non-stolen system memory range, since lmem is local
- * to the gpu.
+ * With stolen lmem, we don't need to request system memory for the
+ * address range since it's local to the gpu.
+ *
+ * Starting MTL, in IGFX devices the stolen memory is exposed via
+ * LMEMBAR and shall be considered similar to stolen lmem.
*/
- if (HAS_LMEM(i915))
+ if (HAS_LMEM(i915) || HAS_LMEMBAR_SMEM_STOLEN(i915))
return 0;
/*
@@ -371,8 +388,6 @@ static void icl_get_stolen_reserved(struct drm_i915_private *i915,
drm_dbg(&i915->drm, "GEN6_STOLEN_RESERVED = 0x%016llx\n", reg_val);
- *base = reg_val & GEN11_STOLEN_RESERVED_ADDR_MASK;
-
switch (reg_val & GEN8_STOLEN_RESERVED_SIZE_MASK) {
case GEN8_STOLEN_RESERVED_1M:
*size = 1024 * 1024;
@@ -390,41 +405,30 @@ static void icl_get_stolen_reserved(struct drm_i915_private *i915,
*size = 8 * 1024 * 1024;
MISSING_CASE(reg_val & GEN8_STOLEN_RESERVED_SIZE_MASK);
}
+
+ if (HAS_LMEMBAR_SMEM_STOLEN(i915))
+ /* the base is initialized to stolen top so subtract size to get base */
+ *base -= *size;
+ else
+ *base = reg_val & GEN11_STOLEN_RESERVED_ADDR_MASK;
}
-static int i915_gem_init_stolen(struct intel_memory_region *mem)
+/*
+ * Initialize i915->dsm_reserved to contain the reserved space within the Data
+ * Stolen Memory. This is a range on the top of DSM that is reserved, not to
+ * be used by driver, so must be excluded from the region passed to the
+ * allocator later. In the spec this is also called as WOPCM.
+ *
+ * Our expectation is that the reserved space is at the top of the stolen
+ * region, as it has been the case for every platform, and *never* at the
+ * bottom, so the calculation here can be simplified.
+ */
+static int init_reserved_stolen(struct drm_i915_private *i915)
{
- struct drm_i915_private *i915 = mem->i915;
struct intel_uncore *uncore = &i915->uncore;
resource_size_t reserved_base, stolen_top;
- resource_size_t reserved_total, reserved_size;
-
- mutex_init(&i915->mm.stolen_lock);
-
- if (intel_vgpu_active(i915)) {
- drm_notice(&i915->drm,
- "%s, disabling use of stolen memory\n",
- "iGVT-g active");
- return 0;
- }
-
- if (i915_vtd_active(i915) && GRAPHICS_VER(i915) < 8) {
- drm_notice(&i915->drm,
- "%s, disabling use of stolen memory\n",
- "DMAR active");
- return 0;
- }
-
- if (resource_size(&mem->region) == 0)
- return 0;
-
- i915->dsm = mem->region;
-
- if (i915_adjust_stolen(i915, &i915->dsm))
- return 0;
-
- GEM_BUG_ON(i915->dsm.start == 0);
- GEM_BUG_ON(i915->dsm.end <= i915->dsm.start);
+ resource_size_t reserved_size;
+ int ret = 0;
stolen_top = i915->dsm.end + 1;
reserved_base = stolen_top;
@@ -455,17 +459,16 @@ static int i915_gem_init_stolen(struct intel_memory_region *mem)
&reserved_base, &reserved_size);
}
- /*
- * Our expectation is that the reserved space is at the top of the
- * stolen region and *never* at the bottom. If we see !reserved_base,
- * it likely means we failed to read the registers correctly.
- */
+ /* No reserved stolen */
+ if (reserved_base == stolen_top)
+ goto bail_out;
+
if (!reserved_base) {
drm_err(&i915->drm,
"inconsistent reservation %pa + %pa; ignoring\n",
&reserved_base, &reserved_size);
- reserved_base = stolen_top;
- reserved_size = 0;
+ ret = -EINVAL;
+ goto bail_out;
}
i915->dsm_reserved =
@@ -475,19 +478,55 @@ static int i915_gem_init_stolen(struct intel_memory_region *mem)
drm_err(&i915->drm,
"Stolen reserved area %pR outside stolen memory %pR\n",
&i915->dsm_reserved, &i915->dsm);
- return 0;
+ ret = -EINVAL;
+ goto bail_out;
}
+ return 0;
+
+bail_out:
+ i915->dsm_reserved =
+ (struct resource)DEFINE_RES_MEM(reserved_base, 0);
+
+ return ret;
+}
+
+static int i915_gem_init_stolen(struct intel_memory_region *mem)
+{
+ struct drm_i915_private *i915 = mem->i915;
+
+ mutex_init(&i915->mm.stolen_lock);
+
+ if (intel_vgpu_active(i915)) {
+ drm_notice(&i915->drm,
+ "%s, disabling use of stolen memory\n",
+ "iGVT-g active");
+ return -ENOSPC;
+ }
+
+ if (i915_vtd_active(i915) && GRAPHICS_VER(i915) < 8) {
+ drm_notice(&i915->drm,
+ "%s, disabling use of stolen memory\n",
+ "DMAR active");
+ return -ENOSPC;
+ }
+
+ if (adjust_stolen(i915, &mem->region))
+ return -ENOSPC;
+
+ if (request_smem_stolen(i915, &mem->region))
+ return -ENOSPC;
+
+ i915->dsm = mem->region;
+
+ if (init_reserved_stolen(i915))
+ return -ENOSPC;
+
/* Exclude the reserved region from driver use */
- mem->region.end = reserved_base - 1;
+ mem->region.end = i915->dsm_reserved.start - 1;
mem->io_size = min(mem->io_size, resource_size(&mem->region));
- /* It is possible for the reserved area to end before the end of stolen
- * memory, so just consider the start. */
- reserved_total = stolen_top - reserved_base;
-
- i915->stolen_usable_size =
- resource_size(&i915->dsm) - reserved_total;
+ i915->stolen_usable_size = resource_size(&mem->region);
drm_dbg(&i915->drm,
"Memory reserved for graphics device: %lluK, usable: %lluK\n",
@@ -495,7 +534,7 @@ static int i915_gem_init_stolen(struct intel_memory_region *mem)
(u64)i915->stolen_usable_size >> 10);
if (i915->stolen_usable_size == 0)
- return 0;
+ return -ENOSPC;
/* Basic memrange allocator for stolen space. */
drm_mm_init(&i915->mm.stolen, 0, i915->stolen_usable_size);
@@ -589,7 +628,7 @@ static int i915_gem_object_get_pages_stolen(struct drm_i915_gem_object *obj)
sg_dma_len(pages->sgl),
POISON_INUSE);
- __i915_gem_object_set_pages(obj, pages, obj->stolen->size);
+ __i915_gem_object_set_pages(obj, pages);
return 0;
}
@@ -733,11 +772,17 @@ i915_gem_object_create_stolen(struct drm_i915_private *i915,
static int init_stolen_smem(struct intel_memory_region *mem)
{
+ int err;
+
/*
* Initialise stolen early so that we may reserve preallocated
* objects for the BIOS to KMS transition.
*/
- return i915_gem_init_stolen(mem);
+ err = i915_gem_init_stolen(mem);
+ if (err)
+ drm_dbg(&mem->i915->drm, "Skip stolen region: failed to setup\n");
+
+ return 0;
}
static int release_stolen_smem(struct intel_memory_region *mem)
@@ -754,27 +799,26 @@ static const struct intel_memory_region_ops i915_region_stolen_smem_ops = {
static int init_stolen_lmem(struct intel_memory_region *mem)
{
+ struct drm_i915_private *i915 = mem->i915;
int err;
if (GEM_WARN_ON(resource_size(&mem->region) == 0))
- return -ENODEV;
+ return 0;
- /*
- * TODO: For stolen lmem we mostly just care about populating the dsm
- * related bits and setting up the drm_mm allocator for the range.
- * Perhaps split up i915_gem_init_stolen() for this.
- */
err = i915_gem_init_stolen(mem);
- if (err)
- return err;
-
- if (mem->io_size && !io_mapping_init_wc(&mem->iomap,
- mem->io_start,
- mem->io_size)) {
- err = -EIO;
- goto err_cleanup;
+ if (err) {
+ drm_dbg(&mem->i915->drm, "Skip stolen region: failed to setup\n");
+ return 0;
}
+ if (mem->io_size &&
+ !io_mapping_init_wc(&mem->iomap, mem->io_start, mem->io_size))
+ goto err_cleanup;
+
+ drm_dbg(&i915->drm, "Stolen Local memory IO start: %pa\n",
+ &mem->io_start);
+ drm_dbg(&i915->drm, "Stolen Local DSM base: %pa\n", &mem->region.start);
+
return 0;
err_cleanup:
@@ -796,6 +840,29 @@ static const struct intel_memory_region_ops i915_region_stolen_lmem_ops = {
.init_object = _i915_gem_object_stolen_init,
};
+static int mtl_get_gms_size(struct intel_uncore *uncore)
+{
+ u16 ggc, gms;
+
+ ggc = intel_uncore_read16(uncore, GGC);
+
+ /* check GGMS, should be fixed 0x3 (8MB) */
+ if ((ggc & GGMS_MASK) != GGMS_MASK)
+ return -EIO;
+
+ /* return valid GMS value, -EIO if invalid */
+ gms = REG_FIELD_GET(GMS_MASK, ggc);
+ switch (gms) {
+ case 0x0 ... 0x04:
+ return gms * 32;
+ case 0xf0 ... 0xfe:
+ return (gms - 0xf0 + 1) * 4;
+ default:
+ MISSING_CASE(gms);
+ return -EIO;
+ }
+}
+
struct intel_memory_region *
i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
u16 instance)
@@ -806,6 +873,7 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
struct intel_memory_region *mem;
resource_size_t io_start, io_size;
resource_size_t min_page_size;
+ int ret;
if (WARN_ON_ONCE(instance))
return ERR_PTR(-ENODEV);
@@ -813,12 +881,8 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
if (!i915_pci_resource_valid(pdev, GEN12_LMEM_BAR))
return ERR_PTR(-ENXIO);
- /* Use DSM base address instead for stolen memory */
- dsm_base = intel_uncore_read64(uncore, GEN12_DSMBASE);
- if (IS_DG1(uncore->i915)) {
+ if (HAS_LMEMBAR_SMEM_STOLEN(i915) || IS_DG1(i915)) {
lmem_size = pci_resource_len(pdev, GEN12_LMEM_BAR);
- if (WARN_ON(lmem_size < dsm_base))
- return ERR_PTR(-ENODEV);
} else {
resource_size_t lmem_range;
@@ -827,13 +891,39 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
lmem_size *= SZ_1G;
}
- dsm_size = lmem_size - dsm_base;
- if (pci_resource_len(pdev, GEN12_LMEM_BAR) < lmem_size) {
+ if (HAS_LMEMBAR_SMEM_STOLEN(i915)) {
+ /*
+ * MTL dsm size is in GGC register.
+ * Also MTL uses offset to DSMBASE in ptes, so i915
+ * uses dsm_base = 0 to setup stolen region.
+ */
+ ret = mtl_get_gms_size(uncore);
+ if (ret < 0) {
+ drm_err(&i915->drm, "invalid MTL GGC register setting\n");
+ return ERR_PTR(ret);
+ }
+
+ dsm_base = 0;
+ dsm_size = (resource_size_t)(ret * SZ_1M);
+
+ GEM_BUG_ON(pci_resource_len(pdev, GEN12_LMEM_BAR) != SZ_256M);
+ GEM_BUG_ON((dsm_size + SZ_8M) > lmem_size);
+ } else {
+ /* Use DSM base address instead for stolen memory */
+ dsm_base = intel_uncore_read64(uncore, GEN12_DSMBASE) & GEN12_BDSM_MASK;
+ if (WARN_ON(lmem_size < dsm_base))
+ return ERR_PTR(-ENODEV);
+ dsm_size = lmem_size - dsm_base;
+ }
+
+ io_size = dsm_size;
+ if (HAS_LMEMBAR_SMEM_STOLEN(i915)) {
+ io_start = pci_resource_start(pdev, GEN12_LMEM_BAR) + SZ_8M;
+ } else if (pci_resource_len(pdev, GEN12_LMEM_BAR) < lmem_size) {
io_start = 0;
io_size = 0;
} else {
io_start = pci_resource_start(pdev, GEN12_LMEM_BAR) + dsm_base;
- io_size = dsm_size;
}
min_page_size = HAS_64K_PAGES(i915) ? I915_GTT_PAGE_SIZE_64K :
@@ -847,16 +937,6 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
if (IS_ERR(mem))
return mem;
- /*
- * TODO: consider creating common helper to just print all the
- * interesting stuff from intel_memory_region, which we can use for all
- * our probed regions.
- */
-
- drm_dbg(&i915->drm, "Stolen Local memory IO start: %pa\n",
- &mem->io_start);
- drm_dbg(&i915->drm, "Stolen Local DSM base: %pa\n", &dsm_base);
-
intel_memory_region_set_name(mem, "stolen-local");
mem->private = true;
@@ -881,6 +961,7 @@ i915_gem_stolen_smem_setup(struct drm_i915_private *i915, u16 type,
intel_memory_region_set_name(mem, "stolen-system");
mem->private = true;
+
return mem;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c
index 0d6d6402..1e50fb0 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c
@@ -279,7 +279,7 @@ static struct ttm_tt *i915_ttm_tt_create(struct ttm_buffer_object *bo,
struct i915_ttm_tt *i915_tt;
int ret;
- if (!obj)
+ if (i915_ttm_is_ghost_object(bo))
return NULL;
i915_tt = kzalloc(sizeof(*i915_tt), GFP_KERNEL);
@@ -362,7 +362,7 @@ static bool i915_ttm_eviction_valuable(struct ttm_buffer_object *bo,
{
struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- if (!obj)
+ if (i915_ttm_is_ghost_object(bo))
return false;
/*
@@ -509,18 +509,9 @@ static int i915_ttm_shrink(struct drm_i915_gem_object *obj, unsigned int flags)
static void i915_ttm_delete_mem_notify(struct ttm_buffer_object *bo)
{
struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
- intel_wakeref_t wakeref = 0;
- if (bo->resource && likely(obj)) {
- /* ttm_bo_release() already has dma_resv_lock */
- if (i915_ttm_cpu_maps_iomem(bo->resource))
- wakeref = intel_runtime_pm_get(&to_i915(obj->base.dev)->runtime_pm);
-
+ if (bo->resource && !i915_ttm_is_ghost_object(bo)) {
__i915_gem_object_pages_fini(obj);
-
- if (wakeref)
- intel_runtime_pm_put(&to_i915(obj->base.dev)->runtime_pm, wakeref);
-
i915_ttm_free_cached_io_rsgt(obj);
}
}
@@ -628,7 +619,7 @@ static void i915_ttm_swap_notify(struct ttm_buffer_object *bo)
struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
int ret;
- if (!obj)
+ if (i915_ttm_is_ghost_object(bo))
return;
ret = i915_ttm_move_notify(bo);
@@ -653,7 +644,7 @@ bool i915_ttm_resource_mappable(struct ttm_resource *res)
if (!i915_ttm_cpu_maps_iomem(res))
return true;
- return bman_res->used_visible_size == bman_res->base.num_pages;
+ return bman_res->used_visible_size == PFN_UP(bman_res->base.size);
}
static int i915_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem)
@@ -661,7 +652,7 @@ static int i915_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource
struct drm_i915_gem_object *obj = i915_ttm_to_gem(mem->bo);
bool unknown_state;
- if (!obj)
+ if (i915_ttm_is_ghost_object(mem->bo))
return -EINVAL;
if (!kref_get_unless_zero(&obj->base.refcount))
@@ -694,7 +685,7 @@ static unsigned long i915_ttm_io_mem_pfn(struct ttm_buffer_object *bo,
unsigned long base;
unsigned int ofs;
- GEM_BUG_ON(!obj);
+ GEM_BUG_ON(i915_ttm_is_ghost_object(bo));
GEM_WARN_ON(bo->ttm);
base = obj->mm.region->iomap.base - obj->mm.region->region.start;
@@ -703,6 +694,50 @@ static unsigned long i915_ttm_io_mem_pfn(struct ttm_buffer_object *bo,
return ((base + sg_dma_address(sg)) >> PAGE_SHIFT) + ofs;
}
+static int i915_ttm_access_memory(struct ttm_buffer_object *bo,
+ unsigned long offset, void *buf,
+ int len, int write)
+{
+ struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
+ resource_size_t iomap = obj->mm.region->iomap.base -
+ obj->mm.region->region.start;
+ unsigned long page = offset >> PAGE_SHIFT;
+ unsigned long bytes_left = len;
+
+ /*
+ * TODO: For now just let it fail if the resource is non-mappable,
+ * otherwise we need to perform the memcpy from the gpu here, without
+ * interfering with the object (like moving the entire thing).
+ */
+ if (!i915_ttm_resource_mappable(bo->resource))
+ return -EIO;
+
+ offset -= page << PAGE_SHIFT;
+ do {
+ unsigned long bytes = min(bytes_left, PAGE_SIZE - offset);
+ void __iomem *ptr;
+ dma_addr_t daddr;
+
+ daddr = i915_gem_object_get_dma_address(obj, page);
+ ptr = ioremap_wc(iomap + daddr + offset, bytes);
+ if (!ptr)
+ return -EIO;
+
+ if (write)
+ memcpy_toio(ptr, buf, bytes);
+ else
+ memcpy_fromio(buf, ptr, bytes);
+ iounmap(ptr);
+
+ page++;
+ buf += bytes;
+ bytes_left -= bytes;
+ offset = 0;
+ } while (bytes_left);
+
+ return len;
+}
+
/*
* All callbacks need to take care not to downcast a struct ttm_buffer_object
* without checking its subclass, since it might be a TTM ghost object.
@@ -719,6 +754,7 @@ static struct ttm_device_funcs i915_ttm_bo_driver = {
.delete_mem_notify = i915_ttm_delete_mem_notify,
.io_mem_reserve = i915_ttm_io_mem_reserve,
.io_mem_pfn = i915_ttm_io_mem_pfn,
+ .access_memory = i915_ttm_access_memory,
};
/**
@@ -783,8 +819,7 @@ static int __i915_ttm_get_pages(struct drm_i915_gem_object *obj,
GEM_BUG_ON(obj->mm.rsgt);
obj->mm.rsgt = rsgt;
- __i915_gem_object_set_pages(obj, &rsgt->table,
- i915_sg_dma_sizes(rsgt->table.sgl));
+ __i915_gem_object_set_pages(obj, &rsgt->table);
}
GEM_BUG_ON(bo->ttm && ((obj->base.size >> PAGE_SHIFT) < bo->ttm->num_pages));
@@ -994,15 +1029,11 @@ static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
struct vm_area_struct *area = vmf->vma;
struct ttm_buffer_object *bo = area->vm_private_data;
struct drm_device *dev = bo->base.dev;
- struct drm_i915_gem_object *obj;
+ struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo);
intel_wakeref_t wakeref = 0;
vm_fault_t ret;
int idx;
- obj = i915_ttm_to_gem(bo);
- if (!obj)
- return VM_FAULT_SIGBUS;
-
/* Sanity check that we allow writing into this object */
if (unlikely(i915_gem_object_is_readonly(obj) &&
area->vm_flags & VM_WRITE))
@@ -1036,7 +1067,8 @@ static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
}
if (err) {
- drm_dbg(dev, "Unable to make resource CPU accessible\n");
+ drm_dbg(dev, "Unable to make resource CPU accessible(err = %pe)\n",
+ ERR_PTR(err));
dma_resv_unlock(bo->base.resv);
ret = VM_FAULT_SIGBUS;
goto out_rpm;
@@ -1057,16 +1089,21 @@ static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
goto out_rpm;
- /* ttm_bo_vm_reserve() already has dma_resv_lock */
+ /*
+ * ttm_bo_vm_reserve() already has dma_resv_lock.
+ * userfault_count is protected by dma_resv lock and rpm wakeref.
+ */
if (ret == VM_FAULT_NOPAGE && wakeref && !obj->userfault_count) {
obj->userfault_count = 1;
- mutex_lock(&to_gt(to_i915(obj->base.dev))->lmem_userfault_lock);
- list_add(&obj->userfault_link, &to_gt(to_i915(obj->base.dev))->lmem_userfault_list);
- mutex_unlock(&to_gt(to_i915(obj->base.dev))->lmem_userfault_lock);
+ spin_lock(&to_i915(obj->base.dev)->runtime_pm.lmem_userfault_lock);
+ list_add(&obj->userfault_link, &to_i915(obj->base.dev)->runtime_pm.lmem_userfault_list);
+ spin_unlock(&to_i915(obj->base.dev)->runtime_pm.lmem_userfault_lock);
+
+ GEM_WARN_ON(!i915_ttm_cpu_maps_iomem(bo->resource));
}
if (wakeref & CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND)
- intel_wakeref_auto(&to_gt(to_i915(obj->base.dev))->userfault_wakeref,
+ intel_wakeref_auto(&to_i915(obj->base.dev)->runtime_pm.userfault_wakeref,
msecs_to_jiffies_timeout(CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND));
i915_ttm_adjust_lru(obj);
@@ -1098,7 +1135,7 @@ static void ttm_vm_open(struct vm_area_struct *vma)
struct drm_i915_gem_object *obj =
i915_ttm_to_gem(vma->vm_private_data);
- GEM_BUG_ON(!obj);
+ GEM_BUG_ON(i915_ttm_is_ghost_object(vma->vm_private_data));
i915_gem_object_get(obj);
}
@@ -1107,7 +1144,7 @@ static void ttm_vm_close(struct vm_area_struct *vma)
struct drm_i915_gem_object *obj =
i915_ttm_to_gem(vma->vm_private_data);
- GEM_BUG_ON(!obj);
+ GEM_BUG_ON(i915_ttm_is_ghost_object(vma->vm_private_data));
i915_gem_object_put(obj);
}
@@ -1128,7 +1165,29 @@ static u64 i915_ttm_mmap_offset(struct drm_i915_gem_object *obj)
static void i915_ttm_unmap_virtual(struct drm_i915_gem_object *obj)
{
+ struct ttm_buffer_object *bo = i915_gem_to_ttm(obj);
+ intel_wakeref_t wakeref = 0;
+
+ assert_object_held_shared(obj);
+
+ if (i915_ttm_cpu_maps_iomem(bo->resource)) {
+ wakeref = intel_runtime_pm_get(&to_i915(obj->base.dev)->runtime_pm);
+
+ /* userfault_count is protected by obj lock and rpm wakeref. */
+ if (obj->userfault_count) {
+ spin_lock(&to_i915(obj->base.dev)->runtime_pm.lmem_userfault_lock);
+ list_del(&obj->userfault_link);
+ spin_unlock(&to_i915(obj->base.dev)->runtime_pm.lmem_userfault_lock);
+ obj->userfault_count = 0;
+ }
+ }
+
+ GEM_WARN_ON(obj->userfault_count);
+
ttm_bo_unmap_virtual(i915_gem_to_ttm(obj));
+
+ if (wakeref)
+ intel_runtime_pm_put(&to_i915(obj->base.dev)->runtime_pm, wakeref);
}
static const struct drm_i915_gem_object_ops i915_gem_ttm_obj_ops = {
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.h b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h
index e4842b4..2a94a99 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h
@@ -28,18 +28,26 @@ i915_gem_to_ttm(struct drm_i915_gem_object *obj)
void i915_ttm_bo_destroy(struct ttm_buffer_object *bo);
/**
+ * i915_ttm_is_ghost_object - Check if the ttm bo is a ghost object.
+ * @bo: Pointer to the ttm buffer object
+ *
+ * Return: True if the ttm bo is not a i915 object but a ghost ttm object,
+ * False otherwise.
+ */
+static inline bool i915_ttm_is_ghost_object(struct ttm_buffer_object *bo)
+{
+ return bo->destroy != i915_ttm_bo_destroy;
+}
+
+/**
* i915_ttm_to_gem - Convert a struct ttm_buffer_object to an embedding
* struct drm_i915_gem_object.
*
- * Return: Pointer to the embedding struct ttm_buffer_object, or NULL
- * if the object was not an i915 ttm object.
+ * Return: Pointer to the embedding struct ttm_buffer_object.
*/
static inline struct drm_i915_gem_object *
i915_ttm_to_gem(struct ttm_buffer_object *bo)
{
- if (bo->destroy != i915_ttm_bo_destroy)
- return NULL;
-
return container_of(bo, struct drm_i915_gem_object, __do_not_access);
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c
index 9a7e505..f59f812 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c
@@ -560,7 +560,7 @@ int i915_ttm_move(struct ttm_buffer_object *bo, bool evict,
bool clear;
int ret;
- if (GEM_WARN_ON(!obj)) {
+ if (GEM_WARN_ON(i915_ttm_is_ghost_object(bo))) {
ttm_bo_move_null(bo, dst_mem);
return 0;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c
index ba14b18..9348b18 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c
@@ -131,7 +131,6 @@ static int i915_gem_userptr_get_pages(struct drm_i915_gem_object *obj)
const unsigned long num_pages = obj->base.size >> PAGE_SHIFT;
unsigned int max_segment = i915_sg_segment_size(obj->base.dev->dev);
struct sg_table *st;
- unsigned int sg_page_sizes;
struct page **pvec;
int ret;
@@ -170,8 +169,7 @@ static int i915_gem_userptr_get_pages(struct drm_i915_gem_object *obj)
if (i915_gem_object_can_bypass_llc(obj))
obj->cache_dirty = true;
- sg_page_sizes = i915_sg_dma_sizes(st->sgl);
- __i915_gem_object_set_pages(obj, st, sg_page_sizes);
+ __i915_gem_object_set_pages(obj, st);
return 0;
@@ -292,7 +290,7 @@ int i915_gem_object_userptr_submit_init(struct drm_i915_gem_object *obj)
if (!i915_gem_object_is_readonly(obj))
gup_flags |= FOLL_WRITE;
- pinned = ret = 0;
+ pinned = 0;
while (pinned < num_pages) {
ret = pin_user_pages_fast(obj->userptr.ptr + pinned * PAGE_SIZE,
num_pages - pinned, gup_flags,
@@ -302,7 +300,6 @@ int i915_gem_object_userptr_submit_init(struct drm_i915_gem_object *obj)
pinned += ret;
}
- ret = 0;
ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c b/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c
index f963b8e..cbd9b62 100644
--- a/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c
+++ b/drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c
@@ -68,7 +68,7 @@ static int huge_get_pages(struct drm_i915_gem_object *obj)
if (i915_gem_gtt_prepare_pages(obj, pages))
goto err;
- __i915_gem_object_set_pages(obj, pages, PAGE_SIZE);
+ __i915_gem_object_set_pages(obj, pages);
return 0;
diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
index c570cf7..beaf27e 100644
--- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
+++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
@@ -136,7 +136,7 @@ static int get_huge_pages(struct drm_i915_gem_object *obj)
goto err;
GEM_BUG_ON(sg_page_sizes != obj->mm.page_mask);
- __i915_gem_object_set_pages(obj, st, sg_page_sizes);
+ __i915_gem_object_set_pages(obj, st);
return 0;
@@ -210,7 +210,6 @@ static int fake_get_huge_pages(struct drm_i915_gem_object *obj)
const u64 max_len = rounddown_pow_of_two(UINT_MAX);
struct sg_table *st;
struct scatterlist *sg;
- unsigned int sg_page_sizes;
u64 rem;
st = kmalloc(sizeof(*st), GFP);
@@ -226,7 +225,6 @@ static int fake_get_huge_pages(struct drm_i915_gem_object *obj)
rem = obj->base.size;
sg = st->sgl;
st->nents = 0;
- sg_page_sizes = 0;
do {
unsigned int page_size = get_largest_page_size(i915, rem);
unsigned int len = min(page_size * div_u64(rem, page_size),
@@ -239,8 +237,6 @@ static int fake_get_huge_pages(struct drm_i915_gem_object *obj)
sg_dma_len(sg) = len;
sg_dma_address(sg) = page_size;
- sg_page_sizes |= len;
-
st->nents++;
rem -= len;
@@ -254,7 +250,7 @@ static int fake_get_huge_pages(struct drm_i915_gem_object *obj)
i915_sg_trim(st);
- __i915_gem_object_set_pages(obj, st, sg_page_sizes);
+ __i915_gem_object_set_pages(obj, st);
return 0;
}
@@ -286,7 +282,7 @@ static int fake_get_huge_pages_single(struct drm_i915_gem_object *obj)
sg_dma_len(sg) = obj->base.size;
sg_dma_address(sg) = page_size;
- __i915_gem_object_set_pages(obj, st, sg->length);
+ __i915_gem_object_set_pages(obj, st);
return 0;
#undef GFP
@@ -1161,7 +1157,8 @@ static int igt_write_huge(struct drm_i915_private *i915,
GEM_BUG_ON(!i915_gem_object_has_pinned_pages(obj));
size = obj->base.size;
- if (obj->mm.page_sizes.sg & I915_GTT_PAGE_SIZE_64K)
+ if (obj->mm.page_sizes.sg & I915_GTT_PAGE_SIZE_64K &&
+ !HAS_64K_PAGES(i915))
size = round_up(size, I915_GTT_PAGE_SIZE_2M);
n = 0;
@@ -1214,6 +1211,10 @@ static int igt_write_huge(struct drm_i915_private *i915,
* size and ensure the vma offset is at the start of the pt
* boundary, however to improve coverage we opt for testing both
* aligned and unaligned offsets.
+ *
+ * With PS64 this is no longer the case, but to ensure we
+ * sometimes get the compact layout for smaller objects, apply
+ * the round_up anyway.
*/
if (obj->mm.page_sizes.sg & I915_GTT_PAGE_SIZE_64K)
offset_low = round_down(offset_low,
@@ -1411,6 +1412,7 @@ static int igt_ppgtt_sanity_check(void *arg)
{ SZ_2M + SZ_4K, SZ_64K | SZ_4K },
{ SZ_2M + SZ_4K, SZ_2M | SZ_4K },
{ SZ_2M + SZ_64K, SZ_2M | SZ_64K },
+ { SZ_2M + SZ_64K, SZ_64K },
};
int i, j;
int err;
@@ -1540,6 +1542,154 @@ static int igt_ppgtt_compact(void *arg)
return err;
}
+static int igt_ppgtt_mixed(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ const unsigned long flags = PIN_OFFSET_FIXED | PIN_USER;
+ struct drm_i915_gem_object *obj, *on;
+ struct i915_gem_engines *engines;
+ struct i915_gem_engines_iter it;
+ struct i915_address_space *vm;
+ struct i915_gem_context *ctx;
+ struct intel_context *ce;
+ struct file *file;
+ I915_RND_STATE(prng);
+ LIST_HEAD(objects);
+ struct intel_memory_region *mr;
+ struct i915_vma *vma;
+ unsigned int count;
+ u32 i, addr;
+ int *order;
+ int n, err;
+
+ /*
+ * Sanity check mixing 4K and 64K pages within the same page-table via
+ * the new PS64 TLB hint.
+ */
+
+ if (!HAS_64K_PAGES(i915)) {
+ pr_info("device lacks PS64, skipping\n");
+ return 0;
+ }
+
+ file = mock_file(i915);
+ if (IS_ERR(file))
+ return PTR_ERR(file);
+
+ ctx = hugepage_ctx(i915, file);
+ if (IS_ERR(ctx)) {
+ err = PTR_ERR(ctx);
+ goto out;
+ }
+ vm = i915_gem_context_get_eb_vm(ctx);
+
+ i = 0;
+ addr = 0;
+ do {
+ u32 sz;
+
+ sz = i915_prandom_u32_max_state(SZ_4M, &prng);
+ sz = max_t(u32, sz, SZ_4K);
+
+ mr = i915->mm.regions[INTEL_REGION_LMEM_0];
+ if (i & 1)
+ mr = i915->mm.regions[INTEL_REGION_SMEM];
+
+ obj = i915_gem_object_create_region(mr, sz, 0, 0);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto out_vm;
+ }
+
+ list_add_tail(&obj->st_link, &objects);
+
+ vma = i915_vma_instance(obj, vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto err_put;
+ }
+
+ addr = round_up(addr, mr->min_page_size);
+ err = i915_vma_pin(vma, 0, 0, addr | flags);
+ if (err)
+ goto err_put;
+
+ if (mr->type == INTEL_MEMORY_LOCAL &&
+ (vma->resource->page_sizes_gtt & I915_GTT_PAGE_SIZE_4K)) {
+ err = -EINVAL;
+ goto err_put;
+ }
+
+ addr += obj->base.size;
+ i++;
+ } while (addr <= SZ_16M);
+
+ n = 0;
+ count = 0;
+ for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) {
+ count++;
+ if (!intel_engine_can_store_dword(ce->engine))
+ continue;
+
+ n++;
+ }
+ i915_gem_context_unlock_engines(ctx);
+ if (!n)
+ goto err_put;
+
+ order = i915_random_order(count * count, &prng);
+ if (!order) {
+ err = -ENOMEM;
+ goto err_put;
+ }
+
+ i = 0;
+ addr = 0;
+ engines = i915_gem_context_lock_engines(ctx);
+ list_for_each_entry(obj, &objects, st_link) {
+ u32 rnd = i915_prandom_u32_max_state(UINT_MAX, &prng);
+
+ addr = round_up(addr, obj->mm.region->min_page_size);
+
+ ce = engines->engines[order[i] % engines->num_engines];
+ i = (i + 1) % (count * count);
+ if (!ce || !intel_engine_can_store_dword(ce->engine))
+ continue;
+
+ err = __igt_write_huge(ce, obj, obj->base.size, addr, 0, rnd);
+ if (err)
+ break;
+
+ err = __igt_write_huge(ce, obj, obj->base.size, addr,
+ offset_in_page(rnd) / sizeof(u32), rnd + 1);
+ if (err)
+ break;
+
+ err = __igt_write_huge(ce, obj, obj->base.size, addr,
+ (PAGE_SIZE / sizeof(u32)) - 1,
+ rnd + 2);
+ if (err)
+ break;
+
+ addr += obj->base.size;
+
+ cond_resched();
+ }
+
+ i915_gem_context_unlock_engines(ctx);
+ kfree(order);
+err_put:
+ list_for_each_entry_safe(obj, on, &objects, st_link) {
+ list_del(&obj->st_link);
+ i915_gem_object_put(obj);
+ }
+out_vm:
+ i915_vm_put(vm);
+out:
+ fput(file);
+ return err;
+}
+
static int igt_tmpfs_fallback(void *arg)
{
struct drm_i915_private *i915 = arg;
@@ -1803,6 +1953,7 @@ int i915_gem_huge_page_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_ppgtt_smoke_huge),
SUBTEST(igt_ppgtt_sanity_check),
SUBTEST(igt_ppgtt_compact),
+ SUBTEST(igt_ppgtt_mixed),
};
if (!HAS_PPGTT(i915)) {
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
index 9a6a6b5..692a1691 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
@@ -13,6 +13,7 @@
#include "gt/intel_gt_regs.h"
#include "gem/i915_gem_lmem.h"
+#include "gem/selftests/igt_gem_utils.h"
#include "selftests/igt_flush_test.h"
#include "selftests/mock_drm.h"
#include "selftests/i915_random.h"
@@ -457,21 +458,6 @@ static int verify_buffer(const struct tiled_blits *t,
return ret;
}
-static int move_to_active(struct i915_vma *vma,
- struct i915_request *rq,
- unsigned int flags)
-{
- int err;
-
- i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, flags);
- i915_vma_unlock(vma);
-
- return err;
-}
-
static int pin_buffer(struct i915_vma *vma, u64 addr)
{
int err;
@@ -525,11 +511,11 @@ tiled_blit(struct tiled_blits *t,
goto err_bb;
}
- err = move_to_active(t->batch, rq, 0);
+ err = igt_vma_move_to_active_unlocked(t->batch, rq, 0);
if (!err)
- err = move_to_active(src->vma, rq, 0);
+ err = igt_vma_move_to_active_unlocked(src->vma, rq, 0);
if (!err)
- err = move_to_active(dst->vma, rq, 0);
+ err = igt_vma_move_to_active_unlocked(dst->vma, rq, 0);
if (!err)
err = rq->engine->emit_bb_start(rq,
t->batch->node.start,
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
index a666d7e6..c228fe4 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
@@ -239,9 +239,7 @@ static int gpu_set(struct context *ctx, unsigned long offset, u32 v)
}
intel_ring_advance(rq, cs);
- err = i915_request_await_object(rq, vma->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
out_rq:
i915_request_add(rq);
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
index c6ad67b..a0ff51d 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
@@ -179,97 +179,108 @@ static int live_nop_switch(void *arg)
}
struct parallel_switch {
- struct task_struct *tsk;
+ struct kthread_worker *worker;
+ struct kthread_work work;
struct intel_context *ce[2];
+ int result;
};
-static int __live_parallel_switch1(void *data)
+static void __live_parallel_switch1(struct kthread_work *work)
{
- struct parallel_switch *arg = data;
+ struct parallel_switch *arg =
+ container_of(work, typeof(*arg), work);
IGT_TIMEOUT(end_time);
unsigned long count;
count = 0;
+ arg->result = 0;
do {
struct i915_request *rq = NULL;
- int err, n;
+ int n;
- err = 0;
- for (n = 0; !err && n < ARRAY_SIZE(arg->ce); n++) {
+ for (n = 0; !arg->result && n < ARRAY_SIZE(arg->ce); n++) {
struct i915_request *prev = rq;
rq = i915_request_create(arg->ce[n]);
if (IS_ERR(rq)) {
i915_request_put(prev);
- return PTR_ERR(rq);
+ arg->result = PTR_ERR(rq);
+ break;
}
i915_request_get(rq);
if (prev) {
- err = i915_request_await_dma_fence(rq, &prev->fence);
+ arg->result =
+ i915_request_await_dma_fence(rq,
+ &prev->fence);
i915_request_put(prev);
}
i915_request_add(rq);
}
+
+ if (IS_ERR_OR_NULL(rq))
+ break;
+
if (i915_request_wait(rq, 0, HZ) < 0)
- err = -ETIME;
+ arg->result = -ETIME;
+
i915_request_put(rq);
- if (err)
- return err;
count++;
- } while (!__igt_timeout(end_time, NULL));
+ } while (!arg->result && !__igt_timeout(end_time, NULL));
- pr_info("%s: %lu switches (sync)\n", arg->ce[0]->engine->name, count);
- return 0;
+ pr_info("%s: %lu switches (sync) <%d>\n",
+ arg->ce[0]->engine->name, count, arg->result);
}
-static int __live_parallel_switchN(void *data)
+static void __live_parallel_switchN(struct kthread_work *work)
{
- struct parallel_switch *arg = data;
+ struct parallel_switch *arg =
+ container_of(work, typeof(*arg), work);
struct i915_request *rq = NULL;
IGT_TIMEOUT(end_time);
unsigned long count;
int n;
count = 0;
+ arg->result = 0;
do {
- for (n = 0; n < ARRAY_SIZE(arg->ce); n++) {
+ for (n = 0; !arg->result && n < ARRAY_SIZE(arg->ce); n++) {
struct i915_request *prev = rq;
- int err = 0;
rq = i915_request_create(arg->ce[n]);
if (IS_ERR(rq)) {
i915_request_put(prev);
- return PTR_ERR(rq);
+ arg->result = PTR_ERR(rq);
+ break;
}
i915_request_get(rq);
if (prev) {
- err = i915_request_await_dma_fence(rq, &prev->fence);
+ arg->result =
+ i915_request_await_dma_fence(rq,
+ &prev->fence);
i915_request_put(prev);
}
i915_request_add(rq);
- if (err) {
- i915_request_put(rq);
- return err;
- }
}
count++;
- } while (!__igt_timeout(end_time, NULL));
- i915_request_put(rq);
+ } while (!arg->result && !__igt_timeout(end_time, NULL));
- pr_info("%s: %lu switches (many)\n", arg->ce[0]->engine->name, count);
- return 0;
+ if (!IS_ERR_OR_NULL(rq))
+ i915_request_put(rq);
+
+ pr_info("%s: %lu switches (many) <%d>\n",
+ arg->ce[0]->engine->name, count, arg->result);
}
static int live_parallel_switch(void *arg)
{
struct drm_i915_private *i915 = arg;
- static int (* const func[])(void *arg) = {
+ static void (* const func[])(struct kthread_work *) = {
__live_parallel_switch1,
__live_parallel_switchN,
NULL,
@@ -277,7 +288,7 @@ static int live_parallel_switch(void *arg)
struct parallel_switch *data = NULL;
struct i915_gem_engines *engines;
struct i915_gem_engines_iter it;
- int (* const *fn)(void *arg);
+ void (* const *fn)(struct kthread_work *);
struct i915_gem_context *ctx;
struct intel_context *ce;
struct file *file;
@@ -348,9 +359,22 @@ static int live_parallel_switch(void *arg)
}
}
+ for (n = 0; n < count; n++) {
+ struct kthread_worker *worker;
+
+ if (!data[n].ce[0])
+ continue;
+
+ worker = kthread_create_worker(0, "igt/parallel:%s",
+ data[n].ce[0]->engine->name);
+ if (IS_ERR(worker))
+ goto out;
+
+ data[n].worker = worker;
+ }
+
for (fn = func; !err && *fn; fn++) {
struct igt_live_test t;
- int n;
err = igt_live_test_begin(&t, i915, __func__, "");
if (err)
@@ -360,30 +384,17 @@ static int live_parallel_switch(void *arg)
if (!data[n].ce[0])
continue;
- data[n].tsk = kthread_run(*fn, &data[n],
- "igt/parallel:%s",
- data[n].ce[0]->engine->name);
- if (IS_ERR(data[n].tsk)) {
- err = PTR_ERR(data[n].tsk);
- break;
- }
- get_task_struct(data[n].tsk);
+ data[n].result = 0;
+ kthread_init_work(&data[n].work, *fn);
+ kthread_queue_work(data[n].worker, &data[n].work);
}
- yield(); /* start all threads before we kthread_stop() */
-
for (n = 0; n < count; n++) {
- int status;
-
- if (IS_ERR_OR_NULL(data[n].tsk))
- continue;
-
- status = kthread_stop(data[n].tsk);
- if (status && !err)
- err = status;
-
- put_task_struct(data[n].tsk);
- data[n].tsk = NULL;
+ if (data[n].ce[0]) {
+ kthread_flush_work(&data[n].work);
+ if (data[n].result && !err)
+ err = data[n].result;
+ }
}
if (igt_live_test_end(&t))
@@ -399,6 +410,9 @@ static int live_parallel_switch(void *arg)
intel_context_unpin(data[n].ce[m]);
intel_context_put(data[n].ce[m]);
}
+
+ if (data[n].worker)
+ kthread_destroy_worker(data[n].worker);
}
kfree(data);
out_file:
@@ -970,15 +984,11 @@ emit_rpcs_query(struct drm_i915_gem_object *obj,
goto err_batch;
}
- err = i915_request_await_object(rq, batch->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(batch, rq, 0);
+ err = i915_vma_move_to_active(batch, rq, 0);
if (err)
goto skip_request;
- err = i915_request_await_object(rq, vma->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
if (err)
goto skip_request;
@@ -1539,9 +1549,7 @@ static int write_to_scratch(struct i915_gem_context *ctx,
}
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
i915_vma_unlock(vma);
if (err)
goto skip_request;
@@ -1675,9 +1683,7 @@ static int read_from_scratch(struct i915_gem_context *ctx,
}
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
i915_vma_unlock(vma);
if (err)
goto skip_request;
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c
index 51ed824..e57f939 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c
@@ -6,8 +6,12 @@
#include "i915_drv.h"
#include "i915_selftest.h"
+#include "gem/i915_gem_context.h"
+#include "mock_context.h"
#include "mock_dmabuf.h"
+#include "igt_gem_utils.h"
+#include "selftests/mock_drm.h"
#include "selftests/mock_gem_device.h"
static int igt_dmabuf_export(void *arg)
@@ -140,6 +144,75 @@ static int igt_dmabuf_import_same_driver_lmem(void *arg)
return err;
}
+static int verify_access(struct drm_i915_private *i915,
+ struct drm_i915_gem_object *native_obj,
+ struct drm_i915_gem_object *import_obj)
+{
+ struct i915_gem_engines_iter it;
+ struct i915_gem_context *ctx;
+ struct intel_context *ce;
+ struct i915_vma *vma;
+ struct file *file;
+ u32 *vaddr;
+ int err = 0, i;
+
+ file = mock_file(i915);
+ if (IS_ERR(file))
+ return PTR_ERR(file);
+
+ ctx = live_context(i915, file);
+ if (IS_ERR(ctx)) {
+ err = PTR_ERR(ctx);
+ goto out_file;
+ }
+
+ for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) {
+ if (intel_engine_can_store_dword(ce->engine))
+ break;
+ }
+ i915_gem_context_unlock_engines(ctx);
+ if (!ce)
+ goto out_file;
+
+ vma = i915_vma_instance(import_obj, ce->vm, NULL);
+ if (IS_ERR(vma)) {
+ err = PTR_ERR(vma);
+ goto out_file;
+ }
+
+ err = i915_vma_pin(vma, 0, 0, PIN_USER);
+ if (err)
+ goto out_file;
+
+ err = igt_gpu_fill_dw(ce, vma, 0,
+ vma->size >> PAGE_SHIFT, 0xdeadbeaf);
+ i915_vma_unpin(vma);
+ if (err)
+ goto out_file;
+
+ err = i915_gem_object_wait(import_obj, 0, MAX_SCHEDULE_TIMEOUT);
+ if (err)
+ goto out_file;
+
+ vaddr = i915_gem_object_pin_map_unlocked(native_obj, I915_MAP_WB);
+ if (IS_ERR(vaddr)) {
+ err = PTR_ERR(vaddr);
+ goto out_file;
+ }
+
+ for (i = 0; i < native_obj->base.size / sizeof(u32); i += PAGE_SIZE / sizeof(u32)) {
+ if (vaddr[i] != 0xdeadbeaf) {
+ pr_err("Data mismatch [%d]=%u\n", i, vaddr[i]);
+ err = -EINVAL;
+ goto out_file;
+ }
+ }
+
+out_file:
+ fput(file);
+ return err;
+}
+
static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915,
struct intel_memory_region **regions,
unsigned int num_regions)
@@ -154,7 +227,7 @@ static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915,
force_different_devices = true;
- obj = __i915_gem_object_create_user(i915, PAGE_SIZE,
+ obj = __i915_gem_object_create_user(i915, SZ_8M,
regions, num_regions);
if (IS_ERR(obj)) {
pr_err("__i915_gem_object_create_user failed with err=%ld\n",
@@ -206,6 +279,10 @@ static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915,
i915_gem_object_unlock(import_obj);
+ err = verify_access(i915, obj, import_obj);
+ if (err)
+ goto out_import;
+
/* Now try a fake an importer */
import_attach = dma_buf_attach(dmabuf, obj->base.dev->dev);
if (IS_ERR(import_attach)) {
@@ -213,7 +290,7 @@ static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915,
goto out_import;
}
- st = dma_buf_map_attachment(import_attach, DMA_BIDIRECTIONAL);
+ st = dma_buf_map_attachment_unlocked(import_attach, DMA_BIDIRECTIONAL);
if (IS_ERR(st)) {
err = PTR_ERR(st);
goto out_detach;
@@ -226,7 +303,7 @@ static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915,
timeout = -ETIME;
}
err = timeout > 0 ? 0 : timeout;
- dma_buf_unmap_attachment(import_attach, st, DMA_BIDIRECTIONAL);
+ dma_buf_unmap_attachment_unlocked(import_attach, st, DMA_BIDIRECTIONAL);
out_detach:
dma_buf_detach(dmabuf, import_attach);
out_import:
@@ -296,7 +373,7 @@ static int igt_dmabuf_import(void *arg)
goto out_obj;
}
- err = dma_buf_vmap(dmabuf, &map);
+ err = dma_buf_vmap_unlocked(dmabuf, &map);
dma_map = err ? NULL : map.vaddr;
if (!dma_map) {
pr_err("dma_buf_vmap failed\n");
@@ -337,7 +414,7 @@ static int igt_dmabuf_import(void *arg)
err = 0;
out_dma_map:
- dma_buf_vunmap(dmabuf, &map);
+ dma_buf_vunmap_unlocked(dmabuf, &map);
out_obj:
i915_gem_object_put(obj);
out_dmabuf:
@@ -358,7 +435,7 @@ static int igt_dmabuf_import_ownership(void *arg)
if (IS_ERR(dmabuf))
return PTR_ERR(dmabuf);
- err = dma_buf_vmap(dmabuf, &map);
+ err = dma_buf_vmap_unlocked(dmabuf, &map);
ptr = err ? NULL : map.vaddr;
if (!ptr) {
pr_err("dma_buf_vmap failed\n");
@@ -367,7 +444,7 @@ static int igt_dmabuf_import_ownership(void *arg)
}
memset(ptr, 0xc5, PAGE_SIZE);
- dma_buf_vunmap(dmabuf, &map);
+ dma_buf_vunmap_unlocked(dmabuf, &map);
obj = to_intel_bo(i915_gem_prime_import(&i915->drm, dmabuf));
if (IS_ERR(obj)) {
@@ -418,7 +495,7 @@ static int igt_dmabuf_export_vmap(void *arg)
}
i915_gem_object_put(obj);
- err = dma_buf_vmap(dmabuf, &map);
+ err = dma_buf_vmap_unlocked(dmabuf, &map);
ptr = err ? NULL : map.vaddr;
if (!ptr) {
pr_err("dma_buf_vmap failed\n");
@@ -435,7 +512,7 @@ static int igt_dmabuf_export_vmap(void *arg)
memset(ptr, 0xc5, dmabuf->size);
err = 0;
- dma_buf_vunmap(dmabuf, &map);
+ dma_buf_vunmap_unlocked(dmabuf, &map);
out:
dma_buf_put(dmabuf);
return err;
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
index b73c91a..3f658d5 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
@@ -8,6 +8,7 @@
#include <linux/prime_numbers.h>
#include "gem/i915_gem_internal.h"
+#include "gem/i915_gem_lmem.h"
#include "gem/i915_gem_region.h"
#include "gem/i915_gem_ttm.h"
#include "gem/i915_gem_ttm_move.h"
@@ -16,6 +17,7 @@
#include "gt/intel_gt.h"
#include "gt/intel_gt_pm.h"
#include "gt/intel_migrate.h"
+#include "i915_reg.h"
#include "i915_ttm_buddy_manager.h"
#include "huge_gem_object.h"
@@ -564,10 +566,8 @@ static int make_obj_busy(struct drm_i915_gem_object *obj)
goto err_unpin;
}
- err = i915_request_await_object(rq, vma->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq,
- EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq,
+ EXEC_OBJECT_WRITE);
i915_request_add(rq);
err_unpin:
@@ -1607,9 +1607,7 @@ static int __igt_mmap_gpu(struct drm_i915_private *i915,
goto out_unpin;
}
- err = i915_request_await_object(rq, vma->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
err = engine->emit_bb_start(rq, vma->node.start, 0, 0);
i915_request_get(rq);
diff --git a/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c b/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c
index 3c55e77..374b10a 100644
--- a/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c
+++ b/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c
@@ -131,17 +131,13 @@ int igt_gpu_fill_dw(struct intel_context *ce,
}
i915_vma_lock(batch);
- err = i915_request_await_object(rq, batch->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(batch, rq, 0);
+ err = i915_vma_move_to_active(batch, rq, 0);
i915_vma_unlock(batch);
if (err)
goto skip_request;
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
i915_vma_unlock(vma);
if (err)
goto skip_request;
diff --git a/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.h b/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.h
index 4221cf8..1379fbc 100644
--- a/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.h
+++ b/drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.h
@@ -9,6 +9,8 @@
#include <linux/types.h>
+#include "i915_vma.h"
+
struct i915_request;
struct i915_gem_context;
struct i915_vma;
@@ -29,4 +31,16 @@ int igt_gpu_fill_dw(struct intel_context *ce,
struct i915_vma *vma, u64 offset,
unsigned long count, u32 val);
+static inline int __must_check
+igt_vma_move_to_active_unlocked(struct i915_vma *vma, struct i915_request *rq,
+ unsigned int flags)
+{
+ int err;
+
+ i915_vma_lock(vma);
+ err = _i915_vma_move_to_active(vma, rq, &rq->fence, flags);
+ i915_vma_unlock(vma);
+ return err;
+}
+
#endif /* __IGT_GEM_UTILS_H__ */
diff --git a/drivers/gpu/drm/i915/gt/gen8_engine_cs.c b/drivers/gpu/drm/i915/gt/gen8_engine_cs.c
index e49fa6f..e1c76e5 100644
--- a/drivers/gpu/drm/i915/gt/gen8_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/gen8_engine_cs.c
@@ -396,15 +396,17 @@ int gen8_emit_init_breadcrumb(struct i915_request *rq)
return 0;
}
-static int __gen125_emit_bb_start(struct i915_request *rq,
- u64 offset, u32 len,
- const unsigned int flags,
- u32 arb)
+static int __xehp_emit_bb_start(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags,
+ u32 arb)
{
struct intel_context *ce = rq->context;
u32 wa_offset = lrc_indirect_bb(ce);
u32 *cs;
+ GEM_BUG_ON(!ce->wa_bb_page);
+
cs = intel_ring_begin(rq, 12);
if (IS_ERR(cs))
return PTR_ERR(cs);
@@ -435,18 +437,18 @@ static int __gen125_emit_bb_start(struct i915_request *rq,
return 0;
}
-int gen125_emit_bb_start_noarb(struct i915_request *rq,
- u64 offset, u32 len,
- const unsigned int flags)
+int xehp_emit_bb_start_noarb(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags)
{
- return __gen125_emit_bb_start(rq, offset, len, flags, MI_ARB_DISABLE);
+ return __xehp_emit_bb_start(rq, offset, len, flags, MI_ARB_DISABLE);
}
-int gen125_emit_bb_start(struct i915_request *rq,
- u64 offset, u32 len,
- const unsigned int flags)
+int xehp_emit_bb_start(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags)
{
- return __gen125_emit_bb_start(rq, offset, len, flags, MI_ARB_ENABLE);
+ return __xehp_emit_bb_start(rq, offset, len, flags, MI_ARB_ENABLE);
}
int gen8_emit_bb_start_noarb(struct i915_request *rq,
@@ -583,6 +585,8 @@ u32 *gen8_emit_fini_breadcrumb_xcs(struct i915_request *rq, u32 *cs)
u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs)
{
cs = gen8_emit_pipe_control(cs,
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_TLB_INVALIDATE |
PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
PIPE_CONTROL_DEPTH_CACHE_FLUSH |
PIPE_CONTROL_DC_FLUSH_ENABLE,
@@ -600,15 +604,21 @@ u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs)
u32 *gen11_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs)
{
+ cs = gen8_emit_pipe_control(cs,
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_TLB_INVALIDATE |
+ PIPE_CONTROL_TILE_CACHE_FLUSH |
+ PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
+ PIPE_CONTROL_DEPTH_CACHE_FLUSH |
+ PIPE_CONTROL_DC_FLUSH_ENABLE,
+ 0);
+
+ /*XXX: Look at gen8_emit_fini_breadcrumb_rcs */
cs = gen8_emit_ggtt_write_rcs(cs,
rq->fence.seqno,
hwsp_offset(rq),
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_TILE_CACHE_FLUSH |
- PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
- PIPE_CONTROL_DEPTH_CACHE_FLUSH |
- PIPE_CONTROL_DC_FLUSH_ENABLE |
- PIPE_CONTROL_FLUSH_ENABLE);
+ PIPE_CONTROL_FLUSH_ENABLE |
+ PIPE_CONTROL_CS_STALL);
return gen8_emit_fini_breadcrumb_tail(rq, cs);
}
@@ -715,6 +725,7 @@ u32 *gen12_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs)
{
struct drm_i915_private *i915 = rq->engine->i915;
u32 flags = (PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_TLB_INVALIDATE |
PIPE_CONTROL_TILE_CACHE_FLUSH |
PIPE_CONTROL_FLUSH_L3 |
PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
@@ -731,11 +742,15 @@ u32 *gen12_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs)
else if (rq->engine->class == COMPUTE_CLASS)
flags &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
+ cs = gen12_emit_pipe_control(cs, PIPE_CONTROL0_HDC_PIPELINE_FLUSH, flags, 0);
+
+ /*XXX: Look at gen8_emit_fini_breadcrumb_rcs */
cs = gen12_emit_ggtt_write_rcs(cs,
rq->fence.seqno,
hwsp_offset(rq),
- PIPE_CONTROL0_HDC_PIPELINE_FLUSH,
- flags);
+ 0,
+ PIPE_CONTROL_FLUSH_ENABLE |
+ PIPE_CONTROL_CS_STALL);
return gen12_emit_fini_breadcrumb_tail(rq, cs);
}
diff --git a/drivers/gpu/drm/i915/gt/gen8_engine_cs.h b/drivers/gpu/drm/i915/gt/gen8_engine_cs.h
index e4d24c8..655e5c0 100644
--- a/drivers/gpu/drm/i915/gt/gen8_engine_cs.h
+++ b/drivers/gpu/drm/i915/gt/gen8_engine_cs.h
@@ -32,12 +32,12 @@ int gen8_emit_bb_start(struct i915_request *rq,
u64 offset, u32 len,
const unsigned int flags);
-int gen125_emit_bb_start_noarb(struct i915_request *rq,
- u64 offset, u32 len,
- const unsigned int flags);
-int gen125_emit_bb_start(struct i915_request *rq,
- u64 offset, u32 len,
- const unsigned int flags);
+int xehp_emit_bb_start_noarb(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags);
+int xehp_emit_bb_start(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags);
u32 *gen8_emit_fini_breadcrumb_xcs(struct i915_request *rq, u32 *cs);
u32 *gen12_emit_fini_breadcrumb_xcs(struct i915_request *rq, u32 *cs);
diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c
index 2128b7a..4daaa6f 100644
--- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c
+++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c
@@ -476,6 +476,7 @@ xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm,
const gen8_pte_t pte_encode = vm->pte_encode(0, cache_level, flags);
unsigned int rem = sg_dma_len(iter->sg);
u64 start = vma_res->start;
+ u64 end = start + vma_res->vma_size;
GEM_BUG_ON(!i915_vm_is_4lvl(vm));
@@ -489,9 +490,10 @@ xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm,
gen8_pte_t encode = pte_encode;
unsigned int page_size;
gen8_pte_t *vaddr;
- u16 index, max;
+ u16 index, max, nent, i;
max = I915_PDES;
+ nent = 1;
if (vma_res->bi.page_sizes.sg & I915_GTT_PAGE_SIZE_2M &&
IS_ALIGNED(iter->dma, I915_GTT_PAGE_SIZE_2M) &&
@@ -503,25 +505,37 @@ xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm,
vaddr = px_vaddr(pd);
} else {
- if (encode & GEN12_PPGTT_PTE_LM) {
- GEM_BUG_ON(__gen8_pte_index(start, 0) % 16);
- GEM_BUG_ON(rem < I915_GTT_PAGE_SIZE_64K);
- GEM_BUG_ON(!IS_ALIGNED(iter->dma,
- I915_GTT_PAGE_SIZE_64K));
+ index = __gen8_pte_index(start, 0);
+ page_size = I915_GTT_PAGE_SIZE;
- index = __gen8_pte_index(start, 0) / 16;
- page_size = I915_GTT_PAGE_SIZE_64K;
+ if (vma_res->bi.page_sizes.sg & I915_GTT_PAGE_SIZE_64K) {
+ /*
+ * Device local-memory on these platforms should
+ * always use 64K pages or larger (including GTT
+ * alignment), therefore if we know the whole
+ * page-table needs to be filled we can always
+ * safely use the compact-layout. Otherwise fall
+ * back to the TLB hint with PS64. If this is
+ * system memory we only bother with PS64.
+ */
+ if ((encode & GEN12_PPGTT_PTE_LM) &&
+ end - start >= SZ_2M && !index) {
+ index = __gen8_pte_index(start, 0) / 16;
+ page_size = I915_GTT_PAGE_SIZE_64K;
- max /= 16;
+ max /= 16;
- vaddr = px_vaddr(pd);
- vaddr[__gen8_pte_index(start, 1)] |= GEN12_PDE_64K;
+ vaddr = px_vaddr(pd);
+ vaddr[__gen8_pte_index(start, 1)] |= GEN12_PDE_64K;
- pt->is_compact = true;
- } else {
- GEM_BUG_ON(pt->is_compact);
- index = __gen8_pte_index(start, 0);
- page_size = I915_GTT_PAGE_SIZE;
+ pt->is_compact = true;
+ } else if (IS_ALIGNED(iter->dma, I915_GTT_PAGE_SIZE_64K) &&
+ rem >= I915_GTT_PAGE_SIZE_64K &&
+ !(index % 16)) {
+ encode |= GEN12_PTE_PS64;
+ page_size = I915_GTT_PAGE_SIZE_64K;
+ nent = 16;
+ }
}
vaddr = px_vaddr(pt);
@@ -529,7 +543,12 @@ xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm,
do {
GEM_BUG_ON(rem < page_size);
- vaddr[index++] = encode | iter->dma;
+
+ for (i = 0; i < nent; i++) {
+ vaddr[index++] =
+ encode | (iter->dma + i *
+ I915_GTT_PAGE_SIZE);
+ }
start += page_size;
iter->dma += page_size;
@@ -745,6 +764,8 @@ static void __xehpsdv_ppgtt_insert_entry_lm(struct i915_address_space *vm,
GEM_BUG_ON(!IS_ALIGNED(addr, SZ_64K));
GEM_BUG_ON(!IS_ALIGNED(offset, SZ_64K));
+ /* XXX: we don't strictly need to use this layout */
+
if (!pt->is_compact) {
vaddr = px_vaddr(pd);
vaddr[gen8_pd_index(idx, 1)] |= GEN12_PDE_64K;
@@ -929,29 +950,18 @@ struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt,
*/
ppgtt->vm.has_read_only = !IS_GRAPHICS_VER(gt->i915, 11, 12);
- if (HAS_LMEM(gt->i915)) {
+ if (HAS_LMEM(gt->i915))
ppgtt->vm.alloc_pt_dma = alloc_pt_lmem;
-
- /*
- * On some platforms the hw has dropped support for 4K GTT pages
- * when dealing with LMEM, and due to the design of 64K GTT
- * pages in the hw, we can only mark the *entire* page-table as
- * operating in 64K GTT mode, since the enable bit is still on
- * the pde, and not the pte. And since we still need to allow
- * 4K GTT pages for SMEM objects, we can't have a "normal" 4K
- * page-table with scratch pointing to LMEM, since that's
- * undefined from the hw pov. The simplest solution is to just
- * move the 64K scratch page to SMEM on such platforms and call
- * it a day, since that should work for all configurations.
- */
- if (HAS_64K_PAGES(gt->i915))
- ppgtt->vm.alloc_scratch_dma = alloc_pt_dma;
- else
- ppgtt->vm.alloc_scratch_dma = alloc_pt_lmem;
- } else {
+ else
ppgtt->vm.alloc_pt_dma = alloc_pt_dma;
- ppgtt->vm.alloc_scratch_dma = alloc_pt_dma;
- }
+
+ /*
+ * Using SMEM here instead of LMEM has the advantage of not reserving
+ * high performance memory for a "never" used filler page. It also
+ * removes the device access that would be required to initialise the
+ * scratch page, reducing pressure on an even scarcer resource.
+ */
+ ppgtt->vm.alloc_scratch_dma = alloc_pt_dma;
ppgtt->vm.pte_encode = gen8_pte_encode;
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index be09fb2..fb62b7b8 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -276,6 +276,14 @@ static inline bool intel_context_is_barrier(const struct intel_context *ce)
return test_bit(CONTEXT_BARRIER_BIT, &ce->flags);
}
+static inline void intel_context_close(struct intel_context *ce)
+{
+ set_bit(CONTEXT_CLOSED_BIT, &ce->flags);
+
+ if (ce->ops->close)
+ ce->ops->close(ce);
+}
+
static inline bool intel_context_is_closed(const struct intel_context *ce)
{
return test_bit(CONTEXT_CLOSED_BIT, &ce->flags);
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 04eacae..e36670f 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -43,6 +43,8 @@ struct intel_context_ops {
void (*revoke)(struct intel_context *ce, struct i915_request *rq,
unsigned int preempt_timeout_ms);
+ void (*close)(struct intel_context *ce);
+
int (*pre_pin)(struct intel_context *ce, struct i915_gem_ww_ctx *ww, void **vaddr);
int (*pin)(struct intel_context *ce, void *vaddr);
void (*unpin)(struct intel_context *ce);
@@ -197,8 +199,6 @@ struct intel_context {
* context's submissions is complete.
*/
struct i915_sw_fence blocked;
- /** @number_committed_requests: number of committed requests */
- int number_committed_requests;
/** @requests: list of active requests on this context */
struct list_head requests;
/** @prio: the context's current guc priority */
@@ -208,6 +208,11 @@ struct intel_context {
* each priority bucket
*/
u32 prio_count[GUC_CLIENT_PRIORITY_NUM];
+ /**
+ * @sched_disable_delay_work: worker to disable scheduling on this
+ * context
+ */
+ struct delayed_work sched_disable_delay_work;
} guc_state;
struct {
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index 04e435b..cbc8b85 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -348,4 +348,10 @@ intel_engine_get_hung_context(struct intel_engine_cs *engine)
return engine->hung_ce;
}
+u64 intel_clamp_heartbeat_interval_ms(struct intel_engine_cs *engine, u64 value);
+u64 intel_clamp_max_busywait_duration_ns(struct intel_engine_cs *engine, u64 value);
+u64 intel_clamp_preempt_timeout_ms(struct intel_engine_cs *engine, u64 value);
+u64 intel_clamp_stop_timeout_ms(struct intel_engine_cs *engine, u64 value);
+u64 intel_clamp_timeslice_duration_ms(struct intel_engine_cs *engine, u64 value);
+
#endif /* _INTEL_RINGBUFFER_H_ */
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 1f71881..c33e0d72 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -13,6 +13,8 @@
#include "i915_cmd_parser.h"
#include "i915_drv.h"
+#include "i915_irq.h"
+#include "i915_reg.h"
#include "intel_breadcrumbs.h"
#include "intel_context.h"
#include "intel_engine.h"
@@ -244,6 +246,13 @@ static const struct engine_info intel_engines[] = {
{ .graphics_ver = 12, .base = GEN12_COMPUTE3_RING_BASE }
}
},
+ [GSC0] = {
+ .class = OTHER_CLASS,
+ .instance = OTHER_GSC_INSTANCE,
+ .mmio_bases = {
+ { .graphics_ver = 12, .base = MTL_GSC_RING_BASE }
+ }
+ },
};
/**
@@ -324,6 +333,7 @@ u32 intel_engine_context_size(struct intel_gt *gt, u8 class)
case VIDEO_DECODE_CLASS:
case VIDEO_ENHANCEMENT_CLASS:
case COPY_ENGINE_CLASS:
+ case OTHER_CLASS:
if (GRAPHICS_VER(gt->i915) < 8)
return 0;
return GEN8_LR_CONTEXT_OTHER_SIZE;
@@ -415,6 +425,7 @@ static u32 get_reset_domain(u8 ver, enum intel_engine_id id)
[CCS1] = GEN11_GRDOM_RENDER,
[CCS2] = GEN11_GRDOM_RENDER,
[CCS3] = GEN11_GRDOM_RENDER,
+ [GSC0] = GEN12_GRDOM_GSC,
};
GEM_BUG_ON(id >= ARRAY_SIZE(engine_reset_domains) ||
!engine_reset_domains[id]);
@@ -486,6 +497,17 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id,
engine->logical_mask = BIT(logical_instance);
__sprint_engine_name(engine);
+ if ((engine->class == COMPUTE_CLASS && !RCS_MASK(engine->gt) &&
+ __ffs(CCS_MASK(engine->gt)) == engine->instance) ||
+ engine->class == RENDER_CLASS)
+ engine->flags |= I915_ENGINE_FIRST_RENDER_COMPUTE;
+
+ /* features common between engines sharing EUs */
+ if (engine->class == RENDER_CLASS || engine->class == COMPUTE_CLASS) {
+ engine->flags |= I915_ENGINE_HAS_RCS_REG_STATE;
+ engine->flags |= I915_ENGINE_HAS_EU_PRIORITY;
+ }
+
engine->props.heartbeat_interval_ms =
CONFIG_DRM_I915_HEARTBEAT_INTERVAL;
engine->props.max_busywait_duration_ns =
@@ -497,20 +519,34 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id,
engine->props.timeslice_duration_ms =
CONFIG_DRM_I915_TIMESLICE_DURATION;
- /* Override to uninterruptible for OpenCL workloads. */
- if (GRAPHICS_VER(i915) == 12 && engine->class == RENDER_CLASS)
- engine->props.preempt_timeout_ms = 0;
+ /*
+ * Mid-thread pre-emption is not available in Gen12. Unfortunately,
+ * some compute workloads run quite long threads. That means they get
+ * reset due to not pre-empting in a timely manner. So, bump the
+ * pre-emption timeout value to be much higher for compute engines.
+ */
+ if (GRAPHICS_VER(i915) == 12 && (engine->flags & I915_ENGINE_HAS_RCS_REG_STATE))
+ engine->props.preempt_timeout_ms = CONFIG_DRM_I915_PREEMPT_TIMEOUT_COMPUTE;
- if ((engine->class == COMPUTE_CLASS && !RCS_MASK(engine->gt) &&
- __ffs(CCS_MASK(engine->gt)) == engine->instance) ||
- engine->class == RENDER_CLASS)
- engine->flags |= I915_ENGINE_FIRST_RENDER_COMPUTE;
+ /* Cap properties according to any system limits */
+#define CLAMP_PROP(field) \
+ do { \
+ u64 clamp = intel_clamp_##field(engine, engine->props.field); \
+ if (clamp != engine->props.field) { \
+ drm_notice(&engine->i915->drm, \
+ "Warning, clamping %s to %lld to prevent overflow\n", \
+ #field, clamp); \
+ engine->props.field = clamp; \
+ } \
+ } while (0)
- /* features common between engines sharing EUs */
- if (engine->class == RENDER_CLASS || engine->class == COMPUTE_CLASS) {
- engine->flags |= I915_ENGINE_HAS_RCS_REG_STATE;
- engine->flags |= I915_ENGINE_HAS_EU_PRIORITY;
- }
+ CLAMP_PROP(heartbeat_interval_ms);
+ CLAMP_PROP(max_busywait_duration_ns);
+ CLAMP_PROP(preempt_timeout_ms);
+ CLAMP_PROP(stop_timeout_ms);
+ CLAMP_PROP(timeslice_duration_ms);
+
+#undef CLAMP_PROP
engine->defaults = engine->props; /* never to change again */
@@ -534,6 +570,55 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id,
return 0;
}
+u64 intel_clamp_heartbeat_interval_ms(struct intel_engine_cs *engine, u64 value)
+{
+ value = min_t(u64, value, jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT));
+
+ return value;
+}
+
+u64 intel_clamp_max_busywait_duration_ns(struct intel_engine_cs *engine, u64 value)
+{
+ value = min(value, jiffies_to_nsecs(2));
+
+ return value;
+}
+
+u64 intel_clamp_preempt_timeout_ms(struct intel_engine_cs *engine, u64 value)
+{
+ /*
+ * NB: The GuC API only supports 32bit values. However, the limit is further
+ * reduced due to internal calculations which would otherwise overflow.
+ */
+ if (intel_guc_submission_is_wanted(&engine->gt->uc.guc))
+ value = min_t(u64, value, guc_policy_max_preempt_timeout_ms());
+
+ value = min_t(u64, value, jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT));
+
+ return value;
+}
+
+u64 intel_clamp_stop_timeout_ms(struct intel_engine_cs *engine, u64 value)
+{
+ value = min_t(u64, value, jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT));
+
+ return value;
+}
+
+u64 intel_clamp_timeslice_duration_ms(struct intel_engine_cs *engine, u64 value)
+{
+ /*
+ * NB: The GuC API only supports 32bit values. However, the limit is further
+ * reduced due to internal calculations which would otherwise overflow.
+ */
+ if (intel_guc_submission_is_wanted(&engine->gt->uc.guc))
+ value = min_t(u64, value, guc_policy_max_exec_quantum_ms());
+
+ value = min_t(u64, value, jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT));
+
+ return value;
+}
+
static void __setup_engine_capabilities(struct intel_engine_cs *engine)
{
struct drm_i915_private *i915 = engine->i915;
@@ -1274,8 +1359,13 @@ int intel_engines_init(struct intel_gt *gt)
return err;
err = setup(engine);
- if (err)
+ if (err) {
+ intel_engine_cleanup_common(engine);
return err;
+ }
+
+ /* The backend should now be responsible for cleanup */
+ GEM_BUG_ON(engine->release == NULL);
err = engine_init_common(engine);
if (err)
@@ -1554,11 +1644,11 @@ void intel_engine_get_instdone(const struct intel_engine_cs *engine,
for_each_ss_steering(iter, engine->gt, slice, subslice) {
instdone->sampler[slice][subslice] =
intel_gt_mcr_read(engine->gt,
- GEN7_SAMPLER_INSTDONE,
+ GEN8_SAMPLER_INSTDONE,
slice, subslice);
instdone->row[slice][subslice] =
intel_gt_mcr_read(engine->gt,
- GEN7_ROW_INSTDONE,
+ GEN8_ROW_INSTDONE,
slice, subslice);
}
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
index a3698f6..9a527e1 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
@@ -22,9 +22,37 @@
static bool next_heartbeat(struct intel_engine_cs *engine)
{
+ struct i915_request *rq;
long delay;
delay = READ_ONCE(engine->props.heartbeat_interval_ms);
+
+ rq = engine->heartbeat.systole;
+
+ /*
+ * FIXME: The final period extension is disabled if the period has been
+ * modified from the default. This is to prevent issues with certain
+ * selftests which override the value and expect specific behaviour.
+ * Once the selftests have been updated to either cope with variable
+ * heartbeat periods (or to override the pre-emption timeout as well,
+ * or just to add a selftest specific override of the extension), the
+ * generic override can be removed.
+ */
+ if (rq && rq->sched.attr.priority >= I915_PRIORITY_BARRIER &&
+ delay == engine->defaults.heartbeat_interval_ms) {
+ long longer;
+
+ /*
+ * The final try is at the highest priority possible. Up until now
+ * a pre-emption might not even have been attempted. So make sure
+ * this last attempt allows enough time for a pre-emption to occur.
+ */
+ longer = READ_ONCE(engine->props.preempt_timeout_ms) * 2;
+ longer = intel_clamp_heartbeat_interval_ms(engine, longer);
+ if (longer > delay)
+ delay = longer;
+ }
+
if (!delay)
return false;
@@ -288,6 +316,17 @@ int intel_engine_set_heartbeat(struct intel_engine_cs *engine,
if (!delay && !intel_engine_has_preempt_reset(engine))
return -ENODEV;
+ /* FIXME: Remove together with equally marked hack in next_heartbeat. */
+ if (delay != engine->defaults.heartbeat_interval_ms &&
+ delay < 2 * engine->props.preempt_timeout_ms) {
+ if (intel_engine_uses_guc(engine))
+ drm_notice(&engine->i915->drm, "%s heartbeat interval adjusted to a non-default value which may downgrade individual engine resets to full GPU resets!\n",
+ engine->name);
+ else
+ drm_notice(&engine->i915->drm, "%s heartbeat interval adjusted to a non-default value which may cause engine resets to target innocent contexts!\n",
+ engine->name);
+ }
+
intel_engine_pm_get(engine);
err = mutex_lock_interruptible(&ce->timeline->mutex);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_regs.h b/drivers/gpu/drm/i915/gt/intel_engine_regs.h
index fe1a0d5..ee3efd0 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_regs.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_regs.h
@@ -201,6 +201,7 @@
#define RING_CONTEXT_STATUS_PTR(base) _MMIO((base) + 0x3a0)
#define RING_CTX_TIMESTAMP(base) _MMIO((base) + 0x3a8) /* gen8+ */
#define RING_PREDICATE_RESULT(base) _MMIO((base) + 0x3b8)
+#define MI_PREDICATE_RESULT_2_ENGINE(base) _MMIO((base) + 0x3bc)
#define RING_FORCE_TO_NONPRIV(base, i) _MMIO(((base) + 0x4D0) + (i) * 4)
#define RING_FORCE_TO_NONPRIV_DENY REG_BIT(30)
#define RING_FORCE_TO_NONPRIV_ADDRESS_MASK REG_GENMASK(25, 2)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 6b5d4ea..4fd54fb 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -136,6 +136,7 @@ enum intel_engine_id {
CCS2,
CCS3,
#define _CCS(n) (CCS0 + (n))
+ GSC0,
I915_NUM_ENGINES
#define INVALID_ENGINE ((enum intel_engine_id)-1)
};
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
index 46a174f..cd4f1b1 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
@@ -140,6 +140,7 @@ const char *intel_engine_class_repr(u8 class)
[COPY_ENGINE_CLASS] = "bcs",
[VIDEO_DECODE_CLASS] = "vcs",
[VIDEO_ENHANCEMENT_CLASS] = "vecs",
+ [OTHER_CLASS] = "other",
[COMPUTE_CLASS] = "ccs",
};
@@ -190,6 +191,15 @@ static void add_legacy_ring(struct legacy_ring *ring,
ring->instance++;
}
+static void engine_rename(struct intel_engine_cs *engine, const char *name, u16 instance)
+{
+ char old[sizeof(engine->name)];
+
+ memcpy(old, engine->name, sizeof(engine->name));
+ scnprintf(engine->name, sizeof(engine->name), "%s%u", name, instance);
+ drm_dbg(&engine->i915->drm, "renamed %s to %s\n", old, engine->name);
+}
+
void intel_engines_driver_register(struct drm_i915_private *i915)
{
struct legacy_ring ring = {};
@@ -205,11 +215,19 @@ void intel_engines_driver_register(struct drm_i915_private *i915)
struct intel_engine_cs *engine =
container_of((struct rb_node *)it, typeof(*engine),
uabi_node);
- char old[sizeof(engine->name)];
if (intel_gt_has_unrecoverable_error(engine->gt))
continue; /* ignore incomplete engines */
+ /*
+ * We don't want to expose the GSC engine to the users, but we
+ * still rename it so it is easier to identify in the debug logs
+ */
+ if (engine->id == GSC0) {
+ engine_rename(engine, "gsc", 0);
+ continue;
+ }
+
GEM_BUG_ON(engine->class >= ARRAY_SIZE(uabi_classes));
engine->uabi_class = uabi_classes[engine->class];
@@ -219,11 +237,9 @@ void intel_engines_driver_register(struct drm_i915_private *i915)
i915->engine_uabi_class_count[engine->uabi_class]++;
/* Replace the internal name with the final user facing name */
- memcpy(old, engine->name, sizeof(engine->name));
- scnprintf(engine->name, sizeof(engine->name), "%s%u",
- intel_engine_class_repr(engine->class),
- engine->uabi_instance);
- DRM_DEBUG_DRIVER("renamed %s to %s\n", old, engine->name);
+ engine_rename(engine,
+ intel_engine_class_repr(engine->class),
+ engine->uabi_instance);
rb_link_node(&engine->uabi_node, prev, p);
rb_insert_color(&engine->uabi_node, &i915->uabi_engines);
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 45b605e..2daffa7 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -110,6 +110,7 @@
#include <linux/string_helpers.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "i915_trace.h"
#include "i915_vgpu.h"
#include "gen8_engine_cs.h"
@@ -3471,9 +3472,9 @@ logical_ring_default_vfuncs(struct intel_engine_cs *engine)
if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 50)) {
if (intel_engine_has_preemption(engine))
- engine->emit_bb_start = gen125_emit_bb_start;
+ engine->emit_bb_start = xehp_emit_bb_start;
else
- engine->emit_bb_start = gen125_emit_bb_start_noarb;
+ engine->emit_bb_start = xehp_emit_bb_start_noarb;
} else {
if (intel_engine_has_preemption(engine))
engine->emit_bb_start = gen8_emit_bb_start;
@@ -3921,6 +3922,7 @@ static struct intel_context *
execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
unsigned long flags)
{
+ struct drm_i915_private *i915 = siblings[0]->i915;
struct virtual_engine *ve;
unsigned int n;
int err;
@@ -3929,7 +3931,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
if (!ve)
return ERR_PTR(-ENOMEM);
- ve->base.i915 = siblings[0]->i915;
+ ve->base.i915 = i915;
ve->base.gt = siblings[0]->gt;
ve->base.uncore = siblings[0]->uncore;
ve->base.id = -1;
@@ -3988,8 +3990,9 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
GEM_BUG_ON(!is_power_of_2(sibling->mask));
if (sibling->mask & ve->base.mask) {
- DRM_DEBUG("duplicate %s entry in load balancer\n",
- sibling->name);
+ drm_dbg(&i915->drm,
+ "duplicate %s entry in load balancer\n",
+ sibling->name);
err = -EINVAL;
goto err_put;
}
@@ -4023,8 +4026,9 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
*/
if (ve->base.class != OTHER_CLASS) {
if (ve->base.class != sibling->class) {
- DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",
- sibling->class, ve->base.class);
+ drm_dbg(&i915->drm,
+ "invalid mixing of engine class, sibling %d, already %d\n",
+ sibling->class, ve->base.class);
err = -EINVAL;
goto err_put;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c
index 2049a00..8145851 100644
--- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
+++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
@@ -560,7 +560,7 @@ static int init_ggtt(struct i915_ggtt *ggtt)
* why.
*/
ggtt->pin_bias = max_t(u32, I915_GTT_PAGE_SIZE,
- intel_wopcm_guc_size(&ggtt->vm.i915->wopcm));
+ intel_wopcm_guc_size(&ggtt->vm.gt->wopcm));
ret = intel_vgt_balloon(ggtt);
if (ret)
@@ -871,8 +871,8 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size)
u32 pte_flags;
int ret;
- GEM_WARN_ON(pci_resource_len(pdev, GTTMMADR_BAR) != gen6_gttmmadr_size(i915));
- phys_addr = pci_resource_start(pdev, GTTMMADR_BAR) + gen6_gttadr_offset(i915);
+ GEM_WARN_ON(pci_resource_len(pdev, GEN4_GTTMMADR_BAR) != gen6_gttmmadr_size(i915));
+ phys_addr = pci_resource_start(pdev, GEN4_GTTMMADR_BAR) + gen6_gttadr_offset(i915);
/*
* On BXT+/ICL+ writes larger than 64 bit to the GTT pagetable range
@@ -931,11 +931,11 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
unsigned int size;
u16 snb_gmch_ctl;
- if (!HAS_LMEM(i915)) {
- if (!i915_pci_resource_valid(pdev, GTT_APERTURE_BAR))
+ if (!HAS_LMEM(i915) && !HAS_LMEMBAR_SMEM_STOLEN(i915)) {
+ if (!i915_pci_resource_valid(pdev, GEN4_GMADR_BAR))
return -ENXIO;
- ggtt->gmadr = pci_resource(pdev, GTT_APERTURE_BAR);
+ ggtt->gmadr = pci_resource(pdev, GEN4_GMADR_BAR);
ggtt->mappable_end = resource_size(&ggtt->gmadr);
}
@@ -986,7 +986,7 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
ggtt->vm.pte_encode = gen8_ggtt_pte_encode;
- setup_private_pat(ggtt->vm.gt->uncore);
+ setup_private_pat(ggtt->vm.gt);
return ggtt_probe_common(ggtt, size);
}
@@ -1089,10 +1089,10 @@ static int gen6_gmch_probe(struct i915_ggtt *ggtt)
unsigned int size;
u16 snb_gmch_ctl;
- if (!i915_pci_resource_valid(pdev, GTT_APERTURE_BAR))
+ if (!i915_pci_resource_valid(pdev, GEN4_GMADR_BAR))
return -ENXIO;
- ggtt->gmadr = pci_resource(pdev, GTT_APERTURE_BAR);
+ ggtt->gmadr = pci_resource(pdev, GEN4_GMADR_BAR);
ggtt->mappable_end = resource_size(&ggtt->gmadr);
/*
@@ -1308,7 +1308,7 @@ void i915_ggtt_resume(struct i915_ggtt *ggtt)
wbinvd_on_all_cpus();
if (GRAPHICS_VER(ggtt->vm.i915) >= 8)
- setup_private_pat(ggtt->vm.gt->uncore);
+ setup_private_pat(ggtt->vm.gt);
intel_ggtt_restore_fences(ggtt);
}
diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt_fencing.c b/drivers/gpu/drm/i915/gt/intel_ggtt_fencing.c
index ea775e6..995082d 100644
--- a/drivers/gpu/drm/i915/gt/intel_ggtt_fencing.c
+++ b/drivers/gpu/drm/i915/gt/intel_ggtt_fencing.c
@@ -816,8 +816,8 @@ i915_gem_object_save_bit_17_swizzle(struct drm_i915_gem_object *obj,
if (obj->bit_17 == NULL) {
obj->bit_17 = bitmap_zalloc(page_count, GFP_KERNEL);
if (obj->bit_17 == NULL) {
- DRM_ERROR("Failed to allocate memory for bit 17 "
- "record\n");
+ drm_err(&to_i915(obj->base.dev)->drm,
+ "Failed to allocate memory for bit 17 record\n");
return;
}
}
diff --git a/drivers/gpu/drm/i915/gt/intel_gpu_commands.h b/drivers/gpu/drm/i915/gt/intel_gpu_commands.h
index d4e9702..f50ea92 100644
--- a/drivers/gpu/drm/i915/gt/intel_gpu_commands.h
+++ b/drivers/gpu/drm/i915/gt/intel_gpu_commands.h
@@ -187,6 +187,10 @@
#define MI_BATCH_RESOURCE_STREAMER REG_BIT(10)
#define MI_BATCH_PREDICATE REG_BIT(15) /* HSW+ on RCS only*/
+#define MI_OPCODE(x) (((x) >> 23) & 0x3f)
+#define IS_MI_LRI_CMD(x) (MI_OPCODE(x) == MI_OPCODE(MI_INSTR(0x22, 0)))
+#define MI_LRI_LEN(x) (((x) & 0xff) + 1)
+
/*
* 3D instructions used by the kernel
*/
diff --git a/drivers/gpu/drm/i915/gt/intel_gsc.c b/drivers/gpu/drm/i915/gt/intel_gsc.c
index 7af6db3..976fdf2 100644
--- a/drivers/gpu/drm/i915/gt/intel_gsc.c
+++ b/drivers/gpu/drm/i915/gt/intel_gsc.c
@@ -7,6 +7,7 @@
#include <linux/mei_aux.h>
#include "i915_drv.h"
#include "i915_reg.h"
+#include "gem/i915_gem_lmem.h"
#include "gem/i915_gem_region.h"
#include "gt/intel_gsc.h"
#include "gt/intel_gt.h"
@@ -142,8 +143,14 @@ static void gsc_destroy_one(struct drm_i915_private *i915,
struct intel_gsc_intf *intf = &gsc->intf[intf_id];
if (intf->adev) {
- auxiliary_device_delete(&intf->adev->aux_dev);
- auxiliary_device_uninit(&intf->adev->aux_dev);
+ struct auxiliary_device *aux_dev = &intf->adev->aux_dev;
+
+ if (intf_id == 0)
+ intel_huc_unregister_gsc_notifier(&gsc_to_gt(gsc)->uc.huc,
+ aux_dev->dev.bus);
+
+ auxiliary_device_delete(aux_dev);
+ auxiliary_device_uninit(aux_dev);
intf->adev = NULL;
}
@@ -242,14 +249,24 @@ static void gsc_init_one(struct drm_i915_private *i915, struct intel_gsc *gsc,
goto fail;
}
+ intf->adev = adev; /* needed by the notifier */
+
+ if (intf_id == 0)
+ intel_huc_register_gsc_notifier(&gsc_to_gt(gsc)->uc.huc,
+ aux_dev->dev.bus);
+
ret = auxiliary_device_add(aux_dev);
if (ret < 0) {
drm_err(&i915->drm, "gsc aux add failed %d\n", ret);
+ if (intf_id == 0)
+ intel_huc_unregister_gsc_notifier(&gsc_to_gt(gsc)->uc.huc,
+ aux_dev->dev.bus);
+ intf->adev = NULL;
+
/* adev will be freed with the put_device() and .release sequence */
auxiliary_device_uninit(aux_dev);
goto fail;
}
- intf->adev = adev;
return;
fail:
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
index 7caa3412..767e329 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt.c
@@ -12,6 +12,7 @@
#include "i915_drv.h"
#include "i915_perf_oa_regs.h"
+#include "i915_reg.h"
#include "intel_context.h"
#include "intel_engine_pm.h"
#include "intel_engine_regs.h"
@@ -40,8 +41,6 @@ void intel_gt_common_init_early(struct intel_gt *gt)
{
spin_lock_init(gt->irq_lock);
- INIT_LIST_HEAD(>->lmem_userfault_list);
- mutex_init(>->lmem_userfault_lock);
INIT_LIST_HEAD(>->closed_vma);
spin_lock_init(>->closed_lock);
@@ -56,6 +55,7 @@ void intel_gt_common_init_early(struct intel_gt *gt)
seqcount_mutex_init(>->tlb.seqno, >->tlb.invalidate_lock);
intel_gt_pm_init_early(gt);
+ intel_wopcm_init_early(>->wopcm);
intel_uc_init_early(>->uc);
intel_rps_init_early(>->rps);
}
@@ -192,7 +192,7 @@ int intel_gt_init_hw(struct intel_gt *gt)
ret = i915_ppgtt_init_hw(gt);
if (ret) {
- DRM_ERROR("Enabling PPGTT failed (%d)\n", ret);
+ drm_err(&i915->drm, "Enabling PPGTT failed (%d)\n", ret);
goto out;
}
@@ -231,6 +231,16 @@ static void gen6_clear_engine_error_register(struct intel_engine_cs *engine)
GEN6_RING_FAULT_REG_POSTING_READ(engine);
}
+i915_reg_t intel_gt_perf_limit_reasons_reg(struct intel_gt *gt)
+{
+ /* GT0_PERF_LIMIT_REASONS is available only for Gen11+ */
+ if (GRAPHICS_VER(gt->i915) < 11)
+ return INVALID_MMIO_REG;
+
+ return gt->type == GT_MEDIA ?
+ MTL_MEDIA_PERF_LIMIT_REASONS : GT0_PERF_LIMIT_REASONS;
+}
+
void
intel_gt_clear_error_registers(struct intel_gt *gt,
intel_engine_mask_t engine_mask)
@@ -254,13 +264,17 @@ intel_gt_clear_error_registers(struct intel_gt *gt,
* some errors might have become stuck,
* mask them.
*/
- DRM_DEBUG_DRIVER("EIR stuck: 0x%08x, masking\n", eir);
+ drm_dbg(>->i915->drm, "EIR stuck: 0x%08x, masking\n", eir);
rmw_set(uncore, EMR, eir);
intel_uncore_write(uncore, GEN2_IIR,
I915_MASTER_ERROR_INTERRUPT);
}
- if (GRAPHICS_VER(i915) >= 12) {
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) {
+ intel_gt_mcr_multicast_rmw(gt, XEHP_RING_FAULT_REG,
+ RING_FAULT_VALID, 0);
+ intel_gt_mcr_read_any(gt, XEHP_RING_FAULT_REG);
+ } else if (GRAPHICS_VER(i915) >= 12) {
rmw_clear(uncore, GEN12_RING_FAULT_REG, RING_FAULT_VALID);
intel_uncore_posting_read(uncore, GEN12_RING_FAULT_REG);
} else if (GRAPHICS_VER(i915) >= 8) {
@@ -298,6 +312,42 @@ static void gen6_check_faults(struct intel_gt *gt)
}
}
+static void xehp_check_faults(struct intel_gt *gt)
+{
+ u32 fault;
+
+ /*
+ * Although the fault register now lives in an MCR register range,
+ * the GAM registers are special and we only truly need to read
+ * the "primary" GAM instance rather than handling each instance
+ * individually. intel_gt_mcr_read_any() will automatically steer
+ * toward the primary instance.
+ */
+ fault = intel_gt_mcr_read_any(gt, XEHP_RING_FAULT_REG);
+ if (fault & RING_FAULT_VALID) {
+ u32 fault_data0, fault_data1;
+ u64 fault_addr;
+
+ fault_data0 = intel_gt_mcr_read_any(gt, XEHP_FAULT_TLB_DATA0);
+ fault_data1 = intel_gt_mcr_read_any(gt, XEHP_FAULT_TLB_DATA1);
+
+ fault_addr = ((u64)(fault_data1 & FAULT_VA_HIGH_BITS) << 44) |
+ ((u64)fault_data0 << 12);
+
+ drm_dbg(>->i915->drm, "Unexpected fault\n"
+ "\tAddr: 0x%08x_%08x\n"
+ "\tAddress space: %s\n"
+ "\tEngine ID: %d\n"
+ "\tSource ID: %d\n"
+ "\tType: %d\n",
+ upper_32_bits(fault_addr), lower_32_bits(fault_addr),
+ fault_data1 & FAULT_GTT_SEL ? "GGTT" : "PPGTT",
+ GEN8_RING_FAULT_ENGINE_ID(fault),
+ RING_FAULT_SRCID(fault),
+ RING_FAULT_FAULT_TYPE(fault));
+ }
+}
+
static void gen8_check_faults(struct intel_gt *gt)
{
struct intel_uncore *uncore = gt->uncore;
@@ -344,7 +394,9 @@ void intel_gt_check_and_clear_faults(struct intel_gt *gt)
struct drm_i915_private *i915 = gt->i915;
/* From GEN8 onwards we only have one 'All Engine Fault Register' */
- if (GRAPHICS_VER(i915) >= 8)
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50))
+ xehp_check_faults(gt);
+ else if (GRAPHICS_VER(i915) >= 8)
gen8_check_faults(gt);
else if (GRAPHICS_VER(i915) >= 6)
gen6_check_faults(gt);
@@ -812,7 +864,6 @@ static int intel_gt_tile_setup(struct intel_gt *gt, phys_addr_t phys_addr)
}
intel_uncore_init_early(gt->uncore, gt);
- intel_wakeref_auto_init(>->userfault_wakeref, gt->uncore->rpm);
ret = intel_uncore_setup_mmio(gt->uncore, phys_addr);
if (ret)
@@ -833,7 +884,7 @@ int intel_gt_probe_all(struct drm_i915_private *i915)
unsigned int i;
int ret;
- mmio_bar = GRAPHICS_VER(i915) == 2 ? GEN2_GTTMMADR_BAR : GTTMMADR_BAR;
+ mmio_bar = intel_mmio_bar(GRAPHICS_VER(i915));
phys_addr = pci_resource_start(pdev, mmio_bar);
/*
@@ -944,7 +995,10 @@ void intel_gt_info_print(const struct intel_gt_info *info,
}
struct reg_and_bit {
- i915_reg_t reg;
+ union {
+ i915_reg_t reg;
+ i915_mcr_reg_t mcr_reg;
+ };
u32 bit;
};
@@ -970,6 +1024,32 @@ get_reg_and_bit(const struct intel_engine_cs *engine, const bool gen8,
return rb;
}
+/*
+ * HW architecture suggest typical invalidation time at 40us,
+ * with pessimistic cases up to 100us and a recommendation to
+ * cap at 1ms. We go a bit higher just in case.
+ */
+#define TLB_INVAL_TIMEOUT_US 100
+#define TLB_INVAL_TIMEOUT_MS 4
+
+/*
+ * On Xe_HP the TLB invalidation registers are located at the same MMIO offsets
+ * but are now considered MCR registers. Since they exist within a GAM range,
+ * the primary instance of the register rolls up the status from each unit.
+ */
+static int wait_for_invalidate(struct intel_gt *gt, struct reg_and_bit rb)
+{
+ if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 50))
+ return intel_gt_mcr_wait_for_reg(gt, rb.mcr_reg, rb.bit, 0,
+ TLB_INVAL_TIMEOUT_US,
+ TLB_INVAL_TIMEOUT_MS);
+ else
+ return __intel_wait_for_register_fw(gt->uncore, rb.reg, rb.bit, 0,
+ TLB_INVAL_TIMEOUT_US,
+ TLB_INVAL_TIMEOUT_MS,
+ NULL);
+}
+
static void mmio_invalidate_full(struct intel_gt *gt)
{
static const i915_reg_t gen8_regs[] = {
@@ -985,6 +1065,13 @@ static void mmio_invalidate_full(struct intel_gt *gt)
[COPY_ENGINE_CLASS] = GEN12_BLT_TLB_INV_CR,
[COMPUTE_CLASS] = GEN12_COMPCTX_TLB_INV_CR,
};
+ static const i915_mcr_reg_t xehp_regs[] = {
+ [RENDER_CLASS] = XEHP_GFX_TLB_INV_CR,
+ [VIDEO_DECODE_CLASS] = XEHP_VD_TLB_INV_CR,
+ [VIDEO_ENHANCEMENT_CLASS] = XEHP_VE_TLB_INV_CR,
+ [COPY_ENGINE_CLASS] = XEHP_BLT_TLB_INV_CR,
+ [COMPUTE_CLASS] = XEHP_COMPCTX_TLB_INV_CR,
+ };
struct drm_i915_private *i915 = gt->i915;
struct intel_uncore *uncore = gt->uncore;
struct intel_engine_cs *engine;
@@ -993,7 +1080,10 @@ static void mmio_invalidate_full(struct intel_gt *gt)
const i915_reg_t *regs;
unsigned int num = 0;
- if (GRAPHICS_VER(i915) == 12) {
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) {
+ regs = NULL;
+ num = ARRAY_SIZE(xehp_regs);
+ } else if (GRAPHICS_VER(i915) == 12) {
regs = gen12_regs;
num = ARRAY_SIZE(gen12_regs);
} else if (GRAPHICS_VER(i915) >= 8 && GRAPHICS_VER(i915) <= 11) {
@@ -1018,16 +1108,22 @@ static void mmio_invalidate_full(struct intel_gt *gt)
if (!intel_engine_pm_is_awake(engine))
continue;
- rb = get_reg_and_bit(engine, regs == gen8_regs, regs, num);
- if (!i915_mmio_reg_offset(rb.reg))
- continue;
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) {
+ intel_gt_mcr_multicast_write_fw(gt,
+ xehp_regs[engine->class],
+ BIT(engine->instance));
+ } else {
+ rb = get_reg_and_bit(engine, regs == gen8_regs, regs, num);
+ if (!i915_mmio_reg_offset(rb.reg))
+ continue;
- if (GRAPHICS_VER(i915) == 12 && (engine->class == VIDEO_DECODE_CLASS ||
- engine->class == VIDEO_ENHANCEMENT_CLASS ||
- engine->class == COMPUTE_CLASS))
- rb.bit = _MASKED_BIT_ENABLE(rb.bit);
+ if (GRAPHICS_VER(i915) == 12 && (engine->class == VIDEO_DECODE_CLASS ||
+ engine->class == VIDEO_ENHANCEMENT_CLASS ||
+ engine->class == COMPUTE_CLASS))
+ rb.bit = _MASKED_BIT_ENABLE(rb.bit);
- intel_uncore_write_fw(uncore, rb.reg, rb.bit);
+ intel_uncore_write_fw(uncore, rb.reg, rb.bit);
+ }
awake |= engine->mask;
}
@@ -1047,22 +1143,17 @@ static void mmio_invalidate_full(struct intel_gt *gt)
for_each_engine_masked(engine, gt, awake, tmp) {
struct reg_and_bit rb;
- /*
- * HW architecture suggest typical invalidation time at 40us,
- * with pessimistic cases up to 100us and a recommendation to
- * cap at 1ms. We go a bit higher just in case.
- */
- const unsigned int timeout_us = 100;
- const unsigned int timeout_ms = 4;
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) {
+ rb.mcr_reg = xehp_regs[engine->class];
+ rb.bit = BIT(engine->instance);
+ } else {
+ rb = get_reg_and_bit(engine, regs == gen8_regs, regs, num);
+ }
- rb = get_reg_and_bit(engine, regs == gen8_regs, regs, num);
- if (__intel_wait_for_register_fw(uncore,
- rb.reg, rb.bit, 0,
- timeout_us, timeout_ms,
- NULL))
+ if (wait_for_invalidate(gt, rb))
drm_err_ratelimited(>->i915->drm,
"%s TLB invalidation did not complete in %ums!\n",
- engine->name, timeout_ms);
+ engine->name, TLB_INVAL_TIMEOUT_MS);
}
/*
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.h b/drivers/gpu/drm/i915/gt/intel_gt.h
index 2ee582e..e0365d55 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt.h
@@ -60,6 +60,7 @@ void intel_gt_driver_late_release_all(struct drm_i915_private *i915);
int intel_gt_wait_for_idle(struct intel_gt *gt, long timeout);
void intel_gt_check_and_clear_faults(struct intel_gt *gt);
+i915_reg_t intel_gt_perf_limit_reasons_reg(struct intel_gt *gt);
void intel_gt_clear_error_registers(struct intel_gt *gt,
intel_engine_mask_t engine_mask);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_clock_utils.c b/drivers/gpu/drm/i915/gt/intel_gt_clock_utils.c
index 3f656d3..2a6a4ca 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_clock_utils.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_clock_utils.c
@@ -107,7 +107,7 @@ static u32 gen9_read_clock_frequency(struct intel_uncore *uncore)
return freq;
}
-static u32 gen5_read_clock_frequency(struct intel_uncore *uncore)
+static u32 gen6_read_clock_frequency(struct intel_uncore *uncore)
{
/*
* PRMs say:
@@ -119,7 +119,27 @@ static u32 gen5_read_clock_frequency(struct intel_uncore *uncore)
return 12500000;
}
-static u32 gen2_read_clock_frequency(struct intel_uncore *uncore)
+static u32 gen5_read_clock_frequency(struct intel_uncore *uncore)
+{
+ /*
+ * 63:32 increments every 1000 ns
+ * 31:0 mbz
+ */
+ return 1000000000 / 1000;
+}
+
+static u32 g4x_read_clock_frequency(struct intel_uncore *uncore)
+{
+ /*
+ * 63:20 increments every 1/4 ns
+ * 19:0 mbz
+ *
+ * -> 63:32 increments every 1024 ns
+ */
+ return 1000000000 / 1024;
+}
+
+static u32 gen4_read_clock_frequency(struct intel_uncore *uncore)
{
/*
* PRMs say:
@@ -127,8 +147,10 @@ static u32 gen2_read_clock_frequency(struct intel_uncore *uncore)
* "The value in this register increments once every 16
* hclks." (through the “Clocking Configuration”
* (“CLKCFG”) MCHBAR register)
+ *
+ * Testing on actual hardware has shown there is no /16.
*/
- return RUNTIME_INFO(uncore->i915)->rawclk_freq * 1000 / 16;
+ return RUNTIME_INFO(uncore->i915)->rawclk_freq * 1000;
}
static u32 read_clock_frequency(struct intel_uncore *uncore)
@@ -137,10 +159,16 @@ static u32 read_clock_frequency(struct intel_uncore *uncore)
return gen11_read_clock_frequency(uncore);
else if (GRAPHICS_VER(uncore->i915) >= 9)
return gen9_read_clock_frequency(uncore);
- else if (GRAPHICS_VER(uncore->i915) >= 5)
+ else if (GRAPHICS_VER(uncore->i915) >= 6)
+ return gen6_read_clock_frequency(uncore);
+ else if (GRAPHICS_VER(uncore->i915) == 5)
return gen5_read_clock_frequency(uncore);
+ else if (IS_G4X(uncore->i915))
+ return g4x_read_clock_frequency(uncore);
+ else if (GRAPHICS_VER(uncore->i915) == 4)
+ return gen4_read_clock_frequency(uncore);
else
- return gen2_read_clock_frequency(uncore);
+ return 0;
}
void intel_gt_init_clock_frequency(struct intel_gt *gt)
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_irq.c b/drivers/gpu/drm/i915/gt/intel_gt_irq.c
index f26882f..6f6b9e0 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_irq.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_irq.c
@@ -17,6 +17,9 @@
static void guc_irq_handler(struct intel_guc *guc, u16 iir)
{
+ if (unlikely(!guc->interrupts.enabled))
+ return;
+
if (iir & GUC_INTR_GUC2HOST)
intel_guc_to_host_event_handler(guc);
}
@@ -44,8 +47,9 @@ gen11_gt_engine_identity(struct intel_gt *gt,
!time_after32(local_clock() >> 10, timeout_ts));
if (unlikely(!(ident & GEN11_INTR_DATA_VALID))) {
- DRM_ERROR("INTR_IDENTITY_REG%u:%u 0x%08x not valid!\n",
- bank, bit, ident);
+ drm_err(>->i915->drm,
+ "INTR_IDENTITY_REG%u:%u 0x%08x not valid!\n",
+ bank, bit, ident);
return 0;
}
@@ -81,35 +85,27 @@ gen11_other_irq_handler(struct intel_gt *gt, const u8 instance,
instance, iir);
}
-static void
-gen11_engine_irq_handler(struct intel_gt *gt, const u8 class,
- const u8 instance, const u16 iir)
+static struct intel_gt *pick_gt(struct intel_gt *gt, u8 class, u8 instance)
{
- struct intel_engine_cs *engine;
+ struct intel_gt *media_gt = gt->i915->media_gt;
- /*
- * Platforms with standalone media have their media engines in another
- * GT.
- */
- if (MEDIA_VER(gt->i915) >= 13 &&
- (class == VIDEO_DECODE_CLASS || class == VIDEO_ENHANCEMENT_CLASS)) {
- if (!gt->i915->media_gt)
- goto err;
+ /* we expect the non-media gt to be passed in */
+ GEM_BUG_ON(gt == media_gt);
- gt = gt->i915->media_gt;
+ if (!media_gt)
+ return gt;
+
+ switch (class) {
+ case VIDEO_DECODE_CLASS:
+ case VIDEO_ENHANCEMENT_CLASS:
+ return media_gt;
+ case OTHER_CLASS:
+ if (instance == OTHER_GSC_INSTANCE && HAS_ENGINE(media_gt, GSC0))
+ return media_gt;
+ fallthrough;
+ default:
+ return gt;
}
-
- if (instance <= MAX_ENGINE_INSTANCE)
- engine = gt->engine_class[class][instance];
- else
- engine = NULL;
-
- if (likely(engine))
- return intel_engine_cs_irq(engine, iir);
-
-err:
- WARN_ONCE(1, "unhandled engine interrupt class=0x%x, instance=0x%x\n",
- class, instance);
}
static void
@@ -122,8 +118,17 @@ gen11_gt_identity_handler(struct intel_gt *gt, const u32 identity)
if (unlikely(!intr))
return;
- if (class <= COPY_ENGINE_CLASS || class == COMPUTE_CLASS)
- return gen11_engine_irq_handler(gt, class, instance, intr);
+ /*
+ * Platforms with standalone media have the media and GSC engines in
+ * another GT.
+ */
+ gt = pick_gt(gt, class, instance);
+
+ if (class <= MAX_ENGINE_CLASS && instance <= MAX_ENGINE_INSTANCE) {
+ struct intel_engine_cs *engine = gt->engine_class[class][instance];
+ if (engine)
+ return intel_engine_cs_irq(engine, intr);
+ }
if (class == OTHER_CLASS)
return gen11_other_irq_handler(gt, instance, intr);
@@ -206,7 +211,7 @@ void gen11_gt_irq_reset(struct intel_gt *gt)
intel_uncore_write(uncore, GEN11_VCS_VECS_INTR_ENABLE, 0);
if (CCS_MASK(gt))
intel_uncore_write(uncore, GEN12_CCS_RSVD_INTR_ENABLE, 0);
- if (HAS_HECI_GSC(gt->i915))
+ if (HAS_HECI_GSC(gt->i915) || HAS_ENGINE(gt, GSC0))
intel_uncore_write(uncore, GEN11_GUNIT_CSME_INTR_ENABLE, 0);
/* Restore masks irqs on RCS, BCS, VCS and VECS engines. */
@@ -233,7 +238,7 @@ void gen11_gt_irq_reset(struct intel_gt *gt)
intel_uncore_write(uncore, GEN12_CCS0_CCS1_INTR_MASK, ~0);
if (HAS_ENGINE(gt, CCS2) || HAS_ENGINE(gt, CCS3))
intel_uncore_write(uncore, GEN12_CCS2_CCS3_INTR_MASK, ~0);
- if (HAS_HECI_GSC(gt->i915))
+ if (HAS_HECI_GSC(gt->i915) || HAS_ENGINE(gt, GSC0))
intel_uncore_write(uncore, GEN11_GUNIT_CSME_INTR_MASK, ~0);
intel_uncore_write(uncore, GEN11_GPM_WGBOXPERF_INTR_ENABLE, 0);
@@ -249,7 +254,8 @@ void gen11_gt_irq_postinstall(struct intel_gt *gt)
{
struct intel_uncore *uncore = gt->uncore;
u32 irqs = GT_RENDER_USER_INTERRUPT;
- const u32 gsc_mask = GSC_IRQ_INTF(0) | GSC_IRQ_INTF(1);
+ u32 guc_mask = intel_uc_wants_guc(>->uc) ? GUC_INTR_GUC2HOST : 0;
+ u32 gsc_mask = 0;
u32 dmask;
u32 smask;
@@ -261,6 +267,11 @@ void gen11_gt_irq_postinstall(struct intel_gt *gt)
dmask = irqs << 16 | irqs;
smask = irqs << 16;
+ if (HAS_ENGINE(gt, GSC0))
+ gsc_mask = irqs;
+ else if (HAS_HECI_GSC(gt->i915))
+ gsc_mask = GSC_IRQ_INTF(0) | GSC_IRQ_INTF(1);
+
BUILD_BUG_ON(irqs & 0xffff0000);
/* Enable RCS, BCS, VCS and VECS class interrupts. */
@@ -268,9 +279,8 @@ void gen11_gt_irq_postinstall(struct intel_gt *gt)
intel_uncore_write(uncore, GEN11_VCS_VECS_INTR_ENABLE, dmask);
if (CCS_MASK(gt))
intel_uncore_write(uncore, GEN12_CCS_RSVD_INTR_ENABLE, smask);
- if (HAS_HECI_GSC(gt->i915))
- intel_uncore_write(uncore, GEN11_GUNIT_CSME_INTR_ENABLE,
- gsc_mask);
+ if (gsc_mask)
+ intel_uncore_write(uncore, GEN11_GUNIT_CSME_INTR_ENABLE, gsc_mask);
/* Unmask irqs on RCS, BCS, VCS and VECS engines. */
intel_uncore_write(uncore, GEN11_RCS0_RSVD_INTR_MASK, ~smask);
@@ -296,9 +306,22 @@ void gen11_gt_irq_postinstall(struct intel_gt *gt)
intel_uncore_write(uncore, GEN12_CCS0_CCS1_INTR_MASK, ~dmask);
if (HAS_ENGINE(gt, CCS2) || HAS_ENGINE(gt, CCS3))
intel_uncore_write(uncore, GEN12_CCS2_CCS3_INTR_MASK, ~dmask);
- if (HAS_HECI_GSC(gt->i915))
+ if (gsc_mask)
intel_uncore_write(uncore, GEN11_GUNIT_CSME_INTR_MASK, ~gsc_mask);
+ if (guc_mask) {
+ /* the enable bit is common for both GTs but the masks are separate */
+ u32 mask = gt->type == GT_MEDIA ?
+ REG_FIELD_PREP(ENGINE0_MASK, guc_mask) :
+ REG_FIELD_PREP(ENGINE1_MASK, guc_mask);
+
+ intel_uncore_write(uncore, GEN11_GUC_SG_INTR_ENABLE,
+ REG_FIELD_PREP(ENGINE1_MASK, guc_mask));
+
+ /* we might not be the first GT to write this reg */
+ intel_uncore_rmw(uncore, MTL_GUC_MGUC_INTR_MASK, mask, 0);
+ }
+
/*
* RPS interrupts will get enabled/disabled on demand when RPS itself
* is enabled/disabled.
@@ -307,10 +330,6 @@ void gen11_gt_irq_postinstall(struct intel_gt *gt)
gt->pm_imr = ~gt->pm_ier;
intel_uncore_write(uncore, GEN11_GPM_WGBOXPERF_INTR_ENABLE, 0);
intel_uncore_write(uncore, GEN11_GPM_WGBOXPERF_INTR_MASK, ~0);
-
- /* Same thing for GuC interrupts */
- intel_uncore_write(uncore, GEN11_GUC_SG_INTR_ENABLE, 0);
- intel_uncore_write(uncore, GEN11_GUC_SG_INTR_MASK, ~0);
}
void gen5_gt_irq_handler(struct intel_gt *gt, u32 gt_iir)
@@ -359,7 +378,8 @@ void gen6_gt_irq_handler(struct intel_gt *gt, u32 gt_iir)
if (gt_iir & (GT_BLT_CS_ERROR_INTERRUPT |
GT_BSD_CS_ERROR_INTERRUPT |
GT_CS_MASTER_ERROR_INTERRUPT))
- DRM_DEBUG("Command parser error, gt_iir 0x%08x\n", gt_iir);
+ drm_dbg(>->i915->drm, "Command parser error, gt_iir 0x%08x\n",
+ gt_iir);
if (gt_iir & GT_PARITY_ERROR(gt->i915))
gen7_parity_error_irq_handler(gt, gt_iir);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_mcr.c b/drivers/gpu/drm/i915/gt/intel_gt_mcr.c
index e79405a..d9a8ff9 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_mcr.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_mcr.c
@@ -40,6 +40,9 @@ static const char * const intel_steering_types[] = {
"L3BANK",
"MSLICE",
"LNCF",
+ "GAM",
+ "DSS",
+ "OADDRM",
"INSTANCE 0",
};
@@ -48,14 +51,23 @@ static const struct intel_mmio_range icl_l3bank_steering_table[] = {
{},
};
+/*
+ * Although the bspec lists more "MSLICE" ranges than shown here, some of those
+ * are of a "GAM" subclass that has special rules. Thus we use a separate
+ * GAM table farther down for those.
+ */
static const struct intel_mmio_range xehpsdv_mslice_steering_table[] = {
- { 0x004000, 0x004AFF },
- { 0x00C800, 0x00CFFF },
{ 0x00DD00, 0x00DDFF },
{ 0x00E900, 0x00FFFF }, /* 0xEA00 - OxEFFF is unused */
{},
};
+static const struct intel_mmio_range xehpsdv_gam_steering_table[] = {
+ { 0x004000, 0x004AFF },
+ { 0x00C800, 0x00CFFF },
+ {},
+};
+
static const struct intel_mmio_range xehpsdv_lncf_steering_table[] = {
{ 0x00B000, 0x00B0FF },
{ 0x00D800, 0x00D8FF },
@@ -89,9 +101,47 @@ static const struct intel_mmio_range pvc_instance0_steering_table[] = {
{},
};
+static const struct intel_mmio_range xelpg_instance0_steering_table[] = {
+ { 0x000B00, 0x000BFF }, /* SQIDI */
+ { 0x001000, 0x001FFF }, /* SQIDI */
+ { 0x004000, 0x0048FF }, /* GAM */
+ { 0x008700, 0x0087FF }, /* SQIDI */
+ { 0x00B000, 0x00B0FF }, /* NODE */
+ { 0x00C800, 0x00CFFF }, /* GAM */
+ { 0x00D880, 0x00D8FF }, /* NODE */
+ { 0x00DD00, 0x00DDFF }, /* OAAL2 */
+ {},
+};
+
+static const struct intel_mmio_range xelpg_l3bank_steering_table[] = {
+ { 0x00B100, 0x00B3FF },
+ {},
+};
+
+/* DSS steering is used for SLICE ranges as well */
+static const struct intel_mmio_range xelpg_dss_steering_table[] = {
+ { 0x005200, 0x0052FF }, /* SLICE */
+ { 0x005500, 0x007FFF }, /* SLICE */
+ { 0x008140, 0x00815F }, /* SLICE (0x8140-0x814F), DSS (0x8150-0x815F) */
+ { 0x0094D0, 0x00955F }, /* SLICE (0x94D0-0x951F), DSS (0x9520-0x955F) */
+ { 0x009680, 0x0096FF }, /* DSS */
+ { 0x00D800, 0x00D87F }, /* SLICE */
+ { 0x00DC00, 0x00DCFF }, /* SLICE */
+ { 0x00DE80, 0x00E8FF }, /* DSS (0xE000-0xE0FF reserved) */
+ {},
+};
+
+static const struct intel_mmio_range xelpmp_oaddrm_steering_table[] = {
+ { 0x393200, 0x39323F },
+ { 0x393400, 0x3934FF },
+ {},
+};
+
void intel_gt_mcr_init(struct intel_gt *gt)
{
struct drm_i915_private *i915 = gt->i915;
+ unsigned long fuse;
+ int i;
/*
* An mslice is unavailable only if both the meml3 for the slice is
@@ -109,14 +159,36 @@ void intel_gt_mcr_init(struct intel_gt *gt)
drm_warn(&i915->drm, "mslice mask all zero!\n");
}
- if (IS_PONTEVECCHIO(i915)) {
+ if (MEDIA_VER(i915) >= 13 && gt->type == GT_MEDIA) {
+ gt->steering_table[OADDRM] = xelpmp_oaddrm_steering_table;
+ } else if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70)) {
+ fuse = REG_FIELD_GET(GT_L3_EXC_MASK,
+ intel_uncore_read(gt->uncore, XEHP_FUSE4));
+
+ /*
+ * Despite the register field being named "exclude mask" the
+ * bits actually represent enabled banks (two banks per bit).
+ */
+ for_each_set_bit(i, &fuse, 3)
+ gt->info.l3bank_mask |= 0x3 << 2 * i;
+
+ gt->steering_table[INSTANCE0] = xelpg_instance0_steering_table;
+ gt->steering_table[L3BANK] = xelpg_l3bank_steering_table;
+ gt->steering_table[DSS] = xelpg_dss_steering_table;
+ } else if (IS_PONTEVECCHIO(i915)) {
gt->steering_table[INSTANCE0] = pvc_instance0_steering_table;
} else if (IS_DG2(i915)) {
gt->steering_table[MSLICE] = xehpsdv_mslice_steering_table;
gt->steering_table[LNCF] = dg2_lncf_steering_table;
+ /*
+ * No need to hook up the GAM table since it has a dedicated
+ * steering control register on DG2 and can use implicit
+ * steering.
+ */
} else if (IS_XEHPSDV(i915)) {
gt->steering_table[MSLICE] = xehpsdv_mslice_steering_table;
gt->steering_table[LNCF] = xehpsdv_lncf_steering_table;
+ gt->steering_table[GAM] = xehpsdv_gam_steering_table;
} else if (GRAPHICS_VER(i915) >= 11 &&
GRAPHICS_VER_FULL(i915) < IP_VER(12, 50)) {
gt->steering_table[L3BANK] = icl_l3bank_steering_table;
@@ -135,6 +207,19 @@ void intel_gt_mcr_init(struct intel_gt *gt)
}
/*
+ * Although the rest of the driver should use MCR-specific functions to
+ * read/write MCR registers, we still use the regular intel_uncore_* functions
+ * internally to implement those, so we need a way for the functions in this
+ * file to "cast" an i915_mcr_reg_t into an i915_reg_t.
+ */
+static i915_reg_t mcr_reg_cast(const i915_mcr_reg_t mcr)
+{
+ i915_reg_t r = { .reg = mcr.reg };
+
+ return r;
+}
+
+/*
* rw_with_mcr_steering_fw - Access a register with specific MCR steering
* @uncore: pointer to struct intel_uncore
* @reg: register being accessed
@@ -148,14 +233,26 @@ void intel_gt_mcr_init(struct intel_gt *gt)
* Caller needs to make sure the relevant forcewake wells are up.
*/
static u32 rw_with_mcr_steering_fw(struct intel_uncore *uncore,
- i915_reg_t reg, u8 rw_flag,
+ i915_mcr_reg_t reg, u8 rw_flag,
int group, int instance, u32 value)
{
u32 mcr_mask, mcr_ss, mcr, old_mcr, val = 0;
lockdep_assert_held(&uncore->lock);
- if (GRAPHICS_VER(uncore->i915) >= 11) {
+ if (GRAPHICS_VER_FULL(uncore->i915) >= IP_VER(12, 70)) {
+ /*
+ * Always leave the hardware in multicast mode when doing reads
+ * (see comment about Wa_22013088509 below) and only change it
+ * to unicast mode when doing writes of a specific instance.
+ *
+ * No need to save old steering reg value.
+ */
+ intel_uncore_write_fw(uncore, MTL_MCR_SELECTOR,
+ REG_FIELD_PREP(MTL_MCR_GROUPID, group) |
+ REG_FIELD_PREP(MTL_MCR_INSTANCEID, instance) |
+ (rw_flag == FW_REG_READ ? GEN11_MCR_MULTICAST : 0));
+ } else if (GRAPHICS_VER(uncore->i915) >= 11) {
mcr_mask = GEN11_MCR_SLICE_MASK | GEN11_MCR_SUBSLICE_MASK;
mcr_ss = GEN11_MCR_SLICE(group) | GEN11_MCR_SUBSLICE(instance);
@@ -173,39 +270,53 @@ static u32 rw_with_mcr_steering_fw(struct intel_uncore *uncore,
*/
if (rw_flag == FW_REG_WRITE)
mcr_mask |= GEN11_MCR_MULTICAST;
+
+ mcr = intel_uncore_read_fw(uncore, GEN8_MCR_SELECTOR);
+ old_mcr = mcr;
+
+ mcr &= ~mcr_mask;
+ mcr |= mcr_ss;
+ intel_uncore_write_fw(uncore, GEN8_MCR_SELECTOR, mcr);
} else {
mcr_mask = GEN8_MCR_SLICE_MASK | GEN8_MCR_SUBSLICE_MASK;
mcr_ss = GEN8_MCR_SLICE(group) | GEN8_MCR_SUBSLICE(instance);
+
+ mcr = intel_uncore_read_fw(uncore, GEN8_MCR_SELECTOR);
+ old_mcr = mcr;
+
+ mcr &= ~mcr_mask;
+ mcr |= mcr_ss;
+ intel_uncore_write_fw(uncore, GEN8_MCR_SELECTOR, mcr);
}
- old_mcr = mcr = intel_uncore_read_fw(uncore, GEN8_MCR_SELECTOR);
-
- mcr &= ~mcr_mask;
- mcr |= mcr_ss;
- intel_uncore_write_fw(uncore, GEN8_MCR_SELECTOR, mcr);
-
if (rw_flag == FW_REG_READ)
- val = intel_uncore_read_fw(uncore, reg);
+ val = intel_uncore_read_fw(uncore, mcr_reg_cast(reg));
else
- intel_uncore_write_fw(uncore, reg, value);
+ intel_uncore_write_fw(uncore, mcr_reg_cast(reg), value);
- mcr &= ~mcr_mask;
- mcr |= old_mcr & mcr_mask;
-
- intel_uncore_write_fw(uncore, GEN8_MCR_SELECTOR, mcr);
+ /*
+ * For pre-MTL platforms, we need to restore the old value of the
+ * steering control register to ensure that implicit steering continues
+ * to behave as expected. For MTL and beyond, we need only reinstate
+ * the 'multicast' bit (and only if we did a write that cleared it).
+ */
+ if (GRAPHICS_VER_FULL(uncore->i915) >= IP_VER(12, 70) && rw_flag == FW_REG_WRITE)
+ intel_uncore_write_fw(uncore, MTL_MCR_SELECTOR, GEN11_MCR_MULTICAST);
+ else if (GRAPHICS_VER_FULL(uncore->i915) < IP_VER(12, 70))
+ intel_uncore_write_fw(uncore, GEN8_MCR_SELECTOR, old_mcr);
return val;
}
static u32 rw_with_mcr_steering(struct intel_uncore *uncore,
- i915_reg_t reg, u8 rw_flag,
+ i915_mcr_reg_t reg, u8 rw_flag,
int group, int instance,
u32 value)
{
enum forcewake_domains fw_domains;
u32 val;
- fw_domains = intel_uncore_forcewake_for_reg(uncore, reg,
+ fw_domains = intel_uncore_forcewake_for_reg(uncore, mcr_reg_cast(reg),
rw_flag);
fw_domains |= intel_uncore_forcewake_for_reg(uncore,
GEN8_MCR_SELECTOR,
@@ -233,7 +344,7 @@ static u32 rw_with_mcr_steering(struct intel_uncore *uncore,
* group/instance.
*/
u32 intel_gt_mcr_read(struct intel_gt *gt,
- i915_reg_t reg,
+ i915_mcr_reg_t reg,
int group, int instance)
{
return rw_with_mcr_steering(gt->uncore, reg, FW_REG_READ, group, instance, 0);
@@ -250,7 +361,7 @@ u32 intel_gt_mcr_read(struct intel_gt *gt,
* Write an MCR register in unicast mode after steering toward a specific
* group/instance.
*/
-void intel_gt_mcr_unicast_write(struct intel_gt *gt, i915_reg_t reg, u32 value,
+void intel_gt_mcr_unicast_write(struct intel_gt *gt, i915_mcr_reg_t reg, u32 value,
int group, int instance)
{
rw_with_mcr_steering(gt->uncore, reg, FW_REG_WRITE, group, instance, value);
@@ -265,9 +376,16 @@ void intel_gt_mcr_unicast_write(struct intel_gt *gt, i915_reg_t reg, u32 value,
* Write an MCR register in multicast mode to update all instances.
*/
void intel_gt_mcr_multicast_write(struct intel_gt *gt,
- i915_reg_t reg, u32 value)
+ i915_mcr_reg_t reg, u32 value)
{
- intel_uncore_write(gt->uncore, reg, value);
+ /*
+ * Ensure we have multicast behavior, just in case some non-i915 agent
+ * left the hardware in unicast mode.
+ */
+ if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 70))
+ intel_uncore_write_fw(gt->uncore, MTL_MCR_SELECTOR, GEN11_MCR_MULTICAST);
+
+ intel_uncore_write(gt->uncore, mcr_reg_cast(reg), value);
}
/**
@@ -281,9 +399,44 @@ void intel_gt_mcr_multicast_write(struct intel_gt *gt,
* domains; use intel_gt_mcr_multicast_write() in cases where forcewake should
* be obtained automatically.
*/
-void intel_gt_mcr_multicast_write_fw(struct intel_gt *gt, i915_reg_t reg, u32 value)
+void intel_gt_mcr_multicast_write_fw(struct intel_gt *gt, i915_mcr_reg_t reg, u32 value)
{
- intel_uncore_write_fw(gt->uncore, reg, value);
+ /*
+ * Ensure we have multicast behavior, just in case some non-i915 agent
+ * left the hardware in unicast mode.
+ */
+ if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 70))
+ intel_uncore_write_fw(gt->uncore, MTL_MCR_SELECTOR, GEN11_MCR_MULTICAST);
+
+ intel_uncore_write_fw(gt->uncore, mcr_reg_cast(reg), value);
+}
+
+/**
+ * intel_gt_mcr_multicast_rmw - Performs a multicast RMW operations
+ * @gt: GT structure
+ * @reg: the MCR register to read and write
+ * @clear: bits to clear during RMW
+ * @set: bits to set during RMW
+ *
+ * Performs a read-modify-write on an MCR register in a multicast manner.
+ * This operation only makes sense on MCR registers where all instances are
+ * expected to have the same value. The read will target any non-terminated
+ * instance and the write will be applied to all instances.
+ *
+ * This function assumes the caller is already holding any necessary forcewake
+ * domains; use intel_gt_mcr_multicast_rmw() in cases where forcewake should
+ * be obtained automatically.
+ *
+ * Returns the old (unmodified) value read.
+ */
+u32 intel_gt_mcr_multicast_rmw(struct intel_gt *gt, i915_mcr_reg_t reg,
+ u32 clear, u32 set)
+{
+ u32 val = intel_gt_mcr_read_any(gt, reg);
+
+ intel_gt_mcr_multicast_write(gt, reg, (val & ~clear) | set);
+
+ return val;
}
/*
@@ -301,7 +454,7 @@ void intel_gt_mcr_multicast_write_fw(struct intel_gt *gt, i915_reg_t reg, u32 va
* for @type steering too.
*/
static bool reg_needs_read_steering(struct intel_gt *gt,
- i915_reg_t reg,
+ i915_mcr_reg_t reg,
enum intel_steering_type type)
{
const u32 offset = i915_mmio_reg_offset(reg);
@@ -332,6 +485,8 @@ static void get_nonterminated_steering(struct intel_gt *gt,
enum intel_steering_type type,
u8 *group, u8 *instance)
{
+ u32 dss;
+
switch (type) {
case L3BANK:
*group = 0; /* unused */
@@ -351,6 +506,15 @@ static void get_nonterminated_steering(struct intel_gt *gt,
*group = __ffs(gt->info.mslice_mask) << 1;
*instance = 0; /* unused */
break;
+ case GAM:
+ *group = IS_DG2(gt->i915) ? 1 : 0;
+ *instance = 0;
+ break;
+ case DSS:
+ dss = intel_sseu_find_first_xehp_dss(>->info.sseu, 0, 0);
+ *group = dss / GEN_DSS_PER_GSLICE;
+ *instance = dss % GEN_DSS_PER_GSLICE;
+ break;
case INSTANCE0:
/*
* There are a lot of MCR types for which instance (0, 0)
@@ -359,6 +523,13 @@ static void get_nonterminated_steering(struct intel_gt *gt,
*group = 0;
*instance = 0;
break;
+ case OADDRM:
+ if ((VDBOX_MASK(gt) | VEBOX_MASK(gt) | gt->info.sfc_mask) & BIT(0))
+ *group = 0;
+ else
+ *group = 1;
+ *instance = 0;
+ break;
default:
MISSING_CASE(type);
*group = 0;
@@ -380,7 +551,7 @@ static void get_nonterminated_steering(struct intel_gt *gt,
* steering.
*/
void intel_gt_mcr_get_nonterminated_steering(struct intel_gt *gt,
- i915_reg_t reg,
+ i915_mcr_reg_t reg,
u8 *group, u8 *instance)
{
int type;
@@ -409,7 +580,7 @@ void intel_gt_mcr_get_nonterminated_steering(struct intel_gt *gt,
*
* Returns the value from a non-terminated instance of @reg.
*/
-u32 intel_gt_mcr_read_any_fw(struct intel_gt *gt, i915_reg_t reg)
+u32 intel_gt_mcr_read_any_fw(struct intel_gt *gt, i915_mcr_reg_t reg)
{
int type;
u8 group, instance;
@@ -423,7 +594,7 @@ u32 intel_gt_mcr_read_any_fw(struct intel_gt *gt, i915_reg_t reg)
}
}
- return intel_uncore_read_fw(gt->uncore, reg);
+ return intel_uncore_read_fw(gt->uncore, mcr_reg_cast(reg));
}
/**
@@ -436,7 +607,7 @@ u32 intel_gt_mcr_read_any_fw(struct intel_gt *gt, i915_reg_t reg)
*
* Returns the value from a non-terminated instance of @reg.
*/
-u32 intel_gt_mcr_read_any(struct intel_gt *gt, i915_reg_t reg)
+u32 intel_gt_mcr_read_any(struct intel_gt *gt, i915_mcr_reg_t reg)
{
int type;
u8 group, instance;
@@ -450,7 +621,7 @@ u32 intel_gt_mcr_read_any(struct intel_gt *gt, i915_reg_t reg)
}
}
- return intel_uncore_read(gt->uncore, reg);
+ return intel_uncore_read(gt->uncore, mcr_reg_cast(reg));
}
static void report_steering_type(struct drm_printer *p,
@@ -483,11 +654,20 @@ static void report_steering_type(struct drm_printer *p,
void intel_gt_mcr_report_steering(struct drm_printer *p, struct intel_gt *gt,
bool dump_table)
{
- drm_printf(p, "Default steering: group=0x%x, instance=0x%x\n",
- gt->default_steering.groupid,
- gt->default_steering.instanceid);
+ /*
+ * Starting with MTL we no longer have default steering;
+ * all ranges are explicitly steered.
+ */
+ if (GRAPHICS_VER_FULL(gt->i915) < IP_VER(12, 70))
+ drm_printf(p, "Default steering: group=0x%x, instance=0x%x\n",
+ gt->default_steering.groupid,
+ gt->default_steering.instanceid);
- if (IS_PONTEVECCHIO(gt->i915)) {
+ if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 70)) {
+ for (int i = 0; i < NUM_STEERING_TYPES; i++)
+ if (gt->steering_table[i])
+ report_steering_type(p, gt, i, dump_table);
+ } else if (IS_PONTEVECCHIO(gt->i915)) {
report_steering_type(p, gt, INSTANCE0, dump_table);
} else if (HAS_MSLICE_STEERING(gt->i915)) {
report_steering_type(p, gt, MSLICE, dump_table);
@@ -520,3 +700,60 @@ void intel_gt_mcr_get_ss_steering(struct intel_gt *gt, unsigned int dss,
return;
}
}
+
+/**
+ * intel_gt_mcr_wait_for_reg_fw - wait until MCR register matches expected state
+ * @gt: GT structure
+ * @reg: the register to read
+ * @mask: mask to apply to register value
+ * @value: value to wait for
+ * @fast_timeout_us: fast timeout in microsecond for atomic/tight wait
+ * @slow_timeout_ms: slow timeout in millisecond
+ *
+ * This routine waits until the target register @reg contains the expected
+ * @value after applying the @mask, i.e. it waits until ::
+ *
+ * (intel_gt_mcr_read_any_fw(gt, reg) & mask) == value
+ *
+ * Otherwise, the wait will timeout after @slow_timeout_ms milliseconds.
+ * For atomic context @slow_timeout_ms must be zero and @fast_timeout_us
+ * must be not larger than 20,0000 microseconds.
+ *
+ * This function is basically an MCR-friendly version of
+ * __intel_wait_for_register_fw(). Generally this function will only be used
+ * on GAM registers which are a bit special --- although they're MCR registers,
+ * reads (e.g., waiting for status updates) are always directed to the primary
+ * instance.
+ *
+ * Note that this routine assumes the caller holds forcewake asserted, it is
+ * not suitable for very long waits.
+ *
+ * Return: 0 if the register matches the desired condition, or -ETIMEDOUT.
+ */
+int intel_gt_mcr_wait_for_reg(struct intel_gt *gt,
+ i915_mcr_reg_t reg,
+ u32 mask,
+ u32 value,
+ unsigned int fast_timeout_us,
+ unsigned int slow_timeout_ms)
+{
+ int ret;
+
+ lockdep_assert_not_held(>->uncore->lock);
+
+#define done ((intel_gt_mcr_read_any(gt, reg) & mask) == value)
+
+ /* Catch any overuse of this function */
+ might_sleep_if(slow_timeout_ms);
+ GEM_BUG_ON(fast_timeout_us > 20000);
+ GEM_BUG_ON(!fast_timeout_us && !slow_timeout_ms);
+
+ ret = -ETIMEDOUT;
+ if (fast_timeout_us && fast_timeout_us <= 20000)
+ ret = _wait_for_atomic(done, fast_timeout_us, 0);
+ if (ret && slow_timeout_ms)
+ ret = wait_for(done, slow_timeout_ms);
+
+ return ret;
+#undef done
+}
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_mcr.h b/drivers/gpu/drm/i915/gt/intel_gt_mcr.h
index 77a8b11..ae93b20 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_mcr.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_mcr.h
@@ -11,21 +11,24 @@
void intel_gt_mcr_init(struct intel_gt *gt);
u32 intel_gt_mcr_read(struct intel_gt *gt,
- i915_reg_t reg,
+ i915_mcr_reg_t reg,
int group, int instance);
-u32 intel_gt_mcr_read_any_fw(struct intel_gt *gt, i915_reg_t reg);
-u32 intel_gt_mcr_read_any(struct intel_gt *gt, i915_reg_t reg);
+u32 intel_gt_mcr_read_any_fw(struct intel_gt *gt, i915_mcr_reg_t reg);
+u32 intel_gt_mcr_read_any(struct intel_gt *gt, i915_mcr_reg_t reg);
void intel_gt_mcr_unicast_write(struct intel_gt *gt,
- i915_reg_t reg, u32 value,
+ i915_mcr_reg_t reg, u32 value,
int group, int instance);
void intel_gt_mcr_multicast_write(struct intel_gt *gt,
- i915_reg_t reg, u32 value);
+ i915_mcr_reg_t reg, u32 value);
void intel_gt_mcr_multicast_write_fw(struct intel_gt *gt,
- i915_reg_t reg, u32 value);
+ i915_mcr_reg_t reg, u32 value);
+
+u32 intel_gt_mcr_multicast_rmw(struct intel_gt *gt, i915_mcr_reg_t reg,
+ u32 clear, u32 set);
void intel_gt_mcr_get_nonterminated_steering(struct intel_gt *gt,
- i915_reg_t reg,
+ i915_mcr_reg_t reg,
u8 *group, u8 *instance);
void intel_gt_mcr_report_steering(struct drm_printer *p, struct intel_gt *gt,
@@ -34,6 +37,13 @@ void intel_gt_mcr_report_steering(struct drm_printer *p, struct intel_gt *gt,
void intel_gt_mcr_get_ss_steering(struct intel_gt *gt, unsigned int dss,
unsigned int *group, unsigned int *instance);
+int intel_gt_mcr_wait_for_reg(struct intel_gt *gt,
+ i915_mcr_reg_t reg,
+ u32 mask,
+ u32 value,
+ unsigned int fast_timeout_us,
+ unsigned int slow_timeout_ms);
+
/*
* Helper for for_each_ss_steering loop. On pre-Xe_HP platforms, subslice
* presence is determined by using the group/instance as direct lookups in the
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.c b/drivers/gpu/drm/i915/gt/intel_gt_pm.c
index f553e21..16db85f 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.c
@@ -7,6 +7,7 @@
#include <linux/suspend.h>
#include "i915_drv.h"
+#include "i915_irq.h"
#include "i915_params.h"
#include "intel_context.h"
#include "intel_engine_pm.h"
@@ -19,10 +20,31 @@
#include "intel_rc6.h"
#include "intel_rps.h"
#include "intel_wakeref.h"
+#include "intel_pcode.h"
#include "pxp/intel_pxp_pm.h"
#define I915_GT_SUSPEND_IDLE_TIMEOUT (HZ / 2)
+static void mtl_media_busy(struct intel_gt *gt)
+{
+ /* Wa_14017073508: mtl */
+ if (IS_MTL_GRAPHICS_STEP(gt->i915, P, STEP_A0, STEP_B0) &&
+ gt->type == GT_MEDIA)
+ snb_pcode_write_p(gt->uncore, PCODE_MBOX_GT_STATE,
+ PCODE_MBOX_GT_STATE_MEDIA_BUSY,
+ PCODE_MBOX_GT_STATE_DOMAIN_MEDIA, 0);
+}
+
+static void mtl_media_idle(struct intel_gt *gt)
+{
+ /* Wa_14017073508: mtl */
+ if (IS_MTL_GRAPHICS_STEP(gt->i915, P, STEP_A0, STEP_B0) &&
+ gt->type == GT_MEDIA)
+ snb_pcode_write_p(gt->uncore, PCODE_MBOX_GT_STATE,
+ PCODE_MBOX_GT_STATE_MEDIA_NOT_BUSY,
+ PCODE_MBOX_GT_STATE_DOMAIN_MEDIA, 0);
+}
+
static void user_forcewake(struct intel_gt *gt, bool suspend)
{
int count = atomic_read(>->user_wakeref);
@@ -70,6 +92,9 @@ static int __gt_unpark(struct intel_wakeref *wf)
GT_TRACE(gt, "\n");
+ /* Wa_14017073508: mtl */
+ mtl_media_busy(gt);
+
/*
* It seems that the DMC likes to transition between the DC states a lot
* when there are no connected displays (no active power domains) during
@@ -119,6 +144,9 @@ static int __gt_park(struct intel_wakeref *wf)
GEM_BUG_ON(!wakeref);
intel_display_power_put_async(i915, POWER_DOMAIN_GT_IRQ, wakeref);
+ /* Wa_14017073508: mtl */
+ mtl_media_idle(gt);
+
return 0;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.c b/drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.c
index 108b9e7..83df4cd 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.c
@@ -83,19 +83,6 @@ static int fw_domains_show(struct seq_file *m, void *data)
}
DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE(fw_domains);
-static void print_rc6_res(struct seq_file *m,
- const char *title,
- const i915_reg_t reg)
-{
- struct intel_gt *gt = m->private;
- intel_wakeref_t wakeref;
-
- with_intel_runtime_pm(gt->uncore->rpm, wakeref)
- seq_printf(m, "%s %u (%llu us)\n", title,
- intel_uncore_read(gt->uncore, reg),
- intel_rc6_residency_us(>->rc6, reg));
-}
-
static int vlv_drpc(struct seq_file *m)
{
struct intel_gt *gt = m->private;
@@ -115,8 +102,8 @@ static int vlv_drpc(struct seq_file *m)
seq_printf(m, "Media Power Well: %s\n",
(pw_status & VLV_GTLC_PW_MEDIA_STATUS_MASK) ? "Up" : "Down");
- print_rc6_res(m, "Render RC6 residency since boot:", GEN6_GT_GFX_RC6);
- print_rc6_res(m, "Media RC6 residency since boot:", VLV_GT_MEDIA_RC6);
+ intel_rc6_print_residency(m, "Render RC6 residency since boot:", INTEL_RC6_RES_RC6);
+ intel_rc6_print_residency(m, "Media RC6 residency since boot:", INTEL_RC6_RES_VLV_MEDIA);
return fw_domains_show(m, NULL);
}
@@ -192,11 +179,11 @@ static int gen6_drpc(struct seq_file *m)
}
/* Not exactly sure what this is */
- print_rc6_res(m, "RC6 \"Locked to RPn\" residency since boot:",
- GEN6_GT_GFX_RC6_LOCKED);
- print_rc6_res(m, "RC6 residency since boot:", GEN6_GT_GFX_RC6);
- print_rc6_res(m, "RC6+ residency since boot:", GEN6_GT_GFX_RC6p);
- print_rc6_res(m, "RC6++ residency since boot:", GEN6_GT_GFX_RC6pp);
+ intel_rc6_print_residency(m, "RC6 \"Locked to RPn\" residency since boot:",
+ INTEL_RC6_RES_RC6_LOCKED);
+ intel_rc6_print_residency(m, "RC6 residency since boot:", INTEL_RC6_RES_RC6);
+ intel_rc6_print_residency(m, "RC6+ residency since boot:", INTEL_RC6_RES_RC6p);
+ intel_rc6_print_residency(m, "RC6++ residency since boot:", INTEL_RC6_RES_RC6pp);
if (GRAPHICS_VER(i915) <= 7) {
seq_printf(m, "RC6 voltage: %dmV\n",
@@ -269,6 +256,61 @@ static int ilk_drpc(struct seq_file *m)
return 0;
}
+static int mtl_drpc(struct seq_file *m)
+{
+ struct intel_gt *gt = m->private;
+ struct intel_uncore *uncore = gt->uncore;
+ u32 gt_core_status, rcctl1, mt_fwake_req;
+ u32 mtl_powergate_enable = 0, mtl_powergate_status = 0;
+
+ mt_fwake_req = intel_uncore_read_fw(uncore, FORCEWAKE_MT);
+ gt_core_status = intel_uncore_read(uncore, MTL_MIRROR_TARGET_WP1);
+
+ rcctl1 = intel_uncore_read(uncore, GEN6_RC_CONTROL);
+ mtl_powergate_enable = intel_uncore_read(uncore, GEN9_PG_ENABLE);
+ mtl_powergate_status = intel_uncore_read(uncore,
+ GEN9_PWRGT_DOMAIN_STATUS);
+
+ seq_printf(m, "RC6 Enabled: %s\n",
+ str_yes_no(rcctl1 & GEN6_RC_CTL_RC6_ENABLE));
+ if (gt->type == GT_MEDIA) {
+ seq_printf(m, "Media Well Gating Enabled: %s\n",
+ str_yes_no(mtl_powergate_enable & GEN9_MEDIA_PG_ENABLE));
+ } else {
+ seq_printf(m, "Render Well Gating Enabled: %s\n",
+ str_yes_no(mtl_powergate_enable & GEN9_RENDER_PG_ENABLE));
+ }
+
+ seq_puts(m, "Current RC state: ");
+ switch (REG_FIELD_GET(MTL_CC_MASK, gt_core_status)) {
+ case MTL_CC0:
+ seq_puts(m, "RC0\n");
+ break;
+ case MTL_CC6:
+ seq_puts(m, "RC6\n");
+ break;
+ default:
+ MISSING_CASE(REG_FIELD_GET(MTL_CC_MASK, gt_core_status));
+ seq_puts(m, "Unknown\n");
+ break;
+ }
+
+ seq_printf(m, "Multi-threaded Forcewake Request: 0x%x\n", mt_fwake_req);
+ if (gt->type == GT_MEDIA)
+ seq_printf(m, "Media Power Well: %s\n",
+ (mtl_powergate_status &
+ GEN9_PWRGT_MEDIA_STATUS_MASK) ? "Up" : "Down");
+ else
+ seq_printf(m, "Render Power Well: %s\n",
+ (mtl_powergate_status &
+ GEN9_PWRGT_RENDER_STATUS_MASK) ? "Up" : "Down");
+
+ /* Works for both render and media gt's */
+ intel_rc6_print_residency(m, "RC6 residency since boot:", INTEL_RC6_RES_RC6);
+
+ return fw_domains_show(m, NULL);
+}
+
static int drpc_show(struct seq_file *m, void *unused)
{
struct intel_gt *gt = m->private;
@@ -277,7 +319,9 @@ static int drpc_show(struct seq_file *m, void *unused)
int err = -ENODEV;
with_intel_runtime_pm(gt->uncore->rpm, wakeref) {
- if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915))
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70))
+ err = mtl_drpc(m);
+ else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915))
err = vlv_drpc(m);
else if (GRAPHICS_VER(i915) >= 6)
err = gen6_drpc(m);
@@ -307,7 +351,7 @@ void intel_gt_pm_frequency_dump(struct intel_gt *gt, struct drm_printer *p)
drm_printf(p, "Current VID: %d\n", (rgvstat & MEMSTAT_VID_MASK) >>
MEMSTAT_VID_SHIFT);
drm_printf(p, "Current P-state: %d\n",
- (rgvstat & MEMSTAT_PSTATE_MASK) >> MEMSTAT_PSTATE_SHIFT);
+ REG_FIELD_GET(MEMSTAT_PSTATE_MASK, rgvstat));
} else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) {
u32 rpmodectl, freq_sts;
@@ -344,162 +388,7 @@ void intel_gt_pm_frequency_dump(struct intel_gt *gt, struct drm_printer *p)
drm_printf(p, "efficient (RPe) frequency: %d MHz\n",
intel_gpu_freq(rps, rps->efficient_freq));
} else if (GRAPHICS_VER(i915) >= 6) {
- u32 rp_state_limits;
- u32 gt_perf_status;
- struct intel_rps_freq_caps caps;
- u32 rpmodectl, rpinclimit, rpdeclimit;
- u32 rpstat, cagf, reqf;
- u32 rpcurupei, rpcurup, rpprevup;
- u32 rpcurdownei, rpcurdown, rpprevdown;
- u32 rpupei, rpupt, rpdownei, rpdownt;
- u32 pm_ier, pm_imr, pm_isr, pm_iir, pm_mask;
-
- rp_state_limits = intel_uncore_read(uncore, GEN6_RP_STATE_LIMITS);
- gen6_rps_get_freq_caps(rps, &caps);
- if (IS_GEN9_LP(i915))
- gt_perf_status = intel_uncore_read(uncore, BXT_GT_PERF_STATUS);
- else
- gt_perf_status = intel_uncore_read(uncore, GEN6_GT_PERF_STATUS);
-
- /* RPSTAT1 is in the GT power well */
- intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL);
-
- reqf = intel_uncore_read(uncore, GEN6_RPNSWREQ);
- if (GRAPHICS_VER(i915) >= 9) {
- reqf >>= 23;
- } else {
- reqf &= ~GEN6_TURBO_DISABLE;
- if (IS_HASWELL(i915) || IS_BROADWELL(i915))
- reqf >>= 24;
- else
- reqf >>= 25;
- }
- reqf = intel_gpu_freq(rps, reqf);
-
- rpmodectl = intel_uncore_read(uncore, GEN6_RP_CONTROL);
- rpinclimit = intel_uncore_read(uncore, GEN6_RP_UP_THRESHOLD);
- rpdeclimit = intel_uncore_read(uncore, GEN6_RP_DOWN_THRESHOLD);
-
- rpstat = intel_uncore_read(uncore, GEN6_RPSTAT1);
- rpcurupei = intel_uncore_read(uncore, GEN6_RP_CUR_UP_EI) & GEN6_CURICONT_MASK;
- rpcurup = intel_uncore_read(uncore, GEN6_RP_CUR_UP) & GEN6_CURBSYTAVG_MASK;
- rpprevup = intel_uncore_read(uncore, GEN6_RP_PREV_UP) & GEN6_CURBSYTAVG_MASK;
- rpcurdownei = intel_uncore_read(uncore, GEN6_RP_CUR_DOWN_EI) & GEN6_CURIAVG_MASK;
- rpcurdown = intel_uncore_read(uncore, GEN6_RP_CUR_DOWN) & GEN6_CURBSYTAVG_MASK;
- rpprevdown = intel_uncore_read(uncore, GEN6_RP_PREV_DOWN) & GEN6_CURBSYTAVG_MASK;
-
- rpupei = intel_uncore_read(uncore, GEN6_RP_UP_EI);
- rpupt = intel_uncore_read(uncore, GEN6_RP_UP_THRESHOLD);
-
- rpdownei = intel_uncore_read(uncore, GEN6_RP_DOWN_EI);
- rpdownt = intel_uncore_read(uncore, GEN6_RP_DOWN_THRESHOLD);
-
- cagf = intel_rps_read_actual_frequency(rps);
-
- intel_uncore_forcewake_put(uncore, FORCEWAKE_ALL);
-
- if (GRAPHICS_VER(i915) >= 11) {
- pm_ier = intel_uncore_read(uncore, GEN11_GPM_WGBOXPERF_INTR_ENABLE);
- pm_imr = intel_uncore_read(uncore, GEN11_GPM_WGBOXPERF_INTR_MASK);
- /*
- * The equivalent to the PM ISR & IIR cannot be read
- * without affecting the current state of the system
- */
- pm_isr = 0;
- pm_iir = 0;
- } else if (GRAPHICS_VER(i915) >= 8) {
- pm_ier = intel_uncore_read(uncore, GEN8_GT_IER(2));
- pm_imr = intel_uncore_read(uncore, GEN8_GT_IMR(2));
- pm_isr = intel_uncore_read(uncore, GEN8_GT_ISR(2));
- pm_iir = intel_uncore_read(uncore, GEN8_GT_IIR(2));
- } else {
- pm_ier = intel_uncore_read(uncore, GEN6_PMIER);
- pm_imr = intel_uncore_read(uncore, GEN6_PMIMR);
- pm_isr = intel_uncore_read(uncore, GEN6_PMISR);
- pm_iir = intel_uncore_read(uncore, GEN6_PMIIR);
- }
- pm_mask = intel_uncore_read(uncore, GEN6_PMINTRMSK);
-
- drm_printf(p, "Video Turbo Mode: %s\n",
- str_yes_no(rpmodectl & GEN6_RP_MEDIA_TURBO));
- drm_printf(p, "HW control enabled: %s\n",
- str_yes_no(rpmodectl & GEN6_RP_ENABLE));
- drm_printf(p, "SW control enabled: %s\n",
- str_yes_no((rpmodectl & GEN6_RP_MEDIA_MODE_MASK) == GEN6_RP_MEDIA_SW_MODE));
-
- drm_printf(p, "PM IER=0x%08x IMR=0x%08x, MASK=0x%08x\n",
- pm_ier, pm_imr, pm_mask);
- if (GRAPHICS_VER(i915) <= 10)
- drm_printf(p, "PM ISR=0x%08x IIR=0x%08x\n",
- pm_isr, pm_iir);
- drm_printf(p, "pm_intrmsk_mbz: 0x%08x\n",
- rps->pm_intrmsk_mbz);
- drm_printf(p, "GT_PERF_STATUS: 0x%08x\n", gt_perf_status);
- drm_printf(p, "Render p-state ratio: %d\n",
- (gt_perf_status & (GRAPHICS_VER(i915) >= 9 ? 0x1ff00 : 0xff00)) >> 8);
- drm_printf(p, "Render p-state VID: %d\n",
- gt_perf_status & 0xff);
- drm_printf(p, "Render p-state limit: %d\n",
- rp_state_limits & 0xff);
- drm_printf(p, "RPSTAT1: 0x%08x\n", rpstat);
- drm_printf(p, "RPMODECTL: 0x%08x\n", rpmodectl);
- drm_printf(p, "RPINCLIMIT: 0x%08x\n", rpinclimit);
- drm_printf(p, "RPDECLIMIT: 0x%08x\n", rpdeclimit);
- drm_printf(p, "RPNSWREQ: %dMHz\n", reqf);
- drm_printf(p, "CAGF: %dMHz\n", cagf);
- drm_printf(p, "RP CUR UP EI: %d (%lldns)\n",
- rpcurupei,
- intel_gt_pm_interval_to_ns(gt, rpcurupei));
- drm_printf(p, "RP CUR UP: %d (%lldns)\n",
- rpcurup, intel_gt_pm_interval_to_ns(gt, rpcurup));
- drm_printf(p, "RP PREV UP: %d (%lldns)\n",
- rpprevup, intel_gt_pm_interval_to_ns(gt, rpprevup));
- drm_printf(p, "Up threshold: %d%%\n",
- rps->power.up_threshold);
- drm_printf(p, "RP UP EI: %d (%lldns)\n",
- rpupei, intel_gt_pm_interval_to_ns(gt, rpupei));
- drm_printf(p, "RP UP THRESHOLD: %d (%lldns)\n",
- rpupt, intel_gt_pm_interval_to_ns(gt, rpupt));
-
- drm_printf(p, "RP CUR DOWN EI: %d (%lldns)\n",
- rpcurdownei,
- intel_gt_pm_interval_to_ns(gt, rpcurdownei));
- drm_printf(p, "RP CUR DOWN: %d (%lldns)\n",
- rpcurdown,
- intel_gt_pm_interval_to_ns(gt, rpcurdown));
- drm_printf(p, "RP PREV DOWN: %d (%lldns)\n",
- rpprevdown,
- intel_gt_pm_interval_to_ns(gt, rpprevdown));
- drm_printf(p, "Down threshold: %d%%\n",
- rps->power.down_threshold);
- drm_printf(p, "RP DOWN EI: %d (%lldns)\n",
- rpdownei, intel_gt_pm_interval_to_ns(gt, rpdownei));
- drm_printf(p, "RP DOWN THRESHOLD: %d (%lldns)\n",
- rpdownt, intel_gt_pm_interval_to_ns(gt, rpdownt));
-
- drm_printf(p, "Lowest (RPN) frequency: %dMHz\n",
- intel_gpu_freq(rps, caps.min_freq));
- drm_printf(p, "Nominal (RP1) frequency: %dMHz\n",
- intel_gpu_freq(rps, caps.rp1_freq));
- drm_printf(p, "Max non-overclocked (RP0) frequency: %dMHz\n",
- intel_gpu_freq(rps, caps.rp0_freq));
- drm_printf(p, "Max overclocked frequency: %dMHz\n",
- intel_gpu_freq(rps, rps->max_freq));
-
- drm_printf(p, "Current freq: %d MHz\n",
- intel_gpu_freq(rps, rps->cur_freq));
- drm_printf(p, "Actual freq: %d MHz\n", cagf);
- drm_printf(p, "Idle freq: %d MHz\n",
- intel_gpu_freq(rps, rps->idle_freq));
- drm_printf(p, "Min freq: %d MHz\n",
- intel_gpu_freq(rps, rps->min_freq));
- drm_printf(p, "Boost freq: %d MHz\n",
- intel_gpu_freq(rps, rps->boost_freq));
- drm_printf(p, "Max freq: %d MHz\n",
- intel_gpu_freq(rps, rps->max_freq));
- drm_printf(p,
- "efficient (RPe) frequency: %d MHz\n",
- intel_gpu_freq(rps, rps->efficient_freq));
+ gen6_rps_frequency_dump(rps, p);
} else {
drm_puts(p, "no P-state info available\n");
}
@@ -655,6 +544,44 @@ static bool rps_eval(void *data)
DEFINE_INTEL_GT_DEBUGFS_ATTRIBUTE(rps_boost);
+static int perf_limit_reasons_get(void *data, u64 *val)
+{
+ struct intel_gt *gt = data;
+ intel_wakeref_t wakeref;
+
+ with_intel_runtime_pm(gt->uncore->rpm, wakeref)
+ *val = intel_uncore_read(gt->uncore, intel_gt_perf_limit_reasons_reg(gt));
+
+ return 0;
+}
+
+static int perf_limit_reasons_clear(void *data, u64 val)
+{
+ struct intel_gt *gt = data;
+ intel_wakeref_t wakeref;
+
+ /*
+ * Clear the upper 16 "log" bits, the lower 16 "status" bits are
+ * read-only. The upper 16 "log" bits are identical to the lower 16
+ * "status" bits except that the "log" bits remain set until cleared.
+ */
+ with_intel_runtime_pm(gt->uncore->rpm, wakeref)
+ intel_uncore_rmw(gt->uncore, intel_gt_perf_limit_reasons_reg(gt),
+ GT0_PERF_LIMIT_REASONS_LOG_MASK, 0);
+
+ return 0;
+}
+
+static bool perf_limit_reasons_eval(void *data)
+{
+ struct intel_gt *gt = data;
+
+ return i915_mmio_reg_valid(intel_gt_perf_limit_reasons_reg(gt));
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(perf_limit_reasons_fops, perf_limit_reasons_get,
+ perf_limit_reasons_clear, "%llu\n");
+
void intel_gt_pm_debugfs_register(struct intel_gt *gt, struct dentry *root)
{
static const struct intel_gt_debugfs_file files[] = {
@@ -664,6 +591,7 @@ void intel_gt_pm_debugfs_register(struct intel_gt *gt, struct dentry *root)
{ "forcewake_user", &forcewake_user_fops, NULL},
{ "llc", &llc_fops, llc_eval },
{ "rps_boost", &rps_boost_fops, rps_eval },
+ { "perf_limit_reasons", &perf_limit_reasons_fops, perf_limit_reasons_eval },
};
intel_gt_debugfs_register_files(root, files, ARRAY_SIZE(files), gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
index 2275ee4..c3cd926 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
@@ -7,6 +7,27 @@
#define __INTEL_GT_REGS__
#include "i915_reg_defs.h"
+#include "display/intel_display_reg_defs.h" /* VLV_DISPLAY_BASE */
+
+#define MCR_REG(offset) ((const i915_mcr_reg_t){ .reg = (offset) })
+
+/*
+ * The perf control registers are technically multicast registers, but the
+ * driver never needs to read/write them directly; we only use them to build
+ * lists of registers (where they're mixed in with other non-MCR registers)
+ * and then operate on the offset directly. For now we'll just define them
+ * as non-multicast so we can place them on the same list, but we may want
+ * to try to come up with a better way to handle heterogeneous lists of
+ * registers in the future.
+ */
+#define PERF_REG(offset) _MMIO(offset)
+
+/* MTL workpoint reg to get core C state and actual freq of 3D, SAMedia */
+#define MTL_MIRROR_TARGET_WP1 _MMIO(0xc60)
+#define MTL_CAGF_MASK REG_GENMASK(8, 0)
+#define MTL_CC0 0x0
+#define MTL_CC6 0x3
+#define MTL_CC_MASK REG_GENMASK(12, 9)
/* RPM unit config (Gen8+) */
#define RPM_CONFIG0 _MMIO(0xd00)
@@ -39,9 +60,17 @@
#define FORCEWAKE_ACK_RENDER_GEN9 _MMIO(0xd84)
#define FORCEWAKE_ACK_MEDIA_GEN9 _MMIO(0xd88)
+#define FORCEWAKE_ACK_GSC _MMIO(0xdf8)
+#define FORCEWAKE_ACK_GT_MTL _MMIO(0xdfc)
+
+#define GMD_ID_GRAPHICS _MMIO(0xd8c)
+#define GMD_ID_MEDIA _MMIO(MTL_MEDIA_GSI_BASE + 0xd8c)
+
#define MCFG_MCR_SELECTOR _MMIO(0xfd0)
+#define MTL_MCR_SELECTOR _MMIO(0xfd4)
#define SF_MCR_SELECTOR _MMIO(0xfd8)
#define GEN8_MCR_SELECTOR _MMIO(0xfdc)
+#define GAM_MCR_SELECTOR _MMIO(0xfe0)
#define GEN8_MCR_SLICE(slice) (((slice) & 3) << 26)
#define GEN8_MCR_SLICE_MASK GEN8_MCR_SLICE(3)
#define GEN8_MCR_SUBSLICE(subslice) (((subslice) & 3) << 24)
@@ -51,6 +80,8 @@
#define GEN11_MCR_SLICE_MASK GEN11_MCR_SLICE(0xf)
#define GEN11_MCR_SUBSLICE(subslice) (((subslice) & 0x7) << 24)
#define GEN11_MCR_SUBSLICE_MASK GEN11_MCR_SUBSLICE(0x7)
+#define MTL_MCR_GROUPID REG_GENMASK(11, 8)
+#define MTL_MCR_INSTANCEID REG_GENMASK(3, 0)
#define IPEIR_I965 _MMIO(0x2064)
#define IPEHR_I965 _MMIO(0x2068)
@@ -326,11 +357,12 @@
#define GEN7_TLB_RD_ADDR _MMIO(0x4700)
#define GEN12_PAT_INDEX(index) _MMIO(0x4800 + (index) * 4)
+#define XEHP_PAT_INDEX(index) MCR_REG(0x4800 + (index) * 4)
-#define XEHP_TILE0_ADDR_RANGE _MMIO(0x4900)
+#define XEHP_TILE0_ADDR_RANGE MCR_REG(0x4900)
#define XEHP_TILE_LMEM_RANGE_SHIFT 8
-#define XEHP_FLAT_CCS_BASE_ADDR _MMIO(0x4910)
+#define XEHP_FLAT_CCS_BASE_ADDR MCR_REG(0x4910)
#define XEHP_CCS_BASE_SHIFT 8
#define GAMTARBMODE _MMIO(0x4a08)
@@ -380,17 +412,18 @@
#define CHICKEN_RASTER_2 _MMIO(0x6208)
#define TBIMR_FAST_CLIP REG_BIT(5)
-#define VFLSKPD _MMIO(0x62a8)
+#define VFLSKPD MCR_REG(0x62a8)
#define DIS_OVER_FETCH_CACHE REG_BIT(1)
#define DIS_MULT_MISS_RD_SQUASH REG_BIT(0)
-#define FF_MODE2 _MMIO(0x6604)
+#define GEN12_FF_MODE2 _MMIO(0x6604)
+#define XEHP_FF_MODE2 MCR_REG(0x6604)
#define FF_MODE2_GS_TIMER_MASK REG_GENMASK(31, 24)
#define FF_MODE2_GS_TIMER_224 REG_FIELD_PREP(FF_MODE2_GS_TIMER_MASK, 224)
#define FF_MODE2_TDS_TIMER_MASK REG_GENMASK(23, 16)
#define FF_MODE2_TDS_TIMER_128 REG_FIELD_PREP(FF_MODE2_TDS_TIMER_MASK, 4)
-#define XEHPG_INSTDONE_GEOM_SVG _MMIO(0x666c)
+#define XEHPG_INSTDONE_GEOM_SVG MCR_REG(0x666c)
#define CACHE_MODE_0_GEN7 _MMIO(0x7000) /* IVB+ */
#define RC_OP_FLUSH_ENABLE (1 << 0)
@@ -418,6 +451,7 @@
#define HIZ_CHICKEN _MMIO(0x7018)
#define CHV_HZ_8X8_MODE_IN_1X REG_BIT(15)
#define DG1_HZ_READ_SUPPRESSION_OPTIMIZATION_DISABLE REG_BIT(14)
+#define HZ_DEPTH_TEST_LE_GE_OPT_DISABLE REG_BIT(13)
#define BDW_HIZ_POWER_COMPILER_CLOCK_GATING_DISABLE REG_BIT(3)
#define GEN8_L3CNTLREG _MMIO(0x7034)
@@ -439,23 +473,16 @@
#define GEN8_HDC_CHICKEN1 _MMIO(0x7304)
#define GEN11_COMMON_SLICE_CHICKEN3 _MMIO(0x7304)
+#define XEHP_COMMON_SLICE_CHICKEN3 MCR_REG(0x7304)
#define DG1_FLOAT_POINT_BLEND_OPT_STRICT_MODE_EN REG_BIT(12)
#define XEHP_DUAL_SIMD8_SEQ_MERGE_DISABLE REG_BIT(12)
#define GEN11_BLEND_EMB_FIX_DISABLE_IN_RCC REG_BIT(11)
#define GEN12_DISABLE_CPS_AWARE_COLOR_PIPE REG_BIT(9)
-/* GEN9 chicken */
-#define SLICE_ECO_CHICKEN0 _MMIO(0x7308)
-#define PIXEL_MASK_CAMMING_DISABLE (1 << 14)
-
-#define GEN9_SLICE_COMMON_ECO_CHICKEN0 _MMIO(0x7308)
-#define DISABLE_PIXEL_MASK_CAMMING (1 << 14)
-
#define GEN9_SLICE_COMMON_ECO_CHICKEN1 _MMIO(0x731c)
-#define GEN11_STATE_CACHE_REDIRECT_TO_CS (1 << 11)
-
-#define SLICE_COMMON_ECO_CHICKEN1 _MMIO(0x731c)
+#define XEHP_SLICE_COMMON_ECO_CHICKEN1 MCR_REG(0x731c)
#define MSC_MSAA_REODER_BUF_BYPASS_DISABLE REG_BIT(14)
+#define GEN11_STATE_CACHE_REDIRECT_TO_CS (1 << 11)
#define GEN9_SLICE_PGCTL_ACK(slice) _MMIO(0x804c + (slice) * 0x4)
#define GEN10_SLICE_PGCTL_ACK(slice) _MMIO(0x804c + ((slice) / 3) * 0x34 + \
@@ -482,9 +509,12 @@
#define VF_PREEMPTION _MMIO(0x83a4)
#define PREEMPTION_VERTEX_COUNT REG_GENMASK(15, 0)
+#define VFG_PREEMPTION_CHICKEN _MMIO(0x83b4)
+#define POLYGON_TRIFAN_LINELOOP_DISABLE REG_BIT(4)
+
#define GEN8_RC6_CTX_INFO _MMIO(0x8504)
-#define GEN12_SQCM _MMIO(0x8724)
+#define XEHP_SQCM MCR_REG(0x8724)
#define EN_32B_ACCESS REG_BIT(30)
#define HSW_IDICR _MMIO(0x9008)
@@ -516,6 +546,8 @@
#define GEN6_MBCTL_BOOT_FETCH_MECH (1 << 0)
/* Fuse readout registers for GT */
+#define XEHP_FUSE4 _MMIO(0x9114)
+#define GT_L3_EXC_MASK REG_GENMASK(6, 4)
#define GEN10_MIRROR_FUSE3 _MMIO(0x9118)
#define GEN10_L3BANK_PAIR_COUNT 4
#define GEN10_L3BANK_MASK 0x0F
@@ -619,6 +651,7 @@
#define XEHPC_GRDOM_BLT3 REG_BIT(26)
#define XEHPC_GRDOM_BLT2 REG_BIT(25)
#define XEHPC_GRDOM_BLT1 REG_BIT(24)
+#define GEN12_GRDOM_GSC REG_BIT(21)
#define GEN11_GRDOM_SFC3 REG_BIT(20)
#define GEN11_GRDOM_SFC2 REG_BIT(19)
#define GEN11_GRDOM_SFC1 REG_BIT(18)
@@ -644,6 +677,9 @@
#define GEN7_MISCCPCTL _MMIO(0x9424)
#define GEN7_DOP_CLOCK_GATE_ENABLE (1 << 0)
+
+#define GEN8_MISCCPCTL MCR_REG(0x9424)
+#define GEN8_DOP_CLOCK_GATE_ENABLE REG_BIT(0)
#define GEN12_DOP_CLOCK_GATE_RENDER_ENABLE REG_BIT(1)
#define GEN8_DOP_CLOCK_GATE_CFCLK_ENABLE (1 << 2)
#define GEN8_DOP_CLOCK_GATE_GUC_ENABLE (1 << 4)
@@ -697,7 +733,8 @@
#define GAMTLBVEBOX0_CLKGATE_DIS REG_BIT(16)
#define LTCDD_CLKGATE_DIS REG_BIT(10)
-#define SLICE_UNIT_LEVEL_CLKGATE _MMIO(0x94d4)
+#define GEN11_SLICE_UNIT_LEVEL_CLKGATE _MMIO(0x94d4)
+#define XEHP_SLICE_UNIT_LEVEL_CLKGATE MCR_REG(0x94d4)
#define SARBUNIT_CLKGATE_DIS (1 << 5)
#define RCCUNIT_CLKGATE_DIS (1 << 7)
#define MSCUNIT_CLKGATE_DIS (1 << 10)
@@ -705,27 +742,27 @@
#define L3_CLKGATE_DIS REG_BIT(16)
#define L3_CR2X_CLKGATE_DIS REG_BIT(17)
-#define SCCGCTL94DC _MMIO(0x94dc)
+#define SCCGCTL94DC MCR_REG(0x94dc)
#define CG3DDISURB REG_BIT(14)
#define UNSLICE_UNIT_LEVEL_CLKGATE2 _MMIO(0x94e4)
#define VSUNIT_CLKGATE_DIS_TGL REG_BIT(19)
#define PSDUNIT_CLKGATE_DIS REG_BIT(5)
-#define SUBSLICE_UNIT_LEVEL_CLKGATE _MMIO(0x9524)
+#define GEN11_SUBSLICE_UNIT_LEVEL_CLKGATE MCR_REG(0x9524)
#define DSS_ROUTER_CLKGATE_DIS REG_BIT(28)
#define GWUNIT_CLKGATE_DIS REG_BIT(16)
-#define SUBSLICE_UNIT_LEVEL_CLKGATE2 _MMIO(0x9528)
+#define SUBSLICE_UNIT_LEVEL_CLKGATE2 MCR_REG(0x9528)
#define CPSSUNIT_CLKGATE_DIS REG_BIT(9)
-#define SSMCGCTL9530 _MMIO(0x9530)
+#define SSMCGCTL9530 MCR_REG(0x9530)
#define RTFUNIT_CLKGATE_DIS REG_BIT(18)
-#define GEN10_DFR_RATIO_EN_AND_CHICKEN _MMIO(0x9550)
+#define GEN10_DFR_RATIO_EN_AND_CHICKEN MCR_REG(0x9550)
#define DFR_DISABLE (1 << 9)
-#define INF_UNIT_LEVEL_CLKGATE _MMIO(0x9560)
+#define INF_UNIT_LEVEL_CLKGATE MCR_REG(0x9560)
#define CGPSF_CLKGATE_DIS (1 << 3)
#define MICRO_BP0_0 _MMIO(0x9800)
@@ -770,12 +807,9 @@
#define GEN6_RP_DOWN_TIMEOUT _MMIO(0xa010)
#define GEN6_RP_INTERRUPT_LIMITS _MMIO(0xa014)
#define GEN6_RPSTAT1 _MMIO(0xa01c)
-#define GEN6_CAGF_SHIFT 8
-#define HSW_CAGF_SHIFT 7
-#define GEN9_CAGF_SHIFT 23
-#define GEN6_CAGF_MASK (0x7f << GEN6_CAGF_SHIFT)
-#define HSW_CAGF_MASK (0x7f << HSW_CAGF_SHIFT)
-#define GEN9_CAGF_MASK (0x1ff << GEN9_CAGF_SHIFT)
+#define GEN6_CAGF_MASK REG_GENMASK(14, 8)
+#define HSW_CAGF_MASK REG_GENMASK(13, 7)
+#define GEN9_CAGF_MASK REG_GENMASK(31, 23)
#define GEN6_RP_CONTROL _MMIO(0xa024)
#define GEN6_RP_MEDIA_TURBO (1 << 11)
#define GEN6_RP_MEDIA_MODE_MASK (3 << 9)
@@ -898,6 +932,8 @@
#define FORCEWAKE_MEDIA_VDBOX_GEN11(n) _MMIO(0xa540 + (n) * 4)
#define FORCEWAKE_MEDIA_VEBOX_GEN11(n) _MMIO(0xa560 + (n) * 4)
+#define FORCEWAKE_REQ_GSC _MMIO(0xa618)
+
#define CHV_POWER_SS0_SIG1 _MMIO(0xa720)
#define CHV_POWER_SS0_SIG2 _MMIO(0xa724)
#define CHV_POWER_SS1_SIG1 _MMIO(0xa728)
@@ -935,7 +971,8 @@
/* MOCS (Memory Object Control State) registers */
#define GEN9_LNCFCMOCS(i) _MMIO(0xb020 + (i) * 4) /* L3 Cache Control */
-#define GEN9_LNCFCMOCS_REG_COUNT 32
+#define XEHP_LNCFCMOCS(i) MCR_REG(0xb020 + (i) * 4)
+#define LNCFCMOCS_REG_COUNT 32
#define GEN7_L3CNTLREG3 _MMIO(0xb024)
@@ -951,15 +988,10 @@
#define GEN7_L3LOG(slice, i) _MMIO(0xb070 + (slice) * 0x200 + (i) * 4)
#define GEN7_L3LOG_SIZE 0x80
-#define GEN10_SCRATCH_LNCF2 _MMIO(0xb0a0)
-#define PMFLUSHDONE_LNICRSDROP (1 << 20)
-#define PMFLUSH_GAPL3UNBLOCK (1 << 21)
-#define PMFLUSHDONE_LNEBLK (1 << 22)
-
-#define XEHP_L3NODEARBCFG _MMIO(0xb0b4)
+#define XEHP_L3NODEARBCFG MCR_REG(0xb0b4)
#define XEHP_LNESPARE REG_BIT(19)
-#define GEN8_L3SQCREG1 _MMIO(0xb100)
+#define GEN8_L3SQCREG1 MCR_REG(0xb100)
/*
* Note that on CHV the following has an off-by-one error wrt. to BSpec.
* Using the formula in BSpec leads to a hang, while the formula here works
@@ -970,31 +1002,28 @@
#define L3_HIGH_PRIO_CREDITS(x) (((x) >> 1) << 14)
#define L3_PRIO_CREDITS_MASK ((0x1f << 19) | (0x1f << 14))
-#define GEN10_L3_CHICKEN_MODE_REGISTER _MMIO(0xb114)
-#define GEN11_I2M_WRITE_DISABLE (1 << 28)
-
-#define GEN8_L3SQCREG4 _MMIO(0xb118)
+#define GEN8_L3SQCREG4 MCR_REG(0xb118)
#define GEN11_LQSC_CLEAN_EVICT_DISABLE (1 << 6)
#define GEN8_LQSC_RO_PERF_DIS (1 << 27)
#define GEN8_LQSC_FLUSH_COHERENT_LINES (1 << 21)
#define GEN8_LQSQ_NONIA_COHERENT_ATOMICS_ENABLE REG_BIT(22)
-#define GEN9_SCRATCH1 _MMIO(0xb11c)
+#define GEN9_SCRATCH1 MCR_REG(0xb11c)
#define EVICTION_PERF_FIX_ENABLE REG_BIT(8)
-#define BDW_SCRATCH1 _MMIO(0xb11c)
+#define BDW_SCRATCH1 MCR_REG(0xb11c)
#define GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE (1 << 2)
-#define GEN11_SCRATCH2 _MMIO(0xb140)
+#define GEN11_SCRATCH2 MCR_REG(0xb140)
#define GEN11_COHERENT_PARTIAL_WRITE_MERGE_ENABLE (1 << 19)
-#define GEN11_L3SQCREG5 _MMIO(0xb158)
+#define XEHP_L3SQCREG5 MCR_REG(0xb158)
#define L3_PWM_TIMER_INIT_VAL_MASK REG_GENMASK(9, 0)
-#define MLTICTXCTL _MMIO(0xb170)
+#define MLTICTXCTL MCR_REG(0xb170)
#define TDONRENDER REG_BIT(2)
-#define XEHP_L3SCQREG7 _MMIO(0xb188)
+#define XEHP_L3SCQREG7 MCR_REG(0xb188)
#define BLEND_FILL_CACHING_OPT_DIS REG_BIT(3)
#define XEHPC_L3SCRUB _MMIO(0xb18c)
@@ -1002,7 +1031,7 @@
#define SCRUB_RATE_PER_BANK_MASK REG_GENMASK(2, 0)
#define SCRUB_RATE_4B_PER_CLK REG_FIELD_PREP(SCRUB_RATE_PER_BANK_MASK, 0x6)
-#define L3SQCREG1_CCS0 _MMIO(0xb200)
+#define L3SQCREG1_CCS0 MCR_REG(0xb200)
#define FLUSHALLNONCOH REG_BIT(5)
#define GEN11_GLBLINVL _MMIO(0xb404)
@@ -1027,11 +1056,14 @@
#define GEN9_BLT_MOCS(i) _MMIO(__GEN9_BCS0_MOCS0 + (i) * 4)
#define GEN12_FAULT_TLB_DATA0 _MMIO(0xceb8)
+#define XEHP_FAULT_TLB_DATA0 MCR_REG(0xceb8)
#define GEN12_FAULT_TLB_DATA1 _MMIO(0xcebc)
+#define XEHP_FAULT_TLB_DATA1 MCR_REG(0xcebc)
#define FAULT_VA_HIGH_BITS (0xf << 0)
#define FAULT_GTT_SEL (1 << 4)
#define GEN12_RING_FAULT_REG _MMIO(0xcec4)
+#define XEHP_RING_FAULT_REG MCR_REG(0xcec4)
#define GEN8_RING_FAULT_ENGINE_ID(x) (((x) >> 12) & 0x7)
#define RING_FAULT_GTTSEL_MASK (1 << 11)
#define RING_FAULT_SRCID(x) (((x) >> 3) & 0xff)
@@ -1039,16 +1071,21 @@
#define RING_FAULT_VALID (1 << 0)
#define GEN12_GFX_TLB_INV_CR _MMIO(0xced8)
+#define XEHP_GFX_TLB_INV_CR MCR_REG(0xced8)
#define GEN12_VD_TLB_INV_CR _MMIO(0xcedc)
+#define XEHP_VD_TLB_INV_CR MCR_REG(0xcedc)
#define GEN12_VE_TLB_INV_CR _MMIO(0xcee0)
+#define XEHP_VE_TLB_INV_CR MCR_REG(0xcee0)
#define GEN12_BLT_TLB_INV_CR _MMIO(0xcee4)
+#define XEHP_BLT_TLB_INV_CR MCR_REG(0xcee4)
#define GEN12_COMPCTX_TLB_INV_CR _MMIO(0xcf04)
+#define XEHP_COMPCTX_TLB_INV_CR MCR_REG(0xcf04)
-#define GEN12_MERT_MOD_CTRL _MMIO(0xcf28)
-#define RENDER_MOD_CTRL _MMIO(0xcf2c)
-#define COMP_MOD_CTRL _MMIO(0xcf30)
-#define VDBX_MOD_CTRL _MMIO(0xcf34)
-#define VEBX_MOD_CTRL _MMIO(0xcf38)
+#define XEHP_MERT_MOD_CTRL MCR_REG(0xcf28)
+#define RENDER_MOD_CTRL MCR_REG(0xcf2c)
+#define COMP_MOD_CTRL MCR_REG(0xcf30)
+#define VDBX_MOD_CTRL MCR_REG(0xcf34)
+#define VEBX_MOD_CTRL MCR_REG(0xcf38)
#define FORCE_MISS_FTLB REG_BIT(3)
#define GEN12_GAMSTLB_CTRL _MMIO(0xcf4c)
@@ -1063,48 +1100,52 @@
#define GEN12_GAM_DONE _MMIO(0xcf68)
#define GEN7_HALF_SLICE_CHICKEN1 _MMIO(0xe100) /* IVB GT1 + VLV */
+#define GEN8_HALF_SLICE_CHICKEN1 MCR_REG(0xe100)
#define GEN7_MAX_PS_THREAD_DEP (8 << 12)
#define GEN7_SINGLE_SUBSCAN_DISPATCH_ENABLE (1 << 10)
#define GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE (1 << 4)
#define GEN7_PSD_SINGLE_PORT_DISPATCH_ENABLE (1 << 3)
#define GEN7_SAMPLER_INSTDONE _MMIO(0xe160)
+#define GEN8_SAMPLER_INSTDONE MCR_REG(0xe160)
#define GEN7_ROW_INSTDONE _MMIO(0xe164)
+#define GEN8_ROW_INSTDONE MCR_REG(0xe164)
-#define HALF_SLICE_CHICKEN2 _MMIO(0xe180)
+#define HALF_SLICE_CHICKEN2 MCR_REG(0xe180)
#define GEN8_ST_PO_DISABLE (1 << 13)
-#define HALF_SLICE_CHICKEN3 _MMIO(0xe184)
+#define HSW_HALF_SLICE_CHICKEN3 _MMIO(0xe184)
+#define GEN8_HALF_SLICE_CHICKEN3 MCR_REG(0xe184)
#define HSW_SAMPLE_C_PERFORMANCE (1 << 9)
#define GEN8_CENTROID_PIXEL_OPT_DIS (1 << 8)
#define GEN9_DISABLE_OCL_OOB_SUPPRESS_LOGIC (1 << 5)
#define GEN8_SAMPLER_POWER_BYPASS_DIS (1 << 1)
-#define GEN9_HALF_SLICE_CHICKEN5 _MMIO(0xe188)
+#define GEN9_HALF_SLICE_CHICKEN5 MCR_REG(0xe188)
#define GEN9_DG_MIRROR_FIX_ENABLE (1 << 5)
#define GEN9_CCS_TLB_PREFETCH_ENABLE (1 << 3)
-#define GEN10_SAMPLER_MODE _MMIO(0xe18c)
+#define GEN10_SAMPLER_MODE MCR_REG(0xe18c)
#define ENABLE_SMALLPL REG_BIT(15)
#define SC_DISABLE_POWER_OPTIMIZATION_EBB REG_BIT(9)
#define GEN11_SAMPLER_ENABLE_HEADLESS_MSG REG_BIT(5)
-#define GEN9_HALF_SLICE_CHICKEN7 _MMIO(0xe194)
+#define GEN9_HALF_SLICE_CHICKEN7 MCR_REG(0xe194)
#define DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA REG_BIT(15)
#define GEN9_SAMPLER_HASH_COMPRESSED_READ_ADDR REG_BIT(8)
#define GEN9_ENABLE_YV12_BUGFIX REG_BIT(4)
#define GEN9_ENABLE_GPGPU_PREEMPTION REG_BIT(2)
-#define GEN10_CACHE_MODE_SS _MMIO(0xe420)
+#define GEN10_CACHE_MODE_SS MCR_REG(0xe420)
#define ENABLE_EU_COUNT_FOR_TDL_FLUSH REG_BIT(10)
#define DISABLE_ECC REG_BIT(5)
#define FLOAT_BLEND_OPTIMIZATION_ENABLE REG_BIT(4)
#define ENABLE_PREFETCH_INTO_IC REG_BIT(3)
-#define EU_PERF_CNTL0 _MMIO(0xe458)
-#define EU_PERF_CNTL4 _MMIO(0xe45c)
+#define EU_PERF_CNTL0 PERF_REG(0xe458)
+#define EU_PERF_CNTL4 PERF_REG(0xe45c)
-#define GEN9_ROW_CHICKEN4 _MMIO(0xe48c)
+#define GEN9_ROW_CHICKEN4 MCR_REG(0xe48c)
#define GEN12_DISABLE_GRF_CLEAR REG_BIT(13)
#define XEHP_DIS_BBL_SYSPIPE REG_BIT(11)
#define GEN12_DISABLE_TDL_PUSH REG_BIT(9)
@@ -1116,7 +1157,7 @@
#define HSW_ROW_CHICKEN3 _MMIO(0xe49c)
#define HSW_ROW_CHICKEN3_L3_GLOBAL_ATOMICS_DISABLE (1 << 6)
-#define GEN8_ROW_CHICKEN _MMIO(0xe4f0)
+#define GEN8_ROW_CHICKEN MCR_REG(0xe4f0)
#define FLOW_CONTROL_ENABLE REG_BIT(15)
#define UGM_BACKUP_MODE REG_BIT(13)
#define MDQ_ARBITRATION_MODE REG_BIT(12)
@@ -1127,42 +1168,43 @@
#define DISABLE_EARLY_EOT REG_BIT(1)
#define GEN7_ROW_CHICKEN2 _MMIO(0xe4f4)
+
+#define GEN8_ROW_CHICKEN2 MCR_REG(0xe4f4)
#define GEN12_DISABLE_READ_SUPPRESSION REG_BIT(15)
#define GEN12_DISABLE_EARLY_READ REG_BIT(14)
#define GEN12_ENABLE_LARGE_GRF_MODE REG_BIT(12)
#define GEN12_PUSH_CONST_DEREF_HOLD_DIS REG_BIT(8)
+#define GEN12_DISABLE_DOP_GATING REG_BIT(0)
-#define RT_CTRL _MMIO(0xe530)
+#define RT_CTRL MCR_REG(0xe530)
#define DIS_NULL_QUERY REG_BIT(10)
#define STACKID_CTRL REG_GENMASK(6, 5)
#define STACKID_CTRL_512 REG_FIELD_PREP(STACKID_CTRL, 0x2)
-#define EU_PERF_CNTL1 _MMIO(0xe558)
-#define EU_PERF_CNTL5 _MMIO(0xe55c)
+#define EU_PERF_CNTL1 PERF_REG(0xe558)
+#define EU_PERF_CNTL5 PERF_REG(0xe55c)
-#define GEN12_HDC_CHICKEN0 _MMIO(0xe5f0)
+#define XEHP_HDC_CHICKEN0 MCR_REG(0xe5f0)
#define LSC_L1_FLUSH_CTL_3D_DATAPORT_FLUSH_EVENTS_MASK REG_GENMASK(13, 11)
-#define ICL_HDC_MODE _MMIO(0xe5f4)
+#define ICL_HDC_MODE MCR_REG(0xe5f4)
-#define EU_PERF_CNTL2 _MMIO(0xe658)
-#define EU_PERF_CNTL6 _MMIO(0xe65c)
-#define EU_PERF_CNTL3 _MMIO(0xe758)
+#define EU_PERF_CNTL2 PERF_REG(0xe658)
+#define EU_PERF_CNTL6 PERF_REG(0xe65c)
+#define EU_PERF_CNTL3 PERF_REG(0xe758)
-#define LSC_CHICKEN_BIT_0 _MMIO(0xe7c8)
+#define LSC_CHICKEN_BIT_0 MCR_REG(0xe7c8)
#define DISABLE_D8_D16_COASLESCE REG_BIT(30)
#define FORCE_1_SUB_MESSAGE_PER_FRAGMENT REG_BIT(15)
-#define LSC_CHICKEN_BIT_0_UDW _MMIO(0xe7c8 + 4)
+#define LSC_CHICKEN_BIT_0_UDW MCR_REG(0xe7c8 + 4)
#define DIS_CHAIN_2XSIMD8 REG_BIT(55 - 32)
#define FORCE_SLM_FENCE_SCOPE_TO_TILE REG_BIT(42 - 32)
#define FORCE_UGM_FENCE_SCOPE_TO_TILE REG_BIT(41 - 32)
#define MAXREQS_PER_BANK REG_GENMASK(39 - 32, 37 - 32)
#define DISABLE_128B_EVICTION_COMMAND_UDW REG_BIT(36 - 32)
-#define SARB_CHICKEN1 _MMIO(0xe90c)
+#define SARB_CHICKEN1 MCR_REG(0xe90c)
#define COMP_CKN_IN REG_GENMASK(30, 29)
-#define GEN7_HALF_SLICE_CHICKEN1_GT2 _MMIO(0xf100)
-
#define GEN7_ROW_CHICKEN2_GT2 _MMIO(0xf4f4)
#define DOP_CLOCK_GATING_DISABLE (1 << 0)
#define PUSH_CONSTANT_DEREF_DISABLE (1 << 8)
@@ -1339,8 +1381,7 @@
#define MEMSTAT_ILK _MMIO(0x111f8)
#define MEMSTAT_VID_MASK 0x7f00
#define MEMSTAT_VID_SHIFT 8
-#define MEMSTAT_PSTATE_MASK 0x00f8
-#define MEMSTAT_PSTATE_SHIFT 3
+#define MEMSTAT_PSTATE_MASK REG_GENMASK(7, 3)
#define MEMSTAT_MON_ACTV (1 << 2)
#define MEMSTAT_SRC_CTL_MASK 0x0003
#define MEMSTAT_SRC_CTL_CORE 0
@@ -1481,6 +1522,8 @@
#define FORCEWAKE_MEDIA_VLV _MMIO(0x1300b8)
#define FORCEWAKE_ACK_MEDIA_VLV _MMIO(0x1300bc)
+#define MTL_MEDIA_MC6 _MMIO(0x138048)
+
#define GEN6_GT_THREAD_STATUS_REG _MMIO(0x13805c)
#define GEN6_GT_THREAD_STATUS_CORE_MASK 0x7
@@ -1510,10 +1553,15 @@
#define VLV_RENDER_C0_COUNT _MMIO(0x138118)
#define VLV_MEDIA_C0_COUNT _MMIO(0x13811c)
+#define GEN12_RPSTAT1 _MMIO(0x1381b4)
+#define GEN12_VOLTAGE_MASK REG_GENMASK(10, 0)
+#define GEN12_CAGF_MASK REG_GENMASK(19, 11)
+
#define GEN11_GT_INTR_DW(x) _MMIO(0x190018 + ((x) * 4))
#define GEN11_CSME (31)
#define GEN11_GUNIT (28)
#define GEN11_GUC (25)
+#define MTL_MGUC (24)
#define GEN11_WDPERF (20)
#define GEN11_KCR (19)
#define GEN11_GTPM (16)
@@ -1568,6 +1616,7 @@
#define GEN11_VECS0_VECS1_INTR_MASK _MMIO(0x1900d0)
#define GEN12_VECS2_VECS3_INTR_MASK _MMIO(0x1900d4)
#define GEN11_GUC_SG_INTR_MASK _MMIO(0x1900e8)
+#define MTL_GUC_MGUC_INTR_MASK _MMIO(0x1900e8) /* MTL+ */
#define GEN11_GPM_WGBOXPERF_INTR_MASK _MMIO(0x1900ec)
#define GEN11_CRYPTO_RSVD_INTR_MASK _MMIO(0x1900f0)
#define GEN11_GUNIT_CSME_INTR_MASK _MMIO(0x1900f4)
@@ -1580,6 +1629,11 @@
#define GEN12_SFC_DONE(n) _MMIO(0x1cc000 + (n) * 0x1000)
+#define GT0_PACKAGE_ENERGY_STATUS _MMIO(0x250004)
+#define GT0_PACKAGE_RAPL_LIMIT _MMIO(0x250008)
+#define GT0_PACKAGE_POWER_SKU_UNIT _MMIO(0x250068)
+#define GT0_PLATFORM_ENERGY_STATUS _MMIO(0x25006c)
+
/*
* Standalone Media's non-engine GT registers are located at their regular GT
* offsets plus 0x380000. This extra offset is stored inside the intel_uncore
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
index d651ccd..9486dd3 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
@@ -22,11 +22,9 @@ bool is_object_gt(struct kobject *kobj)
return !strncmp(kobj->name, "gt", 2);
}
-struct intel_gt *intel_gt_sysfs_get_drvdata(struct device *dev,
+struct intel_gt *intel_gt_sysfs_get_drvdata(struct kobject *kobj,
const char *name)
{
- struct kobject *kobj = &dev->kobj;
-
/*
* We are interested at knowing from where the interface
* has been called, whether it's called from gt/ or from
@@ -38,6 +36,7 @@ struct intel_gt *intel_gt_sysfs_get_drvdata(struct device *dev,
* "struct drm_i915_private *" type.
*/
if (!is_object_gt(kobj)) {
+ struct device *dev = kobj_to_dev(kobj);
struct drm_i915_private *i915 = kdev_minor_to_i915(dev);
return to_gt(i915);
@@ -51,18 +50,18 @@ static struct kobject *gt_get_parent_obj(struct intel_gt *gt)
return >->i915->drm.primary->kdev->kobj;
}
-static ssize_t id_show(struct device *dev,
- struct device_attribute *attr,
+static ssize_t id_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
char *buf)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
return sysfs_emit(buf, "%u\n", gt->info.id);
}
-static DEVICE_ATTR_RO(id);
+static struct kobj_attribute attr_id = __ATTR_RO(id);
static struct attribute *id_attrs[] = {
- &dev_attr_id.attr,
+ &attr_id.attr,
NULL,
};
ATTRIBUTE_GROUPS(id);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.h b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.h
index 6232923..18bab835 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.h
@@ -18,11 +18,6 @@ bool is_object_gt(struct kobject *kobj);
struct drm_i915_private *kobj_to_i915(struct kobject *kobj);
-struct kobject *
-intel_gt_create_kobj(struct intel_gt *gt,
- struct kobject *dir,
- const char *name);
-
static inline struct intel_gt *kobj_to_gt(struct kobject *kobj)
{
return container_of(kobj, struct intel_gt, sysfs_gt);
@@ -30,7 +25,7 @@ static inline struct intel_gt *kobj_to_gt(struct kobject *kobj)
void intel_gt_sysfs_register(struct intel_gt *gt);
void intel_gt_sysfs_unregister(struct intel_gt *gt);
-struct intel_gt *intel_gt_sysfs_get_drvdata(struct device *dev,
+struct intel_gt *intel_gt_sysfs_get_drvdata(struct kobject *kobj,
const char *name);
#endif /* SYSFS_GT_H */
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.c b/drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.c
index 180dd6f..cf71305 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.c
@@ -24,14 +24,15 @@ enum intel_gt_sysfs_op {
};
static int
-sysfs_gt_attribute_w_func(struct device *dev, struct device_attribute *attr,
+sysfs_gt_attribute_w_func(struct kobject *kobj, struct attribute *attr,
int (func)(struct intel_gt *gt, u32 val), u32 val)
{
struct intel_gt *gt;
int ret;
- if (!is_object_gt(&dev->kobj)) {
+ if (!is_object_gt(kobj)) {
int i;
+ struct device *dev = kobj_to_dev(kobj);
struct drm_i915_private *i915 = kdev_minor_to_i915(dev);
for_each_gt(gt, i915, i) {
@@ -40,7 +41,7 @@ sysfs_gt_attribute_w_func(struct device *dev, struct device_attribute *attr,
break;
}
} else {
- gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ gt = intel_gt_sysfs_get_drvdata(kobj, attr->name);
ret = func(gt, val);
}
@@ -48,7 +49,7 @@ sysfs_gt_attribute_w_func(struct device *dev, struct device_attribute *attr,
}
static u32
-sysfs_gt_attribute_r_func(struct device *dev, struct device_attribute *attr,
+sysfs_gt_attribute_r_func(struct kobject *kobj, struct attribute *attr,
u32 (func)(struct intel_gt *gt),
enum intel_gt_sysfs_op op)
{
@@ -57,8 +58,9 @@ sysfs_gt_attribute_r_func(struct device *dev, struct device_attribute *attr,
ret = (op == INTEL_GT_SYSFS_MAX) ? 0 : (u32) -1;
- if (!is_object_gt(&dev->kobj)) {
+ if (!is_object_gt(kobj)) {
int i;
+ struct device *dev = kobj_to_dev(kobj);
struct drm_i915_private *i915 = kdev_minor_to_i915(dev);
for_each_gt(gt, i915, i) {
@@ -77,7 +79,7 @@ sysfs_gt_attribute_r_func(struct device *dev, struct device_attribute *attr,
}
}
} else {
- gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ gt = intel_gt_sysfs_get_drvdata(kobj, attr->name);
ret = func(gt);
}
@@ -92,23 +94,90 @@ sysfs_gt_attribute_r_func(struct device *dev, struct device_attribute *attr,
#define sysfs_gt_attribute_r_max_func(d, a, f) \
sysfs_gt_attribute_r_func(d, a, f, INTEL_GT_SYSFS_MAX)
+#define INTEL_GT_SYSFS_SHOW(_name, _attr_type) \
+ static ssize_t _name##_show_common(struct kobject *kobj, \
+ struct attribute *attr, char *buff) \
+ { \
+ u32 val = sysfs_gt_attribute_r_##_attr_type##_func(kobj, attr, \
+ __##_name##_show); \
+ \
+ return sysfs_emit(buff, "%u\n", val); \
+ } \
+ static ssize_t _name##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buff) \
+ { \
+ return _name ##_show_common(kobj, &attr->attr, buff); \
+ } \
+ static ssize_t _name##_dev_show(struct device *dev, \
+ struct device_attribute *attr, char *buff) \
+ { \
+ return _name##_show_common(&dev->kobj, &attr->attr, buff); \
+ }
+
+#define INTEL_GT_SYSFS_STORE(_name, _func) \
+ static ssize_t _name##_store_common(struct kobject *kobj, \
+ struct attribute *attr, \
+ const char *buff, size_t count) \
+ { \
+ int ret; \
+ u32 val; \
+ \
+ ret = kstrtou32(buff, 0, &val); \
+ if (ret) \
+ return ret; \
+ \
+ ret = sysfs_gt_attribute_w_func(kobj, attr, _func, val); \
+ \
+ return ret ?: count; \
+ } \
+ static ssize_t _name##_store(struct kobject *kobj, \
+ struct kobj_attribute *attr, const char *buff, \
+ size_t count) \
+ { \
+ return _name##_store_common(kobj, &attr->attr, buff, count); \
+ } \
+ static ssize_t _name##_dev_store(struct device *dev, \
+ struct device_attribute *attr, \
+ const char *buff, size_t count) \
+ { \
+ return _name##_store_common(&dev->kobj, &attr->attr, buff, count); \
+ }
+
+#define INTEL_GT_SYSFS_SHOW_MAX(_name) INTEL_GT_SYSFS_SHOW(_name, max)
+#define INTEL_GT_SYSFS_SHOW_MIN(_name) INTEL_GT_SYSFS_SHOW(_name, min)
+
+#define INTEL_GT_ATTR_RW(_name) \
+ static struct kobj_attribute attr_##_name = __ATTR_RW(_name)
+
+#define INTEL_GT_ATTR_RO(_name) \
+ static struct kobj_attribute attr_##_name = __ATTR_RO(_name)
+
+#define INTEL_GT_DUAL_ATTR_RW(_name) \
+ static struct device_attribute dev_attr_##_name = __ATTR(_name, 0644, \
+ _name##_dev_show, \
+ _name##_dev_store); \
+ INTEL_GT_ATTR_RW(_name)
+
+#define INTEL_GT_DUAL_ATTR_RO(_name) \
+ static struct device_attribute dev_attr_##_name = __ATTR(_name, 0444, \
+ _name##_dev_show, \
+ NULL); \
+ INTEL_GT_ATTR_RO(_name)
+
#ifdef CONFIG_PM
-static u32 get_residency(struct intel_gt *gt, i915_reg_t reg)
+static u32 get_residency(struct intel_gt *gt, enum intel_rc6_res_type id)
{
intel_wakeref_t wakeref;
u64 res = 0;
with_intel_runtime_pm(gt->uncore->rpm, wakeref)
- res = intel_rc6_residency_us(>->rc6, reg);
+ res = intel_rc6_residency_us(>->rc6, id);
return DIV_ROUND_CLOSEST_ULL(res, 1000);
}
-static ssize_t rc6_enable_show(struct device *dev,
- struct device_attribute *attr,
- char *buff)
+static u8 get_rc6_mask(struct intel_gt *gt)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
u8 mask = 0;
if (HAS_RC6(gt->i915))
@@ -118,105 +187,105 @@ static ssize_t rc6_enable_show(struct device *dev,
if (HAS_RC6pp(gt->i915))
mask |= BIT(2);
- return sysfs_emit(buff, "%x\n", mask);
+ return mask;
+}
+
+static ssize_t rc6_enable_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buff)
+{
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
+
+ return sysfs_emit(buff, "%x\n", get_rc6_mask(gt));
+}
+
+static ssize_t rc6_enable_dev_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buff)
+{
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(&dev->kobj, attr->attr.name);
+
+ return sysfs_emit(buff, "%x\n", get_rc6_mask(gt));
}
static u32 __rc6_residency_ms_show(struct intel_gt *gt)
{
- return get_residency(gt, GEN6_GT_GFX_RC6);
-}
-
-static ssize_t rc6_residency_ms_show(struct device *dev,
- struct device_attribute *attr,
- char *buff)
-{
- u32 rc6_residency = sysfs_gt_attribute_r_min_func(dev, attr,
- __rc6_residency_ms_show);
-
- return sysfs_emit(buff, "%u\n", rc6_residency);
+ return get_residency(gt, INTEL_RC6_RES_RC6);
}
static u32 __rc6p_residency_ms_show(struct intel_gt *gt)
{
- return get_residency(gt, GEN6_GT_GFX_RC6p);
-}
-
-static ssize_t rc6p_residency_ms_show(struct device *dev,
- struct device_attribute *attr,
- char *buff)
-{
- u32 rc6p_residency = sysfs_gt_attribute_r_min_func(dev, attr,
- __rc6p_residency_ms_show);
-
- return sysfs_emit(buff, "%u\n", rc6p_residency);
+ return get_residency(gt, INTEL_RC6_RES_RC6p);
}
static u32 __rc6pp_residency_ms_show(struct intel_gt *gt)
{
- return get_residency(gt, GEN6_GT_GFX_RC6pp);
-}
-
-static ssize_t rc6pp_residency_ms_show(struct device *dev,
- struct device_attribute *attr,
- char *buff)
-{
- u32 rc6pp_residency = sysfs_gt_attribute_r_min_func(dev, attr,
- __rc6pp_residency_ms_show);
-
- return sysfs_emit(buff, "%u\n", rc6pp_residency);
+ return get_residency(gt, INTEL_RC6_RES_RC6pp);
}
static u32 __media_rc6_residency_ms_show(struct intel_gt *gt)
{
- return get_residency(gt, VLV_GT_MEDIA_RC6);
+ return get_residency(gt, INTEL_RC6_RES_VLV_MEDIA);
}
-static ssize_t media_rc6_residency_ms_show(struct device *dev,
- struct device_attribute *attr,
- char *buff)
-{
- u32 rc6_residency = sysfs_gt_attribute_r_min_func(dev, attr,
- __media_rc6_residency_ms_show);
+INTEL_GT_SYSFS_SHOW_MIN(rc6_residency_ms);
+INTEL_GT_SYSFS_SHOW_MIN(rc6p_residency_ms);
+INTEL_GT_SYSFS_SHOW_MIN(rc6pp_residency_ms);
+INTEL_GT_SYSFS_SHOW_MIN(media_rc6_residency_ms);
- return sysfs_emit(buff, "%u\n", rc6_residency);
-}
-
-static DEVICE_ATTR_RO(rc6_enable);
-static DEVICE_ATTR_RO(rc6_residency_ms);
-static DEVICE_ATTR_RO(rc6p_residency_ms);
-static DEVICE_ATTR_RO(rc6pp_residency_ms);
-static DEVICE_ATTR_RO(media_rc6_residency_ms);
+INTEL_GT_DUAL_ATTR_RO(rc6_enable);
+INTEL_GT_DUAL_ATTR_RO(rc6_residency_ms);
+INTEL_GT_DUAL_ATTR_RO(rc6p_residency_ms);
+INTEL_GT_DUAL_ATTR_RO(rc6pp_residency_ms);
+INTEL_GT_DUAL_ATTR_RO(media_rc6_residency_ms);
static struct attribute *rc6_attrs[] = {
+ &attr_rc6_enable.attr,
+ &attr_rc6_residency_ms.attr,
+ NULL
+};
+
+static struct attribute *rc6p_attrs[] = {
+ &attr_rc6p_residency_ms.attr,
+ &attr_rc6pp_residency_ms.attr,
+ NULL
+};
+
+static struct attribute *media_rc6_attrs[] = {
+ &attr_media_rc6_residency_ms.attr,
+ NULL
+};
+
+static struct attribute *rc6_dev_attrs[] = {
&dev_attr_rc6_enable.attr,
&dev_attr_rc6_residency_ms.attr,
NULL
};
-static struct attribute *rc6p_attrs[] = {
+static struct attribute *rc6p_dev_attrs[] = {
&dev_attr_rc6p_residency_ms.attr,
&dev_attr_rc6pp_residency_ms.attr,
NULL
};
-static struct attribute *media_rc6_attrs[] = {
+static struct attribute *media_rc6_dev_attrs[] = {
&dev_attr_media_rc6_residency_ms.attr,
NULL
};
static const struct attribute_group rc6_attr_group[] = {
{ .attrs = rc6_attrs, },
- { .name = power_group_name, .attrs = rc6_attrs, },
+ { .name = power_group_name, .attrs = rc6_dev_attrs, },
};
static const struct attribute_group rc6p_attr_group[] = {
{ .attrs = rc6p_attrs, },
- { .name = power_group_name, .attrs = rc6p_attrs, },
+ { .name = power_group_name, .attrs = rc6p_dev_attrs, },
};
static const struct attribute_group media_rc6_attr_group[] = {
{ .attrs = media_rc6_attrs, },
- { .name = power_group_name, .attrs = media_rc6_attrs, },
+ { .name = power_group_name, .attrs = media_rc6_dev_attrs, },
};
static int __intel_gt_sysfs_create_group(struct kobject *kobj,
@@ -271,104 +340,34 @@ static u32 __act_freq_mhz_show(struct intel_gt *gt)
return intel_rps_read_actual_frequency(>->rps);
}
-static ssize_t act_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
-{
- u32 actual_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __act_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", actual_freq);
-}
-
static u32 __cur_freq_mhz_show(struct intel_gt *gt)
{
return intel_rps_get_requested_frequency(>->rps);
}
-static ssize_t cur_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
-{
- u32 cur_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __cur_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", cur_freq);
-}
-
static u32 __boost_freq_mhz_show(struct intel_gt *gt)
{
return intel_rps_get_boost_frequency(>->rps);
}
-static ssize_t boost_freq_mhz_show(struct device *dev,
- struct device_attribute *attr,
- char *buff)
-{
- u32 boost_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __boost_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", boost_freq);
-}
-
static int __boost_freq_mhz_store(struct intel_gt *gt, u32 val)
{
return intel_rps_set_boost_frequency(>->rps, val);
}
-static ssize_t boost_freq_mhz_store(struct device *dev,
- struct device_attribute *attr,
- const char *buff, size_t count)
-{
- ssize_t ret;
- u32 val;
-
- ret = kstrtou32(buff, 0, &val);
- if (ret)
- return ret;
-
- return sysfs_gt_attribute_w_func(dev, attr,
- __boost_freq_mhz_store, val) ?: count;
-}
-
-static u32 __rp0_freq_mhz_show(struct intel_gt *gt)
+static u32 __RP0_freq_mhz_show(struct intel_gt *gt)
{
return intel_rps_get_rp0_frequency(>->rps);
}
-static ssize_t RP0_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
-{
- u32 rp0_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __rp0_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", rp0_freq);
-}
-
-static u32 __rp1_freq_mhz_show(struct intel_gt *gt)
-{
- return intel_rps_get_rp1_frequency(>->rps);
-}
-
-static ssize_t RP1_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
-{
- u32 rp1_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __rp1_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", rp1_freq);
-}
-
-static u32 __rpn_freq_mhz_show(struct intel_gt *gt)
+static u32 __RPn_freq_mhz_show(struct intel_gt *gt)
{
return intel_rps_get_rpn_frequency(>->rps);
}
-static ssize_t RPn_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
+static u32 __RP1_freq_mhz_show(struct intel_gt *gt)
{
- u32 rpn_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __rpn_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", rpn_freq);
+ return intel_rps_get_rp1_frequency(>->rps);
}
static u32 __max_freq_mhz_show(struct intel_gt *gt)
@@ -376,71 +375,21 @@ static u32 __max_freq_mhz_show(struct intel_gt *gt)
return intel_rps_get_max_frequency(>->rps);
}
-static ssize_t max_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
-{
- u32 max_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __max_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", max_freq);
-}
-
static int __set_max_freq(struct intel_gt *gt, u32 val)
{
return intel_rps_set_max_frequency(>->rps, val);
}
-static ssize_t max_freq_mhz_store(struct device *dev,
- struct device_attribute *attr,
- const char *buff, size_t count)
-{
- int ret;
- u32 val;
-
- ret = kstrtou32(buff, 0, &val);
- if (ret)
- return ret;
-
- ret = sysfs_gt_attribute_w_func(dev, attr, __set_max_freq, val);
-
- return ret ?: count;
-}
-
static u32 __min_freq_mhz_show(struct intel_gt *gt)
{
return intel_rps_get_min_frequency(>->rps);
}
-static ssize_t min_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
-{
- u32 min_freq = sysfs_gt_attribute_r_min_func(dev, attr,
- __min_freq_mhz_show);
-
- return sysfs_emit(buff, "%u\n", min_freq);
-}
-
static int __set_min_freq(struct intel_gt *gt, u32 val)
{
return intel_rps_set_min_frequency(>->rps, val);
}
-static ssize_t min_freq_mhz_store(struct device *dev,
- struct device_attribute *attr,
- const char *buff, size_t count)
-{
- int ret;
- u32 val;
-
- ret = kstrtou32(buff, 0, &val);
- if (ret)
- return ret;
-
- ret = sysfs_gt_attribute_w_func(dev, attr, __set_min_freq, val);
-
- return ret ?: count;
-}
-
static u32 __vlv_rpe_freq_mhz_show(struct intel_gt *gt)
{
struct intel_rps *rps = >->rps;
@@ -448,23 +397,31 @@ static u32 __vlv_rpe_freq_mhz_show(struct intel_gt *gt)
return intel_gpu_freq(rps, rps->efficient_freq);
}
-static ssize_t vlv_rpe_freq_mhz_show(struct device *dev,
- struct device_attribute *attr, char *buff)
-{
- u32 rpe_freq = sysfs_gt_attribute_r_max_func(dev, attr,
- __vlv_rpe_freq_mhz_show);
+INTEL_GT_SYSFS_SHOW_MAX(act_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MAX(boost_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MAX(cur_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MAX(RP0_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MAX(RP1_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MAX(RPn_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MAX(max_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MIN(min_freq_mhz);
+INTEL_GT_SYSFS_SHOW_MAX(vlv_rpe_freq_mhz);
+INTEL_GT_SYSFS_STORE(boost_freq_mhz, __boost_freq_mhz_store);
+INTEL_GT_SYSFS_STORE(max_freq_mhz, __set_max_freq);
+INTEL_GT_SYSFS_STORE(min_freq_mhz, __set_min_freq);
- return sysfs_emit(buff, "%u\n", rpe_freq);
-}
+#define INTEL_GT_RPS_SYSFS_ATTR(_name, _mode, _show, _store, _show_dev, _store_dev) \
+ static struct device_attribute dev_attr_gt_##_name = __ATTR(gt_##_name, _mode, \
+ _show_dev, _store_dev); \
+ static struct kobj_attribute attr_rps_##_name = __ATTR(rps_##_name, _mode, \
+ _show, _store)
-#define INTEL_GT_RPS_SYSFS_ATTR(_name, _mode, _show, _store) \
- static struct device_attribute dev_attr_gt_##_name = __ATTR(gt_##_name, _mode, _show, _store); \
- static struct device_attribute dev_attr_rps_##_name = __ATTR(rps_##_name, _mode, _show, _store)
-
-#define INTEL_GT_RPS_SYSFS_ATTR_RO(_name) \
- INTEL_GT_RPS_SYSFS_ATTR(_name, 0444, _name##_show, NULL)
-#define INTEL_GT_RPS_SYSFS_ATTR_RW(_name) \
- INTEL_GT_RPS_SYSFS_ATTR(_name, 0644, _name##_show, _name##_store)
+#define INTEL_GT_RPS_SYSFS_ATTR_RO(_name) \
+ INTEL_GT_RPS_SYSFS_ATTR(_name, 0444, _name##_show, NULL, \
+ _name##_dev_show, NULL)
+#define INTEL_GT_RPS_SYSFS_ATTR_RW(_name) \
+ INTEL_GT_RPS_SYSFS_ATTR(_name, 0644, _name##_show, _name##_store, \
+ _name##_dev_show, _name##_dev_store)
/* The below macros generate static structures */
INTEL_GT_RPS_SYSFS_ATTR_RO(act_freq_mhz);
@@ -475,32 +432,31 @@ INTEL_GT_RPS_SYSFS_ATTR_RO(RP1_freq_mhz);
INTEL_GT_RPS_SYSFS_ATTR_RO(RPn_freq_mhz);
INTEL_GT_RPS_SYSFS_ATTR_RW(max_freq_mhz);
INTEL_GT_RPS_SYSFS_ATTR_RW(min_freq_mhz);
+INTEL_GT_RPS_SYSFS_ATTR_RO(vlv_rpe_freq_mhz);
-static DEVICE_ATTR_RO(vlv_rpe_freq_mhz);
-
-#define GEN6_ATTR(s) { \
- &dev_attr_##s##_act_freq_mhz.attr, \
- &dev_attr_##s##_cur_freq_mhz.attr, \
- &dev_attr_##s##_boost_freq_mhz.attr, \
- &dev_attr_##s##_max_freq_mhz.attr, \
- &dev_attr_##s##_min_freq_mhz.attr, \
- &dev_attr_##s##_RP0_freq_mhz.attr, \
- &dev_attr_##s##_RP1_freq_mhz.attr, \
- &dev_attr_##s##_RPn_freq_mhz.attr, \
+#define GEN6_ATTR(p, s) { \
+ &p##attr_##s##_act_freq_mhz.attr, \
+ &p##attr_##s##_cur_freq_mhz.attr, \
+ &p##attr_##s##_boost_freq_mhz.attr, \
+ &p##attr_##s##_max_freq_mhz.attr, \
+ &p##attr_##s##_min_freq_mhz.attr, \
+ &p##attr_##s##_RP0_freq_mhz.attr, \
+ &p##attr_##s##_RP1_freq_mhz.attr, \
+ &p##attr_##s##_RPn_freq_mhz.attr, \
NULL, \
}
-#define GEN6_RPS_ATTR GEN6_ATTR(rps)
-#define GEN6_GT_ATTR GEN6_ATTR(gt)
+#define GEN6_RPS_ATTR GEN6_ATTR(, rps)
+#define GEN6_GT_ATTR GEN6_ATTR(dev_, gt)
static const struct attribute * const gen6_rps_attrs[] = GEN6_RPS_ATTR;
static const struct attribute * const gen6_gt_attrs[] = GEN6_GT_ATTR;
-static ssize_t punit_req_freq_mhz_show(struct device *dev,
- struct device_attribute *attr,
+static ssize_t punit_req_freq_mhz_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
char *buff)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
u32 preq = intel_rps_read_punit_req_frequency(>->rps);
return sysfs_emit(buff, "%u\n", preq);
@@ -508,20 +464,20 @@ static ssize_t punit_req_freq_mhz_show(struct device *dev,
struct intel_gt_bool_throttle_attr {
struct attribute attr;
- ssize_t (*show)(struct device *dev, struct device_attribute *attr,
+ ssize_t (*show)(struct kobject *kobj, struct kobj_attribute *attr,
char *buf);
- i915_reg_t reg32;
+ i915_reg_t (*reg32)(struct intel_gt *gt);
u32 mask;
};
-static ssize_t throttle_reason_bool_show(struct device *dev,
- struct device_attribute *attr,
+static ssize_t throttle_reason_bool_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
char *buff)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
struct intel_gt_bool_throttle_attr *t_attr =
(struct intel_gt_bool_throttle_attr *) attr;
- bool val = rps_read_mask_mmio(>->rps, t_attr->reg32, t_attr->mask);
+ bool val = rps_read_mask_mmio(>->rps, t_attr->reg32(gt), t_attr->mask);
return sysfs_emit(buff, "%u\n", val);
}
@@ -530,11 +486,11 @@ static ssize_t throttle_reason_bool_show(struct device *dev,
struct intel_gt_bool_throttle_attr attr_##sysfs_func__ = { \
.attr = { .name = __stringify(sysfs_func__), .mode = 0444 }, \
.show = throttle_reason_bool_show, \
- .reg32 = GT0_PERF_LIMIT_REASONS, \
+ .reg32 = intel_gt_perf_limit_reasons_reg, \
.mask = mask__, \
}
-static DEVICE_ATTR_RO(punit_req_freq_mhz);
+INTEL_GT_ATTR_RO(punit_req_freq_mhz);
static INTEL_GT_RPS_BOOL_ATTR_RO(throttle_reason_status, GT0_PERF_LIMIT_REASONS_MASK);
static INTEL_GT_RPS_BOOL_ATTR_RO(throttle_reason_pl1, POWER_LIMIT_1_MASK);
static INTEL_GT_RPS_BOOL_ATTR_RO(throttle_reason_pl2, POWER_LIMIT_2_MASK);
@@ -597,8 +553,8 @@ static const struct attribute *throttle_reason_attrs[] = {
#define U8_8_VAL_MASK 0xffff
#define U8_8_SCALE_TO_VALUE "0.00390625"
-static ssize_t freq_factor_scale_show(struct device *dev,
- struct device_attribute *attr,
+static ssize_t freq_factor_scale_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
char *buff)
{
return sysfs_emit(buff, "%s\n", U8_8_SCALE_TO_VALUE);
@@ -610,11 +566,11 @@ static u32 media_ratio_mode_to_factor(u32 mode)
return !mode ? mode : 256 / mode;
}
-static ssize_t media_freq_factor_show(struct device *dev,
- struct device_attribute *attr,
+static ssize_t media_freq_factor_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
char *buff)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
struct intel_guc_slpc *slpc = >->uc.guc.slpc;
intel_wakeref_t wakeref;
u32 mode;
@@ -641,11 +597,11 @@ static ssize_t media_freq_factor_show(struct device *dev,
return sysfs_emit(buff, "%u\n", media_ratio_mode_to_factor(mode));
}
-static ssize_t media_freq_factor_store(struct device *dev,
- struct device_attribute *attr,
+static ssize_t media_freq_factor_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
const char *buff, size_t count)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
struct intel_guc_slpc *slpc = >->uc.guc.slpc;
u32 factor, mode;
int err;
@@ -670,11 +626,11 @@ static ssize_t media_freq_factor_store(struct device *dev,
return err ?: count;
}
-static ssize_t media_RP0_freq_mhz_show(struct device *dev,
- struct device_attribute *attr,
+static ssize_t media_RP0_freq_mhz_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
char *buff)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
u32 val;
int err;
@@ -691,11 +647,11 @@ static ssize_t media_RP0_freq_mhz_show(struct device *dev,
return sysfs_emit(buff, "%u\n", val);
}
-static ssize_t media_RPn_freq_mhz_show(struct device *dev,
- struct device_attribute *attr,
+static ssize_t media_RPn_freq_mhz_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
char *buff)
{
- struct intel_gt *gt = intel_gt_sysfs_get_drvdata(dev, attr->attr.name);
+ struct intel_gt *gt = intel_gt_sysfs_get_drvdata(kobj, attr->attr.name);
u32 val;
int err;
@@ -712,17 +668,17 @@ static ssize_t media_RPn_freq_mhz_show(struct device *dev,
return sysfs_emit(buff, "%u\n", val);
}
-static DEVICE_ATTR_RW(media_freq_factor);
-static struct device_attribute dev_attr_media_freq_factor_scale =
+INTEL_GT_ATTR_RW(media_freq_factor);
+static struct kobj_attribute attr_media_freq_factor_scale =
__ATTR(media_freq_factor.scale, 0444, freq_factor_scale_show, NULL);
-static DEVICE_ATTR_RO(media_RP0_freq_mhz);
-static DEVICE_ATTR_RO(media_RPn_freq_mhz);
+INTEL_GT_ATTR_RO(media_RP0_freq_mhz);
+INTEL_GT_ATTR_RO(media_RPn_freq_mhz);
static const struct attribute *media_perf_power_attrs[] = {
- &dev_attr_media_freq_factor.attr,
- &dev_attr_media_freq_factor_scale.attr,
- &dev_attr_media_RP0_freq_mhz.attr,
- &dev_attr_media_RPn_freq_mhz.attr,
+ &attr_media_freq_factor.attr,
+ &attr_media_freq_factor_scale.attr,
+ &attr_media_RP0_freq_mhz.attr,
+ &attr_media_RPn_freq_mhz.attr,
NULL
};
@@ -754,20 +710,29 @@ static const struct attribute * const rps_defaults_attrs[] = {
NULL
};
-static int intel_sysfs_rps_init(struct intel_gt *gt, struct kobject *kobj,
- const struct attribute * const *attrs)
+static int intel_sysfs_rps_init(struct intel_gt *gt, struct kobject *kobj)
{
+ const struct attribute * const *attrs;
+ struct attribute *vlv_attr;
int ret;
if (GRAPHICS_VER(gt->i915) < 6)
return 0;
+ if (is_object_gt(kobj)) {
+ attrs = gen6_rps_attrs;
+ vlv_attr = &attr_rps_vlv_rpe_freq_mhz.attr;
+ } else {
+ attrs = gen6_gt_attrs;
+ vlv_attr = &dev_attr_gt_vlv_rpe_freq_mhz.attr;
+ }
+
ret = sysfs_create_files(kobj, attrs);
if (ret)
return ret;
if (IS_VALLEYVIEW(gt->i915) || IS_CHERRYVIEW(gt->i915))
- ret = sysfs_create_file(kobj, &dev_attr_vlv_rpe_freq_mhz.attr);
+ ret = sysfs_create_file(kobj, vlv_attr);
return ret;
}
@@ -778,9 +743,7 @@ void intel_gt_sysfs_pm_init(struct intel_gt *gt, struct kobject *kobj)
intel_sysfs_rc6_init(gt, kobj);
- ret = is_object_gt(kobj) ?
- intel_sysfs_rps_init(gt, kobj, gen6_rps_attrs) :
- intel_sysfs_rps_init(gt, kobj, gen6_gt_attrs);
+ ret = intel_sysfs_rps_init(gt, kobj);
if (ret)
drm_warn(>->i915->drm,
"failed to create gt%u RPS sysfs files (%pe)",
@@ -790,13 +753,13 @@ void intel_gt_sysfs_pm_init(struct intel_gt *gt, struct kobject *kobj)
if (!is_object_gt(kobj))
return;
- ret = sysfs_create_file(kobj, &dev_attr_punit_req_freq_mhz.attr);
+ ret = sysfs_create_file(kobj, &attr_punit_req_freq_mhz.attr);
if (ret)
drm_warn(>->i915->drm,
"failed to create gt%u punit_req_freq_mhz sysfs (%pe)",
gt->info.id, ERR_PTR(ret));
- if (GRAPHICS_VER(gt->i915) >= 11) {
+ if (i915_mmio_reg_valid(intel_gt_perf_limit_reasons_reg(gt))) {
ret = sysfs_create_files(kobj, throttle_reason_attrs);
if (ret)
drm_warn(>->i915->drm,
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i915/gt/intel_gt_types.h
index f19c2de..c1d9cd2 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h
@@ -20,6 +20,7 @@
#include "intel_gsc.h"
#include "i915_vma.h"
+#include "i915_perf_types.h"
#include "intel_engine_types.h"
#include "intel_gt_buffer_pool_types.h"
#include "intel_hwconfig.h"
@@ -30,6 +31,7 @@
#include "intel_migrate_types.h"
#include "intel_wakeref.h"
#include "pxp/intel_pxp_types.h"
+#include "intel_wopcm.h"
struct drm_i915_private;
struct i915_ggtt;
@@ -59,6 +61,9 @@ enum intel_steering_type {
L3BANK,
MSLICE,
LNCF,
+ GAM,
+ DSS,
+ OADDRM,
/*
* On some platforms there are multiple types of MCR registers that
@@ -97,6 +102,7 @@ struct intel_gt {
struct intel_uc uc;
struct intel_gsc gsc;
+ struct intel_wopcm wopcm;
struct {
/* Serialize global tlb invalidations */
@@ -141,20 +147,6 @@ struct intel_gt {
struct intel_wakeref wakeref;
atomic_t user_wakeref;
- /**
- * Protects access to lmem usefault list.
- * It is required, if we are outside of the runtime suspend path,
- * access to @lmem_userfault_list requires always first grabbing the
- * runtime pm, to ensure we can't race against runtime suspend.
- * Once we have that we also need to grab @lmem_userfault_lock,
- * at which point we have exclusive access.
- * The runtime suspend path is special since it doesn't really hold any locks,
- * but instead has exclusive access by virtue of all other accesses requiring
- * holding the runtime pm wakeref.
- */
- struct mutex lmem_userfault_lock;
- struct list_head lmem_userfault_list;
-
struct list_head closed_vma;
spinlock_t closed_lock; /* guards the list of closed_vma */
@@ -170,9 +162,6 @@ struct intel_gt {
*/
intel_wakeref_t awake;
- /* Manual runtime pm autosuspend delay for user GGTT/lmem mmaps */
- struct intel_wakeref_auto userfault_wakeref;
-
u32 clock_frequency;
u32 clock_period_ns;
@@ -286,6 +275,8 @@ struct intel_gt {
/* sysfs defaults per gt */
struct gt_defaults defaults;
struct kobject *sysfs_defaults;
+
+ struct i915_perf_gt perf;
};
struct intel_gt_definition {
diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.c b/drivers/gpu/drm/i915/gt/intel_gtt.c
index 2eaeba1..2ba3983 100644
--- a/drivers/gpu/drm/i915/gt/intel_gtt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gtt.c
@@ -12,9 +12,11 @@
#include "gem/i915_gem_internal.h"
#include "gem/i915_gem_lmem.h"
+#include "i915_reg.h"
#include "i915_trace.h"
#include "i915_utils.h"
#include "intel_gt.h"
+#include "intel_gt_mcr.h"
#include "intel_gt_regs.h"
#include "intel_gtt.h"
@@ -269,11 +271,7 @@ void i915_address_space_init(struct i915_address_space *vm, int subclass)
memset64(vm->min_alignment, I915_GTT_MIN_ALIGNMENT,
ARRAY_SIZE(vm->min_alignment));
- if (HAS_64K_PAGES(vm->i915) && NEEDS_COMPACT_PT(vm->i915) &&
- subclass == VM_CLASS_PPGTT) {
- vm->min_alignment[INTEL_MEMORY_LOCAL] = I915_GTT_PAGE_SIZE_2M;
- vm->min_alignment[INTEL_MEMORY_STOLEN_LOCAL] = I915_GTT_PAGE_SIZE_2M;
- } else if (HAS_64K_PAGES(vm->i915)) {
+ if (HAS_64K_PAGES(vm->i915)) {
vm->min_alignment[INTEL_MEMORY_LOCAL] = I915_GTT_PAGE_SIZE_64K;
vm->min_alignment[INTEL_MEMORY_STOLEN_LOCAL] = I915_GTT_PAGE_SIZE_64K;
}
@@ -343,7 +341,8 @@ int setup_scratch_page(struct i915_address_space *vm)
*/
size = I915_GTT_PAGE_SIZE_4K;
if (i915_vm_is_4lvl(vm) &&
- HAS_PAGE_SIZES(vm->i915, I915_GTT_PAGE_SIZE_64K))
+ HAS_PAGE_SIZES(vm->i915, I915_GTT_PAGE_SIZE_64K) &&
+ !HAS_64K_PAGES(vm->i915))
size = I915_GTT_PAGE_SIZE_64K;
do {
@@ -385,18 +384,6 @@ int setup_scratch_page(struct i915_address_space *vm)
if (size == I915_GTT_PAGE_SIZE_4K)
return -ENOMEM;
- /*
- * If we need 64K minimum GTT pages for device local-memory,
- * like on XEHPSDV, then we need to fail the allocation here,
- * otherwise we can't safely support the insertion of
- * local-memory pages for this vm, since the HW expects the
- * correct physical alignment and size when the page-table is
- * operating in 64K GTT mode, which includes any scratch PTEs,
- * since userspace can still touch them.
- */
- if (HAS_64K_PAGES(vm->i915))
- return -ENOMEM;
-
size = I915_GTT_PAGE_SIZE_4K;
} while (1);
}
@@ -493,6 +480,18 @@ static void tgl_setup_private_ppat(struct intel_uncore *uncore)
intel_uncore_write(uncore, GEN12_PAT_INDEX(7), GEN8_PPAT_WB);
}
+static void xehp_setup_private_ppat(struct intel_gt *gt)
+{
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(0), GEN8_PPAT_WB);
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(1), GEN8_PPAT_WC);
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(2), GEN8_PPAT_WT);
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(3), GEN8_PPAT_UC);
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(4), GEN8_PPAT_WB);
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(5), GEN8_PPAT_WB);
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(6), GEN8_PPAT_WB);
+ intel_gt_mcr_multicast_write(gt, XEHP_PAT_INDEX(7), GEN8_PPAT_WB);
+}
+
static void icl_setup_private_ppat(struct intel_uncore *uncore)
{
intel_uncore_write(uncore,
@@ -585,13 +584,16 @@ static void chv_setup_private_ppat(struct intel_uncore *uncore)
intel_uncore_write(uncore, GEN8_PRIVATE_PAT_HI, upper_32_bits(pat));
}
-void setup_private_pat(struct intel_uncore *uncore)
+void setup_private_pat(struct intel_gt *gt)
{
- struct drm_i915_private *i915 = uncore->i915;
+ struct intel_uncore *uncore = gt->uncore;
+ struct drm_i915_private *i915 = gt->i915;
GEM_BUG_ON(GRAPHICS_VER(i915) < 8);
- if (GRAPHICS_VER(i915) >= 12)
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50))
+ xehp_setup_private_ppat(gt);
+ else if (GRAPHICS_VER(i915) >= 12)
tgl_setup_private_ppat(uncore);
else if (GRAPHICS_VER(i915) >= 11)
icl_setup_private_ppat(uncore);
diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h
index c0ca53c..4d75ba4 100644
--- a/drivers/gpu/drm/i915/gt/intel_gtt.h
+++ b/drivers/gpu/drm/i915/gt/intel_gtt.h
@@ -93,6 +93,7 @@ typedef u64 gen8_pte_t;
#define GEN12_GGTT_PTE_LM BIT_ULL(1)
#define GEN12_PDE_64K BIT(6)
+#define GEN12_PTE_PS64 BIT(8)
/*
* Cacheability Control is a 4-bit value. The low three bits are stored in bits
@@ -667,7 +668,7 @@ void ppgtt_unbind_vma(struct i915_address_space *vm,
void gtt_write_workarounds(struct intel_gt *gt);
-void setup_private_pat(struct intel_uncore *uncore);
+void setup_private_pat(struct intel_gt *gt);
int i915_vm_alloc_pt_stash(struct i915_address_space *vm,
struct i915_vm_pt_stash *stash,
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index 3955292..7771a19 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -20,6 +20,30 @@
#include "intel_ring.h"
#include "shmem_utils.h"
+/*
+ * The per-platform tables are u8-encoded in @data. Decode @data and set the
+ * addresses' offset and commands in @regs. The following encoding is used
+ * for each byte. There are 2 steps: decoding commands and decoding addresses.
+ *
+ * Commands:
+ * [7]: create NOPs - number of NOPs are set in lower bits
+ * [6]: When creating MI_LOAD_REGISTER_IMM command, allow to set
+ * MI_LRI_FORCE_POSTED
+ * [5:0]: Number of NOPs or registers to set values to in case of
+ * MI_LOAD_REGISTER_IMM
+ *
+ * Addresses: these are decoded after a MI_LOAD_REGISTER_IMM command by "count"
+ * number of registers. They are set by using the REG/REG16 macros: the former
+ * is used for offsets smaller than 0x200 while the latter is for values bigger
+ * than that. Those macros already set all the bits documented below correctly:
+ *
+ * [7]: When a register offset needs more than 6 bits, use additional bytes, to
+ * follow, for the lower bits
+ * [6:0]: Register offset, without considering the engine base.
+ *
+ * This function only tweaks the commands and register offsets. Values are not
+ * filled out.
+ */
static void set_offsets(u32 *regs,
const u8 *data,
const struct intel_engine_cs *engine,
@@ -264,6 +288,39 @@ static const u8 dg2_xcs_offsets[] = {
END
};
+static const u8 mtl_xcs_offsets[] = {
+ NOP(1),
+ LRI(13, POSTED),
+ REG16(0x244),
+ REG(0x034),
+ REG(0x030),
+ REG(0x038),
+ REG(0x03c),
+ REG(0x168),
+ REG(0x140),
+ REG(0x110),
+ REG(0x1c0),
+ REG(0x1c4),
+ REG(0x1c8),
+ REG(0x180),
+ REG16(0x2b4),
+ NOP(4),
+
+ NOP(1),
+ LRI(9, POSTED),
+ REG16(0x3a8),
+ REG16(0x28c),
+ REG16(0x288),
+ REG16(0x284),
+ REG16(0x280),
+ REG16(0x27c),
+ REG16(0x278),
+ REG16(0x274),
+ REG16(0x270),
+
+ END
+};
+
static const u8 gen8_rcs_offsets[] = {
NOP(1),
LRI(14, POSTED),
@@ -606,6 +663,49 @@ static const u8 dg2_rcs_offsets[] = {
END
};
+static const u8 mtl_rcs_offsets[] = {
+ NOP(1),
+ LRI(15, POSTED),
+ REG16(0x244),
+ REG(0x034),
+ REG(0x030),
+ REG(0x038),
+ REG(0x03c),
+ REG(0x168),
+ REG(0x140),
+ REG(0x110),
+ REG(0x1c0),
+ REG(0x1c4),
+ REG(0x1c8),
+ REG(0x180),
+ REG16(0x2b4),
+ REG(0x120),
+ REG(0x124),
+
+ NOP(1),
+ LRI(9, POSTED),
+ REG16(0x3a8),
+ REG16(0x28c),
+ REG16(0x288),
+ REG16(0x284),
+ REG16(0x280),
+ REG16(0x27c),
+ REG16(0x278),
+ REG16(0x274),
+ REG16(0x270),
+
+ NOP(2),
+ LRI(2, POSTED),
+ REG16(0x5a8),
+ REG16(0x5ac),
+
+ NOP(6),
+ LRI(1, 0),
+ REG(0x0c8),
+
+ END
+};
+
#undef END
#undef REG16
#undef REG
@@ -624,7 +724,9 @@ static const u8 *reg_offsets(const struct intel_engine_cs *engine)
!intel_engine_has_relative_mmio(engine));
if (engine->flags & I915_ENGINE_HAS_RCS_REG_STATE) {
- if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 55))
+ if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 70))
+ return mtl_rcs_offsets;
+ else if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 55))
return dg2_rcs_offsets;
else if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 50))
return xehp_rcs_offsets;
@@ -637,7 +739,9 @@ static const u8 *reg_offsets(const struct intel_engine_cs *engine)
else
return gen8_rcs_offsets;
} else {
- if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 55))
+ if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 70))
+ return mtl_xcs_offsets;
+ else if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 55))
return dg2_xcs_offsets;
else if (GRAPHICS_VER(engine->i915) >= 12)
return gen12_xcs_offsets;
@@ -745,19 +849,18 @@ static int lrc_ring_cmd_buf_cctl(const struct intel_engine_cs *engine)
static u32
lrc_ring_indirect_offset_default(const struct intel_engine_cs *engine)
{
- switch (GRAPHICS_VER(engine->i915)) {
- default:
- MISSING_CASE(GRAPHICS_VER(engine->i915));
- fallthrough;
- case 12:
+ if (GRAPHICS_VER(engine->i915) >= 12)
return GEN12_CTX_RCS_INDIRECT_CTX_OFFSET_DEFAULT;
- case 11:
+ else if (GRAPHICS_VER(engine->i915) >= 11)
return GEN11_CTX_RCS_INDIRECT_CTX_OFFSET_DEFAULT;
- case 9:
+ else if (GRAPHICS_VER(engine->i915) >= 9)
return GEN9_CTX_RCS_INDIRECT_CTX_OFFSET_DEFAULT;
- case 8:
+ else if (GRAPHICS_VER(engine->i915) >= 8)
return GEN8_CTX_RCS_INDIRECT_CTX_OFFSET_DEFAULT;
- }
+
+ GEM_BUG_ON(GRAPHICS_VER(engine->i915) < 8);
+
+ return 0;
}
static void
@@ -1012,7 +1115,7 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)
if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
context_size += I915_GTT_PAGE_SIZE; /* for redzone */
- if (GRAPHICS_VER(engine->i915) == 12) {
+ if (GRAPHICS_VER(engine->i915) >= 12) {
ce->wa_bb_page = context_size / PAGE_SIZE;
context_size += PAGE_SIZE;
}
@@ -1718,24 +1821,16 @@ void lrc_init_wa_ctx(struct intel_engine_cs *engine)
unsigned int i;
int err;
- if (!(engine->flags & I915_ENGINE_HAS_RCS_REG_STATE))
+ if (GRAPHICS_VER(engine->i915) >= 11 ||
+ !(engine->flags & I915_ENGINE_HAS_RCS_REG_STATE))
return;
- switch (GRAPHICS_VER(engine->i915)) {
- case 12:
- case 11:
- return;
- case 9:
+ if (GRAPHICS_VER(engine->i915) == 9) {
wa_bb_fn[0] = gen9_init_indirectctx_bb;
wa_bb_fn[1] = NULL;
- break;
- case 8:
+ } else if (GRAPHICS_VER(engine->i915) == 8) {
wa_bb_fn[0] = gen8_init_indirectctx_bb;
wa_bb_fn[1] = NULL;
- break;
- default:
- MISSING_CASE(GRAPHICS_VER(engine->i915));
- return;
}
err = lrc_create_wa_ctx(engine);
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.h b/drivers/gpu/drm/i915/gt/intel_lrc.h
index a390f081..7111bae 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.h
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.h
@@ -110,6 +110,8 @@ enum {
#define XEHP_SW_CTX_ID_WIDTH 16
#define XEHP_SW_COUNTER_SHIFT 58
#define XEHP_SW_COUNTER_WIDTH 6
+#define GEN12_GUC_SW_CTX_ID_SHIFT 39
+#define GEN12_GUC_SW_CTX_ID_WIDTH 16
static inline void lrc_runtime_start(struct intel_context *ce)
{
diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.c b/drivers/gpu/drm/i915/gt/intel_migrate.c
index aaaf190..b405a041 100644
--- a/drivers/gpu/drm/i915/gt/intel_migrate.c
+++ b/drivers/gpu/drm/i915/gt/intel_migrate.c
@@ -10,6 +10,7 @@
#include "intel_gtt.h"
#include "intel_migrate.h"
#include "intel_ring.h"
+#include "gem/i915_gem_lmem.h"
struct insert_pte_data {
u64 offset;
diff --git a/drivers/gpu/drm/i915/gt/intel_mocs.c b/drivers/gpu/drm/i915/gt/intel_mocs.c
index 152244d..49fdd50 100644
--- a/drivers/gpu/drm/i915/gt/intel_mocs.c
+++ b/drivers/gpu/drm/i915/gt/intel_mocs.c
@@ -7,6 +7,7 @@
#include "intel_engine.h"
#include "intel_gt.h"
+#include "intel_gt_mcr.h"
#include "intel_gt_regs.h"
#include "intel_mocs.h"
#include "intel_ring.h"
@@ -609,14 +610,17 @@ static u32 l3cc_combine(u16 low, u16 high)
0; \
i++)
-static void init_l3cc_table(struct intel_uncore *uncore,
+static void init_l3cc_table(struct intel_gt *gt,
const struct drm_i915_mocs_table *table)
{
unsigned int i;
u32 l3cc;
for_each_l3cc(l3cc, table, i)
- intel_uncore_write_fw(uncore, GEN9_LNCFCMOCS(i), l3cc);
+ if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 50))
+ intel_gt_mcr_multicast_write_fw(gt, XEHP_LNCFCMOCS(i), l3cc);
+ else
+ intel_uncore_write_fw(gt->uncore, GEN9_LNCFCMOCS(i), l3cc);
}
void intel_mocs_init_engine(struct intel_engine_cs *engine)
@@ -636,7 +640,7 @@ void intel_mocs_init_engine(struct intel_engine_cs *engine)
init_mocs_table(engine, &table);
if (flags & HAS_RENDER_L3CC && engine->class == RENDER_CLASS)
- init_l3cc_table(engine->uncore, &table);
+ init_l3cc_table(engine->gt, &table);
}
static u32 global_mocs_offset(void)
@@ -672,7 +676,7 @@ void intel_mocs_init(struct intel_gt *gt)
* memory transactions including guc transactions
*/
if (flags & HAS_RENDER_L3CC)
- init_l3cc_table(gt->uncore, &table);
+ init_l3cc_table(gt, &table);
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
diff --git a/drivers/gpu/drm/i915/gt/intel_rc6.c b/drivers/gpu/drm/i915/gt/intel_rc6.c
index f8d0523..2ee4051e 100644
--- a/drivers/gpu/drm/i915/gt/intel_rc6.c
+++ b/drivers/gpu/drm/i915/gt/intel_rc6.c
@@ -551,6 +551,23 @@ static void __intel_rc6_disable(struct intel_rc6 *rc6)
intel_uncore_forcewake_put(uncore, FORCEWAKE_ALL);
}
+static void rc6_res_reg_init(struct intel_rc6 *rc6)
+{
+ memset(rc6->res_reg, INVALID_MMIO_REG.reg, sizeof(rc6->res_reg));
+
+ switch (rc6_to_gt(rc6)->type) {
+ case GT_MEDIA:
+ rc6->res_reg[INTEL_RC6_RES_RC6] = MTL_MEDIA_MC6;
+ break;
+ default:
+ rc6->res_reg[INTEL_RC6_RES_RC6_LOCKED] = GEN6_GT_GFX_RC6_LOCKED;
+ rc6->res_reg[INTEL_RC6_RES_RC6] = GEN6_GT_GFX_RC6;
+ rc6->res_reg[INTEL_RC6_RES_RC6p] = GEN6_GT_GFX_RC6p;
+ rc6->res_reg[INTEL_RC6_RES_RC6pp] = GEN6_GT_GFX_RC6pp;
+ break;
+ }
+}
+
void intel_rc6_init(struct intel_rc6 *rc6)
{
struct drm_i915_private *i915 = rc6_to_i915(rc6);
@@ -562,6 +579,8 @@ void intel_rc6_init(struct intel_rc6 *rc6)
if (!rc6_supported(rc6))
return;
+ rc6_res_reg_init(rc6);
+
if (IS_CHERRYVIEW(i915))
err = chv_rc6_init(rc6);
else if (IS_VALLEYVIEW(i915))
@@ -736,31 +755,19 @@ static u64 vlv_residency_raw(struct intel_uncore *uncore, const i915_reg_t reg)
return lower | (u64)upper << 8;
}
-u64 intel_rc6_residency_ns(struct intel_rc6 *rc6, const i915_reg_t reg)
+u64 intel_rc6_residency_ns(struct intel_rc6 *rc6, enum intel_rc6_res_type id)
{
struct drm_i915_private *i915 = rc6_to_i915(rc6);
struct intel_uncore *uncore = rc6_to_uncore(rc6);
u64 time_hw, prev_hw, overflow_hw;
+ i915_reg_t reg = rc6->res_reg[id];
unsigned int fw_domains;
unsigned long flags;
- unsigned int i;
u32 mul, div;
if (!rc6->supported)
return 0;
- /*
- * Store previous hw counter values for counter wrap-around handling.
- *
- * There are only four interesting registers and they live next to each
- * other so we can use the relative address, compared to the smallest
- * one as the index into driver storage.
- */
- i = (i915_mmio_reg_offset(reg) -
- i915_mmio_reg_offset(GEN6_GT_GFX_RC6_LOCKED)) / sizeof(u32);
- if (drm_WARN_ON_ONCE(&i915->drm, i >= ARRAY_SIZE(rc6->cur_residency)))
- return 0;
-
fw_domains = intel_uncore_forcewake_for_reg(uncore, reg, FW_REG_READ);
spin_lock_irqsave(&uncore->lock, flags);
@@ -789,11 +796,11 @@ u64 intel_rc6_residency_ns(struct intel_rc6 *rc6, const i915_reg_t reg)
/*
* Counter wrap handling.
*
- * But relying on a sufficient frequency of queries otherwise counters
- * can still wrap.
+ * Store previous hw counter values for counter wrap-around handling. But
+ * relying on a sufficient frequency of queries otherwise counters can still wrap.
*/
- prev_hw = rc6->prev_hw_residency[i];
- rc6->prev_hw_residency[i] = time_hw;
+ prev_hw = rc6->prev_hw_residency[id];
+ rc6->prev_hw_residency[id] = time_hw;
/* RC6 delta from last sample. */
if (time_hw >= prev_hw)
@@ -802,8 +809,8 @@ u64 intel_rc6_residency_ns(struct intel_rc6 *rc6, const i915_reg_t reg)
time_hw += overflow_hw - prev_hw;
/* Add delta to RC6 extended raw driver copy. */
- time_hw += rc6->cur_residency[i];
- rc6->cur_residency[i] = time_hw;
+ time_hw += rc6->cur_residency[id];
+ rc6->cur_residency[id] = time_hw;
intel_uncore_forcewake_put__locked(uncore, fw_domains);
spin_unlock_irqrestore(&uncore->lock, flags);
@@ -811,9 +818,22 @@ u64 intel_rc6_residency_ns(struct intel_rc6 *rc6, const i915_reg_t reg)
return mul_u64_u32_div(time_hw, mul, div);
}
-u64 intel_rc6_residency_us(struct intel_rc6 *rc6, i915_reg_t reg)
+u64 intel_rc6_residency_us(struct intel_rc6 *rc6, enum intel_rc6_res_type id)
{
- return DIV_ROUND_UP_ULL(intel_rc6_residency_ns(rc6, reg), 1000);
+ return DIV_ROUND_UP_ULL(intel_rc6_residency_ns(rc6, id), 1000);
+}
+
+void intel_rc6_print_residency(struct seq_file *m, const char *title,
+ enum intel_rc6_res_type id)
+{
+ struct intel_gt *gt = m->private;
+ i915_reg_t reg = gt->rc6.res_reg[id];
+ intel_wakeref_t wakeref;
+
+ with_intel_runtime_pm(gt->uncore->rpm, wakeref)
+ seq_printf(m, "%s %u (%llu us)\n", title,
+ intel_uncore_read(gt->uncore, reg),
+ intel_rc6_residency_us(>->rc6, id));
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
diff --git a/drivers/gpu/drm/i915/gt/intel_rc6.h b/drivers/gpu/drm/i915/gt/intel_rc6.h
index b6fea71..456fa66 100644
--- a/drivers/gpu/drm/i915/gt/intel_rc6.h
+++ b/drivers/gpu/drm/i915/gt/intel_rc6.h
@@ -6,10 +6,11 @@
#ifndef INTEL_RC6_H
#define INTEL_RC6_H
-#include "i915_reg_defs.h"
+#include <linux/types.h>
-struct intel_engine_cs;
+enum intel_rc6_res_type;
struct intel_rc6;
+struct seq_file;
void intel_rc6_init(struct intel_rc6 *rc6);
void intel_rc6_fini(struct intel_rc6 *rc6);
@@ -21,7 +22,9 @@ void intel_rc6_sanitize(struct intel_rc6 *rc6);
void intel_rc6_enable(struct intel_rc6 *rc6);
void intel_rc6_disable(struct intel_rc6 *rc6);
-u64 intel_rc6_residency_ns(struct intel_rc6 *rc6, i915_reg_t reg);
-u64 intel_rc6_residency_us(struct intel_rc6 *rc6, i915_reg_t reg);
+u64 intel_rc6_residency_ns(struct intel_rc6 *rc6, enum intel_rc6_res_type id);
+u64 intel_rc6_residency_us(struct intel_rc6 *rc6, enum intel_rc6_res_type id);
+void intel_rc6_print_residency(struct seq_file *m, const char *title,
+ enum intel_rc6_res_type id);
#endif /* INTEL_RC6_H */
diff --git a/drivers/gpu/drm/i915/gt/intel_rc6_types.h b/drivers/gpu/drm/i915/gt/intel_rc6_types.h
index e747492..fa23c4d 100644
--- a/drivers/gpu/drm/i915/gt/intel_rc6_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_rc6_types.h
@@ -13,9 +13,20 @@
struct drm_i915_gem_object;
+/* RC6 residency types */
+enum intel_rc6_res_type {
+ INTEL_RC6_RES_RC6_LOCKED,
+ INTEL_RC6_RES_RC6,
+ INTEL_RC6_RES_RC6p,
+ INTEL_RC6_RES_RC6pp,
+ INTEL_RC6_RES_MAX,
+ INTEL_RC6_RES_VLV_MEDIA = INTEL_RC6_RES_RC6p,
+};
+
struct intel_rc6 {
- u64 prev_hw_residency[4];
- u64 cur_residency[4];
+ i915_reg_t res_reg[INTEL_RC6_RES_MAX];
+ u64 prev_hw_residency[INTEL_RC6_RES_MAX];
+ u64 cur_residency[INTEL_RC6_RES_MAX];
u32 ctl_enable;
diff --git a/drivers/gpu/drm/i915/gt/intel_renderstate.c b/drivers/gpu/drm/i915/gt/intel_renderstate.c
index 5121e6d..9c1ae07 100644
--- a/drivers/gpu/drm/i915/gt/intel_renderstate.c
+++ b/drivers/gpu/drm/i915/gt/intel_renderstate.c
@@ -215,9 +215,7 @@ int intel_renderstate_emit(struct intel_renderstate *so,
if (!so->vma)
return 0;
- err = i915_request_await_object(rq, so->vma->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(so->vma, rq, 0);
+ err = i915_vma_move_to_active(so->vma, rq, 0);
if (err)
return err;
diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c
index b366743..24736eb 100644
--- a/drivers/gpu/drm/i915/gt/intel_reset.c
+++ b/drivers/gpu/drm/i915/gt/intel_reset.c
@@ -1278,7 +1278,7 @@ static void intel_gt_reset_global(struct intel_gt *gt,
kobject_uevent_env(kobj, KOBJ_CHANGE, reset_event);
/* Use a watchdog to ensure that our reset completes */
- intel_wedge_on_timeout(&w, gt, 5 * HZ) {
+ intel_wedge_on_timeout(&w, gt, 60 * HZ) {
intel_display_prepare_reset(gt->i915);
intel_gt_reset(gt, engine_mask, reason);
@@ -1407,15 +1407,19 @@ void intel_gt_handle_error(struct intel_gt *gt,
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
}
-int intel_gt_reset_trylock(struct intel_gt *gt, int *srcu)
+static int _intel_gt_reset_lock(struct intel_gt *gt, int *srcu, bool retry)
{
might_lock(>->reset.backoff_srcu);
- might_sleep();
+ if (retry)
+ might_sleep();
rcu_read_lock();
while (test_bit(I915_RESET_BACKOFF, >->reset.flags)) {
rcu_read_unlock();
+ if (!retry)
+ return -EBUSY;
+
if (wait_event_interruptible(gt->reset.queue,
!test_bit(I915_RESET_BACKOFF,
>->reset.flags)))
@@ -1429,6 +1433,16 @@ int intel_gt_reset_trylock(struct intel_gt *gt, int *srcu)
return 0;
}
+int intel_gt_reset_trylock(struct intel_gt *gt, int *srcu)
+{
+ return _intel_gt_reset_lock(gt, srcu, false);
+}
+
+int intel_gt_reset_lock_interruptible(struct intel_gt *gt, int *srcu)
+{
+ return _intel_gt_reset_lock(gt, srcu, true);
+}
+
void intel_gt_reset_unlock(struct intel_gt *gt, int tag)
__releases(>->reset.backoff_srcu)
{
diff --git a/drivers/gpu/drm/i915/gt/intel_reset.h b/drivers/gpu/drm/i915/gt/intel_reset.h
index adc734e..25c975b 100644
--- a/drivers/gpu/drm/i915/gt/intel_reset.h
+++ b/drivers/gpu/drm/i915/gt/intel_reset.h
@@ -39,6 +39,7 @@ int __intel_engine_reset_bh(struct intel_engine_cs *engine,
void __i915_request_reset(struct i915_request *rq, bool guilty);
int __must_check intel_gt_reset_trylock(struct intel_gt *gt, int *srcu);
+int __must_check intel_gt_reset_lock_interruptible(struct intel_gt *gt, int *srcu);
void intel_gt_reset_unlock(struct intel_gt *gt, int tag);
void intel_gt_set_wedged(struct intel_gt *gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
index d5d6f1f..356c787 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
@@ -12,7 +12,9 @@
#include "gen6_ppgtt.h"
#include "gen7_renderclear.h"
#include "i915_drv.h"
+#include "i915_irq.h"
#include "i915_mitigations.h"
+#include "i915_reg.h"
#include "intel_breadcrumbs.h"
#include "intel_context.h"
#include "intel_engine_regs.h"
diff --git a/drivers/gpu/drm/i915/gt/intel_rps.c b/drivers/gpu/drm/i915/gt/intel_rps.c
index 6b86250..9ad3bc7 100644
--- a/drivers/gpu/drm/i915/gt/intel_rps.c
+++ b/drivers/gpu/drm/i915/gt/intel_rps.c
@@ -430,7 +430,8 @@ static int __gen5_rps_set(struct intel_rps *rps, u8 val)
rgvswctl = intel_uncore_read16(uncore, MEMSWCTL);
if (rgvswctl & MEMCTL_CMD_STS) {
- DRM_DEBUG("gpu busy, RCS change rejected\n");
+ drm_dbg(&rps_to_i915(rps)->drm,
+ "gpu busy, RCS change rejected\n");
return -EBUSY; /* still busy with another command */
}
@@ -625,9 +626,7 @@ static void gen5_rps_disable(struct intel_rps *rps)
rgvswctl = intel_uncore_read16(uncore, MEMSWCTL);
/* Ack interrupts, disable EFC interrupt */
- intel_uncore_write(uncore, MEMINTREN,
- intel_uncore_read(uncore, MEMINTREN) &
- ~MEMINT_EVAL_CHG_EN);
+ intel_uncore_rmw(uncore, MEMINTREN, MEMINT_EVAL_CHG_EN, 0);
intel_uncore_write(uncore, MEMINTRSTS, MEMINT_EVAL_CHG);
/* Go back to the starting frequency */
@@ -1016,9 +1015,15 @@ void intel_rps_boost(struct i915_request *rq)
if (rps_uses_slpc(rps)) {
slpc = rps_to_slpc(rps);
+ if (slpc->min_freq_softlimit >= slpc->boost_freq)
+ return;
+
/* Return if old value is non zero */
- if (!atomic_fetch_inc(&slpc->num_waiters))
+ if (!atomic_fetch_inc(&slpc->num_waiters)) {
+ GT_TRACE(rps_to_gt(rps), "boost fence:%llx:%llx\n",
+ rq->fence.context, rq->fence.seqno);
schedule_work(&slpc->boost_work);
+ }
return;
}
@@ -1085,15 +1090,25 @@ static u32 intel_rps_read_state_cap(struct intel_rps *rps)
return intel_uncore_read(uncore, GEN6_RP_STATE_CAP);
}
-/**
- * gen6_rps_get_freq_caps - Get freq caps exposed by HW
- * @rps: the intel_rps structure
- * @caps: returned freq caps
- *
- * Returned "caps" frequencies should be converted to MHz using
- * intel_gpu_freq()
- */
-void gen6_rps_get_freq_caps(struct intel_rps *rps, struct intel_rps_freq_caps *caps)
+static void
+mtl_get_freq_caps(struct intel_rps *rps, struct intel_rps_freq_caps *caps)
+{
+ struct intel_uncore *uncore = rps_to_uncore(rps);
+ u32 rp_state_cap = rps_to_gt(rps)->type == GT_MEDIA ?
+ intel_uncore_read(uncore, MTL_MEDIAP_STATE_CAP) :
+ intel_uncore_read(uncore, MTL_RP_STATE_CAP);
+ u32 rpe = rps_to_gt(rps)->type == GT_MEDIA ?
+ intel_uncore_read(uncore, MTL_MPE_FREQUENCY) :
+ intel_uncore_read(uncore, MTL_GT_RPE_FREQUENCY);
+
+ /* MTL values are in units of 16.67 MHz */
+ caps->rp0_freq = REG_FIELD_GET(MTL_RP0_CAP_MASK, rp_state_cap);
+ caps->min_freq = REG_FIELD_GET(MTL_RPN_CAP_MASK, rp_state_cap);
+ caps->rp1_freq = REG_FIELD_GET(MTL_RPE_MASK, rpe);
+}
+
+static void
+__gen6_rps_get_freq_caps(struct intel_rps *rps, struct intel_rps_freq_caps *caps)
{
struct drm_i915_private *i915 = rps_to_i915(rps);
u32 rp_state_cap;
@@ -1128,6 +1143,24 @@ void gen6_rps_get_freq_caps(struct intel_rps *rps, struct intel_rps_freq_caps *c
}
}
+/**
+ * gen6_rps_get_freq_caps - Get freq caps exposed by HW
+ * @rps: the intel_rps structure
+ * @caps: returned freq caps
+ *
+ * Returned "caps" frequencies should be converted to MHz using
+ * intel_gpu_freq()
+ */
+void gen6_rps_get_freq_caps(struct intel_rps *rps, struct intel_rps_freq_caps *caps)
+{
+ struct drm_i915_private *i915 = rps_to_i915(rps);
+
+ if (IS_METEORLAKE(i915))
+ return mtl_get_freq_caps(rps, caps);
+ else
+ return __gen6_rps_get_freq_caps(rps, caps);
+}
+
static void gen6_rps_init(struct intel_rps *rps)
{
struct drm_i915_private *i915 = rps_to_i915(rps);
@@ -1921,7 +1954,8 @@ void gen6_rps_irq_handler(struct intel_rps *rps, u32 pm_iir)
intel_engine_cs_irq(gt->engine[VECS0], pm_iir >> 10);
if (pm_iir & PM_VEBOX_CS_ERROR_INTERRUPT)
- DRM_DEBUG("Command parser error, pm_iir 0x%08x\n", pm_iir);
+ drm_dbg(&rps_to_i915(rps)->drm,
+ "Command parser error, pm_iir 0x%08x\n", pm_iir);
}
void gen5_rps_irq_handler(struct intel_rps *rps)
@@ -2040,22 +2074,45 @@ void intel_rps_sanitize(struct intel_rps *rps)
rps_disable_interrupts(rps);
}
+u32 intel_rps_read_rpstat_fw(struct intel_rps *rps)
+{
+ struct drm_i915_private *i915 = rps_to_i915(rps);
+ i915_reg_t rpstat;
+
+ rpstat = (GRAPHICS_VER(i915) >= 12) ? GEN12_RPSTAT1 : GEN6_RPSTAT1;
+
+ return intel_uncore_read_fw(rps_to_gt(rps)->uncore, rpstat);
+}
+
+u32 intel_rps_read_rpstat(struct intel_rps *rps)
+{
+ struct drm_i915_private *i915 = rps_to_i915(rps);
+ i915_reg_t rpstat;
+
+ rpstat = (GRAPHICS_VER(i915) >= 12) ? GEN12_RPSTAT1 : GEN6_RPSTAT1;
+
+ return intel_uncore_read(rps_to_gt(rps)->uncore, rpstat);
+}
+
u32 intel_rps_get_cagf(struct intel_rps *rps, u32 rpstat)
{
struct drm_i915_private *i915 = rps_to_i915(rps);
u32 cagf;
- if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915))
- cagf = (rpstat >> 8) & 0xff;
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70))
+ cagf = REG_FIELD_GET(MTL_CAGF_MASK, rpstat);
+ else if (GRAPHICS_VER(i915) >= 12)
+ cagf = REG_FIELD_GET(GEN12_CAGF_MASK, rpstat);
+ else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915))
+ cagf = REG_FIELD_GET(RPE_MASK, rpstat);
else if (GRAPHICS_VER(i915) >= 9)
- cagf = (rpstat & GEN9_CAGF_MASK) >> GEN9_CAGF_SHIFT;
+ cagf = REG_FIELD_GET(GEN9_CAGF_MASK, rpstat);
else if (IS_HASWELL(i915) || IS_BROADWELL(i915))
- cagf = (rpstat & HSW_CAGF_MASK) >> HSW_CAGF_SHIFT;
+ cagf = REG_FIELD_GET(HSW_CAGF_MASK, rpstat);
else if (GRAPHICS_VER(i915) >= 6)
- cagf = (rpstat & GEN6_CAGF_MASK) >> GEN6_CAGF_SHIFT;
+ cagf = REG_FIELD_GET(GEN6_CAGF_MASK, rpstat);
else
- cagf = gen5_invert_freq(rps, (rpstat & MEMSTAT_PSTATE_MASK) >>
- MEMSTAT_PSTATE_SHIFT);
+ cagf = gen5_invert_freq(rps, REG_FIELD_GET(MEMSTAT_PSTATE_MASK, rpstat));
return cagf;
}
@@ -2066,7 +2123,15 @@ static u32 read_cagf(struct intel_rps *rps)
struct intel_uncore *uncore = rps_to_uncore(rps);
u32 freq;
- if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) {
+ /*
+ * For Gen12+ reading freq from HW does not need a forcewake and
+ * registers will return 0 freq when GT is in RC6
+ */
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70)) {
+ freq = intel_uncore_read(uncore, MTL_MIRROR_TARGET_WP1);
+ } else if (GRAPHICS_VER(i915) >= 12) {
+ freq = intel_uncore_read(uncore, GEN12_RPSTAT1);
+ } else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) {
vlv_punit_get(i915);
freq = vlv_punit_read(i915, PUNIT_REG_GPU_FREQ_STS);
vlv_punit_put(i915);
@@ -2191,6 +2256,213 @@ u32 intel_rps_get_rpn_frequency(struct intel_rps *rps)
return intel_gpu_freq(rps, rps->min_freq);
}
+static void rps_frequency_dump(struct intel_rps *rps, struct drm_printer *p)
+{
+ struct intel_gt *gt = rps_to_gt(rps);
+ struct drm_i915_private *i915 = gt->i915;
+ struct intel_uncore *uncore = gt->uncore;
+ struct intel_rps_freq_caps caps;
+ u32 rp_state_limits;
+ u32 gt_perf_status;
+ u32 rpmodectl, rpinclimit, rpdeclimit;
+ u32 rpstat, cagf, reqf;
+ u32 rpcurupei, rpcurup, rpprevup;
+ u32 rpcurdownei, rpcurdown, rpprevdown;
+ u32 rpupei, rpupt, rpdownei, rpdownt;
+ u32 pm_ier, pm_imr, pm_isr, pm_iir, pm_mask;
+
+ rp_state_limits = intel_uncore_read(uncore, GEN6_RP_STATE_LIMITS);
+ gen6_rps_get_freq_caps(rps, &caps);
+ if (IS_GEN9_LP(i915))
+ gt_perf_status = intel_uncore_read(uncore, BXT_GT_PERF_STATUS);
+ else
+ gt_perf_status = intel_uncore_read(uncore, GEN6_GT_PERF_STATUS);
+
+ /* RPSTAT1 is in the GT power well */
+ intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL);
+
+ reqf = intel_uncore_read(uncore, GEN6_RPNSWREQ);
+ if (GRAPHICS_VER(i915) >= 9) {
+ reqf >>= 23;
+ } else {
+ reqf &= ~GEN6_TURBO_DISABLE;
+ if (IS_HASWELL(i915) || IS_BROADWELL(i915))
+ reqf >>= 24;
+ else
+ reqf >>= 25;
+ }
+ reqf = intel_gpu_freq(rps, reqf);
+
+ rpmodectl = intel_uncore_read(uncore, GEN6_RP_CONTROL);
+ rpinclimit = intel_uncore_read(uncore, GEN6_RP_UP_THRESHOLD);
+ rpdeclimit = intel_uncore_read(uncore, GEN6_RP_DOWN_THRESHOLD);
+
+ rpstat = intel_rps_read_rpstat(rps);
+ rpcurupei = intel_uncore_read(uncore, GEN6_RP_CUR_UP_EI) & GEN6_CURICONT_MASK;
+ rpcurup = intel_uncore_read(uncore, GEN6_RP_CUR_UP) & GEN6_CURBSYTAVG_MASK;
+ rpprevup = intel_uncore_read(uncore, GEN6_RP_PREV_UP) & GEN6_CURBSYTAVG_MASK;
+ rpcurdownei = intel_uncore_read(uncore, GEN6_RP_CUR_DOWN_EI) & GEN6_CURIAVG_MASK;
+ rpcurdown = intel_uncore_read(uncore, GEN6_RP_CUR_DOWN) & GEN6_CURBSYTAVG_MASK;
+ rpprevdown = intel_uncore_read(uncore, GEN6_RP_PREV_DOWN) & GEN6_CURBSYTAVG_MASK;
+
+ rpupei = intel_uncore_read(uncore, GEN6_RP_UP_EI);
+ rpupt = intel_uncore_read(uncore, GEN6_RP_UP_THRESHOLD);
+
+ rpdownei = intel_uncore_read(uncore, GEN6_RP_DOWN_EI);
+ rpdownt = intel_uncore_read(uncore, GEN6_RP_DOWN_THRESHOLD);
+
+ cagf = intel_rps_read_actual_frequency(rps);
+
+ intel_uncore_forcewake_put(uncore, FORCEWAKE_ALL);
+
+ if (GRAPHICS_VER(i915) >= 11) {
+ pm_ier = intel_uncore_read(uncore, GEN11_GPM_WGBOXPERF_INTR_ENABLE);
+ pm_imr = intel_uncore_read(uncore, GEN11_GPM_WGBOXPERF_INTR_MASK);
+ /*
+ * The equivalent to the PM ISR & IIR cannot be read
+ * without affecting the current state of the system
+ */
+ pm_isr = 0;
+ pm_iir = 0;
+ } else if (GRAPHICS_VER(i915) >= 8) {
+ pm_ier = intel_uncore_read(uncore, GEN8_GT_IER(2));
+ pm_imr = intel_uncore_read(uncore, GEN8_GT_IMR(2));
+ pm_isr = intel_uncore_read(uncore, GEN8_GT_ISR(2));
+ pm_iir = intel_uncore_read(uncore, GEN8_GT_IIR(2));
+ } else {
+ pm_ier = intel_uncore_read(uncore, GEN6_PMIER);
+ pm_imr = intel_uncore_read(uncore, GEN6_PMIMR);
+ pm_isr = intel_uncore_read(uncore, GEN6_PMISR);
+ pm_iir = intel_uncore_read(uncore, GEN6_PMIIR);
+ }
+ pm_mask = intel_uncore_read(uncore, GEN6_PMINTRMSK);
+
+ drm_printf(p, "Video Turbo Mode: %s\n",
+ str_yes_no(rpmodectl & GEN6_RP_MEDIA_TURBO));
+ drm_printf(p, "HW control enabled: %s\n",
+ str_yes_no(rpmodectl & GEN6_RP_ENABLE));
+ drm_printf(p, "SW control enabled: %s\n",
+ str_yes_no((rpmodectl & GEN6_RP_MEDIA_MODE_MASK) == GEN6_RP_MEDIA_SW_MODE));
+
+ drm_printf(p, "PM IER=0x%08x IMR=0x%08x, MASK=0x%08x\n",
+ pm_ier, pm_imr, pm_mask);
+ if (GRAPHICS_VER(i915) <= 10)
+ drm_printf(p, "PM ISR=0x%08x IIR=0x%08x\n",
+ pm_isr, pm_iir);
+ drm_printf(p, "pm_intrmsk_mbz: 0x%08x\n",
+ rps->pm_intrmsk_mbz);
+ drm_printf(p, "GT_PERF_STATUS: 0x%08x\n", gt_perf_status);
+ drm_printf(p, "Render p-state ratio: %d\n",
+ (gt_perf_status & (GRAPHICS_VER(i915) >= 9 ? 0x1ff00 : 0xff00)) >> 8);
+ drm_printf(p, "Render p-state VID: %d\n",
+ gt_perf_status & 0xff);
+ drm_printf(p, "Render p-state limit: %d\n",
+ rp_state_limits & 0xff);
+ drm_printf(p, "RPSTAT1: 0x%08x\n", rpstat);
+ drm_printf(p, "RPMODECTL: 0x%08x\n", rpmodectl);
+ drm_printf(p, "RPINCLIMIT: 0x%08x\n", rpinclimit);
+ drm_printf(p, "RPDECLIMIT: 0x%08x\n", rpdeclimit);
+ drm_printf(p, "RPNSWREQ: %dMHz\n", reqf);
+ drm_printf(p, "CAGF: %dMHz\n", cagf);
+ drm_printf(p, "RP CUR UP EI: %d (%lldns)\n",
+ rpcurupei,
+ intel_gt_pm_interval_to_ns(gt, rpcurupei));
+ drm_printf(p, "RP CUR UP: %d (%lldns)\n",
+ rpcurup, intel_gt_pm_interval_to_ns(gt, rpcurup));
+ drm_printf(p, "RP PREV UP: %d (%lldns)\n",
+ rpprevup, intel_gt_pm_interval_to_ns(gt, rpprevup));
+ drm_printf(p, "Up threshold: %d%%\n",
+ rps->power.up_threshold);
+ drm_printf(p, "RP UP EI: %d (%lldns)\n",
+ rpupei, intel_gt_pm_interval_to_ns(gt, rpupei));
+ drm_printf(p, "RP UP THRESHOLD: %d (%lldns)\n",
+ rpupt, intel_gt_pm_interval_to_ns(gt, rpupt));
+
+ drm_printf(p, "RP CUR DOWN EI: %d (%lldns)\n",
+ rpcurdownei,
+ intel_gt_pm_interval_to_ns(gt, rpcurdownei));
+ drm_printf(p, "RP CUR DOWN: %d (%lldns)\n",
+ rpcurdown,
+ intel_gt_pm_interval_to_ns(gt, rpcurdown));
+ drm_printf(p, "RP PREV DOWN: %d (%lldns)\n",
+ rpprevdown,
+ intel_gt_pm_interval_to_ns(gt, rpprevdown));
+ drm_printf(p, "Down threshold: %d%%\n",
+ rps->power.down_threshold);
+ drm_printf(p, "RP DOWN EI: %d (%lldns)\n",
+ rpdownei, intel_gt_pm_interval_to_ns(gt, rpdownei));
+ drm_printf(p, "RP DOWN THRESHOLD: %d (%lldns)\n",
+ rpdownt, intel_gt_pm_interval_to_ns(gt, rpdownt));
+
+ drm_printf(p, "Lowest (RPN) frequency: %dMHz\n",
+ intel_gpu_freq(rps, caps.min_freq));
+ drm_printf(p, "Nominal (RP1) frequency: %dMHz\n",
+ intel_gpu_freq(rps, caps.rp1_freq));
+ drm_printf(p, "Max non-overclocked (RP0) frequency: %dMHz\n",
+ intel_gpu_freq(rps, caps.rp0_freq));
+ drm_printf(p, "Max overclocked frequency: %dMHz\n",
+ intel_gpu_freq(rps, rps->max_freq));
+
+ drm_printf(p, "Current freq: %d MHz\n",
+ intel_gpu_freq(rps, rps->cur_freq));
+ drm_printf(p, "Actual freq: %d MHz\n", cagf);
+ drm_printf(p, "Idle freq: %d MHz\n",
+ intel_gpu_freq(rps, rps->idle_freq));
+ drm_printf(p, "Min freq: %d MHz\n",
+ intel_gpu_freq(rps, rps->min_freq));
+ drm_printf(p, "Boost freq: %d MHz\n",
+ intel_gpu_freq(rps, rps->boost_freq));
+ drm_printf(p, "Max freq: %d MHz\n",
+ intel_gpu_freq(rps, rps->max_freq));
+ drm_printf(p,
+ "efficient (RPe) frequency: %d MHz\n",
+ intel_gpu_freq(rps, rps->efficient_freq));
+}
+
+static void slpc_frequency_dump(struct intel_rps *rps, struct drm_printer *p)
+{
+ struct intel_gt *gt = rps_to_gt(rps);
+ struct intel_uncore *uncore = gt->uncore;
+ struct intel_rps_freq_caps caps;
+ u32 pm_mask;
+
+ gen6_rps_get_freq_caps(rps, &caps);
+ pm_mask = intel_uncore_read(uncore, GEN6_PMINTRMSK);
+
+ drm_printf(p, "PM MASK=0x%08x\n", pm_mask);
+ drm_printf(p, "pm_intrmsk_mbz: 0x%08x\n",
+ rps->pm_intrmsk_mbz);
+ drm_printf(p, "RPSTAT1: 0x%08x\n", intel_rps_read_rpstat(rps));
+ drm_printf(p, "RPNSWREQ: %dMHz\n", intel_rps_get_requested_frequency(rps));
+ drm_printf(p, "Lowest (RPN) frequency: %dMHz\n",
+ intel_gpu_freq(rps, caps.min_freq));
+ drm_printf(p, "Nominal (RP1) frequency: %dMHz\n",
+ intel_gpu_freq(rps, caps.rp1_freq));
+ drm_printf(p, "Max non-overclocked (RP0) frequency: %dMHz\n",
+ intel_gpu_freq(rps, caps.rp0_freq));
+ drm_printf(p, "Current freq: %d MHz\n",
+ intel_rps_get_requested_frequency(rps));
+ drm_printf(p, "Actual freq: %d MHz\n",
+ intel_rps_read_actual_frequency(rps));
+ drm_printf(p, "Min freq: %d MHz\n",
+ intel_rps_get_min_frequency(rps));
+ drm_printf(p, "Boost freq: %d MHz\n",
+ intel_rps_get_boost_frequency(rps));
+ drm_printf(p, "Max freq: %d MHz\n",
+ intel_rps_get_max_frequency(rps));
+ drm_printf(p,
+ "efficient (RPe) frequency: %d MHz\n",
+ intel_gpu_freq(rps, caps.rp1_freq));
+}
+
+void gen6_rps_frequency_dump(struct intel_rps *rps, struct drm_printer *p)
+{
+ if (rps_uses_slpc(rps))
+ return slpc_frequency_dump(rps, p);
+ else
+ return rps_frequency_dump(rps, p);
+}
+
static int set_max_freq(struct intel_rps *rps, u32 val)
{
struct drm_i915_private *i915 = rps_to_i915(rps);
diff --git a/drivers/gpu/drm/i915/gt/intel_rps.h b/drivers/gpu/drm/i915/gt/intel_rps.h
index 4509dfd..9e1cad9 100644
--- a/drivers/gpu/drm/i915/gt/intel_rps.h
+++ b/drivers/gpu/drm/i915/gt/intel_rps.h
@@ -10,6 +10,7 @@
#include "i915_reg_defs.h"
struct i915_request;
+struct drm_printer;
void intel_rps_init_early(struct intel_rps *rps);
void intel_rps_init(struct intel_rps *rps);
@@ -47,6 +48,8 @@ u32 intel_rps_get_rp1_frequency(struct intel_rps *rps);
u32 intel_rps_get_rpn_frequency(struct intel_rps *rps);
u32 intel_rps_read_punit_req(struct intel_rps *rps);
u32 intel_rps_read_punit_req_frequency(struct intel_rps *rps);
+u32 intel_rps_read_rpstat(struct intel_rps *rps);
+u32 intel_rps_read_rpstat_fw(struct intel_rps *rps);
void gen6_rps_get_freq_caps(struct intel_rps *rps, struct intel_rps_freq_caps *caps);
void intel_rps_raise_unslice(struct intel_rps *rps);
void intel_rps_lower_unslice(struct intel_rps *rps);
@@ -54,6 +57,8 @@ void intel_rps_lower_unslice(struct intel_rps *rps);
u32 intel_rps_read_throttle_reason(struct intel_rps *rps);
bool rps_read_mask_mmio(struct intel_rps *rps, i915_reg_t reg32, u32 mask);
+void gen6_rps_frequency_dump(struct intel_rps *rps, struct drm_printer *p);
+
void gen5_rps_irq_handler(struct intel_rps *rps);
void gen6_rps_irq_handler(struct intel_rps *rps, u32 pm_iir);
void gen11_rps_irq_handler(struct intel_rps *rps, u32 pm_iir);
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c
index 66f21c7..6c6198a 100644
--- a/drivers/gpu/drm/i915/gt/intel_sseu.c
+++ b/drivers/gpu/drm/i915/gt/intel_sseu.c
@@ -677,8 +677,8 @@ u32 intel_sseu_make_rpcs(struct intel_gt *gt,
* If i915/perf is active, we want a stable powergating configuration
* on the system. Use the configuration pinned by i915/perf.
*/
- if (i915->perf.exclusive_stream)
- req_sseu = &i915->perf.sseu;
+ if (gt->perf.exclusive_stream)
+ req_sseu = >->perf.sseu;
slices = hweight8(req_sseu->slice_mask);
subslices = hweight8(req_sseu->subslice_mask);
diff --git a/drivers/gpu/drm/i915/intel_wopcm.c b/drivers/gpu/drm/i915/gt/intel_wopcm.c
similarity index 87%
rename from drivers/gpu/drm/i915/intel_wopcm.c
rename to drivers/gpu/drm/i915/gt/intel_wopcm.c
index 322fb9e..7ebbcc1 100644
--- a/drivers/gpu/drm/i915/intel_wopcm.c
+++ b/drivers/gpu/drm/i915/gt/intel_wopcm.c
@@ -64,9 +64,9 @@
#define GEN9_GUC_FW_RESERVED SZ_128K
#define GEN9_GUC_WOPCM_OFFSET (GUC_WOPCM_RESERVED + GEN9_GUC_FW_RESERVED)
-static inline struct drm_i915_private *wopcm_to_i915(struct intel_wopcm *wopcm)
+static inline struct intel_gt *wopcm_to_gt(struct intel_wopcm *wopcm)
{
- return container_of(wopcm, struct drm_i915_private, wopcm);
+ return container_of(wopcm, struct intel_gt, wopcm);
}
/**
@@ -77,7 +77,8 @@ static inline struct drm_i915_private *wopcm_to_i915(struct intel_wopcm *wopcm)
*/
void intel_wopcm_init_early(struct intel_wopcm *wopcm)
{
- struct drm_i915_private *i915 = wopcm_to_i915(wopcm);
+ struct intel_gt *gt = wopcm_to_gt(wopcm);
+ struct drm_i915_private *i915 = gt->i915;
if (!HAS_GT_UC(i915))
return;
@@ -157,10 +158,11 @@ static bool check_hw_restrictions(struct drm_i915_private *i915,
return true;
}
-static bool __check_layout(struct drm_i915_private *i915, u32 wopcm_size,
+static bool __check_layout(struct intel_gt *gt, u32 wopcm_size,
u32 guc_wopcm_base, u32 guc_wopcm_size,
u32 guc_fw_size, u32 huc_fw_size)
{
+ struct drm_i915_private *i915 = gt->i915;
const u32 ctx_rsvd = context_reserved_size(i915);
u32 size;
@@ -181,12 +183,14 @@ static bool __check_layout(struct drm_i915_private *i915, u32 wopcm_size,
return false;
}
- size = huc_fw_size + WOPCM_RESERVED_SIZE;
- if (unlikely(guc_wopcm_base < size)) {
- drm_err(&i915->drm, "WOPCM: no space for %s: %uK < %uK\n",
- intel_uc_fw_type_repr(INTEL_UC_FW_TYPE_HUC),
- guc_wopcm_base / SZ_1K, size / SZ_1K);
- return false;
+ if (intel_uc_supports_huc(>->uc)) {
+ size = huc_fw_size + WOPCM_RESERVED_SIZE;
+ if (unlikely(guc_wopcm_base < size)) {
+ drm_err(&i915->drm, "WOPCM: no space for %s: %uK < %uK\n",
+ intel_uc_fw_type_repr(INTEL_UC_FW_TYPE_HUC),
+ guc_wopcm_base / SZ_1K, size / SZ_1K);
+ return false;
+ }
}
return check_hw_restrictions(i915, guc_wopcm_base, guc_wopcm_size,
@@ -228,8 +232,8 @@ static bool __wopcm_regs_writable(struct intel_uncore *uncore)
*/
void intel_wopcm_init(struct intel_wopcm *wopcm)
{
- struct drm_i915_private *i915 = wopcm_to_i915(wopcm);
- struct intel_gt *gt = to_gt(i915);
+ struct intel_gt *gt = wopcm_to_gt(wopcm);
+ struct drm_i915_private *i915 = gt->i915;
u32 guc_fw_size = intel_uc_fw_get_upload_size(>->uc.guc.fw);
u32 huc_fw_size = intel_uc_fw_get_upload_size(>->uc.huc.fw);
u32 ctx_rsvd = context_reserved_size(i915);
@@ -275,6 +279,19 @@ void intel_wopcm_init(struct intel_wopcm *wopcm)
}
/*
+ * On platforms with a media GT, the WOPCM is partitioned between the
+ * two GTs, so we would have to take that into account when doing the
+ * math below. There is also a new section reserved for the GSC context
+ * that would have to be factored in. However, all platforms with a
+ * media GT also have GuC depriv enabled, so the WOPCM regs are
+ * pre-locked and therefore we don't have to do the math ourselves.
+ */
+ if (unlikely(i915->media_gt)) {
+ drm_err(&i915->drm, "Unlocked WOPCM regs with media GT\n");
+ return;
+ }
+
+ /*
* Aligned value of guc_wopcm_base will determine available WOPCM space
* for HuC firmware and mandatory reserved area.
*/
@@ -295,7 +312,7 @@ void intel_wopcm_init(struct intel_wopcm *wopcm)
guc_wopcm_base / SZ_1K, guc_wopcm_size / SZ_1K);
check:
- if (__check_layout(i915, wopcm_size, guc_wopcm_base, guc_wopcm_size,
+ if (__check_layout(gt, wopcm_size, guc_wopcm_base, guc_wopcm_size,
guc_fw_size, huc_fw_size)) {
wopcm->guc.base = guc_wopcm_base;
wopcm->guc.size = guc_wopcm_size;
diff --git a/drivers/gpu/drm/i915/intel_wopcm.h b/drivers/gpu/drm/i915/gt/intel_wopcm.h
similarity index 100%
rename from drivers/gpu/drm/i915/intel_wopcm.h
rename to drivers/gpu/drm/i915/gt/intel_wopcm.h
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c
index a821e3d..2afb4f8 100644
--- a/drivers/gpu/drm/i915/gt/intel_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c
@@ -4,6 +4,7 @@
*/
#include "i915_drv.h"
+#include "i915_reg.h"
#include "intel_context.h"
#include "intel_engine_pm.h"
#include "intel_engine_regs.h"
@@ -17,46 +18,68 @@
/**
* DOC: Hardware workarounds
*
- * This file is intended as a central place to implement most [1]_ of the
- * required workarounds for hardware to work as originally intended. They fall
- * in five basic categories depending on how/when they are applied:
+ * Hardware workarounds are register programming documented to be executed in
+ * the driver that fall outside of the normal programming sequences for a
+ * platform. There are some basic categories of workarounds, depending on
+ * how/when they are applied:
*
- * - Workarounds that touch registers that are saved/restored to/from the HW
- * context image. The list is emitted (via Load Register Immediate commands)
- * everytime a new context is created.
- * - GT workarounds. The list of these WAs is applied whenever these registers
- * revert to default values (on GPU reset, suspend/resume [2]_, etc..).
- * - Display workarounds. The list is applied during display clock-gating
- * initialization.
- * - Workarounds that whitelist a privileged register, so that UMDs can manage
- * them directly. This is just a special case of a MMMIO workaround (as we
- * write the list of these to/be-whitelisted registers to some special HW
- * registers).
- * - Workaround batchbuffers, that get executed automatically by the hardware
- * on every HW context restore.
+ * - Context workarounds: workarounds that touch registers that are
+ * saved/restored to/from the HW context image. The list is emitted (via Load
+ * Register Immediate commands) once when initializing the device and saved in
+ * the default context. That default context is then used on every context
+ * creation to have a "primed golden context", i.e. a context image that
+ * already contains the changes needed to all the registers.
*
- * .. [1] Please notice that there are other WAs that, due to their nature,
- * cannot be applied from a central place. Those are peppered around the rest
- * of the code, as needed.
+ * - Engine workarounds: the list of these WAs is applied whenever the specific
+ * engine is reset. It's also possible that a set of engine classes share a
+ * common power domain and they are reset together. This happens on some
+ * platforms with render and compute engines. In this case (at least) one of
+ * them need to keeep the workaround programming: the approach taken in the
+ * driver is to tie those workarounds to the first compute/render engine that
+ * is registered. When executing with GuC submission, engine resets are
+ * outside of kernel driver control, hence the list of registers involved in
+ * written once, on engine initialization, and then passed to GuC, that
+ * saves/restores their values before/after the reset takes place. See
+ * ``drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c`` for reference.
*
- * .. [2] Technically, some registers are powercontext saved & restored, so they
+ * - GT workarounds: the list of these WAs is applied whenever these registers
+ * revert to their default values: on GPU reset, suspend/resume [1]_, etc.
+ *
+ * - Register whitelist: some workarounds need to be implemented in userspace,
+ * but need to touch privileged registers. The whitelist in the kernel
+ * instructs the hardware to allow the access to happen. From the kernel side,
+ * this is just a special case of a MMIO workaround (as we write the list of
+ * these to/be-whitelisted registers to some special HW registers).
+ *
+ * - Workaround batchbuffers: buffers that get executed automatically by the
+ * hardware on every HW context restore. These buffers are created and
+ * programmed in the default context so the hardware always go through those
+ * programming sequences when switching contexts. The support for workaround
+ * batchbuffers is enabled these hardware mechanisms:
+ *
+ * #. INDIRECT_CTX: A batchbuffer and an offset are provided in the default
+ * context, pointing the hardware to jump to that location when that offset
+ * is reached in the context restore. Workaround batchbuffer in the driver
+ * currently uses this mechanism for all platforms.
+ *
+ * #. BB_PER_CTX_PTR: A batchbuffer is provided in the default context,
+ * pointing the hardware to a buffer to continue executing after the
+ * engine registers are restored in a context restore sequence. This is
+ * currently not used in the driver.
+ *
+ * - Other: There are WAs that, due to their nature, cannot be applied from a
+ * central place. Those are peppered around the rest of the code, as needed.
+ * Workarounds related to the display IP are the main example.
+ *
+ * .. [1] Technically, some registers are powercontext saved & restored, so they
* survive a suspend/resume. In practice, writing them again is not too
- * costly and simplifies things. We can revisit this in the future.
- *
- * Layout
- * ~~~~~~
- *
- * Keep things in this file ordered by WA type, as per the above (context, GT,
- * display, register whitelist, batchbuffer). Then, inside each type, keep the
- * following order:
- *
- * - Infrastructure functions and macros
- * - WAs per platform in standard gen/chrono order
- * - Public functions to init or apply the given workaround type.
+ * costly and simplifies things, so it's the approach taken in the driver.
*/
-static void wa_init_start(struct i915_wa_list *wal, const char *name, const char *engine_name)
+static void wa_init_start(struct i915_wa_list *wal, struct intel_gt *gt,
+ const char *name, const char *engine_name)
{
+ wal->gt = gt;
wal->name = name;
wal->engine_name = engine_name;
}
@@ -80,13 +103,14 @@ static void wa_init_finish(struct i915_wa_list *wal)
if (!wal->count)
return;
- DRM_DEBUG_DRIVER("Initialized %u %s workarounds on %s\n",
- wal->wa_count, wal->name, wal->engine_name);
+ drm_dbg(&wal->gt->i915->drm, "Initialized %u %s workarounds on %s\n",
+ wal->wa_count, wal->name, wal->engine_name);
}
static void _wa_add(struct i915_wa_list *wal, const struct i915_wa *wa)
{
unsigned int addr = i915_mmio_reg_offset(wa->reg);
+ struct drm_i915_private *i915 = wal->gt->i915;
unsigned int start = 0, end = wal->count;
const unsigned int grow = WA_LIST_CHUNK;
struct i915_wa *wa_;
@@ -99,7 +123,7 @@ static void _wa_add(struct i915_wa_list *wal, const struct i915_wa *wa)
list = kmalloc_array(ALIGN(wal->count + 1, grow), sizeof(*wa),
GFP_KERNEL);
if (!list) {
- DRM_ERROR("No space for workaround init!\n");
+ drm_err(&i915->drm, "No space for workaround init!\n");
return;
}
@@ -122,9 +146,10 @@ static void _wa_add(struct i915_wa_list *wal, const struct i915_wa *wa)
wa_ = &wal->list[mid];
if ((wa->clr | wa_->clr) && !(wa->clr & ~wa_->clr)) {
- DRM_ERROR("Discarding overwritten w/a for reg %04x (clear: %08x, set: %08x)\n",
- i915_mmio_reg_offset(wa_->reg),
- wa_->clr, wa_->set);
+ drm_err(&i915->drm,
+ "Discarding overwritten w/a for reg %04x (clear: %08x, set: %08x)\n",
+ i915_mmio_reg_offset(wa_->reg),
+ wa_->clr, wa_->set);
wa_->set &= ~wa->clr;
}
@@ -166,6 +191,21 @@ static void wa_add(struct i915_wa_list *wal, i915_reg_t reg,
_wa_add(wal, &wa);
}
+static void wa_mcr_add(struct i915_wa_list *wal, i915_mcr_reg_t reg,
+ u32 clear, u32 set, u32 read_mask, bool masked_reg)
+{
+ struct i915_wa wa = {
+ .mcr_reg = reg,
+ .clr = clear,
+ .set = set,
+ .read = read_mask,
+ .masked_reg = masked_reg,
+ .is_mcr = 1,
+ };
+
+ _wa_add(wal, &wa);
+}
+
static void
wa_write_clr_set(struct i915_wa_list *wal, i915_reg_t reg, u32 clear, u32 set)
{
@@ -173,6 +213,12 @@ wa_write_clr_set(struct i915_wa_list *wal, i915_reg_t reg, u32 clear, u32 set)
}
static void
+wa_mcr_write_clr_set(struct i915_wa_list *wal, i915_mcr_reg_t reg, u32 clear, u32 set)
+{
+ wa_mcr_add(wal, reg, clear, set, clear, false);
+}
+
+static void
wa_write(struct i915_wa_list *wal, i915_reg_t reg, u32 set)
{
wa_write_clr_set(wal, reg, ~0, set);
@@ -185,11 +231,23 @@ wa_write_or(struct i915_wa_list *wal, i915_reg_t reg, u32 set)
}
static void
+wa_mcr_write_or(struct i915_wa_list *wal, i915_mcr_reg_t reg, u32 set)
+{
+ wa_mcr_write_clr_set(wal, reg, set, set);
+}
+
+static void
wa_write_clr(struct i915_wa_list *wal, i915_reg_t reg, u32 clr)
{
wa_write_clr_set(wal, reg, clr, 0);
}
+static void
+wa_mcr_write_clr(struct i915_wa_list *wal, i915_mcr_reg_t reg, u32 clr)
+{
+ wa_mcr_write_clr_set(wal, reg, clr, 0);
+}
+
/*
* WA operations on "masked register". A masked register has the upper 16 bits
* documented as "masked" in b-spec. Its purpose is to allow writing to just a
@@ -208,18 +266,37 @@ wa_masked_en(struct i915_wa_list *wal, i915_reg_t reg, u32 val)
}
static void
+wa_mcr_masked_en(struct i915_wa_list *wal, i915_mcr_reg_t reg, u32 val)
+{
+ wa_mcr_add(wal, reg, 0, _MASKED_BIT_ENABLE(val), val, true);
+}
+
+static void
wa_masked_dis(struct i915_wa_list *wal, i915_reg_t reg, u32 val)
{
wa_add(wal, reg, 0, _MASKED_BIT_DISABLE(val), val, true);
}
static void
+wa_mcr_masked_dis(struct i915_wa_list *wal, i915_mcr_reg_t reg, u32 val)
+{
+ wa_mcr_add(wal, reg, 0, _MASKED_BIT_DISABLE(val), val, true);
+}
+
+static void
wa_masked_field_set(struct i915_wa_list *wal, i915_reg_t reg,
u32 mask, u32 val)
{
wa_add(wal, reg, 0, _MASKED_FIELD(mask, val), mask, true);
}
+static void
+wa_mcr_masked_field_set(struct i915_wa_list *wal, i915_mcr_reg_t reg,
+ u32 mask, u32 val)
+{
+ wa_mcr_add(wal, reg, 0, _MASKED_FIELD(mask, val), mask, true);
+}
+
static void gen6_ctx_workarounds_init(struct intel_engine_cs *engine,
struct i915_wa_list *wal)
{
@@ -241,8 +318,8 @@ static void gen8_ctx_workarounds_init(struct intel_engine_cs *engine,
wa_masked_en(wal, RING_MI_MODE(RENDER_RING_BASE), ASYNC_FLIP_PERF_DISABLE);
/* WaDisablePartialInstShootdown:bdw,chv */
- wa_masked_en(wal, GEN8_ROW_CHICKEN,
- PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN,
+ PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
/* Use Force Non-Coherent whenever executing a 3D context. This is a
* workaround for a possible hang in the unlikely event a TLB
@@ -288,18 +365,18 @@ static void bdw_ctx_workarounds_init(struct intel_engine_cs *engine,
gen8_ctx_workarounds_init(engine, wal);
/* WaDisableThreadStallDopClockGating:bdw (pre-production) */
- wa_masked_en(wal, GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
/* WaDisableDopClockGating:bdw
*
* Also see the related UCGTCL1 write in bdw_init_clock_gating()
* to disable EUTC clock gating.
*/
- wa_masked_en(wal, GEN7_ROW_CHICKEN2,
- DOP_CLOCK_GATING_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN2,
+ DOP_CLOCK_GATING_DISABLE);
- wa_masked_en(wal, HALF_SLICE_CHICKEN3,
- GEN8_SAMPLER_POWER_BYPASS_DIS);
+ wa_mcr_masked_en(wal, GEN8_HALF_SLICE_CHICKEN3,
+ GEN8_SAMPLER_POWER_BYPASS_DIS);
wa_masked_en(wal, HDC_CHICKEN0,
/* WaForceContextSaveRestoreNonCoherent:bdw */
@@ -314,7 +391,7 @@ static void chv_ctx_workarounds_init(struct intel_engine_cs *engine,
gen8_ctx_workarounds_init(engine, wal);
/* WaDisableThreadStallDopClockGating:chv */
- wa_masked_en(wal, GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
/* Improve HiZ throughput on CHV. */
wa_masked_en(wal, HIZ_CHICKEN, CHV_HZ_8X8_MODE_IN_1X);
@@ -333,21 +410,21 @@ static void gen9_ctx_workarounds_init(struct intel_engine_cs *engine,
*/
wa_masked_en(wal, COMMON_SLICE_CHICKEN2,
GEN9_PBE_COMPRESSED_HASH_SELECTION);
- wa_masked_en(wal, GEN9_HALF_SLICE_CHICKEN7,
- GEN9_SAMPLER_HASH_COMPRESSED_READ_ADDR);
+ wa_mcr_masked_en(wal, GEN9_HALF_SLICE_CHICKEN7,
+ GEN9_SAMPLER_HASH_COMPRESSED_READ_ADDR);
}
/* WaClearFlowControlGpgpuContextSave:skl,bxt,kbl,glk,cfl */
/* WaDisablePartialInstShootdown:skl,bxt,kbl,glk,cfl */
- wa_masked_en(wal, GEN8_ROW_CHICKEN,
- FLOW_CONTROL_ENABLE |
- PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN,
+ FLOW_CONTROL_ENABLE |
+ PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
/* WaEnableYV12BugFixInHalfSliceChicken7:skl,bxt,kbl,glk,cfl */
/* WaEnableSamplerGPGPUPreemptionSupport:skl,bxt,kbl,cfl */
- wa_masked_en(wal, GEN9_HALF_SLICE_CHICKEN7,
- GEN9_ENABLE_YV12_BUGFIX |
- GEN9_ENABLE_GPGPU_PREEMPTION);
+ wa_mcr_masked_en(wal, GEN9_HALF_SLICE_CHICKEN7,
+ GEN9_ENABLE_YV12_BUGFIX |
+ GEN9_ENABLE_GPGPU_PREEMPTION);
/* Wa4x4STCOptimizationDisable:skl,bxt,kbl,glk,cfl */
/* WaDisablePartialResolveInVc:skl,bxt,kbl,cfl */
@@ -356,8 +433,8 @@ static void gen9_ctx_workarounds_init(struct intel_engine_cs *engine,
GEN9_PARTIAL_RESOLVE_IN_VC_DISABLE);
/* WaCcsTlbPrefetchDisable:skl,bxt,kbl,glk,cfl */
- wa_masked_dis(wal, GEN9_HALF_SLICE_CHICKEN5,
- GEN9_CCS_TLB_PREFETCH_ENABLE);
+ wa_mcr_masked_dis(wal, GEN9_HALF_SLICE_CHICKEN5,
+ GEN9_CCS_TLB_PREFETCH_ENABLE);
/* WaForceContextSaveRestoreNonCoherent:skl,bxt,kbl,cfl */
wa_masked_en(wal, HDC_CHICKEN0,
@@ -386,11 +463,11 @@ static void gen9_ctx_workarounds_init(struct intel_engine_cs *engine,
IS_KABYLAKE(i915) ||
IS_COFFEELAKE(i915) ||
IS_COMETLAKE(i915))
- wa_masked_en(wal, HALF_SLICE_CHICKEN3,
- GEN8_SAMPLER_POWER_BYPASS_DIS);
+ wa_mcr_masked_en(wal, GEN8_HALF_SLICE_CHICKEN3,
+ GEN8_SAMPLER_POWER_BYPASS_DIS);
/* WaDisableSTUnitPowerOptimization:skl,bxt,kbl,glk,cfl */
- wa_masked_en(wal, HALF_SLICE_CHICKEN2, GEN8_ST_PO_DISABLE);
+ wa_mcr_masked_en(wal, HALF_SLICE_CHICKEN2, GEN8_ST_PO_DISABLE);
/*
* Supporting preemption with fine-granularity requires changes in the
@@ -469,8 +546,8 @@ static void bxt_ctx_workarounds_init(struct intel_engine_cs *engine,
gen9_ctx_workarounds_init(engine, wal);
/* WaDisableThreadStallDopClockGating:bxt */
- wa_masked_en(wal, GEN8_ROW_CHICKEN,
- STALL_DOP_GATING_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN,
+ STALL_DOP_GATING_DISABLE);
/* WaToEnableHwFixForPushConstHWBug:bxt */
wa_masked_en(wal, COMMON_SLICE_CHICKEN2,
@@ -490,8 +567,8 @@ static void kbl_ctx_workarounds_init(struct intel_engine_cs *engine,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableSbeCacheDispatchPortSharing:kbl */
- wa_masked_en(wal, GEN7_HALF_SLICE_CHICKEN1,
- GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_HALF_SLICE_CHICKEN1,
+ GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
}
static void glk_ctx_workarounds_init(struct intel_engine_cs *engine,
@@ -514,8 +591,8 @@ static void cfl_ctx_workarounds_init(struct intel_engine_cs *engine,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableSbeCacheDispatchPortSharing:cfl */
- wa_masked_en(wal, GEN7_HALF_SLICE_CHICKEN1,
- GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
+ wa_mcr_masked_en(wal, GEN8_HALF_SLICE_CHICKEN1,
+ GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
}
static void icl_ctx_workarounds_init(struct intel_engine_cs *engine,
@@ -534,13 +611,13 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine,
* (the register is whitelisted in hardware now, so UMDs can opt in
* for coherency if they have a good reason).
*/
- wa_masked_en(wal, ICL_HDC_MODE, HDC_FORCE_NON_COHERENT);
+ wa_mcr_masked_en(wal, ICL_HDC_MODE, HDC_FORCE_NON_COHERENT);
/* WaEnableFloatBlendOptimization:icl */
- wa_add(wal, GEN10_CACHE_MODE_SS, 0,
- _MASKED_BIT_ENABLE(FLOAT_BLEND_OPTIMIZATION_ENABLE),
- 0 /* write-only, so skip validation */,
- true);
+ wa_mcr_add(wal, GEN10_CACHE_MODE_SS, 0,
+ _MASKED_BIT_ENABLE(FLOAT_BLEND_OPTIMIZATION_ENABLE),
+ 0 /* write-only, so skip validation */,
+ true);
/* WaDisableGPGPUMidThreadPreemption:icl */
wa_masked_field_set(wal, GEN8_CS_CHICKEN1,
@@ -548,8 +625,8 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine,
GEN9_PREEMPT_GPGPU_THREAD_GROUP_LEVEL);
/* allow headerless messages for preemptible GPGPU context */
- wa_masked_en(wal, GEN10_SAMPLER_MODE,
- GEN11_SAMPLER_ENABLE_HEADLESS_MSG);
+ wa_mcr_masked_en(wal, GEN10_SAMPLER_MODE,
+ GEN11_SAMPLER_ENABLE_HEADLESS_MSG);
/* Wa_1604278689:icl,ehl */
wa_write(wal, IVB_FBC_RT_BASE, 0xFFFFFFFF & ~ILK_FBC_RT_VALID);
@@ -558,7 +635,7 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine,
0xFFFFFFFF);
/* Wa_1406306137:icl,ehl */
- wa_masked_en(wal, GEN9_ROW_CHICKEN4, GEN11_DIS_PICK_2ND_EU);
+ wa_mcr_masked_en(wal, GEN9_ROW_CHICKEN4, GEN11_DIS_PICK_2ND_EU);
}
/*
@@ -569,13 +646,13 @@ static void dg2_ctx_gt_tuning_init(struct intel_engine_cs *engine,
struct i915_wa_list *wal)
{
wa_masked_en(wal, CHICKEN_RASTER_2, TBIMR_FAST_CLIP);
- wa_write_clr_set(wal, GEN11_L3SQCREG5, L3_PWM_TIMER_INIT_VAL_MASK,
- REG_FIELD_PREP(L3_PWM_TIMER_INIT_VAL_MASK, 0x7f));
- wa_add(wal,
- FF_MODE2,
- FF_MODE2_TDS_TIMER_MASK,
- FF_MODE2_TDS_TIMER_128,
- 0, false);
+ wa_mcr_write_clr_set(wal, XEHP_L3SQCREG5, L3_PWM_TIMER_INIT_VAL_MASK,
+ REG_FIELD_PREP(L3_PWM_TIMER_INIT_VAL_MASK, 0x7f));
+ wa_mcr_add(wal,
+ XEHP_FF_MODE2,
+ FF_MODE2_TDS_TIMER_MASK,
+ FF_MODE2_TDS_TIMER_128,
+ 0, false);
}
/*
@@ -599,7 +676,7 @@ static void gen12_ctx_gt_tuning_init(struct intel_engine_cs *engine,
* verification is ignored.
*/
wa_add(wal,
- FF_MODE2,
+ GEN12_FF_MODE2,
FF_MODE2_TDS_TIMER_MASK,
FF_MODE2_TDS_TIMER_128,
0, false);
@@ -608,6 +685,8 @@ static void gen12_ctx_gt_tuning_init(struct intel_engine_cs *engine,
static void gen12_ctx_workarounds_init(struct intel_engine_cs *engine,
struct i915_wa_list *wal)
{
+ struct drm_i915_private *i915 = engine->i915;
+
gen12_ctx_gt_tuning_init(engine, wal);
/*
@@ -637,10 +716,14 @@ static void gen12_ctx_workarounds_init(struct intel_engine_cs *engine,
* to Wa_1608008084.
*/
wa_add(wal,
- FF_MODE2,
+ GEN12_FF_MODE2,
FF_MODE2_GS_TIMER_MASK,
FF_MODE2_GS_TIMER_224,
0, false);
+
+ if (!IS_DG1(i915))
+ /* Wa_1806527549 */
+ wa_masked_en(wal, HIZ_CHICKEN, HZ_DEPTH_TEST_LE_GE_OPT_DISABLE);
}
static void dg1_ctx_workarounds_init(struct intel_engine_cs *engine,
@@ -664,27 +747,27 @@ static void dg2_ctx_workarounds_init(struct intel_engine_cs *engine,
/* Wa_16011186671:dg2_g11 */
if (IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0)) {
- wa_masked_dis(wal, VFLSKPD, DIS_MULT_MISS_RD_SQUASH);
- wa_masked_en(wal, VFLSKPD, DIS_OVER_FETCH_CACHE);
+ wa_mcr_masked_dis(wal, VFLSKPD, DIS_MULT_MISS_RD_SQUASH);
+ wa_mcr_masked_en(wal, VFLSKPD, DIS_OVER_FETCH_CACHE);
}
if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0)) {
/* Wa_14010469329:dg2_g10 */
- wa_masked_en(wal, GEN11_COMMON_SLICE_CHICKEN3,
- XEHP_DUAL_SIMD8_SEQ_MERGE_DISABLE);
+ wa_mcr_masked_en(wal, XEHP_COMMON_SLICE_CHICKEN3,
+ XEHP_DUAL_SIMD8_SEQ_MERGE_DISABLE);
/*
* Wa_22010465075:dg2_g10
* Wa_22010613112:dg2_g10
* Wa_14010698770:dg2_g10
*/
- wa_masked_en(wal, GEN11_COMMON_SLICE_CHICKEN3,
- GEN12_DISABLE_CPS_AWARE_COLOR_PIPE);
+ wa_mcr_masked_en(wal, XEHP_COMMON_SLICE_CHICKEN3,
+ GEN12_DISABLE_CPS_AWARE_COLOR_PIPE);
}
/* Wa_16013271637:dg2 */
- wa_masked_en(wal, SLICE_COMMON_ECO_CHICKEN1,
- MSC_MSAA_REODER_BUF_BYPASS_DISABLE);
+ wa_mcr_masked_en(wal, XEHP_SLICE_COMMON_ECO_CHICKEN1,
+ MSC_MSAA_REODER_BUF_BYPASS_DISABLE);
/* Wa_14014947963:dg2 */
if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_B0, STEP_FOREVER) ||
@@ -768,7 +851,7 @@ __intel_engine_init_ctx_wa(struct intel_engine_cs *engine,
{
struct drm_i915_private *i915 = engine->i915;
- wa_init_start(wal, name, engine->name);
+ wa_init_start(wal, engine->gt, name, engine->name);
/* Applies to all engines */
/*
@@ -1076,18 +1159,23 @@ static void __set_mcr_steering(struct i915_wa_list *wal,
wa_write_clr_set(wal, steering_reg, mcr_mask, mcr);
}
-static void __add_mcr_wa(struct intel_gt *gt, struct i915_wa_list *wal,
- unsigned int slice, unsigned int subslice)
+static void debug_dump_steering(struct intel_gt *gt)
{
struct drm_printer p = drm_debug_printer("MCR Steering:");
+ if (drm_debug_enabled(DRM_UT_DRIVER))
+ intel_gt_mcr_report_steering(&p, gt, false);
+}
+
+static void __add_mcr_wa(struct intel_gt *gt, struct i915_wa_list *wal,
+ unsigned int slice, unsigned int subslice)
+{
__set_mcr_steering(wal, GEN8_MCR_SELECTOR, slice, subslice);
gt->default_steering.groupid = slice;
gt->default_steering.instanceid = subslice;
- if (drm_debug_enabled(DRM_UT_DRIVER))
- intel_gt_mcr_report_steering(&p, gt, false);
+ debug_dump_steering(gt);
}
static void
@@ -1181,6 +1269,9 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal)
gt->steering_table[MSLICE] = NULL;
}
+ if (IS_XEHPSDV(gt->i915) && slice_mask & BIT(0))
+ gt->steering_table[GAM] = NULL;
+
slice = __ffs(slice_mask);
subslice = intel_sseu_find_first_xehp_dss(sseu, GEN_DSS_PER_GSLICE, slice) %
GEN_DSS_PER_GSLICE;
@@ -1198,6 +1289,13 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal)
*/
__set_mcr_steering(wal, MCFG_MCR_SELECTOR, 0, 2);
__set_mcr_steering(wal, SF_MCR_SELECTOR, 0, 2);
+
+ /*
+ * On DG2, GAM registers have a dedicated steering control register
+ * and must always be programmed to a hardcoded groupid of "1."
+ */
+ if (IS_DG2(gt->i915))
+ __set_mcr_steering(wal, GAM_MCR_SELECTOR, 1, 0);
}
static void
@@ -1254,22 +1352,22 @@ icl_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
PSDUNIT_CLKGATE_DIS);
/* Wa_1406680159:icl,ehl */
- wa_write_or(wal,
- SUBSLICE_UNIT_LEVEL_CLKGATE,
- GWUNIT_CLKGATE_DIS);
+ wa_mcr_write_or(wal,
+ GEN11_SUBSLICE_UNIT_LEVEL_CLKGATE,
+ GWUNIT_CLKGATE_DIS);
/* Wa_1607087056:icl,ehl,jsl */
if (IS_ICELAKE(i915) ||
IS_JSL_EHL_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal,
- SLICE_UNIT_LEVEL_CLKGATE,
+ GEN11_SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
/*
* This is not a documented workaround, but rather an optimization
* to reduce sampler power.
*/
- wa_write_clr(wal, GEN10_DFR_RATIO_EN_AND_CHICKEN, DFR_DISABLE);
+ wa_mcr_write_clr(wal, GEN10_DFR_RATIO_EN_AND_CHICKEN, DFR_DISABLE);
}
/*
@@ -1303,7 +1401,7 @@ gen12_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
wa_14011060649(gt, wal);
/* Wa_14011059788:tgl,rkl,adl-s,dg1,adl-p */
- wa_write_or(wal, GEN10_DFR_RATIO_EN_AND_CHICKEN, DFR_DISABLE);
+ wa_mcr_write_or(wal, GEN10_DFR_RATIO_EN_AND_CHICKEN, DFR_DISABLE);
}
static void
@@ -1315,14 +1413,14 @@ tgl_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
/* Wa_1409420604:tgl */
if (IS_TGL_UY_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
- wa_write_or(wal,
- SUBSLICE_UNIT_LEVEL_CLKGATE2,
- CPSSUNIT_CLKGATE_DIS);
+ wa_mcr_write_or(wal,
+ SUBSLICE_UNIT_LEVEL_CLKGATE2,
+ CPSSUNIT_CLKGATE_DIS);
/* Wa_1607087056:tgl also know as BUG:1409180338 */
if (IS_TGL_UY_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal,
- SLICE_UNIT_LEVEL_CLKGATE,
+ GEN11_SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
/* Wa_1408615072:tgl[a0] */
@@ -1341,14 +1439,14 @@ dg1_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
/* Wa_1607087056:dg1 */
if (IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
wa_write_or(wal,
- SLICE_UNIT_LEVEL_CLKGATE,
+ GEN11_SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
/* Wa_1409420604:dg1 */
if (IS_DG1(i915))
- wa_write_or(wal,
- SUBSLICE_UNIT_LEVEL_CLKGATE2,
- CPSSUNIT_CLKGATE_DIS);
+ wa_mcr_write_or(wal,
+ SUBSLICE_UNIT_LEVEL_CLKGATE2,
+ CPSSUNIT_CLKGATE_DIS);
/* Wa_1408615072:dg1 */
/* Empirical testing shows this register is unaffected by engine reset. */
@@ -1365,7 +1463,7 @@ xehpsdv_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
xehp_init_mcr(gt, wal);
/* Wa_1409757795:xehpsdv */
- wa_write_or(wal, SCCGCTL94DC, CG3DDISURB);
+ wa_mcr_write_or(wal, SCCGCTL94DC, CG3DDISURB);
/* Wa_16011155590:xehpsdv */
if (IS_XEHPSDV_GRAPHICS_STEP(i915, STEP_A0, STEP_B0))
@@ -1445,8 +1543,8 @@ dg2_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
CG3DDISCFEG_CLKGATE_DIS);
/* Wa_14011006942:dg2 */
- wa_write_or(wal, SUBSLICE_UNIT_LEVEL_CLKGATE,
- DSS_ROUTER_CLKGATE_DIS);
+ wa_mcr_write_or(wal, GEN11_SUBSLICE_UNIT_LEVEL_CLKGATE,
+ DSS_ROUTER_CLKGATE_DIS);
}
if (IS_DG2_GRAPHICS_STEP(gt->i915, G10, STEP_A0, STEP_B0)) {
@@ -1457,7 +1555,7 @@ dg2_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
wa_write_or(wal, UNSLCGCTL9444, LTCDD_CLKGATE_DIS);
/* Wa_14011371254:dg2_g10 */
- wa_write_or(wal, SLICE_UNIT_LEVEL_CLKGATE, NODEDSS_CLKGATE_DIS);
+ wa_mcr_write_or(wal, XEHP_SLICE_UNIT_LEVEL_CLKGATE, NODEDSS_CLKGATE_DIS);
/* Wa_14011431319:dg2_g10 */
wa_write_or(wal, UNSLCGCTL9440, GAMTLBOACS_CLKGATE_DIS |
@@ -1493,21 +1591,21 @@ dg2_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
GAMEDIA_CLKGATE_DIS);
/* Wa_14011028019:dg2_g10 */
- wa_write_or(wal, SSMCGCTL9530, RTFUNIT_CLKGATE_DIS);
+ wa_mcr_write_or(wal, SSMCGCTL9530, RTFUNIT_CLKGATE_DIS);
}
/* Wa_14014830051:dg2 */
- wa_write_clr(wal, SARB_CHICKEN1, COMP_CKN_IN);
+ wa_mcr_write_clr(wal, SARB_CHICKEN1, COMP_CKN_IN);
/*
* The following are not actually "workarounds" but rather
* recommended tuning settings documented in the bspec's
* performance guide section.
*/
- wa_write_or(wal, GEN12_SQCM, EN_32B_ACCESS);
+ wa_mcr_write_or(wal, XEHP_SQCM, EN_32B_ACCESS);
/* Wa_14015795083 */
- wa_write_clr(wal, GEN7_MISCCPCTL, GEN12_DOP_CLOCK_GATE_RENDER_ENABLE);
+ wa_mcr_write_clr(wal, GEN8_MISCCPCTL, GEN12_DOP_CLOCK_GATE_RENDER_ENABLE);
}
static void
@@ -1516,7 +1614,27 @@ pvc_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
pvc_init_mcr(gt, wal);
/* Wa_14015795083 */
- wa_write_clr(wal, GEN7_MISCCPCTL, GEN12_DOP_CLOCK_GATE_RENDER_ENABLE);
+ wa_mcr_write_clr(wal, GEN8_MISCCPCTL, GEN12_DOP_CLOCK_GATE_RENDER_ENABLE);
+}
+
+static void
+xelpg_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
+{
+ /* FIXME: Actual workarounds will be added in future patch(es) */
+
+ /*
+ * Unlike older platforms, we no longer setup implicit steering here;
+ * all MCR accesses are explicitly steered.
+ */
+ debug_dump_steering(gt);
+}
+
+static void
+xelpmp_gt_workarounds_init(struct intel_gt *gt, struct i915_wa_list *wal)
+{
+ /* FIXME: Actual workarounds will be added in future patch(es) */
+
+ debug_dump_steering(gt);
}
static void
@@ -1524,7 +1642,18 @@ gt_init_workarounds(struct intel_gt *gt, struct i915_wa_list *wal)
{
struct drm_i915_private *i915 = gt->i915;
- if (IS_PONTEVECCHIO(i915))
+ if (gt->type == GT_MEDIA) {
+ if (MEDIA_VER(i915) >= 13)
+ xelpmp_gt_workarounds_init(gt, wal);
+ else
+ MISSING_CASE(MEDIA_VER(i915));
+
+ return;
+ }
+
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70))
+ xelpg_gt_workarounds_init(gt, wal);
+ else if (IS_PONTEVECCHIO(i915))
pvc_gt_workarounds_init(gt, wal);
else if (IS_DG2(i915))
dg2_gt_workarounds_init(gt, wal);
@@ -1572,7 +1701,7 @@ void intel_gt_init_workarounds(struct intel_gt *gt)
{
struct i915_wa_list *wal = >->wa_list;
- wa_init_start(wal, "GT", "global");
+ wa_init_start(wal, gt, "GT", "global");
gt_init_workarounds(gt, wal);
wa_init_finish(wal);
}
@@ -1594,12 +1723,14 @@ wal_get_fw_for_rmw(struct intel_uncore *uncore, const struct i915_wa_list *wal)
}
static bool
-wa_verify(const struct i915_wa *wa, u32 cur, const char *name, const char *from)
+wa_verify(struct intel_gt *gt, const struct i915_wa *wa, u32 cur,
+ const char *name, const char *from)
{
if ((cur ^ wa->set) & wa->read) {
- DRM_ERROR("%s workaround lost on %s! (reg[%x]=0x%x, relevant bits were 0x%x vs expected 0x%x)\n",
- name, from, i915_mmio_reg_offset(wa->reg),
- cur, cur & wa->read, wa->set & wa->read);
+ drm_err(>->i915->drm,
+ "%s workaround lost on %s! (reg[%x]=0x%x, relevant bits were 0x%x vs expected 0x%x)\n",
+ name, from, i915_mmio_reg_offset(wa->reg),
+ cur, cur & wa->read, wa->set & wa->read);
return false;
}
@@ -1607,9 +1738,9 @@ wa_verify(const struct i915_wa *wa, u32 cur, const char *name, const char *from)
return true;
}
-static void
-wa_list_apply(struct intel_gt *gt, const struct i915_wa_list *wal)
+static void wa_list_apply(const struct i915_wa_list *wal)
{
+ struct intel_gt *gt = wal->gt;
struct intel_uncore *uncore = gt->uncore;
enum forcewake_domains fw;
unsigned long flags;
@@ -1628,14 +1759,25 @@ wa_list_apply(struct intel_gt *gt, const struct i915_wa_list *wal)
u32 val, old = 0;
/* open-coded rmw due to steering */
- old = wa->clr ? intel_gt_mcr_read_any_fw(gt, wa->reg) : 0;
+ if (wa->clr)
+ old = wa->is_mcr ?
+ intel_gt_mcr_read_any_fw(gt, wa->mcr_reg) :
+ intel_uncore_read_fw(uncore, wa->reg);
val = (old & ~wa->clr) | wa->set;
- if (val != old || !wa->clr)
- intel_uncore_write_fw(uncore, wa->reg, val);
+ if (val != old || !wa->clr) {
+ if (wa->is_mcr)
+ intel_gt_mcr_multicast_write_fw(gt, wa->mcr_reg, val);
+ else
+ intel_uncore_write_fw(uncore, wa->reg, val);
+ }
- if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
- wa_verify(wa, intel_gt_mcr_read_any_fw(gt, wa->reg),
- wal->name, "application");
+ if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)) {
+ u32 val = wa->is_mcr ?
+ intel_gt_mcr_read_any_fw(gt, wa->mcr_reg) :
+ intel_uncore_read_fw(uncore, wa->reg);
+
+ wa_verify(gt, wa, val, wal->name, "application");
+ }
}
intel_uncore_forcewake_put__locked(uncore, fw);
@@ -1644,7 +1786,7 @@ wa_list_apply(struct intel_gt *gt, const struct i915_wa_list *wal)
void intel_gt_apply_workarounds(struct intel_gt *gt)
{
- wa_list_apply(gt, >->wa_list);
+ wa_list_apply(>->wa_list);
}
static bool wa_list_verify(struct intel_gt *gt,
@@ -1664,8 +1806,9 @@ static bool wa_list_verify(struct intel_gt *gt,
intel_uncore_forcewake_get__locked(uncore, fw);
for (i = 0, wa = wal->list; i < wal->count; i++, wa++)
- ok &= wa_verify(wa,
- intel_gt_mcr_read_any_fw(gt, wa->reg),
+ ok &= wa_verify(wal->gt, wa, wa->is_mcr ?
+ intel_gt_mcr_read_any_fw(gt, wa->mcr_reg) :
+ intel_uncore_read_fw(uncore, wa->reg),
wal->name, from);
intel_uncore_forcewake_put__locked(uncore, fw);
@@ -1712,11 +1855,35 @@ whitelist_reg_ext(struct i915_wa_list *wal, i915_reg_t reg, u32 flags)
}
static void
+whitelist_mcr_reg_ext(struct i915_wa_list *wal, i915_mcr_reg_t reg, u32 flags)
+{
+ struct i915_wa wa = {
+ .mcr_reg = reg,
+ .is_mcr = 1,
+ };
+
+ if (GEM_DEBUG_WARN_ON(wal->count >= RING_MAX_NONPRIV_SLOTS))
+ return;
+
+ if (GEM_DEBUG_WARN_ON(!is_nonpriv_flags_valid(flags)))
+ return;
+
+ wa.mcr_reg.reg |= flags;
+ _wa_add(wal, &wa);
+}
+
+static void
whitelist_reg(struct i915_wa_list *wal, i915_reg_t reg)
{
whitelist_reg_ext(wal, reg, RING_FORCE_TO_NONPRIV_ACCESS_RW);
}
+static void
+whitelist_mcr_reg(struct i915_wa_list *wal, i915_mcr_reg_t reg)
+{
+ whitelist_mcr_reg_ext(wal, reg, RING_FORCE_TO_NONPRIV_ACCESS_RW);
+}
+
static void gen9_whitelist_build(struct i915_wa_list *w)
{
/* WaVFEStateAfterPipeControlwithMediaStateClear:skl,bxt,glk,cfl */
@@ -1742,7 +1909,7 @@ static void skl_whitelist_build(struct intel_engine_cs *engine)
gen9_whitelist_build(w);
/* WaDisableLSQCROPERFforOCL:skl */
- whitelist_reg(w, GEN8_L3SQCREG4);
+ whitelist_mcr_reg(w, GEN8_L3SQCREG4);
}
static void bxt_whitelist_build(struct intel_engine_cs *engine)
@@ -1763,7 +1930,7 @@ static void kbl_whitelist_build(struct intel_engine_cs *engine)
gen9_whitelist_build(w);
/* WaDisableLSQCROPERFforOCL:kbl */
- whitelist_reg(w, GEN8_L3SQCREG4);
+ whitelist_mcr_reg(w, GEN8_L3SQCREG4);
}
static void glk_whitelist_build(struct intel_engine_cs *engine)
@@ -1828,10 +1995,10 @@ static void icl_whitelist_build(struct intel_engine_cs *engine)
switch (engine->class) {
case RENDER_CLASS:
/* WaAllowUMDToModifyHalfSliceChicken7:icl */
- whitelist_reg(w, GEN9_HALF_SLICE_CHICKEN7);
+ whitelist_mcr_reg(w, GEN9_HALF_SLICE_CHICKEN7);
/* WaAllowUMDToModifySamplerMode:icl */
- whitelist_reg(w, GEN10_SAMPLER_MODE);
+ whitelist_mcr_reg(w, GEN10_SAMPLER_MODE);
/* WaEnableStateCacheRedirectToCS:icl */
whitelist_reg(w, GEN9_SLICE_COMMON_ECO_CHICKEN1);
@@ -1987,7 +2154,7 @@ void intel_engine_init_whitelist(struct intel_engine_cs *engine)
struct drm_i915_private *i915 = engine->i915;
struct i915_wa_list *w = &engine->whitelist;
- wa_init_start(w, "whitelist", engine->name);
+ wa_init_start(w, engine->gt, "whitelist", engine->name);
if (IS_PONTEVECCHIO(i915))
pvc_whitelist_build(engine);
@@ -2107,24 +2274,21 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
if (IS_DG2_GRAPHICS_STEP(i915, G11, STEP_A0, STEP_B0)) {
/* Wa_14013392000:dg2_g11 */
- wa_masked_en(wal, GEN7_ROW_CHICKEN2, GEN12_ENABLE_LARGE_GRF_MODE);
-
- /* Wa_16011620976:dg2_g11 */
- wa_write_or(wal, LSC_CHICKEN_BIT_0_UDW, DIS_CHAIN_2XSIMD8);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN2, GEN12_ENABLE_LARGE_GRF_MODE);
}
if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_B0, STEP_FOREVER) ||
IS_DG2_G11(i915) || IS_DG2_G12(i915)) {
/* Wa_1509727124:dg2 */
- wa_masked_en(wal, GEN10_SAMPLER_MODE,
- SC_DISABLE_POWER_OPTIMIZATION_EBB);
+ wa_mcr_masked_en(wal, GEN10_SAMPLER_MODE,
+ SC_DISABLE_POWER_OPTIMIZATION_EBB);
}
if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_A0, STEP_B0) ||
IS_DG2_GRAPHICS_STEP(i915, G11, STEP_A0, STEP_B0)) {
/* Wa_14012419201:dg2 */
- wa_masked_en(wal, GEN9_ROW_CHICKEN4,
- GEN12_DISABLE_HDR_PAST_PAYLOAD_HOLD_FIX);
+ wa_mcr_masked_en(wal, GEN9_ROW_CHICKEN4,
+ GEN12_DISABLE_HDR_PAST_PAYLOAD_HOLD_FIX);
}
if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_B0, STEP_C0) ||
@@ -2133,13 +2297,13 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
* Wa_22012826095:dg2
* Wa_22013059131:dg2
*/
- wa_write_clr_set(wal, LSC_CHICKEN_BIT_0_UDW,
- MAXREQS_PER_BANK,
- REG_FIELD_PREP(MAXREQS_PER_BANK, 2));
+ wa_mcr_write_clr_set(wal, LSC_CHICKEN_BIT_0_UDW,
+ MAXREQS_PER_BANK,
+ REG_FIELD_PREP(MAXREQS_PER_BANK, 2));
/* Wa_22013059131:dg2 */
- wa_write_or(wal, LSC_CHICKEN_BIT_0,
- FORCE_1_SUB_MESSAGE_PER_FRAGMENT);
+ wa_mcr_write_or(wal, LSC_CHICKEN_BIT_0,
+ FORCE_1_SUB_MESSAGE_PER_FRAGMENT);
}
/* Wa_1308578152:dg2_g10 when first gslice is fused off */
@@ -2152,19 +2316,19 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_B0, STEP_FOREVER) ||
IS_DG2_G11(i915) || IS_DG2_G12(i915)) {
/* Wa_22013037850:dg2 */
- wa_write_or(wal, LSC_CHICKEN_BIT_0_UDW,
- DISABLE_128B_EVICTION_COMMAND_UDW);
+ wa_mcr_write_or(wal, LSC_CHICKEN_BIT_0_UDW,
+ DISABLE_128B_EVICTION_COMMAND_UDW);
/* Wa_22012856258:dg2 */
- wa_masked_en(wal, GEN7_ROW_CHICKEN2,
- GEN12_DISABLE_READ_SUPPRESSION);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN2,
+ GEN12_DISABLE_READ_SUPPRESSION);
/*
* Wa_22010960976:dg2
* Wa_14013347512:dg2
*/
- wa_masked_dis(wal, GEN12_HDC_CHICKEN0,
- LSC_L1_FLUSH_CTL_3D_DATAPORT_FLUSH_EVENTS_MASK);
+ wa_mcr_masked_dis(wal, XEHP_HDC_CHICKEN0,
+ LSC_L1_FLUSH_CTL_3D_DATAPORT_FLUSH_EVENTS_MASK);
}
if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_A0, STEP_B0)) {
@@ -2172,8 +2336,8 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
* Wa_1608949956:dg2_g10
* Wa_14010198302:dg2_g10
*/
- wa_masked_en(wal, GEN8_ROW_CHICKEN,
- MDQ_ARBITRATION_MODE | UGM_BACKUP_MODE);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN,
+ MDQ_ARBITRATION_MODE | UGM_BACKUP_MODE);
/*
* Wa_14010918519:dg2_g10
@@ -2181,31 +2345,31 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
* LSC_CHICKEN_BIT_0 always reads back as 0 is this stepping,
* so ignoring verification.
*/
- wa_add(wal, LSC_CHICKEN_BIT_0_UDW, 0,
- FORCE_SLM_FENCE_SCOPE_TO_TILE | FORCE_UGM_FENCE_SCOPE_TO_TILE,
- 0, false);
+ wa_mcr_add(wal, LSC_CHICKEN_BIT_0_UDW, 0,
+ FORCE_SLM_FENCE_SCOPE_TO_TILE | FORCE_UGM_FENCE_SCOPE_TO_TILE,
+ 0, false);
}
if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_A0, STEP_B0)) {
/* Wa_22010430635:dg2 */
- wa_masked_en(wal,
- GEN9_ROW_CHICKEN4,
- GEN12_DISABLE_GRF_CLEAR);
+ wa_mcr_masked_en(wal,
+ GEN9_ROW_CHICKEN4,
+ GEN12_DISABLE_GRF_CLEAR);
/* Wa_14010648519:dg2 */
- wa_write_or(wal, XEHP_L3NODEARBCFG, XEHP_LNESPARE);
+ wa_mcr_write_or(wal, XEHP_L3NODEARBCFG, XEHP_LNESPARE);
}
/* Wa_14013202645:dg2 */
if (IS_DG2_GRAPHICS_STEP(i915, G10, STEP_B0, STEP_C0) ||
IS_DG2_GRAPHICS_STEP(i915, G11, STEP_A0, STEP_B0))
- wa_write_or(wal, RT_CTRL, DIS_NULL_QUERY);
+ wa_mcr_write_or(wal, RT_CTRL, DIS_NULL_QUERY);
/* Wa_22012532006:dg2 */
if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_C0) ||
IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0))
- wa_masked_en(wal, GEN9_HALF_SLICE_CHICKEN7,
- DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA);
+ wa_mcr_masked_en(wal, GEN9_HALF_SLICE_CHICKEN7,
+ DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA);
if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0)) {
/* Wa_14010680813:dg2_g10 */
@@ -2216,17 +2380,16 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
if (IS_DG2_GRAPHICS_STEP(engine->i915, G10, STEP_A0, STEP_B0) ||
IS_DG2_GRAPHICS_STEP(engine->i915, G11, STEP_A0, STEP_B0)) {
/* Wa_14012362059:dg2 */
- wa_write_or(wal, GEN12_MERT_MOD_CTRL, FORCE_MISS_FTLB);
+ wa_mcr_write_or(wal, XEHP_MERT_MOD_CTRL, FORCE_MISS_FTLB);
}
if (IS_DG2_GRAPHICS_STEP(i915, G11, STEP_B0, STEP_FOREVER) ||
IS_DG2_G10(i915)) {
/* Wa_22014600077:dg2 */
- wa_add(wal, GEN10_CACHE_MODE_SS, 0,
- _MASKED_BIT_ENABLE(ENABLE_EU_COUNT_FOR_TDL_FLUSH),
- 0 /* Wa_14012342262 :write-only reg, so skip
- verification */,
- true);
+ wa_mcr_add(wal, GEN10_CACHE_MODE_SS, 0,
+ _MASKED_BIT_ENABLE(ENABLE_EU_COUNT_FOR_TDL_FLUSH),
+ 0 /* Wa_14012342262 write-only reg, so skip verification */,
+ true);
}
if (IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0) ||
@@ -2253,7 +2416,7 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
if (IS_ALDERLAKE_P(i915) || IS_ALDERLAKE_S(i915) || IS_DG1(i915) ||
IS_ROCKETLAKE(i915) || IS_TIGERLAKE(i915)) {
/* Wa_1606931601:tgl,rkl,dg1,adl-s,adl-p */
- wa_masked_en(wal, GEN7_ROW_CHICKEN2, GEN12_DISABLE_EARLY_READ);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN2, GEN12_DISABLE_EARLY_READ);
/*
* Wa_1407928979:tgl A*
@@ -2282,14 +2445,14 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0) ||
IS_ROCKETLAKE(i915) || IS_TIGERLAKE(i915)) {
/* Wa_1409804808:tgl,rkl,dg1[a0],adl-s,adl-p */
- wa_masked_en(wal, GEN7_ROW_CHICKEN2,
- GEN12_PUSH_CONST_DEREF_HOLD_DIS);
+ wa_mcr_masked_en(wal, GEN8_ROW_CHICKEN2,
+ GEN12_PUSH_CONST_DEREF_HOLD_DIS);
/*
* Wa_1409085225:tgl
* Wa_14010229206:tgl,rkl,dg1[a0],adl-s,adl-p
*/
- wa_masked_en(wal, GEN9_ROW_CHICKEN4, GEN12_DISABLE_TDL_PUSH);
+ wa_mcr_masked_en(wal, GEN9_ROW_CHICKEN4, GEN12_DISABLE_TDL_PUSH);
}
if (IS_DG1_GRAPHICS_STEP(i915, STEP_A0, STEP_B0) ||
@@ -2313,9 +2476,9 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
if (IS_DG1(i915) || IS_ROCKETLAKE(i915) || IS_TIGERLAKE(i915) ||
IS_ALDERLAKE_S(i915) || IS_ALDERLAKE_P(i915)) {
/* Wa_1406941453:tgl,rkl,dg1,adl-s,adl-p */
- wa_masked_en(wal,
- GEN10_SAMPLER_MODE,
- ENABLE_SMALLPL);
+ wa_mcr_masked_en(wal,
+ GEN10_SAMPLER_MODE,
+ ENABLE_SMALLPL);
}
if (GRAPHICS_VER(i915) == 11) {
@@ -2349,9 +2512,9 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
* Wa_1405733216:icl
* Formerly known as WaDisableCleanEvicts
*/
- wa_write_or(wal,
- GEN8_L3SQCREG4,
- GEN11_LQSC_CLEAN_EVICT_DISABLE);
+ wa_mcr_write_or(wal,
+ GEN8_L3SQCREG4,
+ GEN11_LQSC_CLEAN_EVICT_DISABLE);
/* Wa_1606682166:icl */
wa_write_or(wal,
@@ -2359,10 +2522,10 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
GEN7_DISABLE_SAMPLER_PREFETCH);
/* Wa_1409178092:icl */
- wa_write_clr_set(wal,
- GEN11_SCRATCH2,
- GEN11_COHERENT_PARTIAL_WRITE_MERGE_ENABLE,
- 0);
+ wa_mcr_write_clr_set(wal,
+ GEN11_SCRATCH2,
+ GEN11_COHERENT_PARTIAL_WRITE_MERGE_ENABLE,
+ 0);
/* WaEnable32PlaneMode:icl */
wa_masked_en(wal, GEN9_CSFE_CHICKEN1_RCS,
@@ -2389,12 +2552,64 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
FF_DOP_CLOCK_GATE_DISABLE);
}
- if (IS_GRAPHICS_VER(i915, 9, 12)) {
- /* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl,tgl */
+ /*
+ * Intel platforms that support fine-grained preemption (i.e., gen9 and
+ * beyond) allow the kernel-mode driver to choose between two different
+ * options for controlling preemption granularity and behavior.
+ *
+ * Option 1 (hardware default):
+ * Preemption settings are controlled in a global manner via
+ * kernel-only register CS_DEBUG_MODE1 (0x20EC). Any granularity
+ * and settings chosen by the kernel-mode driver will apply to all
+ * userspace clients.
+ *
+ * Option 2:
+ * Preemption settings are controlled on a per-context basis via
+ * register CS_CHICKEN1 (0x2580). CS_CHICKEN1 is saved/restored on
+ * context switch and is writable by userspace (e.g., via
+ * MI_LOAD_REGISTER_IMMEDIATE instructions placed in a batch buffer)
+ * which allows different userspace drivers/clients to select
+ * different settings, or to change those settings on the fly in
+ * response to runtime needs. This option was known by name
+ * "FtrPerCtxtPreemptionGranularityControl" at one time, although
+ * that name is somewhat misleading as other non-granularity
+ * preemption settings are also impacted by this decision.
+ *
+ * On Linux, our policy has always been to let userspace drivers
+ * control preemption granularity/settings (Option 2). This was
+ * originally mandatory on gen9 to prevent ABI breakage (old gen9
+ * userspace developed before object-level preemption was enabled would
+ * not behave well if i915 were to go with Option 1 and enable that
+ * preemption in a global manner). On gen9 each context would have
+ * object-level preemption disabled by default (see
+ * WaDisable3DMidCmdPreemption in gen9_ctx_workarounds_init), but
+ * userspace drivers could opt-in to object-level preemption as they
+ * saw fit. For post-gen9 platforms, we continue to utilize Option 2;
+ * even though it is no longer necessary for ABI compatibility when
+ * enabling a new platform, it does ensure that userspace will be able
+ * to implement any workarounds that show up requiring temporary
+ * adjustments to preemption behavior at runtime.
+ *
+ * Notes/Workarounds:
+ * - Wa_14015141709: On DG2 and early steppings of MTL,
+ * CS_CHICKEN1[0] does not disable object-level preemption as
+ * it is supposed to (nor does CS_DEBUG_MODE1[0] if we had been
+ * using Option 1). Effectively this means userspace is unable
+ * to disable object-level preemption on these platforms/steppings
+ * despite the setting here.
+ *
+ * - Wa_16013994831: May require that userspace program
+ * CS_CHICKEN1[10] when certain runtime conditions are true.
+ * Userspace requires Option 2 to be in effect for their update of
+ * CS_CHICKEN1[10] to be effective.
+ *
+ * Other workarounds may appear in the future that will also require
+ * Option 2 behavior to allow proper userspace implementation.
+ */
+ if (GRAPHICS_VER(i915) >= 9)
wa_masked_en(wal,
GEN7_FF_SLICE_CS_CHICKEN1,
GEN9_FFSC_PERCTX_PREEMPT_CTRL);
- }
if (IS_SKYLAKE(i915) ||
IS_KABYLAKE(i915) ||
@@ -2420,36 +2635,36 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE);
/* WaEnableLbsSlaRetryTimerDecrement:skl,bxt,kbl,glk,cfl */
- wa_write_or(wal,
- BDW_SCRATCH1,
- GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE);
+ wa_mcr_write_or(wal,
+ BDW_SCRATCH1,
+ GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE);
/* WaProgramL3SqcReg1DefaultForPerf:bxt,glk */
if (IS_GEN9_LP(i915))
- wa_write_clr_set(wal,
- GEN8_L3SQCREG1,
- L3_PRIO_CREDITS_MASK,
- L3_GENERAL_PRIO_CREDITS(62) |
- L3_HIGH_PRIO_CREDITS(2));
+ wa_mcr_write_clr_set(wal,
+ GEN8_L3SQCREG1,
+ L3_PRIO_CREDITS_MASK,
+ L3_GENERAL_PRIO_CREDITS(62) |
+ L3_HIGH_PRIO_CREDITS(2));
/* WaOCLCoherentLineFlush:skl,bxt,kbl,cfl */
- wa_write_or(wal,
- GEN8_L3SQCREG4,
- GEN8_LQSC_FLUSH_COHERENT_LINES);
+ wa_mcr_write_or(wal,
+ GEN8_L3SQCREG4,
+ GEN8_LQSC_FLUSH_COHERENT_LINES);
/* Disable atomics in L3 to prevent unrecoverable hangs */
wa_write_clr_set(wal, GEN9_SCRATCH_LNCF1,
GEN9_LNCF_NONIA_COHERENT_ATOMICS_ENABLE, 0);
- wa_write_clr_set(wal, GEN8_L3SQCREG4,
- GEN8_LQSQ_NONIA_COHERENT_ATOMICS_ENABLE, 0);
- wa_write_clr_set(wal, GEN9_SCRATCH1,
- EVICTION_PERF_FIX_ENABLE, 0);
+ wa_mcr_write_clr_set(wal, GEN8_L3SQCREG4,
+ GEN8_LQSQ_NONIA_COHERENT_ATOMICS_ENABLE, 0);
+ wa_mcr_write_clr_set(wal, GEN9_SCRATCH1,
+ EVICTION_PERF_FIX_ENABLE, 0);
}
if (IS_HASWELL(i915)) {
/* WaSampleCChickenBitEnable:hsw */
wa_masked_en(wal,
- HALF_SLICE_CHICKEN3, HSW_SAMPLE_C_PERFORMANCE);
+ HSW_HALF_SLICE_CHICKEN3, HSW_SAMPLE_C_PERFORMANCE);
wa_masked_dis(wal,
CACHE_MODE_0_GEN7,
@@ -2657,7 +2872,7 @@ ccs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
{
if (IS_PVC_CT_STEP(engine->i915, STEP_A0, STEP_C0)) {
/* Wa_14014999345:pvc */
- wa_masked_en(wal, GEN10_CACHE_MODE_SS, DISABLE_ECC);
+ wa_mcr_masked_en(wal, GEN10_CACHE_MODE_SS, DISABLE_ECC);
}
}
@@ -2683,8 +2898,8 @@ add_render_compute_tuning_settings(struct drm_i915_private *i915,
}
if (IS_DG2(i915)) {
- wa_write_or(wal, XEHP_L3SCQREG7, BLEND_FILL_CACHING_OPT_DIS);
- wa_write_clr_set(wal, RT_CTRL, STACKID_CTRL, STACKID_CTRL_512);
+ wa_mcr_write_or(wal, XEHP_L3SCQREG7, BLEND_FILL_CACHING_OPT_DIS);
+ wa_mcr_write_clr_set(wal, RT_CTRL, STACKID_CTRL, STACKID_CTRL_512);
/*
* This is also listed as Wa_22012654132 for certain DG2
@@ -2695,10 +2910,10 @@ add_render_compute_tuning_settings(struct drm_i915_private *i915,
* back for verification on DG2 (due to Wa_14012342262), so
* we need to explicitly skip the readback.
*/
- wa_add(wal, GEN10_CACHE_MODE_SS, 0,
- _MASKED_BIT_ENABLE(ENABLE_PREFETCH_INTO_IC),
- 0 /* write-only, so skip validation */,
- true);
+ wa_mcr_add(wal, GEN10_CACHE_MODE_SS, 0,
+ _MASKED_BIT_ENABLE(ENABLE_PREFETCH_INTO_IC),
+ 0 /* write-only, so skip validation */,
+ true);
}
/*
@@ -2707,8 +2922,8 @@ add_render_compute_tuning_settings(struct drm_i915_private *i915,
* platforms.
*/
if (INTEL_INFO(i915)->tuning_thread_rr_after_dep)
- wa_masked_field_set(wal, GEN9_ROW_CHICKEN4, THREAD_EX_ARB_MODE,
- THREAD_EX_ARB_MODE_RR_AFTER_DEP);
+ wa_mcr_masked_field_set(wal, GEN9_ROW_CHICKEN4, THREAD_EX_ARB_MODE,
+ THREAD_EX_ARB_MODE_RR_AFTER_DEP);
}
/*
@@ -2734,30 +2949,30 @@ general_render_compute_wa_init(struct intel_engine_cs *engine, struct i915_wa_li
if (IS_XEHPSDV(i915)) {
/* Wa_1409954639 */
- wa_masked_en(wal,
- GEN8_ROW_CHICKEN,
- SYSTOLIC_DOP_CLOCK_GATING_DIS);
+ wa_mcr_masked_en(wal,
+ GEN8_ROW_CHICKEN,
+ SYSTOLIC_DOP_CLOCK_GATING_DIS);
/* Wa_1607196519 */
- wa_masked_en(wal,
- GEN9_ROW_CHICKEN4,
- GEN12_DISABLE_GRF_CLEAR);
+ wa_mcr_masked_en(wal,
+ GEN9_ROW_CHICKEN4,
+ GEN12_DISABLE_GRF_CLEAR);
/* Wa_14010670810:xehpsdv */
- wa_write_or(wal, XEHP_L3NODEARBCFG, XEHP_LNESPARE);
+ wa_mcr_write_or(wal, XEHP_L3NODEARBCFG, XEHP_LNESPARE);
/* Wa_14010449647:xehpsdv */
- wa_masked_en(wal, GEN7_HALF_SLICE_CHICKEN1,
- GEN7_PSD_SINGLE_PORT_DISPATCH_ENABLE);
+ wa_mcr_masked_en(wal, GEN8_HALF_SLICE_CHICKEN1,
+ GEN7_PSD_SINGLE_PORT_DISPATCH_ENABLE);
/* Wa_18011725039:xehpsdv */
if (IS_XEHPSDV_GRAPHICS_STEP(i915, STEP_A1, STEP_B0)) {
- wa_masked_dis(wal, MLTICTXCTL, TDONRENDER);
- wa_write_or(wal, L3SQCREG1_CCS0, FLUSHALLNONCOH);
+ wa_mcr_masked_dis(wal, MLTICTXCTL, TDONRENDER);
+ wa_mcr_write_or(wal, L3SQCREG1_CCS0, FLUSHALLNONCOH);
}
/* Wa_14012362059:xehpsdv */
- wa_write_or(wal, GEN12_MERT_MOD_CTRL, FORCE_MISS_FTLB);
+ wa_mcr_write_or(wal, XEHP_MERT_MOD_CTRL, FORCE_MISS_FTLB);
/* Wa_14014368820:xehpsdv */
wa_write_or(wal, GEN12_GAMCNTRL_CTRL, INVALIDATION_BROADCAST_MODE_DIS |
@@ -2766,26 +2981,37 @@ general_render_compute_wa_init(struct intel_engine_cs *engine, struct i915_wa_li
if (IS_DG2(i915) || IS_PONTEVECCHIO(i915)) {
/* Wa_14015227452:dg2,pvc */
- wa_masked_en(wal, GEN9_ROW_CHICKEN4, XEHP_DIS_BBL_SYSPIPE);
+ wa_mcr_masked_en(wal, GEN9_ROW_CHICKEN4, XEHP_DIS_BBL_SYSPIPE);
/* Wa_22014226127:dg2,pvc */
- wa_write_or(wal, LSC_CHICKEN_BIT_0, DISABLE_D8_D16_COASLESCE);
+ wa_mcr_write_or(wal, LSC_CHICKEN_BIT_0, DISABLE_D8_D16_COASLESCE);
/* Wa_16015675438:dg2,pvc */
wa_masked_en(wal, FF_SLICE_CS_CHICKEN2, GEN12_PERF_FIX_BALANCING_CFE_DISABLE);
/* Wa_18018781329:dg2,pvc */
- wa_write_or(wal, RENDER_MOD_CTRL, FORCE_MISS_FTLB);
- wa_write_or(wal, COMP_MOD_CTRL, FORCE_MISS_FTLB);
- wa_write_or(wal, VDBX_MOD_CTRL, FORCE_MISS_FTLB);
- wa_write_or(wal, VEBX_MOD_CTRL, FORCE_MISS_FTLB);
+ wa_mcr_write_or(wal, RENDER_MOD_CTRL, FORCE_MISS_FTLB);
+ wa_mcr_write_or(wal, COMP_MOD_CTRL, FORCE_MISS_FTLB);
+ wa_mcr_write_or(wal, VDBX_MOD_CTRL, FORCE_MISS_FTLB);
+ wa_mcr_write_or(wal, VEBX_MOD_CTRL, FORCE_MISS_FTLB);
+ }
+
+ if (IS_DG2(i915)) {
+ /*
+ * Wa_16011620976:dg2_g11
+ * Wa_22015475538:dg2
+ */
+ wa_mcr_write_or(wal, LSC_CHICKEN_BIT_0_UDW, DIS_CHAIN_2XSIMD8);
+
+ /* Wa_18017747507:dg2 */
+ wa_masked_en(wal, VFG_PREEMPTION_CHICKEN, POLYGON_TRIFAN_LINELOOP_DISABLE);
}
}
static void
engine_init_workarounds(struct intel_engine_cs *engine, struct i915_wa_list *wal)
{
- if (I915_SELFTEST_ONLY(GRAPHICS_VER(engine->i915) < 4))
+ if (GRAPHICS_VER(engine->i915) < 4)
return;
engine_fake_wa_init(engine, wal);
@@ -2810,17 +3036,14 @@ void intel_engine_init_workarounds(struct intel_engine_cs *engine)
{
struct i915_wa_list *wal = &engine->wa_list;
- if (GRAPHICS_VER(engine->i915) < 4)
- return;
-
- wa_init_start(wal, "engine", engine->name);
+ wa_init_start(wal, engine->gt, "engine", engine->name);
engine_init_workarounds(engine, wal);
wa_init_finish(wal);
}
void intel_engine_apply_workarounds(struct intel_engine_cs *engine)
{
- wa_list_apply(engine->gt, &engine->wa_list);
+ wa_list_apply(&engine->wa_list);
}
static const struct i915_range mcr_ranges_gen8[] = {
@@ -2964,9 +3187,7 @@ static int engine_wa_list_verify(struct intel_context *ce,
goto err_vma;
}
- err = i915_request_await_object(rq, vma->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
if (err == 0)
err = wa_list_srm(rq, wal, vma);
@@ -2994,7 +3215,7 @@ static int engine_wa_list_verify(struct intel_context *ce,
if (mcr_range(rq->engine->i915, i915_mmio_reg_offset(wa->reg)))
continue;
- if (!wa_verify(wa, results[i], wal->name, from))
+ if (!wa_verify(wal->gt, wa, results[i], wal->name, from))
err = -ENXIO;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds_types.h b/drivers/gpu/drm/i915/gt/intel_workarounds_types.h
index 8a4b6de..e141881 100644
--- a/drivers/gpu/drm/i915/gt/intel_workarounds_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_workarounds_types.h
@@ -10,15 +10,23 @@
#include "i915_reg_defs.h"
+struct intel_gt;
+
struct i915_wa {
- i915_reg_t reg;
+ union {
+ i915_reg_t reg;
+ i915_mcr_reg_t mcr_reg;
+ };
u32 clr;
u32 set;
u32 read;
- bool masked_reg;
+
+ u32 masked_reg:1;
+ u32 is_mcr:1;
};
struct i915_wa_list {
+ struct intel_gt *gt;
const char *name;
const char *engine_name;
struct i915_wa *list;
diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_cs.c b/drivers/gpu/drm/i915/gt/selftest_engine_cs.c
index 1b75f47..881b64f 100644
--- a/drivers/gpu/drm/i915/gt/selftest_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/selftest_engine_cs.c
@@ -39,6 +39,16 @@ static int perf_end(struct intel_gt *gt)
return igt_flush_test(gt->i915);
}
+static i915_reg_t timestamp_reg(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *i915 = engine->i915;
+
+ if (GRAPHICS_VER(i915) == 5 || IS_G4X(i915))
+ return RING_TIMESTAMP_UDW(engine->mmio_base);
+ else
+ return RING_TIMESTAMP(engine->mmio_base);
+}
+
static int write_timestamp(struct i915_request *rq, int slot)
{
struct intel_timeline *tl =
@@ -55,7 +65,7 @@ static int write_timestamp(struct i915_request *rq, int slot)
if (GRAPHICS_VER(rq->engine->i915) >= 8)
cmd++;
*cs++ = cmd;
- *cs++ = i915_mmio_reg_offset(RING_TIMESTAMP(rq->engine->mmio_base));
+ *cs++ = i915_mmio_reg_offset(timestamp_reg(rq->engine));
*cs++ = tl->hwsp_offset + slot * sizeof(u32);
*cs++ = 0;
@@ -125,7 +135,7 @@ static int perf_mi_bb_start(void *arg)
enum intel_engine_id id;
int err = 0;
- if (GRAPHICS_VER(gt->i915) < 7) /* for per-engine CS_TIMESTAMP */
+ if (GRAPHICS_VER(gt->i915) < 4) /* Any CS_TIMESTAMP? */
return 0;
perf_begin(gt);
@@ -135,6 +145,9 @@ static int perf_mi_bb_start(void *arg)
u32 cycles[COUNT];
int i;
+ if (GRAPHICS_VER(engine->i915) < 7 && engine->id != RCS0)
+ continue;
+
intel_engine_pm_get(engine);
batch = create_empty_batch(ce);
@@ -249,7 +262,7 @@ static int perf_mi_noop(void *arg)
enum intel_engine_id id;
int err = 0;
- if (GRAPHICS_VER(gt->i915) < 7) /* for per-engine CS_TIMESTAMP */
+ if (GRAPHICS_VER(gt->i915) < 4) /* Any CS_TIMESTAMP? */
return 0;
perf_begin(gt);
@@ -259,6 +272,9 @@ static int perf_mi_noop(void *arg)
u32 cycles[COUNT];
int i;
+ if (GRAPHICS_VER(engine->i915) < 7 && engine->id != RCS0)
+ continue;
+
intel_engine_pm_get(engine);
base = create_empty_batch(ce);
diff --git a/drivers/gpu/drm/i915/gt/selftest_engine_pm.c b/drivers/gpu/drm/i915/gt/selftest_engine_pm.c
index 0dcb3ed4..87c9431 100644
--- a/drivers/gpu/drm/i915/gt/selftest_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/selftest_engine_pm.c
@@ -317,7 +317,7 @@ static int live_engine_busy_stats(void *arg)
ENGINE_TRACE(engine, "measuring busy time\n");
preempt_disable();
de = intel_engine_get_busy_time(engine, &t[0]);
- mdelay(10);
+ mdelay(100);
de = ktime_sub(intel_engine_get_busy_time(engine, &t[1]), de);
preempt_enable();
dt = ktime_sub(t[1], t[0]);
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 1e08b24..ab2e9a6 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -85,8 +85,6 @@ static int wait_for_reset(struct intel_engine_cs *engine,
break;
} while (time_before(jiffies, timeout));
- flush_scheduled_work();
-
if (rq->fence.error != -EIO) {
pr_err("%s: hanging request %llx:%lld not reset\n",
engine->name,
@@ -2766,9 +2764,7 @@ static int create_gang(struct intel_engine_cs *engine,
i915_request_get(rq);
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, false);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
if (!err)
err = rq->engine->emit_bb_start(rq,
vma->node.start,
@@ -3182,15 +3178,11 @@ create_gpr_client(struct intel_engine_cs *engine,
}
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, false);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
i915_vma_unlock(vma);
i915_vma_lock(batch);
if (!err)
- err = i915_request_await_object(rq, batch->obj, false);
- if (!err)
err = i915_vma_move_to_active(batch, rq, 0);
if (!err)
err = rq->engine->emit_bb_start(rq,
@@ -3475,12 +3467,14 @@ static int random_priority(struct rnd_state *rnd)
struct preempt_smoke {
struct intel_gt *gt;
+ struct kthread_work work;
struct i915_gem_context **contexts;
struct intel_engine_cs *engine;
struct drm_i915_gem_object *batch;
unsigned int ncontext;
struct rnd_state prng;
unsigned long count;
+ int result;
};
static struct i915_gem_context *smoke_context(struct preempt_smoke *smoke)
@@ -3521,9 +3515,7 @@ static int smoke_submit(struct preempt_smoke *smoke,
if (vma) {
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, false);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
if (!err)
err = rq->engine->emit_bb_start(rq,
vma->node.start,
@@ -3540,34 +3532,31 @@ static int smoke_submit(struct preempt_smoke *smoke,
return err;
}
-static int smoke_crescendo_thread(void *arg)
+static void smoke_crescendo_work(struct kthread_work *work)
{
- struct preempt_smoke *smoke = arg;
+ struct preempt_smoke *smoke = container_of(work, typeof(*smoke), work);
IGT_TIMEOUT(end_time);
unsigned long count;
count = 0;
do {
struct i915_gem_context *ctx = smoke_context(smoke);
- int err;
- err = smoke_submit(smoke,
- ctx, count % I915_PRIORITY_MAX,
- smoke->batch);
- if (err)
- return err;
+ smoke->result = smoke_submit(smoke, ctx,
+ count % I915_PRIORITY_MAX,
+ smoke->batch);
count++;
- } while (count < smoke->ncontext && !__igt_timeout(end_time, NULL));
+ } while (!smoke->result && count < smoke->ncontext &&
+ !__igt_timeout(end_time, NULL));
smoke->count = count;
- return 0;
}
static int smoke_crescendo(struct preempt_smoke *smoke, unsigned int flags)
#define BATCH BIT(0)
{
- struct task_struct *tsk[I915_NUM_ENGINES] = {};
+ struct kthread_worker *worker[I915_NUM_ENGINES] = {};
struct preempt_smoke *arg;
struct intel_engine_cs *engine;
enum intel_engine_id id;
@@ -3578,6 +3567,8 @@ static int smoke_crescendo(struct preempt_smoke *smoke, unsigned int flags)
if (!arg)
return -ENOMEM;
+ memset(arg, 0, I915_NUM_ENGINES * sizeof(*arg));
+
for_each_engine(engine, smoke->gt, id) {
arg[id] = *smoke;
arg[id].engine = engine;
@@ -3585,31 +3576,28 @@ static int smoke_crescendo(struct preempt_smoke *smoke, unsigned int flags)
arg[id].batch = NULL;
arg[id].count = 0;
- tsk[id] = kthread_run(smoke_crescendo_thread, arg,
- "igt/smoke:%d", id);
- if (IS_ERR(tsk[id])) {
- err = PTR_ERR(tsk[id]);
+ worker[id] = kthread_create_worker(0, "igt/smoke:%d", id);
+ if (IS_ERR(worker[id])) {
+ err = PTR_ERR(worker[id]);
break;
}
- get_task_struct(tsk[id]);
- }
- yield(); /* start all threads before we kthread_stop() */
+ kthread_init_work(&arg[id].work, smoke_crescendo_work);
+ kthread_queue_work(worker[id], &arg[id].work);
+ }
count = 0;
for_each_engine(engine, smoke->gt, id) {
- int status;
-
- if (IS_ERR_OR_NULL(tsk[id]))
+ if (IS_ERR_OR_NULL(worker[id]))
continue;
- status = kthread_stop(tsk[id]);
- if (status && !err)
- err = status;
+ kthread_flush_work(&arg[id].work);
+ if (arg[id].result && !err)
+ err = arg[id].result;
count += arg[id].count;
- put_task_struct(tsk[id]);
+ kthread_destroy_worker(worker[id]);
}
pr_info("Submitted %lu crescendo:%x requests across %d engines and %d contexts\n",
diff --git a/drivers/gpu/drm/i915/gt/selftest_gt_pm.c b/drivers/gpu/drm/i915/gt/selftest_gt_pm.c
index be94f86..b46425a 100644
--- a/drivers/gpu/drm/i915/gt/selftest_gt_pm.c
+++ b/drivers/gpu/drm/i915/gt/selftest_gt_pm.c
@@ -36,6 +36,19 @@ static int cmp_u32(const void *A, const void *B)
return 0;
}
+static u32 read_timestamp(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *i915 = engine->i915;
+
+ /* On i965 the first read tends to give a stale value */
+ ENGINE_READ_FW(engine, RING_TIMESTAMP);
+
+ if (GRAPHICS_VER(i915) == 5 || IS_G4X(i915))
+ return ENGINE_READ_FW(engine, RING_TIMESTAMP_UDW);
+ else
+ return ENGINE_READ_FW(engine, RING_TIMESTAMP);
+}
+
static void measure_clocks(struct intel_engine_cs *engine,
u32 *out_cycles, ktime_t *out_dt)
{
@@ -45,13 +58,13 @@ static void measure_clocks(struct intel_engine_cs *engine,
for (i = 0; i < 5; i++) {
local_irq_disable();
- cycles[i] = -ENGINE_READ_FW(engine, RING_TIMESTAMP);
+ cycles[i] = -read_timestamp(engine);
dt[i] = ktime_get();
udelay(1000);
dt[i] = ktime_sub(ktime_get(), dt[i]);
- cycles[i] += ENGINE_READ_FW(engine, RING_TIMESTAMP);
+ cycles[i] += read_timestamp(engine);
local_irq_enable();
}
@@ -78,25 +91,6 @@ static int live_gt_clocks(void *arg)
if (GRAPHICS_VER(gt->i915) < 4) /* Any CS_TIMESTAMP? */
return 0;
- if (GRAPHICS_VER(gt->i915) == 5)
- /*
- * XXX CS_TIMESTAMP low dword is dysfunctional?
- *
- * Ville's experiments indicate the high dword still works,
- * but at a correspondingly reduced frequency.
- */
- return 0;
-
- if (GRAPHICS_VER(gt->i915) == 4)
- /*
- * XXX CS_TIMESTAMP appears gibberish
- *
- * Ville's experiments indicate that it mostly appears 'stuck'
- * in that we see the register report the same cycle count
- * for a couple of reads.
- */
- return 0;
-
intel_gt_pm_get(gt);
intel_uncore_forcewake_get(gt->uncore, FORCEWAKE_ALL);
diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
index 7f3bb1d..bc05ef4 100644
--- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
+++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
@@ -99,22 +99,6 @@ static u64 hws_address(const struct i915_vma *hws,
return hws->node.start + offset_in_page(sizeof(u32)*rq->fence.context);
}
-static int move_to_active(struct i915_vma *vma,
- struct i915_request *rq,
- unsigned int flags)
-{
- int err;
-
- i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj,
- flags & EXEC_OBJECT_WRITE);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, flags);
- i915_vma_unlock(vma);
-
- return err;
-}
-
static struct i915_request *
hang_create_request(struct hang *h, struct intel_engine_cs *engine)
{
@@ -175,11 +159,11 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine)
goto unpin_hws;
}
- err = move_to_active(vma, rq, 0);
+ err = igt_vma_move_to_active_unlocked(vma, rq, 0);
if (err)
goto cancel_rq;
- err = move_to_active(hws, rq, 0);
+ err = igt_vma_move_to_active_unlocked(hws, rq, 0);
if (err)
goto cancel_rq;
@@ -866,10 +850,13 @@ static int igt_reset_active_engine(void *arg)
}
struct active_engine {
- struct task_struct *task;
+ struct kthread_worker *worker;
+ struct kthread_work work;
struct intel_engine_cs *engine;
unsigned long resets;
unsigned int flags;
+ bool stop;
+ int result;
};
#define TEST_ACTIVE BIT(0)
@@ -900,10 +887,10 @@ static int active_request_put(struct i915_request *rq)
return err;
}
-static int active_engine(void *data)
+static void active_engine(struct kthread_work *work)
{
I915_RND_STATE(prng);
- struct active_engine *arg = data;
+ struct active_engine *arg = container_of(work, typeof(*arg), work);
struct intel_engine_cs *engine = arg->engine;
struct i915_request *rq[8] = {};
struct intel_context *ce[ARRAY_SIZE(rq)];
@@ -913,16 +900,17 @@ static int active_engine(void *data)
for (count = 0; count < ARRAY_SIZE(ce); count++) {
ce[count] = intel_context_create(engine);
if (IS_ERR(ce[count])) {
- err = PTR_ERR(ce[count]);
- pr_err("[%s] Create context #%ld failed: %d!\n", engine->name, count, err);
+ arg->result = PTR_ERR(ce[count]);
+ pr_err("[%s] Create context #%ld failed: %d!\n",
+ engine->name, count, arg->result);
while (--count)
intel_context_put(ce[count]);
- return err;
+ return;
}
}
count = 0;
- while (!kthread_should_stop()) {
+ while (!READ_ONCE(arg->stop)) {
unsigned int idx = count++ & (ARRAY_SIZE(rq) - 1);
struct i915_request *old = rq[idx];
struct i915_request *new;
@@ -967,7 +955,7 @@ static int active_engine(void *data)
intel_context_put(ce[count]);
}
- return err;
+ arg->result = err;
}
static int __igt_reset_engines(struct intel_gt *gt,
@@ -1022,7 +1010,7 @@ static int __igt_reset_engines(struct intel_gt *gt,
memset(threads, 0, sizeof(*threads) * I915_NUM_ENGINES);
for_each_engine(other, gt, tmp) {
- struct task_struct *tsk;
+ struct kthread_worker *worker;
threads[tmp].resets =
i915_reset_engine_count(global, other);
@@ -1036,19 +1024,21 @@ static int __igt_reset_engines(struct intel_gt *gt,
threads[tmp].engine = other;
threads[tmp].flags = flags;
- tsk = kthread_run(active_engine, &threads[tmp],
- "igt/%s", other->name);
- if (IS_ERR(tsk)) {
- err = PTR_ERR(tsk);
- pr_err("[%s] Thread spawn failed: %d!\n", engine->name, err);
+ worker = kthread_create_worker(0, "igt/%s",
+ other->name);
+ if (IS_ERR(worker)) {
+ err = PTR_ERR(worker);
+ pr_err("[%s] Worker create failed: %d!\n",
+ engine->name, err);
goto unwind;
}
- threads[tmp].task = tsk;
- get_task_struct(tsk);
- }
+ threads[tmp].worker = worker;
- yield(); /* start all threads before we begin */
+ kthread_init_work(&threads[tmp].work, active_engine);
+ kthread_queue_work(threads[tmp].worker,
+ &threads[tmp].work);
+ }
st_engine_heartbeat_disable_no_pm(engine);
GEM_BUG_ON(test_and_set_bit(I915_RESET_ENGINE + id,
@@ -1197,17 +1187,20 @@ static int __igt_reset_engines(struct intel_gt *gt,
for_each_engine(other, gt, tmp) {
int ret;
- if (!threads[tmp].task)
+ if (!threads[tmp].worker)
continue;
- ret = kthread_stop(threads[tmp].task);
+ WRITE_ONCE(threads[tmp].stop, true);
+ kthread_flush_work(&threads[tmp].work);
+ ret = READ_ONCE(threads[tmp].result);
if (ret) {
pr_err("kthread for other engine %s failed, err=%d\n",
other->name, ret);
if (!err)
err = ret;
}
- put_task_struct(threads[tmp].task);
+
+ kthread_destroy_worker(threads[tmp].worker);
/* GuC based resets are not logged per engine */
if (!using_guc) {
@@ -1510,18 +1503,9 @@ static int __igt_reset_evict_vma(struct intel_gt *gt,
}
}
- i915_vma_lock(arg.vma);
- err = i915_request_await_object(rq, arg.vma->obj,
- flags & EXEC_OBJECT_WRITE);
- if (err == 0) {
- err = i915_vma_move_to_active(arg.vma, rq, flags);
- if (err)
- pr_err("[%s] Move to active failed: %d!\n", engine->name, err);
- } else {
- pr_err("[%s] Request await failed: %d!\n", engine->name, err);
- }
-
- i915_vma_unlock(arg.vma);
+ err = igt_vma_move_to_active_unlocked(arg.vma, rq, flags);
+ if (err)
+ pr_err("[%s] Move to active failed: %d!\n", engine->name, err);
if (flags & EXEC_OBJECT_NEEDS_FENCE)
i915_vma_unpin_fence(arg.vma);
diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
index 82d3f80..7c56ffd 100644
--- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
+++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
@@ -452,9 +452,7 @@ static int __live_lrc_state(struct intel_engine_cs *engine,
*cs++ = i915_ggtt_offset(scratch) + RING_TAIL_IDX * sizeof(u32);
*cs++ = 0;
- err = i915_request_await_object(rq, scratch->obj, true);
- if (!err)
- err = i915_vma_move_to_active(scratch, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(scratch, rq, EXEC_OBJECT_WRITE);
i915_request_get(rq);
i915_request_add(rq);
@@ -602,9 +600,7 @@ __gpr_read(struct intel_context *ce, struct i915_vma *scratch, u32 *slot)
}
i915_vma_lock(scratch);
- err = i915_request_await_object(rq, scratch->obj, true);
- if (!err)
- err = i915_vma_move_to_active(scratch, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(scratch, rq, EXEC_OBJECT_WRITE);
i915_vma_unlock(scratch);
i915_request_get(rq);
@@ -1053,21 +1049,6 @@ store_context(struct intel_context *ce, struct i915_vma *scratch)
return batch;
}
-static int move_to_active(struct i915_request *rq,
- struct i915_vma *vma,
- unsigned int flags)
-{
- int err;
-
- i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, flags);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, flags);
- i915_vma_unlock(vma);
-
- return err;
-}
-
static struct i915_request *
record_registers(struct intel_context *ce,
struct i915_vma *before,
@@ -1093,19 +1074,19 @@ record_registers(struct intel_context *ce,
if (IS_ERR(rq))
goto err_after;
- err = move_to_active(rq, before, EXEC_OBJECT_WRITE);
+ err = igt_vma_move_to_active_unlocked(before, rq, EXEC_OBJECT_WRITE);
if (err)
goto err_rq;
- err = move_to_active(rq, b_before, 0);
+ err = igt_vma_move_to_active_unlocked(b_before, rq, 0);
if (err)
goto err_rq;
- err = move_to_active(rq, after, EXEC_OBJECT_WRITE);
+ err = igt_vma_move_to_active_unlocked(after, rq, EXEC_OBJECT_WRITE);
if (err)
goto err_rq;
- err = move_to_active(rq, b_after, 0);
+ err = igt_vma_move_to_active_unlocked(b_after, rq, 0);
if (err)
goto err_rq;
@@ -1243,7 +1224,7 @@ static int poison_registers(struct intel_context *ce, u32 poison, u32 *sema)
goto err_batch;
}
- err = move_to_active(rq, batch, 0);
+ err = igt_vma_move_to_active_unlocked(batch, rq, 0);
if (err)
goto err_rq;
diff --git a/drivers/gpu/drm/i915/gt/selftest_migrate.c b/drivers/gpu/drm/i915/gt/selftest_migrate.c
index 2b0c879..0dc5309 100644
--- a/drivers/gpu/drm/i915/gt/selftest_migrate.c
+++ b/drivers/gpu/drm/i915/gt/selftest_migrate.c
@@ -6,6 +6,7 @@
#include <linux/sort.h>
#include "gem/i915_gem_internal.h"
+#include "gem/i915_gem_lmem.h"
#include "selftests/i915_random.h"
diff --git a/drivers/gpu/drm/i915/gt/selftest_mocs.c b/drivers/gpu/drm/i915/gt/selftest_mocs.c
index c1d8613..f27cc28 100644
--- a/drivers/gpu/drm/i915/gt/selftest_mocs.c
+++ b/drivers/gpu/drm/i915/gt/selftest_mocs.c
@@ -7,6 +7,7 @@
#include "gt/intel_gpu_commands.h"
#include "i915_selftest.h"
+#include "gem/selftests/igt_gem_utils.h"
#include "gem/selftests/mock_context.h"
#include "selftests/igt_reset.h"
#include "selftests/igt_spinner.h"
@@ -228,9 +229,7 @@ static int check_mocs_engine(struct live_mocs *arg,
return PTR_ERR(rq);
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, true);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
i915_vma_unlock(vma);
/* Read the mocs tables back using SRM */
diff --git a/drivers/gpu/drm/i915/gt/selftest_rc6.c b/drivers/gpu/drm/i915/gt/selftest_rc6.c
index 8c70b7e..2ceeade 100644
--- a/drivers/gpu/drm/i915/gt/selftest_rc6.c
+++ b/drivers/gpu/drm/i915/gt/selftest_rc6.c
@@ -19,11 +19,11 @@ static u64 rc6_residency(struct intel_rc6 *rc6)
/* XXX VLV_GT_MEDIA_RC6? */
- result = intel_rc6_residency_ns(rc6, GEN6_GT_GFX_RC6);
+ result = intel_rc6_residency_ns(rc6, INTEL_RC6_RES_RC6);
if (HAS_RC6p(rc6_to_i915(rc6)))
- result += intel_rc6_residency_ns(rc6, GEN6_GT_GFX_RC6p);
+ result += intel_rc6_residency_ns(rc6, INTEL_RC6_RES_RC6p);
if (HAS_RC6pp(rc6_to_i915(rc6)))
- result += intel_rc6_residency_ns(rc6, GEN6_GT_GFX_RC6pp);
+ result += intel_rc6_residency_ns(rc6, INTEL_RC6_RES_RC6pp);
return result;
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_rps.c b/drivers/gpu/drm/i915/gt/selftest_rps.c
index cfb4708..39f1b75 100644
--- a/drivers/gpu/drm/i915/gt/selftest_rps.c
+++ b/drivers/gpu/drm/i915/gt/selftest_rps.c
@@ -652,9 +652,7 @@ int live_rps_frequency_cs(void *arg)
goto err_vma;
}
- err = i915_request_await_object(rq, vma->obj, false);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
if (!err)
err = rq->engine->emit_bb_start(rq,
vma->node.start,
@@ -793,9 +791,7 @@ int live_rps_frequency_srm(void *arg)
goto err_vma;
}
- err = i915_request_await_object(rq, vma->obj, false);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
if (!err)
err = rq->engine->emit_bb_start(rq,
vma->node.start,
@@ -1107,21 +1103,27 @@ static u64 __measure_power(int duration_ms)
return div64_u64(1000 * 1000 * dE, dt);
}
-static u64 measure_power_at(struct intel_rps *rps, int *freq)
+static u64 measure_power(struct intel_rps *rps, int *freq)
{
u64 x[5];
int i;
- *freq = rps_set_check(rps, *freq);
for (i = 0; i < 5; i++)
x[i] = __measure_power(5);
- *freq = (*freq + read_cagf(rps)) / 2;
+
+ *freq = (*freq + intel_rps_read_actual_frequency(rps)) / 2;
/* A simple triangle filter for better result stability */
sort(x, 5, sizeof(*x), cmp_u64, NULL);
return div_u64(x[1] + 2 * x[2] + x[3], 4);
}
+static u64 measure_power_at(struct intel_rps *rps, int *freq)
+{
+ *freq = rps_set_check(rps, *freq);
+ return measure_power(rps, freq);
+}
+
int live_rps_power(void *arg)
{
struct intel_gt *gt = arg;
diff --git a/drivers/gpu/drm/i915/gt/selftest_slpc.c b/drivers/gpu/drm/i915/gt/selftest_slpc.c
index f8a1d27..bd44ce7 100644
--- a/drivers/gpu/drm/i915/gt/selftest_slpc.c
+++ b/drivers/gpu/drm/i915/gt/selftest_slpc.c
@@ -11,7 +11,16 @@
enum test_type {
VARY_MIN,
VARY_MAX,
- MAX_GRANTED
+ MAX_GRANTED,
+ SLPC_POWER,
+ TILE_INTERACTION,
+};
+
+struct slpc_thread {
+ struct kthread_worker *worker;
+ struct kthread_work work;
+ struct intel_gt *gt;
+ int result;
};
static int slpc_set_min_freq(struct intel_guc_slpc *slpc, u32 freq)
@@ -41,6 +50,39 @@ static int slpc_set_max_freq(struct intel_guc_slpc *slpc, u32 freq)
return ret;
}
+static int slpc_set_freq(struct intel_gt *gt, u32 freq)
+{
+ int err;
+ struct intel_guc_slpc *slpc = >->uc.guc.slpc;
+
+ err = slpc_set_max_freq(slpc, freq);
+ if (err) {
+ pr_err("Unable to update max freq");
+ return err;
+ }
+
+ err = slpc_set_min_freq(slpc, freq);
+ if (err) {
+ pr_err("Unable to update min freq");
+ return err;
+ }
+
+ return err;
+}
+
+static u64 measure_power_at_freq(struct intel_gt *gt, int *freq, u64 *power)
+{
+ int err = 0;
+
+ err = slpc_set_freq(gt, *freq);
+ if (err)
+ return err;
+ *freq = intel_rps_read_actual_frequency(>->rps);
+ *power = measure_power(>->rps, freq);
+
+ return err;
+}
+
static int vary_max_freq(struct intel_guc_slpc *slpc, struct intel_rps *rps,
u32 *max_act_freq)
{
@@ -113,6 +155,58 @@ static int vary_min_freq(struct intel_guc_slpc *slpc, struct intel_rps *rps,
return err;
}
+static int slpc_power(struct intel_gt *gt, struct intel_engine_cs *engine)
+{
+ struct intel_guc_slpc *slpc = >->uc.guc.slpc;
+ struct {
+ u64 power;
+ int freq;
+ } min, max;
+ int err = 0;
+
+ /*
+ * Our fundamental assumption is that running at lower frequency
+ * actually saves power. Let's see if our RAPL measurement supports
+ * that theory.
+ */
+ if (!librapl_supported(gt->i915))
+ return 0;
+
+ min.freq = slpc->min_freq;
+ err = measure_power_at_freq(gt, &min.freq, &min.power);
+
+ if (err)
+ return err;
+
+ max.freq = slpc->rp0_freq;
+ err = measure_power_at_freq(gt, &max.freq, &max.power);
+
+ if (err)
+ return err;
+
+ pr_info("%s: min:%llumW @ %uMHz, max:%llumW @ %uMHz\n",
+ engine->name,
+ min.power, min.freq,
+ max.power, max.freq);
+
+ if (10 * min.freq >= 9 * max.freq) {
+ pr_notice("Could not control frequency, ran at [%uMHz, %uMhz]\n",
+ min.freq, max.freq);
+ }
+
+ if (11 * min.power > 10 * max.power) {
+ pr_err("%s: did not conserve power when setting lower frequency!\n",
+ engine->name);
+ err = -EINVAL;
+ }
+
+ /* Restore min/max frequencies */
+ slpc_set_max_freq(slpc, slpc->rp0_freq);
+ slpc_set_min_freq(slpc, slpc->min_freq);
+
+ return err;
+}
+
static int max_granted_freq(struct intel_guc_slpc *slpc, struct intel_rps *rps, u32 *max_act_freq)
{
struct intel_gt *gt = rps_to_gt(rps);
@@ -126,7 +220,8 @@ static int max_granted_freq(struct intel_guc_slpc *slpc, struct intel_rps *rps,
*max_act_freq = intel_rps_read_actual_frequency(rps);
if (*max_act_freq != slpc->rp0_freq) {
/* Check if there was some throttling by pcode */
- perf_limit_reasons = intel_uncore_read(gt->uncore, GT0_PERF_LIMIT_REASONS);
+ perf_limit_reasons = intel_uncore_read(gt->uncore,
+ intel_gt_perf_limit_reasons_reg(gt));
/* If not, this is an error */
if (!(perf_limit_reasons & GT0_PERF_LIMIT_REASONS_MASK)) {
@@ -153,6 +248,11 @@ static int run_test(struct intel_gt *gt, int test_type)
if (!intel_uc_uses_guc_slpc(>->uc))
return 0;
+ if (slpc->min_freq == slpc->rp0_freq) {
+ pr_err("Min/Max are fused to the same value\n");
+ return -EINVAL;
+ }
+
if (igt_spinner_init(&spin, gt))
return -ENOMEM;
@@ -167,17 +267,14 @@ static int run_test(struct intel_gt *gt, int test_type)
}
/*
- * FIXME: With efficient frequency enabled, GuC can request
- * frequencies higher than the SLPC max. While this is fixed
- * in GuC, we level set these tests with RPn as min.
+ * Set min frequency to RPn so that we can test the whole
+ * range of RPn-RP0. This also turns off efficient freq
+ * usage and makes results more predictable.
*/
err = slpc_set_min_freq(slpc, slpc->min_freq);
- if (err)
+ if (err) {
+ pr_err("Unable to update min freq!");
return err;
-
- if (slpc->min_freq == slpc->rp0_freq) {
- pr_err("Min/Max are fused to the same value\n");
- return -EINVAL;
}
intel_gt_pm_wait_for_idle(gt);
@@ -222,9 +319,10 @@ static int run_test(struct intel_gt *gt, int test_type)
break;
case MAX_GRANTED:
+ case TILE_INTERACTION:
/* Media engines have a different RP0 */
- if (engine->class == VIDEO_DECODE_CLASS ||
- engine->class == VIDEO_ENHANCEMENT_CLASS) {
+ if (gt->type != GT_MEDIA && (engine->class == VIDEO_DECODE_CLASS ||
+ engine->class == VIDEO_ENHANCEMENT_CLASS)) {
igt_spinner_end(&spin);
st_engine_heartbeat_enable(engine);
err = 0;
@@ -233,17 +331,24 @@ static int run_test(struct intel_gt *gt, int test_type)
err = max_granted_freq(slpc, rps, &max_act_freq);
break;
+
+ case SLPC_POWER:
+ err = slpc_power(gt, engine);
+ break;
}
- pr_info("Max actual frequency for %s was %d\n",
- engine->name, max_act_freq);
+ if (test_type != SLPC_POWER) {
+ pr_info("Max actual frequency for %s was %d\n",
+ engine->name, max_act_freq);
- /* Actual frequency should rise above min */
- if (max_act_freq <= slpc_min_freq) {
- pr_err("Actual freq did not rise above min\n");
- pr_err("Perf Limit Reasons: 0x%x\n",
- intel_uncore_read(gt->uncore, GT0_PERF_LIMIT_REASONS));
- err = -EINVAL;
+ /* Actual frequency should rise above min */
+ if (max_act_freq <= slpc->min_freq) {
+ pr_err("Actual freq did not rise above min\n");
+ pr_err("Perf Limit Reasons: 0x%x\n",
+ intel_uncore_read(gt->uncore,
+ intel_gt_perf_limit_reasons_reg(gt)));
+ err = -EINVAL;
+ }
}
igt_spinner_end(&spin);
@@ -270,26 +375,116 @@ static int run_test(struct intel_gt *gt, int test_type)
static int live_slpc_vary_min(void *arg)
{
struct drm_i915_private *i915 = arg;
- struct intel_gt *gt = to_gt(i915);
+ struct intel_gt *gt;
+ unsigned int i;
+ int ret;
- return run_test(gt, VARY_MIN);
+ for_each_gt(gt, i915, i) {
+ ret = run_test(gt, VARY_MIN);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
}
static int live_slpc_vary_max(void *arg)
{
struct drm_i915_private *i915 = arg;
- struct intel_gt *gt = to_gt(i915);
+ struct intel_gt *gt;
+ unsigned int i;
+ int ret;
- return run_test(gt, VARY_MAX);
+ for_each_gt(gt, i915, i) {
+ ret = run_test(gt, VARY_MAX);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
}
/* check if pcode can grant RP0 */
static int live_slpc_max_granted(void *arg)
{
struct drm_i915_private *i915 = arg;
- struct intel_gt *gt = to_gt(i915);
+ struct intel_gt *gt;
+ unsigned int i;
+ int ret;
- return run_test(gt, MAX_GRANTED);
+ for_each_gt(gt, i915, i) {
+ ret = run_test(gt, MAX_GRANTED);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+static int live_slpc_power(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct intel_gt *gt;
+ unsigned int i;
+ int ret;
+
+ for_each_gt(gt, i915, i) {
+ ret = run_test(gt, SLPC_POWER);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+static void slpc_spinner_thread(struct kthread_work *work)
+{
+ struct slpc_thread *thread = container_of(work, typeof(*thread), work);
+
+ thread->result = run_test(thread->gt, TILE_INTERACTION);
+}
+
+static int live_slpc_tile_interaction(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct intel_gt *gt;
+ struct slpc_thread *threads;
+ int i = 0, ret = 0;
+
+ threads = kcalloc(I915_MAX_GT, sizeof(*threads), GFP_KERNEL);
+ if (!threads)
+ return -ENOMEM;
+
+ for_each_gt(gt, i915, i) {
+ threads[i].worker = kthread_create_worker(0, "igt/slpc_parallel:%d", gt->info.id);
+
+ if (IS_ERR(threads[i].worker)) {
+ ret = PTR_ERR(threads[i].worker);
+ break;
+ }
+
+ threads[i].gt = gt;
+ kthread_init_work(&threads[i].work, slpc_spinner_thread);
+ kthread_queue_work(threads[i].worker, &threads[i].work);
+ }
+
+ for_each_gt(gt, i915, i) {
+ int status;
+
+ if (IS_ERR_OR_NULL(threads[i].worker))
+ continue;
+
+ kthread_flush_work(&threads[i].work);
+ status = READ_ONCE(threads[i].result);
+ if (status && !ret) {
+ pr_err("%s GT %d failed ", __func__, gt->info.id);
+ ret = status;
+ }
+ kthread_destroy_worker(threads[i].worker);
+ }
+
+ kfree(threads);
+ return ret;
}
int intel_slpc_live_selftests(struct drm_i915_private *i915)
@@ -298,10 +493,17 @@ int intel_slpc_live_selftests(struct drm_i915_private *i915)
SUBTEST(live_slpc_vary_max),
SUBTEST(live_slpc_vary_min),
SUBTEST(live_slpc_max_granted),
+ SUBTEST(live_slpc_power),
+ SUBTEST(live_slpc_tile_interaction),
};
- if (intel_gt_is_wedged(to_gt(i915)))
- return 0;
+ struct intel_gt *gt;
+ unsigned int i;
+
+ for_each_gt(gt, i915, i) {
+ if (intel_gt_is_wedged(gt))
+ return 0;
+ }
return i915_live_subtests(tests, i915);
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_workarounds.c b/drivers/gpu/drm/i915/gt/selftest_workarounds.c
index 67a9aab..96e3861 100644
--- a/drivers/gpu/drm/i915/gt/selftest_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/selftest_workarounds.c
@@ -66,14 +66,14 @@ reference_lists_init(struct intel_gt *gt, struct wa_lists *lists)
memset(lists, 0, sizeof(*lists));
- wa_init_start(&lists->gt_wa_list, "GT_REF", "global");
+ wa_init_start(&lists->gt_wa_list, gt, "GT_REF", "global");
gt_init_workarounds(gt, &lists->gt_wa_list);
wa_init_finish(&lists->gt_wa_list);
for_each_engine(engine, gt, id) {
struct i915_wa_list *wal = &lists->engine[id].wa_list;
- wa_init_start(wal, "REF", engine->name);
+ wa_init_start(wal, gt, "REF", engine->name);
engine_init_workarounds(engine, wal);
wa_init_finish(wal);
@@ -139,9 +139,7 @@ read_nonprivs(struct intel_context *ce)
}
i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
i915_vma_unlock(vma);
if (err)
goto err_req;
@@ -632,16 +630,12 @@ static int check_dirty_whitelist(struct intel_context *ce)
goto err_request;
}
- err = i915_request_await_object(rq, batch->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(batch, rq, 0);
+ err = i915_vma_move_to_active(batch, rq, 0);
if (err)
goto err_request;
- err = i915_request_await_object(rq, scratch->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(scratch, rq,
- EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(scratch, rq,
+ EXEC_OBJECT_WRITE);
if (err)
goto err_request;
@@ -860,9 +854,7 @@ static int read_whitelisted_registers(struct intel_context *ce,
return PTR_ERR(rq);
i915_vma_lock(results);
- err = i915_request_await_object(rq, results->obj, true);
- if (err == 0)
- err = i915_vma_move_to_active(results, rq, EXEC_OBJECT_WRITE);
+ err = i915_vma_move_to_active(results, rq, EXEC_OBJECT_WRITE);
i915_vma_unlock(results);
if (err)
goto err_req;
@@ -944,9 +936,7 @@ static int scrub_whitelisted_registers(struct intel_context *ce)
}
i915_vma_lock(batch);
- err = i915_request_await_object(rq, batch->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(batch, rq, 0);
+ err = i915_vma_move_to_active(batch, rq, 0);
i915_vma_unlock(batch);
if (err)
goto err_request;
@@ -991,7 +981,7 @@ static bool pardon_reg(struct drm_i915_private *i915, i915_reg_t reg)
/* Alas, we must pardon some whitelists. Mistakes already made */
static const struct regmask pardon[] = {
{ GEN9_CTX_PREEMPT_REG, 9 },
- { GEN8_L3SQCREG4, 9 },
+ { _MMIO(0xb118), 9 }, /* GEN8_L3SQCREG4 */
};
return find_reg(i915, reg, pardon, ARRAY_SIZE(pardon));
diff --git a/drivers/gpu/drm/i915/gt/sysfs_engines.c b/drivers/gpu/drm/i915/gt/sysfs_engines.c
index 9670310..f2d9858 100644
--- a/drivers/gpu/drm/i915/gt/sysfs_engines.c
+++ b/drivers/gpu/drm/i915/gt/sysfs_engines.c
@@ -144,7 +144,7 @@ max_spin_store(struct kobject *kobj, struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct intel_engine_cs *engine = kobj_to_engine(kobj);
- unsigned long long duration;
+ unsigned long long duration, clamped;
int err;
/*
@@ -168,7 +168,8 @@ max_spin_store(struct kobject *kobj, struct kobj_attribute *attr,
if (err)
return err;
- if (duration > jiffies_to_nsecs(2))
+ clamped = intel_clamp_max_busywait_duration_ns(engine, duration);
+ if (duration != clamped)
return -EINVAL;
WRITE_ONCE(engine->props.max_busywait_duration_ns, duration);
@@ -203,7 +204,7 @@ timeslice_store(struct kobject *kobj, struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct intel_engine_cs *engine = kobj_to_engine(kobj);
- unsigned long long duration;
+ unsigned long long duration, clamped;
int err;
/*
@@ -218,7 +219,8 @@ timeslice_store(struct kobject *kobj, struct kobj_attribute *attr,
if (err)
return err;
- if (duration > jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT))
+ clamped = intel_clamp_timeslice_duration_ms(engine, duration);
+ if (duration != clamped)
return -EINVAL;
WRITE_ONCE(engine->props.timeslice_duration_ms, duration);
@@ -256,7 +258,7 @@ stop_store(struct kobject *kobj, struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct intel_engine_cs *engine = kobj_to_engine(kobj);
- unsigned long long duration;
+ unsigned long long duration, clamped;
int err;
/*
@@ -272,7 +274,8 @@ stop_store(struct kobject *kobj, struct kobj_attribute *attr,
if (err)
return err;
- if (duration > jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT))
+ clamped = intel_clamp_stop_timeout_ms(engine, duration);
+ if (duration != clamped)
return -EINVAL;
WRITE_ONCE(engine->props.stop_timeout_ms, duration);
@@ -306,7 +309,7 @@ preempt_timeout_store(struct kobject *kobj, struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct intel_engine_cs *engine = kobj_to_engine(kobj);
- unsigned long long timeout;
+ unsigned long long timeout, clamped;
int err;
/*
@@ -322,7 +325,8 @@ preempt_timeout_store(struct kobject *kobj, struct kobj_attribute *attr,
if (err)
return err;
- if (timeout > jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT))
+ clamped = intel_clamp_preempt_timeout_ms(engine, timeout);
+ if (timeout != clamped)
return -EINVAL;
WRITE_ONCE(engine->props.preempt_timeout_ms, timeout);
@@ -362,7 +366,7 @@ heartbeat_store(struct kobject *kobj, struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct intel_engine_cs *engine = kobj_to_engine(kobj);
- unsigned long long delay;
+ unsigned long long delay, clamped;
int err;
/*
@@ -379,7 +383,8 @@ heartbeat_store(struct kobject *kobj, struct kobj_attribute *attr,
if (err)
return err;
- if (delay >= jiffies_to_msecs(MAX_SCHEDULE_TIMEOUT))
+ clamped = intel_clamp_heartbeat_interval_ms(engine, delay);
+ if (delay != clamped)
return -EINVAL;
err = intel_engine_set_heartbeat(engine, delay);
diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
index 29ef8af..f359bef 100644
--- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
+++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
@@ -117,6 +117,7 @@ enum intel_guc_action {
INTEL_GUC_ACTION_ENTER_S_STATE = 0x501,
INTEL_GUC_ACTION_EXIT_S_STATE = 0x502,
INTEL_GUC_ACTION_GLOBAL_SCHED_POLICY_CHANGE = 0x506,
+ INTEL_GUC_ACTION_UPDATE_SCHEDULING_POLICIES_KLV = 0x509,
INTEL_GUC_ACTION_SCHED_CONTEXT = 0x1000,
INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_SET = 0x1001,
INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE = 0x1002,
diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_slpc_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_slpc_abi.h
index 4c840a2..811add1 100644
--- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_slpc_abi.h
+++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_slpc_abi.h
@@ -128,6 +128,15 @@ enum slpc_media_ratio_mode {
SLPC_MEDIA_RATIO_MODE_FIXED_ONE_TO_TWO = 2,
};
+enum slpc_gucrc_mode {
+ SLPC_GUCRC_MODE_HW = 0,
+ SLPC_GUCRC_MODE_GUCRC_NO_RC6 = 1,
+ SLPC_GUCRC_MODE_GUCRC_STATIC_TIMEOUT = 2,
+ SLPC_GUCRC_MODE_GUCRC_DYNAMIC_HYSTERESIS = 3,
+
+ SLPC_GUCRC_MODE_MAX,
+};
+
enum slpc_event_id {
SLPC_EVENT_RESET = 0,
SLPC_EVENT_SHUTDOWN = 1,
diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h
index 4a59478..58012ed 100644
--- a/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h
+++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h
@@ -82,9 +82,16 @@
#define GUC_KLV_SELF_CFG_G2H_CTB_SIZE_LEN 1u
/*
+ * Global scheduling policy update keys.
+ */
+enum {
+ GUC_SCHEDULING_POLICIES_KLV_ID_RENDER_COMPUTE_YIELD = 0x1001,
+};
+
+/*
* Per context scheduling policy update keys.
*/
-enum {
+enum {
GUC_CONTEXT_POLICIES_KLV_ID_EXECUTION_QUANTUM = 0x2001,
GUC_CONTEXT_POLICIES_KLV_ID_PREEMPTION_TIMEOUT = 0x2002,
GUC_CONTEXT_POLICIES_KLV_ID_SCHEDULING_PRIORITY = 0x2003,
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
index bac06e3..52aede3 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
@@ -98,6 +98,8 @@ static void gen9_enable_guc_interrupts(struct intel_guc *guc)
gt->pm_guc_events);
gen6_gt_pm_enable_irq(gt, gt->pm_guc_events);
spin_unlock_irq(gt->irq_lock);
+
+ guc->interrupts.enabled = true;
}
static void gen9_disable_guc_interrupts(struct intel_guc *guc)
@@ -105,6 +107,7 @@ static void gen9_disable_guc_interrupts(struct intel_guc *guc)
struct intel_gt *gt = guc_to_gt(guc);
assert_rpm_wakelock_held(>->i915->runtime_pm);
+ guc->interrupts.enabled = false;
spin_lock_irq(gt->irq_lock);
@@ -116,39 +119,39 @@ static void gen9_disable_guc_interrupts(struct intel_guc *guc)
gen9_reset_guc_interrupts(guc);
}
+static bool __gen11_reset_guc_interrupts(struct intel_gt *gt)
+{
+ u32 irq = gt->type == GT_MEDIA ? MTL_MGUC : GEN11_GUC;
+
+ lockdep_assert_held(gt->irq_lock);
+ return gen11_gt_reset_one_iir(gt, 0, irq);
+}
+
static void gen11_reset_guc_interrupts(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
spin_lock_irq(gt->irq_lock);
- gen11_gt_reset_one_iir(gt, 0, GEN11_GUC);
+ __gen11_reset_guc_interrupts(gt);
spin_unlock_irq(gt->irq_lock);
}
static void gen11_enable_guc_interrupts(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
- u32 events = REG_FIELD_PREP(ENGINE1_MASK, GUC_INTR_GUC2HOST);
spin_lock_irq(gt->irq_lock);
- WARN_ON_ONCE(gen11_gt_reset_one_iir(gt, 0, GEN11_GUC));
- intel_uncore_write(gt->uncore,
- GEN11_GUC_SG_INTR_ENABLE, events);
- intel_uncore_write(gt->uncore,
- GEN11_GUC_SG_INTR_MASK, ~events);
+ __gen11_reset_guc_interrupts(gt);
spin_unlock_irq(gt->irq_lock);
+
+ guc->interrupts.enabled = true;
}
static void gen11_disable_guc_interrupts(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
- spin_lock_irq(gt->irq_lock);
-
- intel_uncore_write(gt->uncore, GEN11_GUC_SG_INTR_MASK, ~0);
- intel_uncore_write(gt->uncore, GEN11_GUC_SG_INTR_ENABLE, 0);
-
- spin_unlock_irq(gt->irq_lock);
+ guc->interrupts.enabled = false;
intel_synchronize_irq(gt->i915);
gen11_reset_guc_interrupts(guc);
@@ -156,7 +159,8 @@ static void gen11_disable_guc_interrupts(struct intel_guc *guc)
void intel_guc_init_early(struct intel_guc *guc)
{
- struct drm_i915_private *i915 = guc_to_gt(guc)->i915;
+ struct intel_gt *gt = guc_to_gt(guc);
+ struct drm_i915_private *i915 = gt->i915;
intel_uc_fw_init_early(&guc->fw, INTEL_UC_FW_TYPE_GUC);
intel_guc_ct_init_early(&guc->ct);
@@ -168,12 +172,17 @@ void intel_guc_init_early(struct intel_guc *guc)
mutex_init(&guc->send_mutex);
spin_lock_init(&guc->irq_lock);
if (GRAPHICS_VER(i915) >= 11) {
- guc->notify_reg = GEN11_GUC_HOST_INTERRUPT;
guc->interrupts.reset = gen11_reset_guc_interrupts;
guc->interrupts.enable = gen11_enable_guc_interrupts;
guc->interrupts.disable = gen11_disable_guc_interrupts;
- guc->send_regs.base =
- i915_mmio_reg_offset(GEN11_SOFT_SCRATCH(0));
+ if (gt->type == GT_MEDIA) {
+ guc->notify_reg = MEDIA_GUC_HOST_INTERRUPT;
+ guc->send_regs.base = i915_mmio_reg_offset(MEDIA_SOFT_SCRATCH(0));
+ } else {
+ guc->notify_reg = GEN11_GUC_HOST_INTERRUPT;
+ guc->send_regs.base = i915_mmio_reg_offset(GEN11_SOFT_SCRATCH(0));
+ }
+
guc->send_regs.count = GEN11_SOFT_SCRATCH_COUNT;
} else {
@@ -441,6 +450,7 @@ int intel_guc_init(struct intel_guc *guc)
err_fw:
intel_uc_fw_fini(&guc->fw);
out:
+ intel_uc_fw_change_status(&guc->fw, INTEL_UC_FIRMWARE_INIT_FAIL);
i915_probe_error(gt->i915, "failed with %d\n", ret);
return ret;
}
@@ -870,14 +880,14 @@ void intel_guc_load_status(struct intel_guc *guc, struct drm_printer *p)
u32 status = intel_uncore_read(uncore, GUC_STATUS);
u32 i;
- drm_printf(p, "\nGuC status 0x%08x:\n", status);
+ drm_printf(p, "GuC status 0x%08x:\n", status);
drm_printf(p, "\tBootrom status = 0x%x\n",
(status & GS_BOOTROM_MASK) >> GS_BOOTROM_SHIFT);
drm_printf(p, "\tuKernel status = 0x%x\n",
(status & GS_UKERNEL_MASK) >> GS_UKERNEL_SHIFT);
drm_printf(p, "\tMIA Core status = 0x%x\n",
(status & GS_MIA_MASK) >> GS_MIA_SHIFT);
- drm_puts(p, "\nScratch registers:\n");
+ drm_puts(p, "Scratch registers:\n");
for (i = 0; i < 16; i++) {
drm_printf(p, "\t%2d: \t0x%x\n",
i, intel_uncore_read(uncore, SOFT_SCRATCH(i)));
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 804133d..1bb3f98 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -78,6 +78,7 @@ struct intel_guc {
/** @interrupts: pointers to GuC interrupt-managing functions. */
struct {
+ bool enabled;
void (*reset)(struct intel_guc *guc);
void (*enable)(struct intel_guc *guc);
void (*disable)(struct intel_guc *guc);
@@ -113,6 +114,10 @@ struct intel_guc {
*/
struct list_head guc_id_list;
/**
+ * @guc_ids_in_use: Number single-lrc guc_ids in use
+ */
+ unsigned int guc_ids_in_use;
+ /**
* @destroyed_contexts: list of contexts waiting to be destroyed
* (deregistered with the GuC)
*/
@@ -132,6 +137,16 @@ struct intel_guc {
* @reset_fail_mask: mask of engines that failed to reset
*/
intel_engine_mask_t reset_fail_mask;
+ /**
+ * @sched_disable_delay_ms: schedule disable delay, in ms, for
+ * contexts
+ */
+ unsigned int sched_disable_delay_ms;
+ /**
+ * @sched_disable_gucid_threshold: threshold of min remaining available
+ * guc_ids before we start bypassing the schedule disable delay
+ */
+ unsigned int sched_disable_gucid_threshold;
} submission_state;
/**
@@ -316,9 +331,11 @@ static inline int intel_guc_send_busy_loop(struct intel_guc *guc,
return err;
}
+/* Only call this from the interrupt handler code */
static inline void intel_guc_to_host_event_handler(struct intel_guc *guc)
{
- intel_guc_ct_event_handler(&guc->ct);
+ if (guc->interrupts.enabled)
+ intel_guc_ct_event_handler(&guc->ct);
}
/* GuC addresses above GUC_GGTT_TOP also don't map through the GTT */
@@ -466,4 +483,6 @@ void intel_guc_write_barrier(struct intel_guc *guc);
void intel_guc_dump_time_info(struct intel_guc *guc, struct drm_printer *p);
+int intel_guc_sched_disable_gucid_threshold_max(struct intel_guc *guc);
+
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
index 74cbe8e..a7f737c 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
@@ -5,6 +5,7 @@
#include <linux/bsearch.h>
+#include "gem/i915_gem_lmem.h"
#include "gt/intel_engine_regs.h"
#include "gt/intel_gt.h"
#include "gt/intel_gt_mcr.h"
@@ -277,24 +278,16 @@ __mmio_reg_add(struct temp_regset *regset, struct guc_mmio_reg *reg)
return slot;
}
-#define GUC_REGSET_STEERING(group, instance) ( \
- FIELD_PREP(GUC_REGSET_STEERING_GROUP, (group)) | \
- FIELD_PREP(GUC_REGSET_STEERING_INSTANCE, (instance)) | \
- GUC_REGSET_NEEDS_STEERING \
-)
-
static long __must_check guc_mmio_reg_add(struct intel_gt *gt,
struct temp_regset *regset,
- i915_reg_t reg, u32 flags)
+ u32 offset, u32 flags)
{
u32 count = regset->storage_used - (regset->registers - regset->storage);
- u32 offset = i915_mmio_reg_offset(reg);
struct guc_mmio_reg entry = {
.offset = offset,
.flags = flags,
};
struct guc_mmio_reg *slot;
- u8 group, inst;
/*
* The mmio list is built using separate lists within the driver.
@@ -306,17 +299,6 @@ static long __must_check guc_mmio_reg_add(struct intel_gt *gt,
sizeof(entry), guc_mmio_reg_cmp))
return 0;
- /*
- * The GuC doesn't have a default steering, so we need to explicitly
- * steer all registers that need steering. However, we do not keep track
- * of all the steering ranges, only of those that have a chance of using
- * a non-default steering from the i915 pov. Instead of adding such
- * tracking, it is easier to just program the default steering for all
- * regs that don't need a non-default one.
- */
- intel_gt_mcr_get_nonterminated_steering(gt, reg, &group, &inst);
- entry.flags |= GUC_REGSET_STEERING(group, inst);
-
slot = __mmio_reg_add(regset, &entry);
if (IS_ERR(slot))
return PTR_ERR(slot);
@@ -335,6 +317,38 @@ static long __must_check guc_mmio_reg_add(struct intel_gt *gt,
#define GUC_MMIO_REG_ADD(gt, regset, reg, masked) \
guc_mmio_reg_add(gt, \
regset, \
+ i915_mmio_reg_offset(reg), \
+ (masked) ? GUC_REGSET_MASKED : 0)
+
+#define GUC_REGSET_STEERING(group, instance) ( \
+ FIELD_PREP(GUC_REGSET_STEERING_GROUP, (group)) | \
+ FIELD_PREP(GUC_REGSET_STEERING_INSTANCE, (instance)) | \
+ GUC_REGSET_NEEDS_STEERING \
+)
+
+static long __must_check guc_mcr_reg_add(struct intel_gt *gt,
+ struct temp_regset *regset,
+ i915_mcr_reg_t reg, u32 flags)
+{
+ u8 group, inst;
+
+ /*
+ * The GuC doesn't have a default steering, so we need to explicitly
+ * steer all registers that need steering. However, we do not keep track
+ * of all the steering ranges, only of those that have a chance of using
+ * a non-default steering from the i915 pov. Instead of adding such
+ * tracking, it is easier to just program the default steering for all
+ * regs that don't need a non-default one.
+ */
+ intel_gt_mcr_get_nonterminated_steering(gt, reg, &group, &inst);
+ flags |= GUC_REGSET_STEERING(group, inst);
+
+ return guc_mmio_reg_add(gt, regset, i915_mmio_reg_offset(reg), flags);
+}
+
+#define GUC_MCR_REG_ADD(gt, regset, reg, masked) \
+ guc_mcr_reg_add(gt, \
+ regset, \
(reg), \
(masked) ? GUC_REGSET_MASKED : 0)
@@ -372,8 +386,21 @@ static int guc_mmio_regset_init(struct temp_regset *regset,
false);
/* add in local MOCS registers */
- for (i = 0; i < GEN9_LNCFCMOCS_REG_COUNT; i++)
- ret |= GUC_MMIO_REG_ADD(gt, regset, GEN9_LNCFCMOCS(i), false);
+ for (i = 0; i < LNCFCMOCS_REG_COUNT; i++)
+ if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 50))
+ ret |= GUC_MCR_REG_ADD(gt, regset, XEHP_LNCFCMOCS(i), false);
+ else
+ ret |= GUC_MMIO_REG_ADD(gt, regset, GEN9_LNCFCMOCS(i), false);
+
+ if (GRAPHICS_VER(engine->i915) >= 12) {
+ ret |= GUC_MMIO_REG_ADD(gt, regset, EU_PERF_CNTL0, false);
+ ret |= GUC_MMIO_REG_ADD(gt, regset, EU_PERF_CNTL1, false);
+ ret |= GUC_MMIO_REG_ADD(gt, regset, EU_PERF_CNTL2, false);
+ ret |= GUC_MMIO_REG_ADD(gt, regset, EU_PERF_CNTL3, false);
+ ret |= GUC_MMIO_REG_ADD(gt, regset, EU_PERF_CNTL4, false);
+ ret |= GUC_MMIO_REG_ADD(gt, regset, EU_PERF_CNTL5, false);
+ ret |= GUC_MMIO_REG_ADD(gt, regset, EU_PERF_CNTL6, false);
+ }
return ret ? -1 : 0;
}
@@ -461,6 +488,11 @@ static void fill_engine_enable_masks(struct intel_gt *gt,
info_map_write(info_map, engine_enabled_masks[GUC_BLITTER_CLASS], BCS_MASK(gt));
info_map_write(info_map, engine_enabled_masks[GUC_VIDEO_CLASS], VDBOX_MASK(gt));
info_map_write(info_map, engine_enabled_masks[GUC_VIDEOENHANCE_CLASS], VEBOX_MASK(gt));
+
+ /* The GSC engine is an instance (6) of OTHER_CLASS */
+ if (gt->engine[GSC0])
+ info_map_write(info_map, engine_enabled_masks[GUC_GSC_OTHER_CLASS],
+ BIT(gt->engine[GSC0]->instance));
}
#define LR_HW_CONTEXT_SIZE (80 * sizeof(u32))
@@ -502,9 +534,6 @@ static int guc_prep_golden_context(struct intel_guc *guc)
}
for (engine_class = 0; engine_class <= MAX_ENGINE_CLASS; ++engine_class) {
- if (engine_class == OTHER_CLASS)
- continue;
-
guc_class = engine_class_to_guc_class(engine_class);
if (!info_map_read(&info_map, engine_enabled_masks[guc_class]))
@@ -582,9 +611,6 @@ static void guc_init_golden_context(struct intel_guc *guc)
addr_ggtt = intel_guc_ggtt_offset(guc, guc->ads_vma) + offset;
for (engine_class = 0; engine_class <= MAX_ENGINE_CLASS; ++engine_class) {
- if (engine_class == OTHER_CLASS)
- continue;
-
guc_class = engine_class_to_guc_class(engine_class);
if (!ads_blob_read(guc, system_info.engine_enabled_masks[guc_class]))
continue;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_capture.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_capture.c
index 8f11651..1c1b850 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_capture.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_capture.c
@@ -132,6 +132,11 @@ static const struct __guc_mmio_reg_descr xe_lpd_blt_inst_regs[] = {
COMMON_BASE_ENGINE_INSTANCE,
};
+/* XE_LPD - GSC Per-Engine-Instance */
+static const struct __guc_mmio_reg_descr xe_lpd_gsc_inst_regs[] = {
+ COMMON_BASE_ENGINE_INSTANCE,
+};
+
/* GEN9 - Global */
static const struct __guc_mmio_reg_descr default_global_regs[] = {
COMMON_BASE_GLOBAL,
@@ -165,16 +170,20 @@ static const struct __guc_mmio_reg_descr empty_regs_list[] = {
}
/* List of lists */
-static struct __guc_mmio_reg_descr_group default_lists[] = {
+static const struct __guc_mmio_reg_descr_group default_lists[] = {
MAKE_REGLIST(default_global_regs, PF, GLOBAL, 0),
MAKE_REGLIST(default_rc_class_regs, PF, ENGINE_CLASS, GUC_RENDER_CLASS),
MAKE_REGLIST(xe_lpd_rc_inst_regs, PF, ENGINE_INSTANCE, GUC_RENDER_CLASS),
+ MAKE_REGLIST(default_rc_class_regs, PF, ENGINE_CLASS, GUC_COMPUTE_CLASS),
+ MAKE_REGLIST(xe_lpd_rc_inst_regs, PF, ENGINE_INSTANCE, GUC_COMPUTE_CLASS),
MAKE_REGLIST(empty_regs_list, PF, ENGINE_CLASS, GUC_VIDEO_CLASS),
MAKE_REGLIST(xe_lpd_vd_inst_regs, PF, ENGINE_INSTANCE, GUC_VIDEO_CLASS),
MAKE_REGLIST(empty_regs_list, PF, ENGINE_CLASS, GUC_VIDEOENHANCE_CLASS),
MAKE_REGLIST(xe_lpd_vec_inst_regs, PF, ENGINE_INSTANCE, GUC_VIDEOENHANCE_CLASS),
MAKE_REGLIST(empty_regs_list, PF, ENGINE_CLASS, GUC_BLITTER_CLASS),
MAKE_REGLIST(xe_lpd_blt_inst_regs, PF, ENGINE_INSTANCE, GUC_BLITTER_CLASS),
+ MAKE_REGLIST(empty_regs_list, PF, ENGINE_CLASS, GUC_GSC_OTHER_CLASS),
+ MAKE_REGLIST(xe_lpd_gsc_inst_regs, PF, ENGINE_INSTANCE, GUC_GSC_OTHER_CLASS),
{}
};
@@ -182,12 +191,16 @@ static const struct __guc_mmio_reg_descr_group xe_lpd_lists[] = {
MAKE_REGLIST(xe_lpd_global_regs, PF, GLOBAL, 0),
MAKE_REGLIST(xe_lpd_rc_class_regs, PF, ENGINE_CLASS, GUC_RENDER_CLASS),
MAKE_REGLIST(xe_lpd_rc_inst_regs, PF, ENGINE_INSTANCE, GUC_RENDER_CLASS),
+ MAKE_REGLIST(xe_lpd_rc_class_regs, PF, ENGINE_CLASS, GUC_COMPUTE_CLASS),
+ MAKE_REGLIST(xe_lpd_rc_inst_regs, PF, ENGINE_INSTANCE, GUC_COMPUTE_CLASS),
MAKE_REGLIST(empty_regs_list, PF, ENGINE_CLASS, GUC_VIDEO_CLASS),
MAKE_REGLIST(xe_lpd_vd_inst_regs, PF, ENGINE_INSTANCE, GUC_VIDEO_CLASS),
MAKE_REGLIST(xe_lpd_vec_class_regs, PF, ENGINE_CLASS, GUC_VIDEOENHANCE_CLASS),
MAKE_REGLIST(xe_lpd_vec_inst_regs, PF, ENGINE_INSTANCE, GUC_VIDEOENHANCE_CLASS),
MAKE_REGLIST(empty_regs_list, PF, ENGINE_CLASS, GUC_BLITTER_CLASS),
MAKE_REGLIST(xe_lpd_blt_inst_regs, PF, ENGINE_INSTANCE, GUC_BLITTER_CLASS),
+ MAKE_REGLIST(empty_regs_list, PF, ENGINE_CLASS, GUC_GSC_OTHER_CLASS),
+ MAKE_REGLIST(xe_lpd_gsc_inst_regs, PF, ENGINE_INSTANCE, GUC_GSC_OTHER_CLASS),
{}
};
@@ -240,19 +253,19 @@ static void guc_capture_free_extlists(struct __guc_mmio_reg_descr_group *reglist
struct __ext_steer_reg {
const char *name;
- i915_reg_t reg;
+ i915_mcr_reg_t reg;
};
static const struct __ext_steer_reg xe_extregs[] = {
- {"GEN7_SAMPLER_INSTDONE", GEN7_SAMPLER_INSTDONE},
- {"GEN7_ROW_INSTDONE", GEN7_ROW_INSTDONE}
+ {"GEN8_SAMPLER_INSTDONE", GEN8_SAMPLER_INSTDONE},
+ {"GEN8_ROW_INSTDONE", GEN8_ROW_INSTDONE}
};
static void __fill_ext_reg(struct __guc_mmio_reg_descr *ext,
const struct __ext_steer_reg *extlist,
int slice_id, int subslice_id)
{
- ext->reg = extlist->reg;
+ ext->reg = _MMIO(i915_mmio_reg_offset(extlist->reg));
ext->flags = FIELD_PREP(GUC_REGSET_STEERING_GROUP, slice_id);
ext->flags |= FIELD_PREP(GUC_REGSET_STEERING_INSTANCE, subslice_id);
ext->regname = extlist->name;
@@ -419,6 +432,46 @@ guc_capture_get_device_reglist(struct intel_guc *guc)
return default_lists;
}
+static const char *
+__stringify_type(u32 type)
+{
+ switch (type) {
+ case GUC_CAPTURE_LIST_TYPE_GLOBAL:
+ return "Global";
+ case GUC_CAPTURE_LIST_TYPE_ENGINE_CLASS:
+ return "Class";
+ case GUC_CAPTURE_LIST_TYPE_ENGINE_INSTANCE:
+ return "Instance";
+ default:
+ break;
+ }
+
+ return "unknown";
+}
+
+static const char *
+__stringify_engclass(u32 class)
+{
+ switch (class) {
+ case GUC_RENDER_CLASS:
+ return "Render";
+ case GUC_VIDEO_CLASS:
+ return "Video";
+ case GUC_VIDEOENHANCE_CLASS:
+ return "VideoEnhance";
+ case GUC_BLITTER_CLASS:
+ return "Blitter";
+ case GUC_COMPUTE_CLASS:
+ return "Compute";
+ case GUC_GSC_OTHER_CLASS:
+ return "GSC-Other";
+ default:
+ break;
+ }
+
+ return "unknown";
+}
+
static int
guc_capture_list_init(struct intel_guc *guc, u32 owner, u32 type, u32 classid,
struct guc_mmio_reg *ptr, u16 num_entries)
@@ -482,32 +535,55 @@ guc_cap_list_num_regs(struct intel_guc_state_capture *gc, u32 owner, u32 type, u
return num_regs;
}
-int
-intel_guc_capture_getlistsize(struct intel_guc *guc, u32 owner, u32 type, u32 classid,
- size_t *size)
+static int
+guc_capture_getlistsize(struct intel_guc *guc, u32 owner, u32 type, u32 classid,
+ size_t *size, bool is_purpose_est)
{
struct intel_guc_state_capture *gc = guc->capture;
+ struct drm_i915_private *i915 = guc_to_gt(guc)->i915;
struct __guc_capture_ads_cache *cache = &gc->ads_cache[owner][type][classid];
int num_regs;
- if (!gc->reglists)
+ if (!gc->reglists) {
+ drm_warn(&i915->drm, "GuC-capture: No reglist on this device\n");
return -ENODEV;
+ }
if (cache->is_valid) {
*size = cache->size;
return cache->status;
}
+ if (!is_purpose_est && owner == GUC_CAPTURE_LIST_INDEX_PF &&
+ !guc_capture_get_one_list(gc->reglists, owner, type, classid)) {
+ if (type == GUC_CAPTURE_LIST_TYPE_GLOBAL)
+ drm_warn(&i915->drm, "Missing GuC-Err-Cap reglist Global!\n");
+ else
+ drm_warn(&i915->drm, "Missing GuC-Err-Cap reglist %s(%u):%s(%u)!\n",
+ __stringify_type(type), type,
+ __stringify_engclass(classid), classid);
+ return -ENODATA;
+ }
+
num_regs = guc_cap_list_num_regs(gc, owner, type, classid);
+ /* intentional empty lists can exist depending on hw config */
if (!num_regs)
return -ENODATA;
- *size = PAGE_ALIGN((sizeof(struct guc_debug_capture_list)) +
- (num_regs * sizeof(struct guc_mmio_reg)));
+ if (size)
+ *size = PAGE_ALIGN((sizeof(struct guc_debug_capture_list)) +
+ (num_regs * sizeof(struct guc_mmio_reg)));
return 0;
}
+int
+intel_guc_capture_getlistsize(struct intel_guc *guc, u32 owner, u32 type, u32 classid,
+ size_t *size)
+{
+ return guc_capture_getlistsize(guc, owner, type, classid, size, false);
+}
+
static void guc_capture_create_prealloc_nodes(struct intel_guc *guc);
int
@@ -606,7 +682,7 @@ guc_capture_output_min_size_est(struct intel_guc *guc)
struct intel_gt *gt = guc_to_gt(guc);
struct intel_engine_cs *engine;
enum intel_engine_id id;
- int worst_min_size = 0, num_regs = 0;
+ int worst_min_size = 0;
size_t tmp = 0;
if (!guc->capture)
@@ -627,21 +703,19 @@ guc_capture_output_min_size_est(struct intel_guc *guc)
worst_min_size += sizeof(struct guc_state_capture_group_header_t) +
(3 * sizeof(struct guc_state_capture_header_t));
- if (!intel_guc_capture_getlistsize(guc, 0, GUC_CAPTURE_LIST_TYPE_GLOBAL, 0, &tmp))
- num_regs += tmp;
+ if (!guc_capture_getlistsize(guc, 0, GUC_CAPTURE_LIST_TYPE_GLOBAL, 0, &tmp, true))
+ worst_min_size += tmp;
- if (!intel_guc_capture_getlistsize(guc, 0, GUC_CAPTURE_LIST_TYPE_ENGINE_CLASS,
- engine->class, &tmp)) {
- num_regs += tmp;
+ if (!guc_capture_getlistsize(guc, 0, GUC_CAPTURE_LIST_TYPE_ENGINE_CLASS,
+ engine->class, &tmp, true)) {
+ worst_min_size += tmp;
}
- if (!intel_guc_capture_getlistsize(guc, 0, GUC_CAPTURE_LIST_TYPE_ENGINE_INSTANCE,
- engine->class, &tmp)) {
- num_regs += tmp;
+ if (!guc_capture_getlistsize(guc, 0, GUC_CAPTURE_LIST_TYPE_ENGINE_INSTANCE,
+ engine->class, &tmp, true)) {
+ worst_min_size += tmp;
}
}
- worst_min_size += (num_regs * sizeof(struct guc_mmio_reg));
-
return worst_min_size;
}
@@ -658,15 +732,23 @@ static void check_guc_capture_size(struct intel_guc *guc)
int spare_size = min_size * GUC_CAPTURE_OVERBUFFER_MULTIPLIER;
u32 buffer_size = intel_guc_log_section_size_capture(&guc->log);
+ /*
+ * NOTE: min_size is much smaller than the capture region allocation (DG2: <80K vs 1MB)
+ * Additionally, its based on space needed to fit all engines getting reset at once
+ * within the same G2H handler task slot. This is very unlikely. However, if GuC really
+ * does run out of space for whatever reason, we will see an separate warning message
+ * when processing the G2H event capture-notification, search for:
+ * INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_NOSPACE.
+ */
if (min_size < 0)
drm_warn(&i915->drm, "Failed to calculate GuC error state capture buffer minimum size: %d!\n",
min_size);
else if (min_size > buffer_size)
- drm_warn(&i915->drm, "GuC error state capture buffer is too small: %d < %d\n",
+ drm_warn(&i915->drm, "GuC error state capture buffer maybe small: %d < %d\n",
buffer_size, min_size);
else if (spare_size > buffer_size)
- drm_notice(&i915->drm, "GuC error state capture buffer maybe too small: %d < %d (min = %d)\n",
- buffer_size, spare_size, min_size);
+ drm_dbg(&i915->drm, "GuC error state capture buffer lacks spare size: %d < %d (min = %d)\n",
+ buffer_size, spare_size, min_size);
}
/*
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.c
index 25f09a4..7269eb0 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.c
@@ -71,12 +71,73 @@ static bool intel_eval_slpc_support(void *data)
return intel_guc_slpc_is_used(guc);
}
+static int guc_sched_disable_delay_ms_get(void *data, u64 *val)
+{
+ struct intel_guc *guc = data;
+
+ if (!intel_guc_submission_is_used(guc))
+ return -ENODEV;
+
+ *val = (u64)guc->submission_state.sched_disable_delay_ms;
+
+ return 0;
+}
+
+static int guc_sched_disable_delay_ms_set(void *data, u64 val)
+{
+ struct intel_guc *guc = data;
+
+ if (!intel_guc_submission_is_used(guc))
+ return -ENODEV;
+
+ /* clamp to a practical limit, 1 minute is reasonable for a longest delay */
+ guc->submission_state.sched_disable_delay_ms = min_t(u64, val, 60000);
+
+ return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(guc_sched_disable_delay_ms_fops,
+ guc_sched_disable_delay_ms_get,
+ guc_sched_disable_delay_ms_set, "%lld\n");
+
+static int guc_sched_disable_gucid_threshold_get(void *data, u64 *val)
+{
+ struct intel_guc *guc = data;
+
+ if (!intel_guc_submission_is_used(guc))
+ return -ENODEV;
+
+ *val = guc->submission_state.sched_disable_gucid_threshold;
+ return 0;
+}
+
+static int guc_sched_disable_gucid_threshold_set(void *data, u64 val)
+{
+ struct intel_guc *guc = data;
+
+ if (!intel_guc_submission_is_used(guc))
+ return -ENODEV;
+
+ if (val > intel_guc_sched_disable_gucid_threshold_max(guc))
+ guc->submission_state.sched_disable_gucid_threshold =
+ intel_guc_sched_disable_gucid_threshold_max(guc);
+ else
+ guc->submission_state.sched_disable_gucid_threshold = val;
+
+ return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(guc_sched_disable_gucid_threshold_fops,
+ guc_sched_disable_gucid_threshold_get,
+ guc_sched_disable_gucid_threshold_set, "%lld\n");
+
void intel_guc_debugfs_register(struct intel_guc *guc, struct dentry *root)
{
static const struct intel_gt_debugfs_file files[] = {
{ "guc_info", &guc_info_fops, NULL },
{ "guc_registered_contexts", &guc_registered_contexts_fops, NULL },
{ "guc_slpc_info", &guc_slpc_info_fops, &intel_eval_slpc_support},
+ { "guc_sched_disable_delay_ms", &guc_sched_disable_delay_ms_fops, NULL },
+ { "guc_sched_disable_gucid_threshold", &guc_sched_disable_gucid_threshold_fops,
+ NULL },
};
if (!intel_guc_is_supported(guc))
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
index a0372735..5b86b2e2 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
@@ -10,12 +10,15 @@
*/
#include "gt/intel_gt.h"
+#include "gt/intel_gt_mcr.h"
#include "gt/intel_gt_regs.h"
#include "intel_guc_fw.h"
#include "i915_drv.h"
-static void guc_prepare_xfer(struct intel_uncore *uncore)
+static void guc_prepare_xfer(struct intel_gt *gt)
{
+ struct intel_uncore *uncore = gt->uncore;
+
u32 shim_flags = GUC_ENABLE_READ_CACHE_LOGIC |
GUC_ENABLE_READ_CACHE_FOR_SRAM_DATA |
GUC_ENABLE_READ_CACHE_FOR_WOPCM_DATA |
@@ -35,8 +38,9 @@ static void guc_prepare_xfer(struct intel_uncore *uncore)
if (GRAPHICS_VER(uncore->i915) == 9) {
/* DOP Clock Gating Enable for GuC clocks */
- intel_uncore_rmw(uncore, GEN7_MISCCPCTL,
- 0, GEN8_DOP_CLOCK_GATE_GUC_ENABLE);
+ intel_gt_mcr_multicast_write(gt, GEN8_MISCCPCTL,
+ GEN8_DOP_CLOCK_GATE_GUC_ENABLE |
+ intel_gt_mcr_read_any(gt, GEN8_MISCCPCTL));
/* allows for 5us (in 10ns units) before GT can go to RC6 */
intel_uncore_write(uncore, GUC_ARAT_C6DIS, 0x1FF);
@@ -168,7 +172,7 @@ int intel_guc_fw_upload(struct intel_guc *guc)
struct intel_uncore *uncore = gt->uncore;
int ret;
- guc_prepare_xfer(uncore);
+ guc_prepare_xfer(gt);
/*
* Note that GuC needs the CSS header plus uKernel code to be copied
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index 323b055..4ae5fc2 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -47,7 +47,8 @@
#define GUC_VIDEOENHANCE_CLASS 2
#define GUC_BLITTER_CLASS 3
#define GUC_COMPUTE_CLASS 4
-#define GUC_LAST_ENGINE_CLASS GUC_COMPUTE_CLASS
+#define GUC_GSC_OTHER_CLASS 5
+#define GUC_LAST_ENGINE_CLASS GUC_GSC_OTHER_CLASS
#define GUC_MAX_ENGINE_CLASSES 16
#define GUC_MAX_INSTANCES_PER_CLASS 32
@@ -169,6 +170,7 @@ static u8 engine_class_guc_class_map[] = {
[COPY_ENGINE_CLASS] = GUC_BLITTER_CLASS,
[VIDEO_DECODE_CLASS] = GUC_VIDEO_CLASS,
[VIDEO_ENHANCEMENT_CLASS] = GUC_VIDEOENHANCE_CLASS,
+ [OTHER_CLASS] = GUC_GSC_OTHER_CLASS,
[COMPUTE_CLASS] = GUC_COMPUTE_CLASS,
};
@@ -178,12 +180,13 @@ static u8 guc_class_engine_class_map[] = {
[GUC_VIDEO_CLASS] = VIDEO_DECODE_CLASS,
[GUC_VIDEOENHANCE_CLASS] = VIDEO_ENHANCEMENT_CLASS,
[GUC_COMPUTE_CLASS] = COMPUTE_CLASS,
+ [GUC_GSC_OTHER_CLASS] = OTHER_CLASS,
};
static inline u8 engine_class_to_guc_class(u8 class)
{
BUILD_BUG_ON(ARRAY_SIZE(engine_class_guc_class_map) != MAX_ENGINE_CLASS + 1);
- GEM_BUG_ON(class > MAX_ENGINE_CLASS || class == OTHER_CLASS);
+ GEM_BUG_ON(class > MAX_ENGINE_CLASS);
return engine_class_guc_class_map[class];
}
@@ -290,6 +293,25 @@ struct guc_update_context_policy {
struct guc_klv_generic_dw_t klv[GUC_CONTEXT_POLICIES_KLV_NUM_IDS];
} __packed;
+/* Format of the UPDATE_SCHEDULING_POLICIES H2G data packet */
+struct guc_update_scheduling_policy_header {
+ u32 action;
+} __packed;
+
+/*
+ * Can't dynmically allocate memory for the scheduling policy KLV because
+ * it will be sent from within the reset path. Need a fixed size lump on
+ * the stack instead :(.
+ *
+ * Currently, there is only one KLV defined, which has 1 word of KL + 2 words of V.
+ */
+#define MAX_SCHEDULING_POLICY_SIZE 3
+
+struct guc_update_scheduling_policy {
+ struct guc_update_scheduling_policy_header header;
+ u32 data[MAX_SCHEDULING_POLICY_SIZE];
+} __packed;
+
#define GUC_POWER_UNSPECIFIED 0
#define GUC_POWER_D0 1
#define GUC_POWER_D1 2
@@ -298,6 +320,9 @@ struct guc_update_context_policy {
/* Scheduling policy settings */
+#define GLOBAL_SCHEDULE_POLICY_RC_YIELD_DURATION 100 /* in ms */
+#define GLOBAL_SCHEDULE_POLICY_RC_YIELD_RATIO 50 /* in percent */
+
#define GLOBAL_POLICY_MAX_NUM_WI 15
/* Don't reset an engine upon preemption failure */
@@ -305,6 +330,27 @@ struct guc_update_context_policy {
#define GLOBAL_POLICY_DEFAULT_DPC_PROMOTE_TIME_US 500000
+/*
+ * GuC converts the timeout to clock ticks internally. Different platforms have
+ * different GuC clocks. Thus, the maximum value before overflow is platform
+ * dependent. Current worst case scenario is about 110s. So, the spec says to
+ * limit to 100s to be safe.
+ */
+#define GUC_POLICY_MAX_EXEC_QUANTUM_US (100 * 1000 * 1000UL)
+#define GUC_POLICY_MAX_PREEMPT_TIMEOUT_US (100 * 1000 * 1000UL)
+
+static inline u32 guc_policy_max_exec_quantum_ms(void)
+{
+ BUILD_BUG_ON(GUC_POLICY_MAX_EXEC_QUANTUM_US >= UINT_MAX);
+ return GUC_POLICY_MAX_EXEC_QUANTUM_US / 1000;
+}
+
+static inline u32 guc_policy_max_preempt_timeout_ms(void)
+{
+ BUILD_BUG_ON(GUC_POLICY_MAX_PREEMPT_TIMEOUT_US >= UINT_MAX);
+ return GUC_POLICY_MAX_PREEMPT_TIMEOUT_US / 1000;
+}
+
struct guc_policies {
u32 submission_queue_depth[GUC_MAX_ENGINE_CLASSES];
/* In micro seconds. How much time to allow before DPC processing is
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_log.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_log.c
index 55d3ef9..68331c53 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_log.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_log.c
@@ -16,15 +16,15 @@
#if defined(CONFIG_DRM_I915_DEBUG_GUC)
#define GUC_LOG_DEFAULT_CRASH_BUFFER_SIZE SZ_2M
#define GUC_LOG_DEFAULT_DEBUG_BUFFER_SIZE SZ_16M
-#define GUC_LOG_DEFAULT_CAPTURE_BUFFER_SIZE SZ_4M
+#define GUC_LOG_DEFAULT_CAPTURE_BUFFER_SIZE SZ_1M
#elif defined(CONFIG_DRM_I915_DEBUG_GEM)
#define GUC_LOG_DEFAULT_CRASH_BUFFER_SIZE SZ_1M
#define GUC_LOG_DEFAULT_DEBUG_BUFFER_SIZE SZ_2M
-#define GUC_LOG_DEFAULT_CAPTURE_BUFFER_SIZE SZ_4M
+#define GUC_LOG_DEFAULT_CAPTURE_BUFFER_SIZE SZ_1M
#else
#define GUC_LOG_DEFAULT_CRASH_BUFFER_SIZE SZ_8K
#define GUC_LOG_DEFAULT_DEBUG_BUFFER_SIZE SZ_64K
-#define GUC_LOG_DEFAULT_CAPTURE_BUFFER_SIZE SZ_2M
+#define GUC_LOG_DEFAULT_CAPTURE_BUFFER_SIZE SZ_1M
#endif
static void guc_log_copy_debuglogs_for_relay(struct intel_guc_log *log);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_rc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_rc.c
index 8f8dd05..b585509 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_rc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_rc.c
@@ -11,9 +11,20 @@
static bool __guc_rc_supported(struct intel_guc *guc)
{
+ struct intel_gt *gt = guc_to_gt(guc);
+
+ /*
+ * Wa_14017073508: mtl
+ * Do not enable gucrc to avoid additional interrupts which
+ * may disrupt pcode wa.
+ */
+ if (IS_MTL_GRAPHICS_STEP(gt->i915, P, STEP_A0, STEP_B0) &&
+ gt->type == GT_MEDIA)
+ return false;
+
/* GuC RC is unavailable for pre-Gen12 */
return guc->submission_supported &&
- GRAPHICS_VER(guc_to_gt(guc)->i915) >= 12;
+ GRAPHICS_VER(gt->i915) >= 12;
}
static bool __guc_rc_selected(struct intel_guc *guc)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_reg.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_reg.h
index a7092f7..9915de3 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_reg.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_reg.h
@@ -36,6 +36,7 @@
#define SOFT_SCRATCH_COUNT 16
#define GEN11_SOFT_SCRATCH(n) _MMIO(0x190240 + (n) * 4)
+#define MEDIA_SOFT_SCRATCH(n) _MMIO(0x190310 + (n) * 4)
#define GEN11_SOFT_SCRATCH_COUNT 4
#define UOS_RSA_SCRATCH(i) _MMIO(0xc200 + (i) * 4)
@@ -101,6 +102,7 @@
#define GUC_SEND_INTERRUPT _MMIO(0xc4c8)
#define GUC_SEND_TRIGGER (1<<0)
#define GEN11_GUC_HOST_INTERRUPT _MMIO(0x1901f0)
+#define MEDIA_GUC_HOST_INTERRUPT _MMIO(0x190304)
#define GEN12_GUC_SEM_INTR_ENABLES _MMIO(0xc71c)
#define GUC_SEM_INTR_ROUTE_TO_GUC BIT(31)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c
index fdd895f..6346493 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.c
@@ -137,6 +137,17 @@ static int guc_action_slpc_set_param(struct intel_guc *guc, u8 id, u32 value)
return ret > 0 ? -EPROTO : ret;
}
+static int guc_action_slpc_unset_param(struct intel_guc *guc, u8 id)
+{
+ u32 request[] = {
+ GUC_ACTION_HOST2GUC_PC_SLPC_REQUEST,
+ SLPC_EVENT(SLPC_EVENT_PARAMETER_UNSET, 1),
+ id,
+ };
+
+ return intel_guc_send(guc, request, ARRAY_SIZE(request));
+}
+
static bool slpc_is_running(struct intel_guc_slpc *slpc)
{
return slpc_get_state(slpc) == SLPC_GLOBAL_STATE_RUNNING;
@@ -190,6 +201,15 @@ static int slpc_set_param(struct intel_guc_slpc *slpc, u8 id, u32 value)
return ret;
}
+static int slpc_unset_param(struct intel_guc_slpc *slpc, u8 id)
+{
+ struct intel_guc *guc = slpc_to_guc(slpc);
+
+ GEM_BUG_ON(id >= SLPC_MAX_PARAM);
+
+ return guc_action_slpc_unset_param(guc, id);
+}
+
static int slpc_force_min_freq(struct intel_guc_slpc *slpc, u32 freq)
{
struct drm_i915_private *i915 = slpc_to_i915(slpc);
@@ -263,6 +283,7 @@ int intel_guc_slpc_init(struct intel_guc_slpc *slpc)
slpc->max_freq_softlimit = 0;
slpc->min_freq_softlimit = 0;
+ slpc->min_is_rpmax = false;
slpc->boost_freq = 0;
atomic_set(&slpc->num_waiters, 0);
@@ -588,6 +609,39 @@ static int slpc_set_softlimits(struct intel_guc_slpc *slpc)
return 0;
}
+static bool is_slpc_min_freq_rpmax(struct intel_guc_slpc *slpc)
+{
+ struct drm_i915_private *i915 = slpc_to_i915(slpc);
+ int slpc_min_freq;
+ int ret;
+
+ ret = intel_guc_slpc_get_min_freq(slpc, &slpc_min_freq);
+ if (ret) {
+ drm_err(&i915->drm,
+ "Failed to get min freq: (%d)\n",
+ ret);
+ return false;
+ }
+
+ if (slpc_min_freq == SLPC_MAX_FREQ_MHZ)
+ return true;
+ else
+ return false;
+}
+
+static void update_server_min_softlimit(struct intel_guc_slpc *slpc)
+{
+ /* For server parts, SLPC min will be at RPMax.
+ * Use min softlimit to clamp it to RP0 instead.
+ */
+ if (!slpc->min_freq_softlimit &&
+ is_slpc_min_freq_rpmax(slpc)) {
+ slpc->min_is_rpmax = true;
+ slpc->min_freq_softlimit = slpc->rp0_freq;
+ (slpc_to_gt(slpc))->defaults.min_freq = slpc->min_freq_softlimit;
+ }
+}
+
static int slpc_use_fused_rp0(struct intel_guc_slpc *slpc)
{
/* Force SLPC to used platform rp0 */
@@ -610,6 +664,52 @@ static void slpc_get_rp_values(struct intel_guc_slpc *slpc)
slpc->boost_freq = slpc->rp0_freq;
}
+/**
+ * intel_guc_slpc_override_gucrc_mode() - override GUCRC mode
+ * @slpc: pointer to intel_guc_slpc.
+ * @mode: new value of the mode.
+ *
+ * This function will override the GUCRC mode.
+ *
+ * Return: 0 on success, non-zero error code on failure.
+ */
+int intel_guc_slpc_override_gucrc_mode(struct intel_guc_slpc *slpc, u32 mode)
+{
+ int ret;
+ struct drm_i915_private *i915 = slpc_to_i915(slpc);
+ intel_wakeref_t wakeref;
+
+ if (mode >= SLPC_GUCRC_MODE_MAX)
+ return -EINVAL;
+
+ with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
+ ret = slpc_set_param(slpc, SLPC_PARAM_PWRGATE_RC_MODE, mode);
+ if (ret)
+ drm_err(&i915->drm,
+ "Override gucrc mode %d failed %d\n",
+ mode, ret);
+ }
+
+ return ret;
+}
+
+int intel_guc_slpc_unset_gucrc_mode(struct intel_guc_slpc *slpc)
+{
+ struct drm_i915_private *i915 = slpc_to_i915(slpc);
+ intel_wakeref_t wakeref;
+ int ret = 0;
+
+ with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
+ ret = slpc_unset_param(slpc, SLPC_PARAM_PWRGATE_RC_MODE);
+ if (ret)
+ drm_err(&i915->drm,
+ "Unsetting gucrc mode failed %d\n",
+ ret);
+ }
+
+ return ret;
+}
+
/*
* intel_guc_slpc_enable() - Start SLPC
* @slpc: pointer to intel_guc_slpc.
@@ -647,6 +747,9 @@ int intel_guc_slpc_enable(struct intel_guc_slpc *slpc)
slpc_get_rp_values(slpc);
+ /* Handle the case where min=max=RPmax */
+ update_server_min_softlimit(slpc);
+
/* Set SLPC max limit to RP0 */
ret = slpc_use_fused_rp0(slpc);
if (unlikely(ret)) {
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h
index 82a98f7..17ed515 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h
@@ -9,6 +9,8 @@
#include "intel_guc_submission.h"
#include "intel_guc_slpc_types.h"
+#define SLPC_MAX_FREQ_MHZ 4250
+
struct intel_gt;
struct drm_printer;
@@ -42,5 +44,7 @@ int intel_guc_slpc_set_media_ratio_mode(struct intel_guc_slpc *slpc, u32 val);
void intel_guc_pm_intrmsk_enable(struct intel_gt *gt);
void intel_guc_slpc_boost(struct intel_guc_slpc *slpc);
void intel_guc_slpc_dec_waiters(struct intel_guc_slpc *slpc);
+int intel_guc_slpc_unset_gucrc_mode(struct intel_guc_slpc *slpc);
+int intel_guc_slpc_override_gucrc_mode(struct intel_guc_slpc *slpc, u32 mode);
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h
index 73d2081..a6ef53b 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h
@@ -19,6 +19,9 @@ struct intel_guc_slpc {
bool supported;
bool selected;
+ /* Indicates this is a server part */
+ bool min_is_rpmax;
+
/* platform frequency limits */
u32 min_freq;
u32 rp0_freq;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 1db59ee..0a42f18 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -6,6 +6,7 @@
#include <linux/circ_buf.h>
#include "gem/i915_gem_context.h"
+#include "gem/i915_gem_lmem.h"
#include "gt/gen8_engine_cs.h"
#include "gt/intel_breadcrumbs.h"
#include "gt/intel_context.h"
@@ -29,6 +30,7 @@
#include "intel_guc_submission.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "i915_trace.h"
/**
@@ -65,7 +67,13 @@
* corresponding G2H returns indicating the scheduling disable operation has
* completed it is safe to unpin the context. While a disable is in flight it
* isn't safe to resubmit the context so a fence is used to stall all future
- * requests of that context until the G2H is returned.
+ * requests of that context until the G2H is returned. Because this interaction
+ * with the GuC takes a non-zero amount of time we delay the disabling of
+ * scheduling after the pin count goes to zero by a configurable period of time
+ * (see SCHED_DISABLE_DELAY_MS). The thought is this gives the user a window of
+ * time to resubmit something on the context before doing this costly operation.
+ * This delay is only done if the context isn't closed and the guc_id usage is
+ * less than a threshold (see NUM_SCHED_DISABLE_GUC_IDS_THRESHOLD).
*
* Context deregistration:
* Before a context can be destroyed or if we steal its guc_id we must
@@ -163,7 +171,8 @@ guc_create_parallel(struct intel_engine_cs **engines,
#define SCHED_STATE_PENDING_ENABLE BIT(5)
#define SCHED_STATE_REGISTERED BIT(6)
#define SCHED_STATE_POLICY_REQUIRED BIT(7)
-#define SCHED_STATE_BLOCKED_SHIFT 8
+#define SCHED_STATE_CLOSED BIT(8)
+#define SCHED_STATE_BLOCKED_SHIFT 9
#define SCHED_STATE_BLOCKED BIT(SCHED_STATE_BLOCKED_SHIFT)
#define SCHED_STATE_BLOCKED_MASK (0xfff << SCHED_STATE_BLOCKED_SHIFT)
@@ -173,12 +182,20 @@ static inline void init_sched_state(struct intel_context *ce)
ce->guc_state.sched_state &= SCHED_STATE_BLOCKED_MASK;
}
+/*
+ * Kernel contexts can have SCHED_STATE_REGISTERED after suspend.
+ * A context close can race with the submission path, so SCHED_STATE_CLOSED
+ * can be set immediately before we try to register.
+ */
+#define SCHED_STATE_VALID_INIT \
+ (SCHED_STATE_BLOCKED_MASK | \
+ SCHED_STATE_CLOSED | \
+ SCHED_STATE_REGISTERED)
+
__maybe_unused
static bool sched_state_is_init(struct intel_context *ce)
{
- /* Kernel contexts can have SCHED_STATE_REGISTERED after suspend. */
- return !(ce->guc_state.sched_state &
- ~(SCHED_STATE_BLOCKED_MASK | SCHED_STATE_REGISTERED));
+ return !(ce->guc_state.sched_state & ~SCHED_STATE_VALID_INIT);
}
static inline bool
@@ -319,6 +336,17 @@ static inline void clr_context_policy_required(struct intel_context *ce)
ce->guc_state.sched_state &= ~SCHED_STATE_POLICY_REQUIRED;
}
+static inline bool context_close_done(struct intel_context *ce)
+{
+ return ce->guc_state.sched_state & SCHED_STATE_CLOSED;
+}
+
+static inline void set_context_close_done(struct intel_context *ce)
+{
+ lockdep_assert_held(&ce->guc_state.lock);
+ ce->guc_state.sched_state |= SCHED_STATE_CLOSED;
+}
+
static inline u32 context_blocked(struct intel_context *ce)
{
return (ce->guc_state.sched_state & SCHED_STATE_BLOCKED_MASK) >>
@@ -343,25 +371,6 @@ static inline void decr_context_blocked(struct intel_context *ce)
ce->guc_state.sched_state -= SCHED_STATE_BLOCKED;
}
-static inline bool context_has_committed_requests(struct intel_context *ce)
-{
- return !!ce->guc_state.number_committed_requests;
-}
-
-static inline void incr_context_committed_requests(struct intel_context *ce)
-{
- lockdep_assert_held(&ce->guc_state.lock);
- ++ce->guc_state.number_committed_requests;
- GEM_BUG_ON(ce->guc_state.number_committed_requests < 0);
-}
-
-static inline void decr_context_committed_requests(struct intel_context *ce)
-{
- lockdep_assert_held(&ce->guc_state.lock);
- --ce->guc_state.number_committed_requests;
- GEM_BUG_ON(ce->guc_state.number_committed_requests < 0);
-}
-
static struct intel_context *
request_to_scheduling_context(struct i915_request *rq)
{
@@ -1067,6 +1076,12 @@ static void scrub_guc_desc_for_outstanding_g2h(struct intel_guc *guc)
xa_unlock(&guc->context_lookup);
+ if (test_bit(CONTEXT_GUC_INIT, &ce->flags) &&
+ (cancel_delayed_work(&ce->guc_state.sched_disable_delay_work))) {
+ /* successful cancel so jump straight to close it */
+ intel_context_sched_disable_unpin(ce);
+ }
+
spin_lock(&ce->guc_state.lock);
/*
@@ -1387,7 +1402,9 @@ static void guc_timestamp_ping(struct work_struct *wrk)
/*
* Synchronize with gt reset to make sure the worker does not
- * corrupt the engine/guc stats.
+ * corrupt the engine/guc stats. NB: can't actually block waiting
+ * for a reset to complete as the reset requires flushing out
+ * this worker thread if started. So waiting would deadlock.
*/
ret = intel_gt_reset_trylock(gt, &srcu);
if (ret)
@@ -1994,6 +2011,9 @@ static int new_guc_id(struct intel_guc *guc, struct intel_context *ce)
if (unlikely(ret < 0))
return ret;
+ if (!intel_context_is_parent(ce))
+ ++guc->submission_state.guc_ids_in_use;
+
ce->guc_id.id = ret;
return 0;
}
@@ -2003,14 +2023,16 @@ static void __release_guc_id(struct intel_guc *guc, struct intel_context *ce)
GEM_BUG_ON(intel_context_is_child(ce));
if (!context_guc_id_invalid(ce)) {
- if (intel_context_is_parent(ce))
+ if (intel_context_is_parent(ce)) {
bitmap_release_region(guc->submission_state.guc_ids_bitmap,
ce->guc_id.id,
order_base_2(ce->parallel.number_children
+ 1));
- else
+ } else {
+ --guc->submission_state.guc_ids_in_use;
ida_simple_remove(&guc->submission_state.guc_ids,
ce->guc_id.id);
+ }
clr_ctx_id_mapping(guc, ce->guc_id.id);
set_context_guc_id_invalid(ce);
}
@@ -2429,6 +2451,10 @@ static int guc_context_policy_init_v70(struct intel_context *ce, bool loop)
int ret;
/* NB: For both of these, zero means disabled. */
+ GEM_BUG_ON(overflows_type(engine->props.timeslice_duration_ms * 1000,
+ execution_quantum));
+ GEM_BUG_ON(overflows_type(engine->props.preempt_timeout_ms * 1000,
+ preemption_timeout));
execution_quantum = engine->props.timeslice_duration_ms * 1000;
preemption_timeout = engine->props.preempt_timeout_ms * 1000;
@@ -2462,6 +2488,10 @@ static void guc_context_policy_init_v69(struct intel_engine_cs *engine,
desc->policy_flags |= CONTEXT_POLICY_FLAG_PREEMPT_TO_IDLE_V69;
/* NB: For both of these, zero means disabled. */
+ GEM_BUG_ON(overflows_type(engine->props.timeslice_duration_ms * 1000,
+ desc->execution_quantum));
+ GEM_BUG_ON(overflows_type(engine->props.preempt_timeout_ms * 1000,
+ desc->preemption_timeout));
desc->execution_quantum = engine->props.timeslice_duration_ms * 1000;
desc->preemption_timeout = engine->props.preempt_timeout_ms * 1000;
}
@@ -2998,41 +3028,104 @@ guc_context_revoke(struct intel_context *ce, struct i915_request *rq,
}
}
-static void guc_context_sched_disable(struct intel_context *ce)
+static void do_sched_disable(struct intel_guc *guc, struct intel_context *ce,
+ unsigned long flags)
+ __releases(ce->guc_state.lock)
{
- struct intel_guc *guc = ce_to_guc(ce);
- unsigned long flags;
struct intel_runtime_pm *runtime_pm = &ce->engine->gt->i915->runtime_pm;
intel_wakeref_t wakeref;
u16 guc_id;
- GEM_BUG_ON(intel_context_is_child(ce));
-
- spin_lock_irqsave(&ce->guc_state.lock, flags);
-
- /*
- * We have to check if the context has been disabled by another thread,
- * check if submssion has been disabled to seal a race with reset and
- * finally check if any more requests have been committed to the
- * context ensursing that a request doesn't slip through the
- * 'context_pending_disable' fence.
- */
- if (unlikely(!context_enabled(ce) || submission_disabled(guc) ||
- context_has_committed_requests(ce))) {
- clr_context_enabled(ce);
- spin_unlock_irqrestore(&ce->guc_state.lock, flags);
- goto unpin;
- }
+ lockdep_assert_held(&ce->guc_state.lock);
guc_id = prep_context_pending_disable(ce);
spin_unlock_irqrestore(&ce->guc_state.lock, flags);
with_intel_runtime_pm(runtime_pm, wakeref)
__guc_context_sched_disable(guc, ce, guc_id);
+}
- return;
-unpin:
- intel_context_sched_disable_unpin(ce);
+static bool bypass_sched_disable(struct intel_guc *guc,
+ struct intel_context *ce)
+{
+ lockdep_assert_held(&ce->guc_state.lock);
+ GEM_BUG_ON(intel_context_is_child(ce));
+
+ if (submission_disabled(guc) || context_guc_id_invalid(ce) ||
+ !ctx_id_mapped(guc, ce->guc_id.id)) {
+ clr_context_enabled(ce);
+ return true;
+ }
+
+ return !context_enabled(ce);
+}
+
+static void __delay_sched_disable(struct work_struct *wrk)
+{
+ struct intel_context *ce =
+ container_of(wrk, typeof(*ce), guc_state.sched_disable_delay_work.work);
+ struct intel_guc *guc = ce_to_guc(ce);
+ unsigned long flags;
+
+ spin_lock_irqsave(&ce->guc_state.lock, flags);
+
+ if (bypass_sched_disable(guc, ce)) {
+ spin_unlock_irqrestore(&ce->guc_state.lock, flags);
+ intel_context_sched_disable_unpin(ce);
+ } else {
+ do_sched_disable(guc, ce, flags);
+ }
+}
+
+static bool guc_id_pressure(struct intel_guc *guc, struct intel_context *ce)
+{
+ /*
+ * parent contexts are perma-pinned, if we are unpinning do schedule
+ * disable immediately.
+ */
+ if (intel_context_is_parent(ce))
+ return true;
+
+ /*
+ * If we are beyond the threshold for avail guc_ids, do schedule disable immediately.
+ */
+ return guc->submission_state.guc_ids_in_use >
+ guc->submission_state.sched_disable_gucid_threshold;
+}
+
+static void guc_context_sched_disable(struct intel_context *ce)
+{
+ struct intel_guc *guc = ce_to_guc(ce);
+ u64 delay = guc->submission_state.sched_disable_delay_ms;
+ unsigned long flags;
+
+ spin_lock_irqsave(&ce->guc_state.lock, flags);
+
+ if (bypass_sched_disable(guc, ce)) {
+ spin_unlock_irqrestore(&ce->guc_state.lock, flags);
+ intel_context_sched_disable_unpin(ce);
+ } else if (!intel_context_is_closed(ce) && !guc_id_pressure(guc, ce) &&
+ delay) {
+ spin_unlock_irqrestore(&ce->guc_state.lock, flags);
+ mod_delayed_work(system_unbound_wq,
+ &ce->guc_state.sched_disable_delay_work,
+ msecs_to_jiffies(delay));
+ } else {
+ do_sched_disable(guc, ce, flags);
+ }
+}
+
+static void guc_context_close(struct intel_context *ce)
+{
+ unsigned long flags;
+
+ if (test_bit(CONTEXT_GUC_INIT, &ce->flags) &&
+ cancel_delayed_work(&ce->guc_state.sched_disable_delay_work))
+ __delay_sched_disable(&ce->guc_state.sched_disable_delay_work.work);
+
+ spin_lock_irqsave(&ce->guc_state.lock, flags);
+ set_context_close_done(ce);
+ spin_unlock_irqrestore(&ce->guc_state.lock, flags);
}
static inline void guc_lrc_desc_unpin(struct intel_context *ce)
@@ -3071,7 +3164,6 @@ static void __guc_context_destroy(struct intel_context *ce)
ce->guc_state.prio_count[GUC_CLIENT_PRIORITY_HIGH] ||
ce->guc_state.prio_count[GUC_CLIENT_PRIORITY_KMD_NORMAL] ||
ce->guc_state.prio_count[GUC_CLIENT_PRIORITY_NORMAL]);
- GEM_BUG_ON(ce->guc_state.number_committed_requests);
lrc_fini(ce);
intel_context_fini(ce);
@@ -3340,8 +3432,6 @@ static void remove_from_context(struct i915_request *rq)
guc_prio_fini(rq, ce);
- decr_context_committed_requests(ce);
-
spin_unlock_irq(&ce->guc_state.lock);
atomic_dec(&ce->guc_id.ref);
@@ -3351,6 +3441,8 @@ static void remove_from_context(struct i915_request *rq)
static const struct intel_context_ops guc_context_ops = {
.alloc = guc_context_alloc,
+ .close = guc_context_close,
+
.pre_pin = guc_context_pre_pin,
.pin = guc_context_pin,
.unpin = guc_context_unpin,
@@ -3433,6 +3525,10 @@ static void guc_context_init(struct intel_context *ce)
rcu_read_unlock();
ce->guc_state.prio = map_i915_prio_to_guc_prio(prio);
+
+ INIT_DELAYED_WORK(&ce->guc_state.sched_disable_delay_work,
+ __delay_sched_disable);
+
set_bit(CONTEXT_GUC_INIT, &ce->flags);
}
@@ -3471,6 +3567,26 @@ static int guc_request_alloc(struct i915_request *rq)
guc_context_init(ce);
/*
+ * If the context gets closed while the execbuf is ongoing, the context
+ * close code will race with the below code to cancel the delayed work.
+ * If the context close wins the race and cancels the work, it will
+ * immediately call the sched disable (see guc_context_close), so there
+ * is a chance we can get past this check while the sched_disable code
+ * is being executed. To make sure that code completes before we check
+ * the status further down, we wait for the close process to complete.
+ * Else, this code path could send a request down thinking that the
+ * context is still in a schedule-enable mode while the GuC ends up
+ * dropping the request completely because the disable did go from the
+ * context_close path right to GuC just prior. In the event the CT is
+ * full, we could potentially need to wait up to 1.5 seconds.
+ */
+ if (cancel_delayed_work_sync(&ce->guc_state.sched_disable_delay_work))
+ intel_context_sched_disable_unpin(ce);
+ else if (intel_context_is_closed(ce))
+ if (wait_for(context_close_done(ce), 1500))
+ drm_warn(&guc_to_gt(guc)->i915->drm,
+ "timed out waiting on context sched close before realloc\n");
+ /*
* Call pin_guc_id here rather than in the pinning step as with
* dma_resv, contexts can be repeatedly pinned / unpinned trashing the
* guc_id and creating horrible race conditions. This is especially bad
@@ -3524,7 +3640,6 @@ static int guc_request_alloc(struct i915_request *rq)
list_add_tail(&rq->guc_fence_link, &ce->guc_state.fences);
}
- incr_context_committed_requests(ce);
spin_unlock_irqrestore(&ce->guc_state.lock, flags);
return 0;
@@ -3600,6 +3715,8 @@ static int guc_virtual_context_alloc(struct intel_context *ce)
static const struct intel_context_ops virtual_guc_context_ops = {
.alloc = guc_virtual_context_alloc,
+ .close = guc_context_close,
+
.pre_pin = guc_virtual_context_pre_pin,
.pin = guc_virtual_context_pin,
.unpin = guc_virtual_context_unpin,
@@ -3689,6 +3806,8 @@ static void guc_child_context_destroy(struct kref *kref)
static const struct intel_context_ops virtual_parent_context_ops = {
.alloc = guc_virtual_context_alloc,
+ .close = guc_context_close,
+
.pre_pin = guc_context_pre_pin,
.pin = guc_parent_context_pin,
.unpin = guc_parent_context_unpin,
@@ -3995,6 +4114,9 @@ static inline void guc_kernel_context_pin(struct intel_guc *guc,
if (context_guc_id_invalid(ce))
pin_guc_id(guc, ce);
+ if (!test_bit(CONTEXT_GUC_INIT, &ce->flags))
+ guc_context_init(ce);
+
try_context_registration(ce, true);
}
@@ -4093,7 +4215,7 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
engine->emit_bb_start = gen8_emit_bb_start;
if (GRAPHICS_VER_FULL(engine->i915) >= IP_VER(12, 50))
- engine->emit_bb_start = gen125_emit_bb_start;
+ engine->emit_bb_start = xehp_emit_bb_start;
}
static void rcs_submission_override(struct intel_engine_cs *engine)
@@ -4177,6 +4299,98 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
return 0;
}
+struct scheduling_policy {
+ /* internal data */
+ u32 max_words, num_words;
+ u32 count;
+ /* API data */
+ struct guc_update_scheduling_policy h2g;
+};
+
+static u32 __guc_scheduling_policy_action_size(struct scheduling_policy *policy)
+{
+ u32 *start = (void *)&policy->h2g;
+ u32 *end = policy->h2g.data + policy->num_words;
+ size_t delta = end - start;
+
+ return delta;
+}
+
+static struct scheduling_policy *__guc_scheduling_policy_start_klv(struct scheduling_policy *policy)
+{
+ policy->h2g.header.action = INTEL_GUC_ACTION_UPDATE_SCHEDULING_POLICIES_KLV;
+ policy->max_words = ARRAY_SIZE(policy->h2g.data);
+ policy->num_words = 0;
+ policy->count = 0;
+
+ return policy;
+}
+
+static void __guc_scheduling_policy_add_klv(struct scheduling_policy *policy,
+ u32 action, u32 *data, u32 len)
+{
+ u32 *klv_ptr = policy->h2g.data + policy->num_words;
+
+ GEM_BUG_ON((policy->num_words + 1 + len) > policy->max_words);
+ *(klv_ptr++) = FIELD_PREP(GUC_KLV_0_KEY, action) |
+ FIELD_PREP(GUC_KLV_0_LEN, len);
+ memcpy(klv_ptr, data, sizeof(u32) * len);
+ policy->num_words += 1 + len;
+ policy->count++;
+}
+
+static int __guc_action_set_scheduling_policies(struct intel_guc *guc,
+ struct scheduling_policy *policy)
+{
+ int ret;
+
+ ret = intel_guc_send(guc, (u32 *)&policy->h2g,
+ __guc_scheduling_policy_action_size(policy));
+ if (ret < 0)
+ return ret;
+
+ if (ret != policy->count) {
+ drm_warn(&guc_to_gt(guc)->i915->drm, "GuC global scheduler policy processed %d of %d KLVs!",
+ ret, policy->count);
+ if (ret > policy->count)
+ return -EPROTO;
+ }
+
+ return 0;
+}
+
+static int guc_init_global_schedule_policy(struct intel_guc *guc)
+{
+ struct scheduling_policy policy;
+ struct intel_gt *gt = guc_to_gt(guc);
+ intel_wakeref_t wakeref;
+ int ret = 0;
+
+ if (GET_UC_VER(guc) < MAKE_UC_VER(70, 3, 0))
+ return 0;
+
+ __guc_scheduling_policy_start_klv(&policy);
+
+ with_intel_runtime_pm(>->i915->runtime_pm, wakeref) {
+ u32 yield[] = {
+ GLOBAL_SCHEDULE_POLICY_RC_YIELD_DURATION,
+ GLOBAL_SCHEDULE_POLICY_RC_YIELD_RATIO,
+ };
+
+ __guc_scheduling_policy_add_klv(&policy,
+ GUC_SCHEDULING_POLICIES_KLV_ID_RENDER_COMPUTE_YIELD,
+ yield, ARRAY_SIZE(yield));
+
+ ret = __guc_action_set_scheduling_policies(guc, &policy);
+ if (ret)
+ i915_probe_error(gt->i915,
+ "Failed to configure global scheduling policies: %pe!\n",
+ ERR_PTR(ret));
+ }
+
+ return ret;
+}
+
void intel_guc_submission_enable(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
@@ -4189,6 +4403,7 @@ void intel_guc_submission_enable(struct intel_guc *guc)
guc_init_lrc_mapping(guc);
guc_init_engine_stats(guc);
+ guc_init_global_schedule_policy(guc);
}
void intel_guc_submission_disable(struct intel_guc *guc)
@@ -4219,6 +4434,26 @@ static bool __guc_submission_selected(struct intel_guc *guc)
return i915->params.enable_guc & ENABLE_GUC_SUBMISSION;
}
+int intel_guc_sched_disable_gucid_threshold_max(struct intel_guc *guc)
+{
+ return guc->submission_state.num_guc_ids - NUMBER_MULTI_LRC_GUC_ID(guc);
+}
+
+/*
+ * This default value of 33 milisecs (+1 milisec round up) ensures 30fps or higher
+ * workloads are able to enjoy the latency reduction when delaying the schedule-disable
+ * operation. This matches the 30fps game-render + encode (real world) workload this
+ * knob was tested against.
+ */
+#define SCHED_DISABLE_DELAY_MS 34
+
+/*
+ * A threshold of 75% is a reasonable starting point considering that real world apps
+ * generally don't get anywhere near this.
+ */
+#define NUM_SCHED_DISABLE_GUCIDS_DEFAULT_THRESHOLD(__guc) \
+ (((intel_guc_sched_disable_gucid_threshold_max(guc)) * 3) / 4)
+
void intel_guc_submission_init_early(struct intel_guc *guc)
{
xa_init_flags(&guc->context_lookup, XA_FLAGS_LOCK_IRQ);
@@ -4235,7 +4470,10 @@ void intel_guc_submission_init_early(struct intel_guc *guc)
spin_lock_init(&guc->timestamp.lock);
INIT_DELAYED_WORK(&guc->timestamp.work, guc_timestamp_ping);
+ guc->submission_state.sched_disable_delay_ms = SCHED_DISABLE_DELAY_MS;
guc->submission_state.num_guc_ids = GUC_MAX_CONTEXT_ID;
+ guc->submission_state.sched_disable_gucid_threshold =
+ NUM_SCHED_DISABLE_GUCIDS_DEFAULT_THRESHOLD(guc);
guc->submission_supported = __guc_submission_supported(guc);
guc->submission_selected = __guc_submission_selected(guc);
}
@@ -4669,7 +4907,7 @@ void intel_guc_submission_print_info(struct intel_guc *guc,
drm_printf(p, "GuC Number Outstanding Submission G2H: %u\n",
atomic_read(&guc->outstanding_submission_g2h));
- drm_printf(p, "GuC tasklet count: %u\n\n",
+ drm_printf(p, "GuC tasklet count: %u\n",
atomic_read(&sched_engine->tasklet.count));
spin_lock_irqsave(&sched_engine->lock, flags);
@@ -4717,7 +4955,7 @@ static inline void guc_log_context(struct drm_printer *p,
atomic_read(&ce->pin_count));
drm_printf(p, "\t\tGuC ID Ref Count: %u\n",
atomic_read(&ce->guc_id.ref));
- drm_printf(p, "\t\tSchedule State: 0x%x\n\n",
+ drm_printf(p, "\t\tSchedule State: 0x%x\n",
ce->guc_state.sched_state);
}
@@ -4746,7 +4984,7 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
READ_ONCE(*ce->parallel.guc.wq_head));
drm_printf(p, "\t\tWQI Tail: %u\n",
READ_ONCE(*ce->parallel.guc.wq_tail));
- drm_printf(p, "\t\tWQI Status: %u\n\n",
+ drm_printf(p, "\t\tWQI Status: %u\n",
READ_ONCE(*ce->parallel.guc.wq_status));
}
@@ -4754,7 +4992,7 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
emit_bb_start_parent_no_preempt_mid_batch) {
u8 i;
- drm_printf(p, "\t\tChildren Go: %u\n\n",
+ drm_printf(p, "\t\tChildren Go: %u\n",
get_children_go_value(ce));
for (i = 0; i < ce->parallel.number_children; ++i)
drm_printf(p, "\t\tChildren Join: %u\n",
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_huc.c b/drivers/gpu/drm/i915/gt/uc/intel_huc.c
index 3bb8838..410905d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_huc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_huc.c
@@ -10,6 +10,9 @@
#include "intel_huc.h"
#include "i915_drv.h"
+#include <linux/device/bus.h>
+#include <linux/mei_aux.h>
+
/**
* DOC: HuC
*
@@ -42,12 +45,240 @@
* HuC-specific commands.
*/
+/*
+ * MEI-GSC load is an async process. The probing of the exposed aux device
+ * (see intel_gsc.c) usually happens a few seconds after i915 probe, depending
+ * on when the kernel schedules it. Unless something goes terribly wrong, we're
+ * guaranteed for this to happen during boot, so the big timeout is a safety net
+ * that we never expect to need.
+ * MEI-PXP + HuC load usually takes ~300ms, but if the GSC needs to be resumed
+ * and/or reset, this can take longer. Note that the kernel might schedule
+ * other work between the i915 init/resume and the MEI one, which can add to
+ * the delay.
+ */
+#define GSC_INIT_TIMEOUT_MS 10000
+#define PXP_INIT_TIMEOUT_MS 5000
+
+static int sw_fence_dummy_notify(struct i915_sw_fence *sf,
+ enum i915_sw_fence_notify state)
+{
+ return NOTIFY_DONE;
+}
+
+static void __delayed_huc_load_complete(struct intel_huc *huc)
+{
+ if (!i915_sw_fence_done(&huc->delayed_load.fence))
+ i915_sw_fence_complete(&huc->delayed_load.fence);
+}
+
+static void delayed_huc_load_complete(struct intel_huc *huc)
+{
+ hrtimer_cancel(&huc->delayed_load.timer);
+ __delayed_huc_load_complete(huc);
+}
+
+static void __gsc_init_error(struct intel_huc *huc)
+{
+ huc->delayed_load.status = INTEL_HUC_DELAYED_LOAD_ERROR;
+ __delayed_huc_load_complete(huc);
+}
+
+static void gsc_init_error(struct intel_huc *huc)
+{
+ hrtimer_cancel(&huc->delayed_load.timer);
+ __gsc_init_error(huc);
+}
+
+static void gsc_init_done(struct intel_huc *huc)
+{
+ hrtimer_cancel(&huc->delayed_load.timer);
+
+ /* MEI-GSC init is done, now we wait for MEI-PXP to bind */
+ huc->delayed_load.status = INTEL_HUC_WAITING_ON_PXP;
+ if (!i915_sw_fence_done(&huc->delayed_load.fence))
+ hrtimer_start(&huc->delayed_load.timer,
+ ms_to_ktime(PXP_INIT_TIMEOUT_MS),
+ HRTIMER_MODE_REL);
+}
+
+static enum hrtimer_restart huc_delayed_load_timer_callback(struct hrtimer *hrtimer)
+{
+ struct intel_huc *huc = container_of(hrtimer, struct intel_huc, delayed_load.timer);
+
+ if (!intel_huc_is_authenticated(huc)) {
+ if (huc->delayed_load.status == INTEL_HUC_WAITING_ON_GSC)
+ drm_notice(&huc_to_gt(huc)->i915->drm,
+ "timed out waiting for MEI GSC init to load HuC\n");
+ else if (huc->delayed_load.status == INTEL_HUC_WAITING_ON_PXP)
+ drm_notice(&huc_to_gt(huc)->i915->drm,
+ "timed out waiting for MEI PXP init to load HuC\n");
+ else
+ MISSING_CASE(huc->delayed_load.status);
+
+ __gsc_init_error(huc);
+ }
+
+ return HRTIMER_NORESTART;
+}
+
+static void huc_delayed_load_start(struct intel_huc *huc)
+{
+ ktime_t delay;
+
+ GEM_BUG_ON(intel_huc_is_authenticated(huc));
+
+ /*
+ * On resume we don't have to wait for MEI-GSC to be re-probed, but we
+ * do need to wait for MEI-PXP to reset & re-bind
+ */
+ switch (huc->delayed_load.status) {
+ case INTEL_HUC_WAITING_ON_GSC:
+ delay = ms_to_ktime(GSC_INIT_TIMEOUT_MS);
+ break;
+ case INTEL_HUC_WAITING_ON_PXP:
+ delay = ms_to_ktime(PXP_INIT_TIMEOUT_MS);
+ break;
+ default:
+ gsc_init_error(huc);
+ return;
+ }
+
+ /*
+ * This fence is always complete unless we're waiting for the
+ * GSC device to come up to load the HuC. We arm the fence here
+ * and complete it when we confirm that the HuC is loaded from
+ * the PXP bind callback.
+ */
+ GEM_BUG_ON(!i915_sw_fence_done(&huc->delayed_load.fence));
+ i915_sw_fence_fini(&huc->delayed_load.fence);
+ i915_sw_fence_reinit(&huc->delayed_load.fence);
+ i915_sw_fence_await(&huc->delayed_load.fence);
+ i915_sw_fence_commit(&huc->delayed_load.fence);
+
+ hrtimer_start(&huc->delayed_load.timer, delay, HRTIMER_MODE_REL);
+}
+
+static int gsc_notifier(struct notifier_block *nb, unsigned long action, void *data)
+{
+ struct device *dev = data;
+ struct intel_huc *huc = container_of(nb, struct intel_huc, delayed_load.nb);
+ struct intel_gsc_intf *intf = &huc_to_gt(huc)->gsc.intf[0];
+
+ if (!intf->adev || &intf->adev->aux_dev.dev != dev)
+ return 0;
+
+ switch (action) {
+ case BUS_NOTIFY_BOUND_DRIVER: /* mei driver bound to aux device */
+ gsc_init_done(huc);
+ break;
+
+ case BUS_NOTIFY_DRIVER_NOT_BOUND: /* mei driver fails to be bound */
+ case BUS_NOTIFY_UNBIND_DRIVER: /* mei driver about to be unbound */
+ drm_info(&huc_to_gt(huc)->i915->drm,
+ "mei driver not bound, disabling HuC load\n");
+ gsc_init_error(huc);
+ break;
+ }
+
+ return 0;
+}
+
+void intel_huc_register_gsc_notifier(struct intel_huc *huc, struct bus_type *bus)
+{
+ int ret;
+
+ if (!intel_huc_is_loaded_by_gsc(huc))
+ return;
+
+ huc->delayed_load.nb.notifier_call = gsc_notifier;
+ ret = bus_register_notifier(bus, &huc->delayed_load.nb);
+ if (ret) {
+ drm_err(&huc_to_gt(huc)->i915->drm,
+ "failed to register GSC notifier\n");
+ huc->delayed_load.nb.notifier_call = NULL;
+ gsc_init_error(huc);
+ }
+}
+
+void intel_huc_unregister_gsc_notifier(struct intel_huc *huc, struct bus_type *bus)
+{
+ if (!huc->delayed_load.nb.notifier_call)
+ return;
+
+ delayed_huc_load_complete(huc);
+
+ bus_unregister_notifier(bus, &huc->delayed_load.nb);
+ huc->delayed_load.nb.notifier_call = NULL;
+}
+
+static void delayed_huc_load_init(struct intel_huc *huc)
+{
+ /*
+ * Initialize fence to be complete as this is expected to be complete
+ * unless there is a delayed HuC load in progress.
+ */
+ i915_sw_fence_init(&huc->delayed_load.fence,
+ sw_fence_dummy_notify);
+ i915_sw_fence_commit(&huc->delayed_load.fence);
+
+ hrtimer_init(&huc->delayed_load.timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ huc->delayed_load.timer.function = huc_delayed_load_timer_callback;
+}
+
+static void delayed_huc_load_fini(struct intel_huc *huc)
+{
+ /*
+ * the fence is initialized in init_early, so we need to clean it up
+ * even if HuC loading is off.
+ */
+ delayed_huc_load_complete(huc);
+ i915_sw_fence_fini(&huc->delayed_load.fence);
+}
+
+static bool vcs_supported(struct intel_gt *gt)
+{
+ intel_engine_mask_t mask = gt->info.engine_mask;
+
+ /*
+ * We reach here from i915_driver_early_probe for the primary GT before
+ * its engine mask is set, so we use the device info engine mask for it;
+ * this means we're not taking VCS fusing into account, but if the
+ * primary GT supports VCS engines we expect at least one of them to
+ * remain unfused so we're fine.
+ * For other GTs we expect the GT-specific mask to be set before we
+ * call this function.
+ */
+ GEM_BUG_ON(!gt_is_root(gt) && !gt->info.engine_mask);
+
+ if (gt_is_root(gt))
+ mask = RUNTIME_INFO(gt->i915)->platform_engine_mask;
+ else
+ mask = gt->info.engine_mask;
+
+ return __ENGINE_INSTANCES_MASK(mask, VCS0, I915_MAX_VCS);
+}
+
void intel_huc_init_early(struct intel_huc *huc)
{
struct drm_i915_private *i915 = huc_to_gt(huc)->i915;
+ struct intel_gt *gt = huc_to_gt(huc);
intel_uc_fw_init_early(&huc->fw, INTEL_UC_FW_TYPE_HUC);
+ /*
+ * we always init the fence as already completed, even if HuC is not
+ * supported. This way we don't have to distinguish between HuC not
+ * supported/disabled or already loaded, and can focus on if the load
+ * is currently in progress (fence not complete) or not, which is what
+ * we care about for stalling userspace submissions.
+ */
+ delayed_huc_load_init(huc);
+
+ if (!vcs_supported(gt)) {
+ intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_NOT_SUPPORTED);
+ return;
+ }
+
if (GRAPHICS_VER(i915) >= 11) {
huc->status.reg = GEN11_HUC_KERNEL_LOAD_INFO;
huc->status.mask = HUC_LOAD_SUCCESSFUL;
@@ -113,16 +344,59 @@ int intel_huc_init(struct intel_huc *huc)
return 0;
out:
+ intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_INIT_FAIL);
drm_info(&i915->drm, "HuC init failed with %d\n", err);
return err;
}
void intel_huc_fini(struct intel_huc *huc)
{
+ /*
+ * the fence is initialized in init_early, so we need to clean it up
+ * even if HuC loading is off.
+ */
+ delayed_huc_load_fini(huc);
+
+ if (intel_uc_fw_is_loadable(&huc->fw))
+ intel_uc_fw_fini(&huc->fw);
+}
+
+void intel_huc_suspend(struct intel_huc *huc)
+{
if (!intel_uc_fw_is_loadable(&huc->fw))
return;
- intel_uc_fw_fini(&huc->fw);
+ /*
+ * in the unlikely case that we're suspending before the GSC has
+ * completed its loading sequence, just stop waiting. We'll restart
+ * on resume.
+ */
+ delayed_huc_load_complete(huc);
+}
+
+int intel_huc_wait_for_auth_complete(struct intel_huc *huc)
+{
+ struct intel_gt *gt = huc_to_gt(huc);
+ int ret;
+
+ ret = __intel_wait_for_register(gt->uncore,
+ huc->status.reg,
+ huc->status.mask,
+ huc->status.value,
+ 2, 50, NULL);
+
+ /* mark the load process as complete even if the wait failed */
+ delayed_huc_load_complete(huc);
+
+ if (ret) {
+ drm_err(>->i915->drm, "HuC: Firmware not verified %d\n", ret);
+ intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_LOAD_FAIL);
+ return ret;
+ }
+
+ intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_RUNNING);
+ drm_info(>->i915->drm, "HuC authenticated\n");
+ return 0;
}
/**
@@ -161,27 +435,18 @@ int intel_huc_auth(struct intel_huc *huc)
}
/* Check authentication status, it should be done by now */
- ret = __intel_wait_for_register(gt->uncore,
- huc->status.reg,
- huc->status.mask,
- huc->status.value,
- 2, 50, NULL);
- if (ret) {
- DRM_ERROR("HuC: Firmware not verified %d\n", ret);
+ ret = intel_huc_wait_for_auth_complete(huc);
+ if (ret)
goto fail;
- }
- intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_RUNNING);
- drm_info(>->i915->drm, "HuC authenticated\n");
return 0;
fail:
i915_probe_error(gt->i915, "HuC: Authentication failed %d\n", ret);
- intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_LOAD_FAIL);
return ret;
}
-static bool huc_is_authenticated(struct intel_huc *huc)
+bool intel_huc_is_authenticated(struct intel_huc *huc)
{
struct intel_gt *gt = huc_to_gt(huc);
intel_wakeref_t wakeref;
@@ -200,13 +465,8 @@ static bool huc_is_authenticated(struct intel_huc *huc)
* This function reads status register to verify if HuC
* firmware was successfully loaded.
*
- * Returns:
- * * -ENODEV if HuC is not present on this platform,
- * * -EOPNOTSUPP if HuC firmware is disabled,
- * * -ENOPKG if HuC firmware was not installed,
- * * -ENOEXEC if HuC firmware is invalid or mismatched,
- * * 0 if HuC firmware is not running,
- * * 1 if HuC firmware is authenticated and running.
+ * The return values match what is expected for the I915_PARAM_HUC_STATUS
+ * getparam.
*/
int intel_huc_check_status(struct intel_huc *huc)
{
@@ -219,11 +479,21 @@ int intel_huc_check_status(struct intel_huc *huc)
return -ENOPKG;
case INTEL_UC_FIRMWARE_ERROR:
return -ENOEXEC;
+ case INTEL_UC_FIRMWARE_INIT_FAIL:
+ return -ENOMEM;
+ case INTEL_UC_FIRMWARE_LOAD_FAIL:
+ return -EIO;
default:
break;
}
- return huc_is_authenticated(huc);
+ return intel_huc_is_authenticated(huc);
+}
+
+static bool huc_has_delayed_load(struct intel_huc *huc)
+{
+ return intel_huc_is_loaded_by_gsc(huc) &&
+ (huc->delayed_load.status != INTEL_HUC_DELAYED_LOAD_ERROR);
}
void intel_huc_update_auth_status(struct intel_huc *huc)
@@ -231,9 +501,11 @@ void intel_huc_update_auth_status(struct intel_huc *huc)
if (!intel_uc_fw_is_loadable(&huc->fw))
return;
- if (huc_is_authenticated(huc))
+ if (intel_huc_is_authenticated(huc))
intel_uc_fw_change_status(&huc->fw,
INTEL_UC_FIRMWARE_RUNNING);
+ else if (huc_has_delayed_load(huc))
+ huc_delayed_load_start(huc);
}
/**
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_huc.h b/drivers/gpu/drm/i915/gt/uc/intel_huc.h
index d7e25b6..52db036 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_huc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_huc.h
@@ -7,9 +7,21 @@
#define _INTEL_HUC_H_
#include "i915_reg_defs.h"
+#include "i915_sw_fence.h"
#include "intel_uc_fw.h"
#include "intel_huc_fw.h"
+#include <linux/notifier.h>
+#include <linux/hrtimer.h>
+
+struct bus_type;
+
+enum intel_huc_delayed_load_status {
+ INTEL_HUC_WAITING_ON_GSC = 0,
+ INTEL_HUC_WAITING_ON_PXP,
+ INTEL_HUC_DELAYED_LOAD_ERROR,
+};
+
struct intel_huc {
/* Generic uC firmware management */
struct intel_uc_fw fw;
@@ -20,14 +32,27 @@ struct intel_huc {
u32 mask;
u32 value;
} status;
+
+ struct {
+ struct i915_sw_fence fence;
+ struct hrtimer timer;
+ struct notifier_block nb;
+ enum intel_huc_delayed_load_status status;
+ } delayed_load;
};
void intel_huc_init_early(struct intel_huc *huc);
int intel_huc_init(struct intel_huc *huc);
void intel_huc_fini(struct intel_huc *huc);
+void intel_huc_suspend(struct intel_huc *huc);
int intel_huc_auth(struct intel_huc *huc);
+int intel_huc_wait_for_auth_complete(struct intel_huc *huc);
int intel_huc_check_status(struct intel_huc *huc);
void intel_huc_update_auth_status(struct intel_huc *huc);
+bool intel_huc_is_authenticated(struct intel_huc *huc);
+
+void intel_huc_register_gsc_notifier(struct intel_huc *huc, struct bus_type *bus);
+void intel_huc_unregister_gsc_notifier(struct intel_huc *huc, struct bus_type *bus);
static inline int intel_huc_sanitize(struct intel_huc *huc)
{
@@ -56,6 +81,12 @@ static inline bool intel_huc_is_loaded_by_gsc(const struct intel_huc *huc)
return huc->fw.loaded_via_gsc;
}
+static inline bool intel_huc_wait_required(struct intel_huc *huc)
+{
+ return intel_huc_is_used(huc) && intel_huc_is_loaded_by_gsc(huc) &&
+ !intel_huc_is_authenticated(huc);
+}
+
void intel_huc_load_status(struct intel_huc *huc, struct drm_printer *p);
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.c
index 9d6ab1e..4f24641 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.c
@@ -3,9 +3,43 @@
* Copyright © 2014-2019 Intel Corporation
*/
+#include "gt/intel_gsc.h"
#include "gt/intel_gt.h"
+#include "intel_huc.h"
#include "intel_huc_fw.h"
#include "i915_drv.h"
+#include "pxp/intel_pxp_huc.h"
+
+int intel_huc_fw_load_and_auth_via_gsc(struct intel_huc *huc)
+{
+ int ret;
+
+ if (!intel_huc_is_loaded_by_gsc(huc))
+ return -ENODEV;
+
+ if (!intel_uc_fw_is_loadable(&huc->fw))
+ return -ENOEXEC;
+
+ /*
+ * If we abort a suspend, HuC might still be loaded when the mei
+ * component gets re-bound and this function called again. If so, just
+ * mark the HuC as loaded.
+ */
+ if (intel_huc_is_authenticated(huc)) {
+ intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_RUNNING);
+ return 0;
+ }
+
+ GEM_WARN_ON(intel_uc_fw_is_loaded(&huc->fw));
+
+ ret = intel_pxp_huc_load_and_auth(&huc_to_gt(huc)->pxp);
+ if (ret)
+ return ret;
+
+ intel_uc_fw_change_status(&huc->fw, INTEL_UC_FIRMWARE_TRANSFERRED);
+
+ return intel_huc_wait_for_auth_complete(huc);
+}
/**
* intel_huc_fw_upload() - load HuC uCode to device via DMA transfer
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.h b/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.h
index 12f264e..db42e23 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_huc_fw.h
@@ -8,6 +8,7 @@
struct intel_huc;
+int intel_huc_fw_load_and_auth_via_gsc(struct intel_huc *huc);
int intel_huc_fw_upload(struct intel_huc *huc);
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc.c b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
index dbd048b..2a508b1 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_uc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
@@ -357,8 +357,8 @@ static int uc_init_wopcm(struct intel_uc *uc)
{
struct intel_gt *gt = uc_to_gt(uc);
struct intel_uncore *uncore = gt->uncore;
- u32 base = intel_wopcm_guc_base(>->i915->wopcm);
- u32 size = intel_wopcm_guc_size(>->i915->wopcm);
+ u32 base = intel_wopcm_guc_base(>->wopcm);
+ u32 size = intel_wopcm_guc_size(>->wopcm);
u32 huc_agent = intel_uc_uses_huc(uc) ? HUC_LOADING_AGENT_GUC : 0;
u32 mask;
int err;
@@ -636,8 +636,10 @@ void intel_uc_runtime_suspend(struct intel_uc *uc)
{
struct intel_guc *guc = &uc->guc;
- if (!intel_guc_is_ready(guc))
+ if (!intel_guc_is_ready(guc)) {
+ guc->interrupts.enabled = false;
return;
+ }
/*
* Wait for any outstanding CTB before tearing down communication /w the
@@ -657,8 +659,10 @@ void intel_uc_suspend(struct intel_uc *uc)
intel_wakeref_t wakeref;
int err;
- if (!intel_guc_is_ready(guc))
+ if (!intel_guc_is_ready(guc)) {
+ guc->interrupts.enabled = false;
return;
+ }
with_intel_runtime_pm(&uc_to_gt(uc)->i915->runtime_pm, wakeref) {
err = intel_guc_suspend(guc);
@@ -718,6 +722,7 @@ int intel_uc_runtime_resume(struct intel_uc *uc)
static const struct intel_uc_ops uc_ops_off = {
.init_hw = __uc_check_hw,
+ .fini = __uc_fini, /* to clean-up the init_early initialization */
};
static const struct intel_uc_ops uc_ops_on = {
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c
index b91ad4a..0c80ba5 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c
@@ -93,7 +93,8 @@ void intel_uc_fw_change_status(struct intel_uc_fw *uc_fw,
fw_def(BROXTON, 0, guc_mmp(bxt, 70, 1, 1)) \
fw_def(SKYLAKE, 0, guc_mmp(skl, 70, 1, 1))
-#define INTEL_HUC_FIRMWARE_DEFS(fw_def, huc_raw, huc_mmp) \
+#define INTEL_HUC_FIRMWARE_DEFS(fw_def, huc_raw, huc_mmp, huc_gsc) \
+ fw_def(DG2, 0, huc_gsc(dg2)) \
fw_def(ALDERLAKE_P, 0, huc_raw(tgl)) \
fw_def(ALDERLAKE_P, 0, huc_mmp(tgl, 7, 9, 3)) \
fw_def(ALDERLAKE_S, 0, huc_raw(tgl)) \
@@ -141,6 +142,9 @@ void intel_uc_fw_change_status(struct intel_uc_fw *uc_fw,
#define MAKE_HUC_FW_PATH_BLANK(prefix_) \
__MAKE_UC_FW_PATH_BLANK(prefix_, "_huc")
+#define MAKE_HUC_FW_PATH_GSC(prefix_) \
+ __MAKE_UC_FW_PATH_BLANK(prefix_, "_huc_gsc")
+
#define MAKE_HUC_FW_PATH_MMP(prefix_, major_, minor_, patch_) \
__MAKE_UC_FW_PATH_MMP(prefix_, "_huc_", major_, minor_, patch_)
@@ -153,7 +157,7 @@ void intel_uc_fw_change_status(struct intel_uc_fw *uc_fw,
MODULE_FIRMWARE(uc_);
INTEL_GUC_FIRMWARE_DEFS(INTEL_UC_MODULE_FW, MAKE_GUC_FW_PATH_MAJOR, MAKE_GUC_FW_PATH_MMP)
-INTEL_HUC_FIRMWARE_DEFS(INTEL_UC_MODULE_FW, MAKE_HUC_FW_PATH_BLANK, MAKE_HUC_FW_PATH_MMP)
+INTEL_HUC_FIRMWARE_DEFS(INTEL_UC_MODULE_FW, MAKE_HUC_FW_PATH_BLANK, MAKE_HUC_FW_PATH_MMP, MAKE_HUC_FW_PATH_GSC)
/*
* The next expansion of the table macros (in __uc_fw_auto_select below) provides
@@ -168,6 +172,7 @@ struct __packed uc_fw_blob {
u8 major;
u8 minor;
u8 patch;
+ bool loaded_via_gsc;
};
#define UC_FW_BLOB_BASE(major_, minor_, patch_, path_) \
@@ -176,16 +181,16 @@ struct __packed uc_fw_blob {
.patch = patch_, \
.path = path_,
-#define UC_FW_BLOB_NEW(major_, minor_, patch_, path_) \
+#define UC_FW_BLOB_NEW(major_, minor_, patch_, gsc_, path_) \
{ UC_FW_BLOB_BASE(major_, minor_, patch_, path_) \
- .legacy = false }
+ .legacy = false, .loaded_via_gsc = gsc_ }
#define UC_FW_BLOB_OLD(major_, minor_, patch_, path_) \
{ UC_FW_BLOB_BASE(major_, minor_, patch_, path_) \
.legacy = true }
#define GUC_FW_BLOB(prefix_, major_, minor_) \
- UC_FW_BLOB_NEW(major_, minor_, 0, \
+ UC_FW_BLOB_NEW(major_, minor_, 0, false, \
MAKE_GUC_FW_PATH_MAJOR(prefix_, major_, minor_))
#define GUC_FW_BLOB_MMP(prefix_, major_, minor_, patch_) \
@@ -193,12 +198,15 @@ struct __packed uc_fw_blob {
MAKE_GUC_FW_PATH_MMP(prefix_, major_, minor_, patch_))
#define HUC_FW_BLOB(prefix_) \
- UC_FW_BLOB_NEW(0, 0, 0, MAKE_HUC_FW_PATH_BLANK(prefix_))
+ UC_FW_BLOB_NEW(0, 0, 0, false, MAKE_HUC_FW_PATH_BLANK(prefix_))
#define HUC_FW_BLOB_MMP(prefix_, major_, minor_, patch_) \
UC_FW_BLOB_OLD(major_, minor_, patch_, \
MAKE_HUC_FW_PATH_MMP(prefix_, major_, minor_, patch_))
+#define HUC_FW_BLOB_GSC(prefix_) \
+ UC_FW_BLOB_NEW(0, 0, 0, true, MAKE_HUC_FW_PATH_GSC(prefix_))
+
struct __packed uc_fw_platform_requirement {
enum intel_platform p;
u8 rev; /* first platform rev using this FW */
@@ -224,7 +232,7 @@ __uc_fw_auto_select(struct drm_i915_private *i915, struct intel_uc_fw *uc_fw)
INTEL_GUC_FIRMWARE_DEFS(MAKE_FW_LIST, GUC_FW_BLOB, GUC_FW_BLOB_MMP)
};
static const struct uc_fw_platform_requirement blobs_huc[] = {
- INTEL_HUC_FIRMWARE_DEFS(MAKE_FW_LIST, HUC_FW_BLOB, HUC_FW_BLOB_MMP)
+ INTEL_HUC_FIRMWARE_DEFS(MAKE_FW_LIST, HUC_FW_BLOB, HUC_FW_BLOB_MMP, HUC_FW_BLOB_GSC)
};
static const struct fw_blobs_by_type blobs_all[INTEL_UC_FW_NUM_TYPES] = {
[INTEL_UC_FW_TYPE_GUC] = { blobs_guc, ARRAY_SIZE(blobs_guc) },
@@ -272,6 +280,7 @@ __uc_fw_auto_select(struct drm_i915_private *i915, struct intel_uc_fw *uc_fw)
uc_fw->file_wanted.path = blob->path;
uc_fw->file_wanted.major_ver = blob->major;
uc_fw->file_wanted.minor_ver = blob->minor;
+ uc_fw->loaded_via_gsc = blob->loaded_via_gsc;
found = true;
break;
}
@@ -469,10 +478,11 @@ static int check_gsc_manifest(const struct firmware *fw,
return 0;
}
-static int check_ccs_header(struct drm_i915_private *i915,
+static int check_ccs_header(struct intel_gt *gt,
const struct firmware *fw,
struct intel_uc_fw *uc_fw)
{
+ struct drm_i915_private *i915 = gt->i915;
struct uc_css_header *css;
size_t size;
@@ -514,10 +524,10 @@ static int check_ccs_header(struct drm_i915_private *i915,
/* Sanity check whether this fw is not larger than whole WOPCM memory */
size = __intel_uc_fw_get_upload_size(uc_fw);
- if (unlikely(size >= i915->wopcm.size)) {
+ if (unlikely(size >= gt->wopcm.size)) {
drm_warn(&i915->drm, "%s firmware %s: invalid size: %zu > %zu\n",
intel_uc_fw_type_repr(uc_fw->type), uc_fw->file_selected.path,
- size, (size_t)i915->wopcm.size);
+ size, (size_t)gt->wopcm.size);
return -E2BIG;
}
@@ -545,7 +555,8 @@ static int check_ccs_header(struct drm_i915_private *i915,
*/
int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw)
{
- struct drm_i915_private *i915 = __uc_fw_to_gt(uc_fw)->i915;
+ struct intel_gt *gt = __uc_fw_to_gt(uc_fw);
+ struct drm_i915_private *i915 = gt->i915;
struct intel_uc_fw_file file_ideal;
struct device *dev = i915->drm.dev;
struct drm_i915_gem_object *obj;
@@ -553,7 +564,7 @@ int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw)
bool old_ver = false;
int err;
- GEM_BUG_ON(!i915->wopcm.size);
+ GEM_BUG_ON(!gt->wopcm.size);
GEM_BUG_ON(!intel_uc_fw_is_enabled(uc_fw));
err = i915_inject_probe_error(i915, -ENXIO);
@@ -566,6 +577,17 @@ int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw)
err = firmware_request_nowarn(&fw, uc_fw->file_selected.path, dev);
memcpy(&file_ideal, &uc_fw->file_wanted, sizeof(file_ideal));
+ if (!err && fw->size > INTEL_UC_RSVD_GGTT_PER_FW) {
+ drm_err(&i915->drm,
+ "%s firmware %s: size (%zuKB) exceeds max supported size (%uKB)\n",
+ intel_uc_fw_type_repr(uc_fw->type), uc_fw->file_selected.path,
+ fw->size / SZ_1K, INTEL_UC_RSVD_GGTT_PER_FW / SZ_1K);
+
+ /* try to find another blob to load */
+ release_firmware(fw);
+ err = -ENOENT;
+ }
+
/* Any error is terminal if overriding. Don't bother searching for older versions */
if (err && intel_uc_fw_is_overridden(uc_fw))
goto fail;
@@ -595,7 +617,7 @@ int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw)
if (uc_fw->loaded_via_gsc)
err = check_gsc_manifest(fw, uc_fw);
else
- err = check_ccs_header(i915, fw, uc_fw);
+ err = check_ccs_header(gt, fw, uc_fw);
if (err)
goto fail;
@@ -668,14 +690,30 @@ int intel_uc_fw_fetch(struct intel_uc_fw *uc_fw)
static u32 uc_fw_ggtt_offset(struct intel_uc_fw *uc_fw)
{
- struct i915_ggtt *ggtt = __uc_fw_to_gt(uc_fw)->ggtt;
+ struct intel_gt *gt = __uc_fw_to_gt(uc_fw);
+ struct i915_ggtt *ggtt = gt->ggtt;
struct drm_mm_node *node = &ggtt->uc_fw;
+ u32 offset = uc_fw->type * INTEL_UC_RSVD_GGTT_PER_FW;
+
+ /*
+ * The media GT shares the GGTT with the root GT, which means that
+ * we need to use different offsets for the binaries on the media GT.
+ * To keep the math simple, we use 8MB for the root tile and 8MB for
+ * the media one. This will need to be updated if we ever have more
+ * than 1 media GT.
+ */
+ BUILD_BUG_ON(INTEL_UC_FW_NUM_TYPES * INTEL_UC_RSVD_GGTT_PER_FW > SZ_8M);
+ GEM_BUG_ON(gt->type == GT_MEDIA && gt->info.id > 1);
+ if (gt->type == GT_MEDIA)
+ offset += SZ_8M;
GEM_BUG_ON(!drm_mm_node_allocated(node));
GEM_BUG_ON(upper_32_bits(node->start));
GEM_BUG_ON(upper_32_bits(node->start + node->size - 1));
+ GEM_BUG_ON(offset + uc_fw->obj->base.size > node->size);
+ GEM_BUG_ON(uc_fw->obj->base.size > INTEL_UC_RSVD_GGTT_PER_FW);
- return lower_32_bits(node->start);
+ return lower_32_bits(node->start + offset);
}
static void uc_fw_bind_ggtt(struct intel_uc_fw *uc_fw)
@@ -690,7 +728,6 @@ static void uc_fw_bind_ggtt(struct intel_uc_fw *uc_fw)
dummy->bi.pages = obj->mm.pages;
GEM_BUG_ON(!i915_gem_object_has_pinned_pages(obj));
- GEM_BUG_ON(dummy->node_size > ggtt->uc_fw.size);
/* uc_fw->obj cache domains were not controlled across suspend */
if (i915_gem_object_has_struct_page(obj))
@@ -904,7 +941,6 @@ int intel_uc_fw_init(struct intel_uc_fw *uc_fw)
out_unpin:
i915_gem_object_unpin_pages(uc_fw->obj);
out:
- intel_uc_fw_change_status(uc_fw, INTEL_UC_FIRMWARE_INIT_FAIL);
return err;
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h
index cb586f7..bc898ba 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h
@@ -6,6 +6,7 @@
#ifndef _INTEL_UC_FW_H_
#define _INTEL_UC_FW_H_
+#include <linux/sizes.h>
#include <linux/types.h>
#include "intel_uc_fw_abi.h"
#include "intel_device_info.h"
@@ -114,6 +115,19 @@ struct intel_uc_fw {
(uc)->fw.file_selected.minor_ver, \
(uc)->fw.file_selected.patch_ver))
+/*
+ * When we load the uC binaries, we pin them in a reserved section at the top of
+ * the GGTT, which is ~18 MBs. On multi-GT systems where the GTs share the GGTT,
+ * we also need to make sure that each binary is pinned to a unique location
+ * during load, because the different GT can go through the FW load at the same
+ * time (see uc_fw_ggtt_offset() for details).
+ * Given that the available space is much greater than what is required by the
+ * binaries, to keep things simple instead of dynamically partitioning the
+ * reserved section to make space for all the blobs we can just reserve a static
+ * chunk for each binary.
+ */
+#define INTEL_UC_RSVD_GGTT_PER_FW SZ_2M
+
#ifdef CONFIG_DRM_I915_DEBUG_GUC
void intel_uc_fw_change_status(struct intel_uc_fw *uc_fw,
enum intel_uc_fw_status status);
diff --git a/drivers/gpu/drm/i915/gt/uc/selftest_guc_hangcheck.c b/drivers/gpu/drm/i915/gt/uc/selftest_guc_hangcheck.c
index 01f8cd3..d91b58f 100644
--- a/drivers/gpu/drm/i915/gt/uc/selftest_guc_hangcheck.c
+++ b/drivers/gpu/drm/i915/gt/uc/selftest_guc_hangcheck.c
@@ -35,11 +35,14 @@ static int intel_hang_guc(void *arg)
struct i915_request *rq;
intel_wakeref_t wakeref;
struct i915_gpu_error *global = >->i915->gpu_error;
- struct intel_engine_cs *engine;
+ struct intel_engine_cs *engine = intel_selftest_find_any_engine(gt);
unsigned int reset_count;
u32 guc_status;
u32 old_beat;
+ if (!engine)
+ return 0;
+
ctx = kernel_context(gt->i915, NULL);
if (IS_ERR(ctx)) {
drm_err(>->i915->drm, "Failed get kernel context: %ld\n", PTR_ERR(ctx));
@@ -48,14 +51,13 @@ static int intel_hang_guc(void *arg)
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
- ce = intel_context_create(gt->engine[BCS0]);
+ ce = intel_context_create(engine);
if (IS_ERR(ce)) {
ret = PTR_ERR(ce);
drm_err(>->i915->drm, "Failed to create spinner request: %d\n", ret);
goto err;
}
- engine = ce->engine;
reset_count = i915_reset_count(global);
old_beat = engine->props.heartbeat_interval_ms;
diff --git a/drivers/gpu/drm/i915/gvt/cfg_space.c b/drivers/gpu/drm/i915/gvt/cfg_space.c
index eef3bba..9bafac1 100644
--- a/drivers/gpu/drm/i915/gvt/cfg_space.c
+++ b/drivers/gpu/drm/i915/gvt/cfg_space.c
@@ -244,7 +244,7 @@ static void emulate_pci_bar_write(struct intel_vgpu *vgpu, unsigned int offset,
}
/**
- * intel_vgpu_emulate_cfg_read - emulate vGPU configuration space write
+ * intel_vgpu_emulate_cfg_write - emulate vGPU configuration space write
* @vgpu: target vgpu
* @offset: offset
* @p_data: write data ptr
@@ -354,9 +354,9 @@ void intel_vgpu_init_cfg_space(struct intel_vgpu *vgpu,
memset(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_OPREGION, 0, 4);
vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_GTTMMIO].size =
- pci_resource_len(pdev, GTTMMADR_BAR);
+ pci_resource_len(pdev, GEN4_GTTMMADR_BAR);
vgpu->cfg_space.bar[INTEL_GVT_PCI_BAR_APERTURE].size =
- pci_resource_len(pdev, GTT_APERTURE_BAR);
+ pci_resource_len(pdev, GEN4_GMADR_BAR);
memset(vgpu_cfg_space(vgpu) + PCI_ROM_ADDRESS, 0, 4);
diff --git a/drivers/gpu/drm/i915/gvt/cmd_parser.c b/drivers/gpu/drm/i915/gvt/cmd_parser.c
index de13f10..0ebf5fb 100644
--- a/drivers/gpu/drm/i915/gvt/cmd_parser.c
+++ b/drivers/gpu/drm/i915/gvt/cmd_parser.c
@@ -37,6 +37,7 @@
#include <linux/slab.h>
#include "i915_drv.h"
+#include "i915_reg.h"
#include "gt/intel_engine_regs.h"
#include "gt/intel_gpu_commands.h"
#include "gt/intel_gt_regs.h"
diff --git a/drivers/gpu/drm/i915/gvt/display.c b/drivers/gpu/drm/i915/gvt/display.c
index c7722c8..c033249 100644
--- a/drivers/gpu/drm/i915/gvt/display.c
+++ b/drivers/gpu/drm/i915/gvt/display.c
@@ -36,6 +36,8 @@
#include "i915_reg.h"
#include "gvt.h"
+#include "display/intel_dpio_phy.h"
+
static int get_edp_pipe(struct intel_vgpu *vgpu)
{
u32 data = vgpu_vreg(vgpu, _TRANS_DDI_FUNC_CTL_EDP);
diff --git a/drivers/gpu/drm/i915/gvt/dmabuf.c b/drivers/gpu/drm/i915/gvt/dmabuf.c
index 01e54b45..355f1c0 100644
--- a/drivers/gpu/drm/i915/gvt/dmabuf.c
+++ b/drivers/gpu/drm/i915/gvt/dmabuf.c
@@ -88,7 +88,7 @@ static int vgpu_gem_get_pages(
sg_dma_address(sg) = dma_addr;
}
- __i915_gem_object_set_pages(obj, st, PAGE_SIZE);
+ __i915_gem_object_set_pages(obj, st);
out:
if (ret) {
dma_addr_t dma_addr;
diff --git a/drivers/gpu/drm/i915/gvt/dmabuf.h b/drivers/gpu/drm/i915/gvt/dmabuf.h
index 5f8f03f..3dcdb65 100644
--- a/drivers/gpu/drm/i915/gvt/dmabuf.h
+++ b/drivers/gpu/drm/i915/gvt/dmabuf.h
@@ -48,7 +48,7 @@ struct intel_vgpu_fb_info {
struct intel_vgpu_dmabuf_obj *obj;
};
-/**
+/*
* struct intel_vgpu_dmabuf_obj- Intel vGPU device buffer object
*/
struct intel_vgpu_dmabuf_obj {
diff --git a/drivers/gpu/drm/i915/gvt/firmware.c b/drivers/gpu/drm/i915/gvt/firmware.c
index 54fe442..a683c22 100644
--- a/drivers/gpu/drm/i915/gvt/firmware.c
+++ b/drivers/gpu/drm/i915/gvt/firmware.c
@@ -104,7 +104,7 @@ static int expose_firmware_sysfs(struct intel_gvt *gvt)
memcpy(p, gvt->firmware.mmio, info->mmio_size);
- crc32_start = offsetof(struct gvt_firmware_header, crc32) + 4;
+ crc32_start = offsetof(struct gvt_firmware_header, version);
h->crc32 = crc32_le(0, firmware + crc32_start, size - crc32_start);
firmware_attr.size = size;
diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
index ce0eb03..51e5e8f 100644
--- a/drivers/gpu/drm/i915/gvt/gtt.c
+++ b/drivers/gpu/drm/i915/gvt/gtt.c
@@ -282,11 +282,6 @@ static inline int get_next_pt_type(int type)
return gtt_type_table[type].next_pt_type;
}
-static inline int get_pt_type(int type)
-{
- return gtt_type_table[type].pt_type;
-}
-
static inline int get_entry_type(int type)
{
return gtt_type_table[type].entry_type;
@@ -2785,7 +2780,7 @@ int intel_gvt_init_gtt(struct intel_gvt *gvt)
* intel_gvt_clean_gtt - clean up mm components of a GVT device
* @gvt: GVT device
*
- * This function is called at the driver unloading stage, to clean up the
+ * This function is called at the driver unloading stage, to clean up
* the mm components of a GVT device.
*
*/
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index dbf8d74..62823c0 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -227,8 +227,6 @@ struct intel_vgpu {
unsigned long nr_cache_entries;
struct mutex cache_lock;
- atomic_t released;
-
struct kvm_page_track_notifier_node track_node;
#define NR_BKT (1 << 18)
struct hlist_head ptable[NR_BKT];
diff --git a/drivers/gpu/drm/i915/gvt/handlers.c b/drivers/gpu/drm/i915/gvt/handlers.c
index daac205..735fc83 100644
--- a/drivers/gpu/drm/i915/gvt/handlers.c
+++ b/drivers/gpu/drm/i915/gvt/handlers.c
@@ -43,6 +43,7 @@
#include "intel_mchbar_regs.h"
#include "display/intel_display_types.h"
#include "display/intel_dmc_regs.h"
+#include "display/intel_dpio_phy.h"
#include "display/intel_fbc.h"
#include "display/vlv_dsi_pll_regs.h"
#include "gt/intel_gt_regs.h"
@@ -734,7 +735,7 @@ static i915_reg_t force_nonpriv_white_list[] = {
_MMIO(0x770c),
_MMIO(0x83a8),
_MMIO(0xb110),
- GEN8_L3SQCREG4,//_MMIO(0xb118)
+ _MMIO(0xb118),
_MMIO(0xe100),
_MMIO(0xe18c),
_MMIO(0xe48c),
@@ -2257,7 +2258,7 @@ static int init_generic_mmio_info(struct intel_gvt *gvt)
MMIO_DFH(_MMIO(0x2438), D_ALL, F_CMD_ACCESS, NULL, NULL);
MMIO_DFH(_MMIO(0x243c), D_ALL, F_CMD_ACCESS, NULL, NULL);
MMIO_DFH(_MMIO(0x7018), D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL);
- MMIO_DFH(HALF_SLICE_CHICKEN3, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL);
+ MMIO_DFH(HSW_HALF_SLICE_CHICKEN3, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL);
MMIO_DFH(GEN7_HALF_SLICE_CHICKEN1, D_ALL, F_MODE_MASK | F_CMD_ACCESS, NULL, NULL);
/* display */
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 714221f..077892a 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -683,7 +683,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
intel_gvt_activate_vgpu(vgpu);
- atomic_set(&vgpu->released, 0);
return 0;
}
@@ -705,9 +704,6 @@ static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
if (!vgpu->attached)
return;
- if (atomic_cmpxchg(&vgpu->released, 0, 1))
- return;
-
intel_gvt_release_vgpu(vgpu);
debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME, vgpu->debugfs));
diff --git a/drivers/gpu/drm/i915/gvt/mmio.c b/drivers/gpu/drm/i915/gvt/mmio.c
index 9acc005..5b5def6 100644
--- a/drivers/gpu/drm/i915/gvt/mmio.c
+++ b/drivers/gpu/drm/i915/gvt/mmio.c
@@ -37,6 +37,7 @@
#include "i915_reg.h"
#include "gvt.h"
+#include "display/intel_dpio_phy.h"
#include "gt/intel_gt_regs.h"
/**
diff --git a/drivers/gpu/drm/i915/gvt/mmio_context.c b/drivers/gpu/drm/i915/gvt/mmio_context.c
index 1c6e941..490e8ae 100644
--- a/drivers/gpu/drm/i915/gvt/mmio_context.c
+++ b/drivers/gpu/drm/i915/gvt/mmio_context.c
@@ -34,6 +34,7 @@
*/
#include "i915_drv.h"
+#include "i915_reg.h"
#include "gt/intel_context.h"
#include "gt/intel_engine_regs.h"
#include "gt/intel_gpu_commands.h"
@@ -106,15 +107,15 @@ static struct engine_mmio gen9_engine_mmio_list[] __cacheline_aligned = {
{RCS0, GEN8_CS_CHICKEN1, 0xffff, true}, /* 0x2580 */
{RCS0, COMMON_SLICE_CHICKEN2, 0xffff, true}, /* 0x7014 */
{RCS0, GEN9_CS_DEBUG_MODE1, 0xffff, false}, /* 0x20ec */
- {RCS0, GEN8_L3SQCREG4, 0, false}, /* 0xb118 */
- {RCS0, GEN9_SCRATCH1, 0, false}, /* 0xb11c */
+ {RCS0, _MMIO(0xb118), 0, false}, /* GEN8_L3SQCREG4 */
+ {RCS0, _MMIO(0xb11c), 0, false}, /* GEN9_SCRATCH1 */
{RCS0, GEN9_SCRATCH_LNCF1, 0, false}, /* 0xb008 */
{RCS0, GEN7_HALF_SLICE_CHICKEN1, 0xffff, true}, /* 0xe100 */
- {RCS0, HALF_SLICE_CHICKEN2, 0xffff, true}, /* 0xe180 */
- {RCS0, HALF_SLICE_CHICKEN3, 0xffff, true}, /* 0xe184 */
- {RCS0, GEN9_HALF_SLICE_CHICKEN5, 0xffff, true}, /* 0xe188 */
- {RCS0, GEN9_HALF_SLICE_CHICKEN7, 0xffff, true}, /* 0xe194 */
- {RCS0, GEN8_ROW_CHICKEN, 0xffff, true}, /* 0xe4f0 */
+ {RCS0, _MMIO(0xe180), 0xffff, true}, /* HALF_SLICE_CHICKEN2 */
+ {RCS0, _MMIO(0xe184), 0xffff, true}, /* GEN8_HALF_SLICE_CHICKEN3 */
+ {RCS0, _MMIO(0xe188), 0xffff, true}, /* GEN9_HALF_SLICE_CHICKEN5 */
+ {RCS0, _MMIO(0xe194), 0xffff, true}, /* GEN9_HALF_SLICE_CHICKEN7 */
+ {RCS0, _MMIO(0xe4f0), 0xffff, true}, /* GEN8_ROW_CHICKEN */
{RCS0, TRVATTL3PTRDW(0), 0, true}, /* 0x4de0 */
{RCS0, TRVATTL3PTRDW(1), 0, true}, /* 0x4de4 */
{RCS0, TRNULLDETCT, 0, true}, /* 0x4de8 */
diff --git a/drivers/gpu/drm/i915/gvt/page_track.c b/drivers/gpu/drm/i915/gvt/page_track.c
index 3375b51..df34e73 100644
--- a/drivers/gpu/drm/i915/gvt/page_track.c
+++ b/drivers/gpu/drm/i915/gvt/page_track.c
@@ -120,7 +120,7 @@ int intel_vgpu_enable_page_track(struct intel_vgpu *vgpu, unsigned long gfn)
}
/**
- * intel_vgpu_enable_page_track - cancel write-protection on guest page
+ * intel_vgpu_disable_page_track - cancel write-protection on guest page
* @vgpu: a vGPU
* @gfn: the gfn of guest page
*
diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c
index d6fe94c..9cd8fcb 100644
--- a/drivers/gpu/drm/i915/gvt/scheduler.c
+++ b/drivers/gpu/drm/i915/gvt/scheduler.c
@@ -570,9 +570,8 @@ static int prepare_shadow_batch_buffer(struct intel_vgpu_workload *workload)
if (gmadr_bytes == 8)
bb->bb_start_cmd_va[2] = 0;
- ret = i915_vma_move_to_active(bb->vma,
- workload->req,
- 0);
+ ret = i915_vma_move_to_active(bb->vma, workload->req,
+ __EXEC_OBJECT_NO_REQUEST_AWAIT);
if (ret)
goto err;
diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c
index 56c7147..3c529c2 100644
--- a/drivers/gpu/drm/i915/gvt/vgpu.c
+++ b/drivers/gpu/drm/i915/gvt/vgpu.c
@@ -158,7 +158,7 @@ void intel_gvt_clean_vgpu_types(struct intel_gvt *gvt)
}
/**
- * intel_gvt_active_vgpu - activate a virtual GPU
+ * intel_gvt_activate_vgpu - activate a virtual GPU
* @vgpu: virtual GPU
*
* This function is called when user wants to activate a virtual GPU.
@@ -172,7 +172,7 @@ void intel_gvt_activate_vgpu(struct intel_vgpu *vgpu)
}
/**
- * intel_gvt_deactive_vgpu - deactivate a virtual GPU
+ * intel_gvt_deactivate_vgpu - deactivate a virtual GPU
* @vgpu: virtual GPU
*
* This function is called when user wants to deactivate a virtual GPU.
@@ -295,7 +295,7 @@ struct intel_vgpu *intel_gvt_create_idle_vgpu(struct intel_gvt *gvt)
}
/**
- * intel_gvt_destroy_vgpu - destroy an idle virtual GPU
+ * intel_gvt_destroy_idle_vgpu - destroy an idle virtual GPU
* @vgpu: virtual GPU
*
* This function is called when user wants to destroy an idle virtual GPU.
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index ae987e9..6c7ac73 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -688,8 +688,8 @@ i915_drop_caches_set(void *data, u64 val)
unsigned int flags;
int ret;
- DRM_DEBUG("Dropping caches: 0x%08llx [0x%08llx]\n",
- val, val & DROP_ALL);
+ drm_dbg(&i915->drm, "Dropping caches: 0x%08llx [0x%08llx]\n",
+ val, val & DROP_ALL);
ret = gt_drop_caches(to_gt(i915), val);
if (ret)
diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c
index f2a15d8..69103ae 100644
--- a/drivers/gpu/drm/i915/i915_driver.c
+++ b/drivers/gpu/drm/i915/i915_driver.c
@@ -81,6 +81,7 @@
#include "i915_drm_client.h"
#include "i915_drv.h"
#include "i915_getparam.h"
+#include "i915_hwmon.h"
#include "i915_ioc32.h"
#include "i915_ioctl.h"
#include "i915_irq.h"
@@ -337,7 +338,8 @@ static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
if (i915_inject_probe_failure(dev_priv))
return -ENODEV;
- intel_device_info_subplatform_init(dev_priv);
+ intel_device_info_runtime_init_early(dev_priv);
+
intel_step_init(dev_priv);
intel_uncore_mmio_debug_init_early(dev_priv);
@@ -370,8 +372,6 @@ static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
if (ret)
goto err_ttm;
- intel_wopcm_init_early(&dev_priv->wopcm);
-
ret = intel_root_gt_init_early(dev_priv);
if (ret < 0)
goto err_rootgt;
@@ -739,7 +739,6 @@ static void i915_driver_hw_remove(struct drm_i915_private *dev_priv)
*/
static void i915_driver_register(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_gt *gt;
unsigned int i;
@@ -749,7 +748,7 @@ static void i915_driver_register(struct drm_i915_private *dev_priv)
intel_vgpu_register(dev_priv);
/* Reveal our presence to userspace */
- if (drm_dev_register(dev, 0)) {
+ if (drm_dev_register(&dev_priv->drm, 0)) {
drm_err(&dev_priv->drm,
"Failed to register driver for userspace access!\n");
return;
@@ -764,6 +763,8 @@ static void i915_driver_register(struct drm_i915_private *dev_priv)
for_each_gt(gt, dev_priv, i)
intel_gt_driver_register(gt);
+ i915_hwmon_register(dev_priv);
+
intel_display_driver_register(dev_priv);
intel_power_domains_enable(dev_priv);
@@ -796,6 +797,8 @@ static void i915_driver_unregister(struct drm_i915_private *dev_priv)
for_each_gt(gt, dev_priv, i)
intel_gt_driver_unregister(gt);
+ i915_hwmon_unregister(dev_priv);
+
i915_perf_unregister(dev_priv);
i915_pmu_unregister(dev_priv);
@@ -894,10 +897,6 @@ int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (IS_ERR(i915))
return PTR_ERR(i915);
- /* Disable nuclear pageflip by default on pre-ILK */
- if (!i915->params.nuclear_pageflip && DISPLAY_VER(i915) < 5)
- i915->drm.driver_features &= ~DRIVER_ATOMIC;
-
ret = pci_enable_device(pdev);
if (ret)
goto out_fini;
@@ -1093,32 +1092,30 @@ static void i915_driver_postclose(struct drm_device *dev, struct drm_file *file)
static void intel_suspend_encoders(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_encoder *encoder;
if (!HAS_DISPLAY(dev_priv))
return;
- drm_modeset_lock_all(dev);
- for_each_intel_encoder(dev, encoder)
+ drm_modeset_lock_all(&dev_priv->drm);
+ for_each_intel_encoder(&dev_priv->drm, encoder)
if (encoder->suspend)
encoder->suspend(encoder);
- drm_modeset_unlock_all(dev);
+ drm_modeset_unlock_all(&dev_priv->drm);
}
static void intel_shutdown_encoders(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
struct intel_encoder *encoder;
if (!HAS_DISPLAY(dev_priv))
return;
- drm_modeset_lock_all(dev);
- for_each_intel_encoder(dev, encoder)
+ drm_modeset_lock_all(&dev_priv->drm);
+ for_each_intel_encoder(&dev_priv->drm, encoder)
if (encoder->shutdown)
encoder->shutdown(encoder);
- drm_modeset_unlock_all(dev);
+ drm_modeset_unlock_all(&dev_priv->drm);
}
void i915_driver_shutdown(struct drm_i915_private *i915)
@@ -1662,7 +1659,8 @@ static int intel_runtime_suspend(struct device *kdev)
intel_runtime_pm_enable_interrupts(dev_priv);
- intel_gt_runtime_resume(to_gt(dev_priv));
+ for_each_gt(gt, dev_priv, i)
+ intel_gt_runtime_resume(gt);
enable_rpm_wakeref_asserts(rpm);
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index bdc81db..a380db3 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -40,7 +40,6 @@
#include "display/intel_display_core.h"
#include "gem/i915_gem_context_types.h"
-#include "gem/i915_gem_lmem.h"
#include "gem/i915_gem_shrinker.h"
#include "gem/i915_gem_stolen.h"
@@ -63,7 +62,6 @@
#include "intel_runtime_pm.h"
#include "intel_step.h"
#include "intel_uncore.h"
-#include "intel_wopcm.h"
struct drm_i915_clock_gating_funcs;
struct drm_i915_gem_object;
@@ -75,9 +73,6 @@ struct intel_limit;
struct intel_overlay_error_state;
struct vlv_s0ix_state;
-/* Threshold == 5 for long IRQs, 50 for short */
-#define HPD_STORM_DEFAULT_THRESHOLD 50
-
#define I915_GEM_GPU_DOMAINS \
(I915_GEM_DOMAIN_RENDER | \
I915_GEM_DOMAIN_SAMPLER | \
@@ -239,8 +234,6 @@ struct drm_i915_private {
struct intel_gvt *gvt;
- struct intel_wopcm wopcm;
-
struct pci_dev *bridge_dev;
struct rb_root uabi_engines;
@@ -291,28 +284,13 @@ struct drm_i915_private {
unsigned long gem_quirks;
- struct drm_atomic_state *modeset_restore_state;
- struct drm_modeset_acquire_ctx reset_ctx;
-
struct i915_gem_mm mm;
- /* Kernel Modesetting */
-
- struct list_head global_obj_list;
-
bool mchbar_need_disable;
struct intel_l3_parity l3_parity;
/*
- * HTI (aka HDPORT) state read during initial hw readout. Most
- * platforms don't have HTI, so this will just stay 0. Those that do
- * will use this later to figure out which PLLs and PHYs are unavailable
- * for driver usage.
- */
- u32 hti_state;
-
- /*
* edram size in MB.
* Cannot be determined by PCIID. You must always read a register.
*/
@@ -353,6 +331,8 @@ struct drm_i915_private {
struct i915_perf perf;
+ struct i915_hwmon *hwmon;
+
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
struct intel_gt gt0;
@@ -742,6 +722,10 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define IS_XEHPSDV_GRAPHICS_STEP(__i915, since, until) \
(IS_XEHPSDV(__i915) && IS_GRAPHICS_STEP(__i915, since, until))
+#define IS_MTL_GRAPHICS_STEP(__i915, variant, since, until) \
+ (IS_SUBPLATFORM(__i915, INTEL_METEORLAKE, INTEL_SUBPLATFORM_##variant) && \
+ IS_GRAPHICS_STEP(__i915, since, until))
+
/*
* DG2 hardware steppings are a bit unusual. The hardware design was forked to
* create three variants (G10, G11, and G12) which each have distinct
@@ -780,12 +764,15 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define __HAS_ENGINE(engine_mask, id) ((engine_mask) & BIT(id))
#define HAS_ENGINE(gt, id) __HAS_ENGINE((gt)->info.engine_mask, id)
-#define ENGINE_INSTANCES_MASK(gt, first, count) ({ \
+#define __ENGINE_INSTANCES_MASK(mask, first, count) ({ \
unsigned int first__ = (first); \
unsigned int count__ = (count); \
- ((gt)->info.engine_mask & \
- GENMASK(first__ + count__ - 1, first__)) >> first__; \
+ ((mask) & GENMASK(first__ + count__ - 1, first__)) >> first__; \
})
+
+#define ENGINE_INSTANCES_MASK(gt, first, count) \
+ __ENGINE_INSTANCES_MASK((gt)->info.engine_mask, first, count)
+
#define RCS_MASK(gt) \
ENGINE_INSTANCES_MASK(gt, RCS0, I915_MAX_RCS)
#define BCS_MASK(gt) \
@@ -871,6 +858,7 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_DOUBLE_BUFFERED_M_N(dev_priv) (DISPLAY_VER(dev_priv) >= 9 || IS_BROADWELL(dev_priv))
#define HAS_CDCLK_CRAWL(dev_priv) (INTEL_INFO(dev_priv)->display.has_cdclk_crawl)
+#define HAS_CDCLK_SQUASH(dev_priv) (INTEL_INFO(dev_priv)->display.has_cdclk_squash)
#define HAS_DDI(dev_priv) (INTEL_INFO(dev_priv)->display.has_ddi)
#define HAS_FPGA_DBG_UNCLAIMED(dev_priv) (INTEL_INFO(dev_priv)->display.has_fpga_dbg)
#define HAS_PSR(dev_priv) (INTEL_INFO(dev_priv)->display.has_psr)
@@ -900,19 +888,17 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_RUNTIME_PM(dev_priv) (INTEL_INFO(dev_priv)->has_runtime_pm)
#define HAS_64BIT_RELOC(dev_priv) (INTEL_INFO(dev_priv)->has_64bit_reloc)
+#define HAS_OA_BPC_REPORTING(dev_priv) \
+ (INTEL_INFO(dev_priv)->has_oa_bpc_reporting)
+#define HAS_OA_SLICE_CONTRIB_LIMITS(dev_priv) \
+ (INTEL_INFO(dev_priv)->has_oa_slice_contrib_limits)
+
/*
* Set this flag, when platform requires 64K GTT page sizes or larger for
* device local memory access.
*/
#define HAS_64K_PAGES(dev_priv) (INTEL_INFO(dev_priv)->has_64k_pages)
-/*
- * Set this flag when platform doesn't allow both 64k pages and 4k pages in
- * the same PT. this flag means we need to support compact PT layout for the
- * ppGTT when using the 64K GTT pages.
- */
-#define NEEDS_COMPACT_PT(dev_priv) (INTEL_INFO(dev_priv)->needs_compact_pt)
-
#define HAS_IPC(dev_priv) (INTEL_INFO(dev_priv)->display.has_ipc)
#define HAS_REGION(i915, i) (RUNTIME_INFO(i915)->memory_regions & (i))
@@ -938,6 +924,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_GMCH(dev_priv) (INTEL_INFO(dev_priv)->display.has_gmch)
+#define HAS_GMD_ID(i915) (INTEL_INFO(i915)->has_gmd_id)
+
#define HAS_LSPCON(dev_priv) (IS_DISPLAY_VER(dev_priv, 9, 10))
#define HAS_L3_CCS_READ(i915) (INTEL_INFO(i915)->has_l3_ccs_read)
@@ -976,6 +964,9 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_ONE_EU_PER_FUSE_BIT(i915) (INTEL_INFO(i915)->has_one_eu_per_fuse_bit)
+#define HAS_LMEMBAR_SMEM_STOLEN(i915) (!HAS_LMEM(i915) && \
+ GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70))
+
/* intel_device_info.c */
static inline struct intel_device_info *
mkwrite_device_info(struct drm_i915_private *dev_priv)
@@ -983,16 +974,4 @@ mkwrite_device_info(struct drm_i915_private *dev_priv)
return (struct intel_device_info *)INTEL_INFO(dev_priv);
}
-static inline enum i915_map_type
-i915_coherent_map_type(struct drm_i915_private *i915,
- struct drm_i915_gem_object *obj, bool always_coherent)
-{
- if (i915_gem_object_is_lmem(obj))
- return I915_MAP_WC;
- if (HAS_LLC(i915) || always_coherent)
- return I915_MAP_WB;
- else
- return I915_MAP_WC;
-}
-
#endif
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 2bdddb6..8468ca9 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -843,7 +843,7 @@ void i915_gem_runtime_suspend(struct drm_i915_private *i915)
__i915_gem_object_release_mmap_gtt(obj);
list_for_each_entry_safe(obj, on,
- &to_gt(i915)->lmem_userfault_list, userfault_link)
+ &i915->runtime_pm.lmem_userfault_list, userfault_link)
i915_gem_object_runtime_pm_release_mmap_offset(obj);
/*
@@ -1128,6 +1128,8 @@ void i915_gem_drain_workqueue(struct drm_i915_private *i915)
int i915_gem_init(struct drm_i915_private *dev_priv)
{
+ struct intel_gt *gt;
+ unsigned int i;
int ret;
/* We need to fallback to 4K pages if host doesn't support huge gtt. */
@@ -1138,8 +1140,10 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
if (ret)
return ret;
- intel_uc_fetch_firmwares(&to_gt(dev_priv)->uc);
- intel_wopcm_init(&dev_priv->wopcm);
+ for_each_gt(gt, dev_priv, i) {
+ intel_uc_fetch_firmwares(>->uc);
+ intel_wopcm_init(>->wopcm);
+ }
ret = i915_init_ggtt(dev_priv);
if (ret) {
@@ -1158,9 +1162,11 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
*/
intel_init_clock_gating(dev_priv);
- ret = intel_gt_init(to_gt(dev_priv));
- if (ret)
- goto err_unlock;
+ for_each_gt(gt, dev_priv, i) {
+ ret = intel_gt_init(gt);
+ if (ret)
+ goto err_unlock;
+ }
return 0;
@@ -1173,8 +1179,13 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
err_unlock:
i915_gem_drain_workqueue(dev_priv);
- if (ret != -EIO)
- intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc);
+ if (ret != -EIO) {
+ for_each_gt(gt, dev_priv, i) {
+ intel_gt_driver_remove(gt);
+ intel_gt_driver_release(gt);
+ intel_uc_cleanup_firmwares(>->uc);
+ }
+ }
if (ret == -EIO) {
/*
@@ -1182,10 +1193,12 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
* as wedged. But we only want to do this when the GPU is angry,
* for all other failure, such as an allocation failure, bail.
*/
- if (!intel_gt_is_wedged(to_gt(dev_priv))) {
- i915_probe_error(dev_priv,
- "Failed to initialize GPU, declaring it wedged!\n");
- intel_gt_set_wedged(to_gt(dev_priv));
+ for_each_gt(gt, dev_priv, i) {
+ if (!intel_gt_is_wedged(gt)) {
+ i915_probe_error(dev_priv,
+ "Failed to initialize GPU, declaring it wedged!\n");
+ intel_gt_set_wedged(gt);
+ }
}
/* Minimal basic recovery for KMS */
@@ -1213,23 +1226,27 @@ void i915_gem_driver_unregister(struct drm_i915_private *i915)
void i915_gem_driver_remove(struct drm_i915_private *dev_priv)
{
- intel_wakeref_auto_fini(&to_gt(dev_priv)->userfault_wakeref);
+ struct intel_gt *gt;
+ unsigned int i;
i915_gem_suspend_late(dev_priv);
- intel_gt_driver_remove(to_gt(dev_priv));
+ for_each_gt(gt, dev_priv, i)
+ intel_gt_driver_remove(gt);
dev_priv->uabi_engines = RB_ROOT;
/* Flush any outstanding unpin_work. */
i915_gem_drain_workqueue(dev_priv);
-
- i915_gem_drain_freed_objects(dev_priv);
}
void i915_gem_driver_release(struct drm_i915_private *dev_priv)
{
- intel_gt_driver_release(to_gt(dev_priv));
+ struct intel_gt *gt;
+ unsigned int i;
- intel_uc_cleanup_firmwares(&to_gt(dev_priv)->uc);
+ for_each_gt(gt, dev_priv, i) {
+ intel_gt_driver_release(gt);
+ intel_uc_cleanup_firmwares(>->uc);
+ }
/* Flush any outstanding work, including i915_gem_context.release_work. */
i915_gem_drain_workqueue(dev_priv);
@@ -1259,7 +1276,7 @@ void i915_gem_init_early(struct drm_i915_private *dev_priv)
void i915_gem_cleanup_early(struct drm_i915_private *dev_priv)
{
- i915_gem_drain_freed_objects(dev_priv);
+ i915_gem_drain_workqueue(dev_priv);
GEM_BUG_ON(!llist_empty(&dev_priv->mm.free_list));
GEM_BUG_ON(atomic_read(&dev_priv->mm.free_count));
drm_WARN_ON(&dev_priv->drm, dev_priv->mm.shrink_count);
@@ -1271,7 +1288,7 @@ int i915_gem_open(struct drm_i915_private *i915, struct drm_file *file)
struct i915_drm_client *client;
int ret = -ENOMEM;
- DRM_DEBUG("\n");
+ drm_dbg(&i915->drm, "\n");
file_priv = kzalloc(sizeof(*file_priv), GFP_KERNEL);
if (!file_priv)
diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c
index 342c8ca..61ef2d9 100644
--- a/drivers/gpu/drm/i915/i915_getparam.c
+++ b/drivers/gpu/drm/i915/i915_getparam.c
@@ -175,8 +175,11 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data,
case I915_PARAM_PERF_REVISION:
value = i915_perf_ioctl_version();
break;
+ case I915_PARAM_OA_TIMESTAMP_FREQUENCY:
+ value = i915_perf_oa_timestamp_frequency(i915);
+ break;
default:
- DRM_DEBUG("Unknown parameter %d\n", param->param);
+ drm_dbg(&i915->drm, "Unknown parameter %d\n", param->param);
return -EINVAL;
}
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 9ea2fe3..9d5d5a3 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -55,6 +55,7 @@
#include "i915_drv.h"
#include "i915_gpu_error.h"
#include "i915_memcpy.h"
+#include "i915_reg.h"
#include "i915_scatterlist.h"
#include "i915_utils.h"
@@ -1221,7 +1222,10 @@ static void engine_record_registers(struct intel_engine_coredump *ee)
if (GRAPHICS_VER(i915) >= 6) {
ee->rc_psmi = ENGINE_READ(engine, RING_PSMI_CTL);
- if (GRAPHICS_VER(i915) >= 12)
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50))
+ ee->fault_reg = intel_gt_mcr_read_any(engine->gt,
+ XEHP_RING_FAULT_REG);
+ else if (GRAPHICS_VER(i915) >= 12)
ee->fault_reg = intel_uncore_read(engine->uncore,
GEN12_RING_FAULT_REG);
else if (GRAPHICS_VER(i915) >= 8)
@@ -1820,7 +1824,12 @@ static void gt_record_global_regs(struct intel_gt_coredump *gt)
if (GRAPHICS_VER(i915) == 7)
gt->err_int = intel_uncore_read(uncore, GEN7_ERR_INT);
- if (GRAPHICS_VER(i915) >= 12) {
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) {
+ gt->fault_data0 = intel_gt_mcr_read_any((struct intel_gt *)gt->_gt,
+ XEHP_FAULT_TLB_DATA0);
+ gt->fault_data1 = intel_gt_mcr_read_any((struct intel_gt *)gt->_gt,
+ XEHP_FAULT_TLB_DATA1);
+ } else if (GRAPHICS_VER(i915) >= 12) {
gt->fault_data0 = intel_uncore_read(uncore,
GEN12_FAULT_TLB_DATA0);
gt->fault_data1 = intel_uncore_read(uncore,
diff --git a/drivers/gpu/drm/i915/i915_hwmon.c b/drivers/gpu/drm/i915/i915_hwmon.c
new file mode 100644
index 0000000..c588a17
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_hwmon.c
@@ -0,0 +1,732 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#include <linux/hwmon.h>
+#include <linux/hwmon-sysfs.h>
+#include <linux/types.h>
+
+#include "i915_drv.h"
+#include "i915_hwmon.h"
+#include "i915_reg.h"
+#include "intel_mchbar_regs.h"
+#include "intel_pcode.h"
+#include "gt/intel_gt.h"
+#include "gt/intel_gt_regs.h"
+
+/*
+ * SF_* - scale factors for particular quantities according to hwmon spec.
+ * - voltage - millivolts
+ * - power - microwatts
+ * - curr - milliamperes
+ * - energy - microjoules
+ * - time - milliseconds
+ */
+#define SF_VOLTAGE 1000
+#define SF_POWER 1000000
+#define SF_CURR 1000
+#define SF_ENERGY 1000000
+#define SF_TIME 1000
+
+struct hwm_reg {
+ i915_reg_t gt_perf_status;
+ i915_reg_t pkg_power_sku_unit;
+ i915_reg_t pkg_power_sku;
+ i915_reg_t pkg_rapl_limit;
+ i915_reg_t energy_status_all;
+ i915_reg_t energy_status_tile;
+};
+
+struct hwm_energy_info {
+ u32 reg_val_prev;
+ long accum_energy; /* Accumulated energy for energy1_input */
+};
+
+struct hwm_drvdata {
+ struct i915_hwmon *hwmon;
+ struct intel_uncore *uncore;
+ struct device *hwmon_dev;
+ struct hwm_energy_info ei; /* Energy info for energy1_input */
+ char name[12];
+ int gt_n;
+};
+
+struct i915_hwmon {
+ struct hwm_drvdata ddat;
+ struct hwm_drvdata ddat_gt[I915_MAX_GT];
+ struct mutex hwmon_lock; /* counter overflow logic and rmw */
+ struct hwm_reg rg;
+ int scl_shift_power;
+ int scl_shift_energy;
+ int scl_shift_time;
+};
+
+static void
+hwm_locked_with_pm_intel_uncore_rmw(struct hwm_drvdata *ddat,
+ i915_reg_t reg, u32 clear, u32 set)
+{
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ struct intel_uncore *uncore = ddat->uncore;
+ intel_wakeref_t wakeref;
+
+ mutex_lock(&hwmon->hwmon_lock);
+
+ with_intel_runtime_pm(uncore->rpm, wakeref)
+ intel_uncore_rmw(uncore, reg, clear, set);
+
+ mutex_unlock(&hwmon->hwmon_lock);
+}
+
+/*
+ * This function's return type of u64 allows for the case where the scaling
+ * of the field taken from the 32-bit register value might cause a result to
+ * exceed 32 bits.
+ */
+static u64
+hwm_field_read_and_scale(struct hwm_drvdata *ddat, i915_reg_t rgadr,
+ u32 field_msk, int nshift, u32 scale_factor)
+{
+ struct intel_uncore *uncore = ddat->uncore;
+ intel_wakeref_t wakeref;
+ u32 reg_value;
+
+ with_intel_runtime_pm(uncore->rpm, wakeref)
+ reg_value = intel_uncore_read(uncore, rgadr);
+
+ reg_value = REG_FIELD_GET(field_msk, reg_value);
+
+ return mul_u64_u32_shr(reg_value, scale_factor, nshift);
+}
+
+static void
+hwm_field_scale_and_write(struct hwm_drvdata *ddat, i915_reg_t rgadr,
+ int nshift, unsigned int scale_factor, long lval)
+{
+ u32 nval;
+
+ /* Computation in 64-bits to avoid overflow. Round to nearest. */
+ nval = DIV_ROUND_CLOSEST_ULL((u64)lval << nshift, scale_factor);
+
+ hwm_locked_with_pm_intel_uncore_rmw(ddat, rgadr,
+ PKG_PWR_LIM_1,
+ REG_FIELD_PREP(PKG_PWR_LIM_1, nval));
+}
+
+/*
+ * hwm_energy - Obtain energy value
+ *
+ * The underlying energy hardware register is 32-bits and is subject to
+ * overflow. How long before overflow? For example, with an example
+ * scaling bit shift of 14 bits (see register *PACKAGE_POWER_SKU_UNIT) and
+ * a power draw of 1000 watts, the 32-bit counter will overflow in
+ * approximately 4.36 minutes.
+ *
+ * Examples:
+ * 1 watt: (2^32 >> 14) / 1 W / (60 * 60 * 24) secs/day -> 3 days
+ * 1000 watts: (2^32 >> 14) / 1000 W / 60 secs/min -> 4.36 minutes
+ *
+ * The function significantly increases overflow duration (from 4.36
+ * minutes) by accumulating the energy register into a 'long' as allowed by
+ * the hwmon API. Using x86_64 128 bit arithmetic (see mul_u64_u32_shr()),
+ * a 'long' of 63 bits, SF_ENERGY of 1e6 (~20 bits) and
+ * hwmon->scl_shift_energy of 14 bits we have 57 (63 - 20 + 14) bits before
+ * energy1_input overflows. This at 1000 W is an overflow duration of 278 years.
+ */
+static void
+hwm_energy(struct hwm_drvdata *ddat, long *energy)
+{
+ struct intel_uncore *uncore = ddat->uncore;
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ struct hwm_energy_info *ei = &ddat->ei;
+ intel_wakeref_t wakeref;
+ i915_reg_t rgaddr;
+ u32 reg_val;
+
+ if (ddat->gt_n >= 0)
+ rgaddr = hwmon->rg.energy_status_tile;
+ else
+ rgaddr = hwmon->rg.energy_status_all;
+
+ mutex_lock(&hwmon->hwmon_lock);
+
+ with_intel_runtime_pm(uncore->rpm, wakeref)
+ reg_val = intel_uncore_read(uncore, rgaddr);
+
+ if (reg_val >= ei->reg_val_prev)
+ ei->accum_energy += reg_val - ei->reg_val_prev;
+ else
+ ei->accum_energy += UINT_MAX - ei->reg_val_prev + reg_val;
+ ei->reg_val_prev = reg_val;
+
+ *energy = mul_u64_u32_shr(ei->accum_energy, SF_ENERGY,
+ hwmon->scl_shift_energy);
+ mutex_unlock(&hwmon->hwmon_lock);
+}
+
+static ssize_t
+hwm_power1_max_interval_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct hwm_drvdata *ddat = dev_get_drvdata(dev);
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ intel_wakeref_t wakeref;
+ u32 r, x, y, x_w = 2; /* 2 bits */
+ u64 tau4, out;
+
+ with_intel_runtime_pm(ddat->uncore->rpm, wakeref)
+ r = intel_uncore_read(ddat->uncore, hwmon->rg.pkg_rapl_limit);
+
+ x = REG_FIELD_GET(PKG_PWR_LIM_1_TIME_X, r);
+ y = REG_FIELD_GET(PKG_PWR_LIM_1_TIME_Y, r);
+ /*
+ * tau = 1.x * power(2,y), x = bits(23:22), y = bits(21:17)
+ * = (4 | x) << (y - 2)
+ * where (y - 2) ensures a 1.x fixed point representation of 1.x
+ * However because y can be < 2, we compute
+ * tau4 = (4 | x) << y
+ * but add 2 when doing the final right shift to account for units
+ */
+ tau4 = ((1 << x_w) | x) << y;
+ /* val in hwmon interface units (millisec) */
+ out = mul_u64_u32_shr(tau4, SF_TIME, hwmon->scl_shift_time + x_w);
+
+ return sysfs_emit(buf, "%llu\n", out);
+}
+
+static ssize_t
+hwm_power1_max_interval_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct hwm_drvdata *ddat = dev_get_drvdata(dev);
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ u32 x, y, rxy, x_w = 2; /* 2 bits */
+ u64 tau4, r, max_win;
+ unsigned long val;
+ int ret;
+
+ ret = kstrtoul(buf, 0, &val);
+ if (ret)
+ return ret;
+
+ /*
+ * Max HW supported tau in '1.x * power(2,y)' format, x = 0, y = 0x12
+ * The hwmon->scl_shift_time default of 0xa results in a max tau of 256 seconds
+ */
+#define PKG_MAX_WIN_DEFAULT 0x12ull
+
+ /*
+ * val must be < max in hwmon interface units. The steps below are
+ * explained in i915_power1_max_interval_show()
+ */
+ r = FIELD_PREP(PKG_MAX_WIN, PKG_MAX_WIN_DEFAULT);
+ x = REG_FIELD_GET(PKG_MAX_WIN_X, r);
+ y = REG_FIELD_GET(PKG_MAX_WIN_Y, r);
+ tau4 = ((1 << x_w) | x) << y;
+ max_win = mul_u64_u32_shr(tau4, SF_TIME, hwmon->scl_shift_time + x_w);
+
+ if (val > max_win)
+ return -EINVAL;
+
+ /* val in hw units */
+ val = DIV_ROUND_CLOSEST_ULL((u64)val << hwmon->scl_shift_time, SF_TIME);
+ /* Convert to 1.x * power(2,y) */
+ if (!val)
+ return -EINVAL;
+ y = ilog2(val);
+ /* x = (val - (1 << y)) >> (y - 2); */
+ x = (val - (1ul << y)) << x_w >> y;
+
+ rxy = REG_FIELD_PREP(PKG_PWR_LIM_1_TIME_X, x) | REG_FIELD_PREP(PKG_PWR_LIM_1_TIME_Y, y);
+
+ hwm_locked_with_pm_intel_uncore_rmw(ddat, hwmon->rg.pkg_rapl_limit,
+ PKG_PWR_LIM_1_TIME, rxy);
+ return count;
+}
+
+static SENSOR_DEVICE_ATTR(power1_max_interval, 0664,
+ hwm_power1_max_interval_show,
+ hwm_power1_max_interval_store, 0);
+
+static struct attribute *hwm_attributes[] = {
+ &sensor_dev_attr_power1_max_interval.dev_attr.attr,
+ NULL
+};
+
+static umode_t hwm_attributes_visible(struct kobject *kobj,
+ struct attribute *attr, int index)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct hwm_drvdata *ddat = dev_get_drvdata(dev);
+ struct i915_hwmon *hwmon = ddat->hwmon;
+
+ if (attr == &sensor_dev_attr_power1_max_interval.dev_attr.attr)
+ return i915_mmio_reg_valid(hwmon->rg.pkg_rapl_limit) ? attr->mode : 0;
+
+ return 0;
+}
+
+static const struct attribute_group hwm_attrgroup = {
+ .attrs = hwm_attributes,
+ .is_visible = hwm_attributes_visible,
+};
+
+static const struct attribute_group *hwm_groups[] = {
+ &hwm_attrgroup,
+ NULL
+};
+
+static const struct hwmon_channel_info *hwm_info[] = {
+ HWMON_CHANNEL_INFO(in, HWMON_I_INPUT),
+ HWMON_CHANNEL_INFO(power, HWMON_P_MAX | HWMON_P_RATED_MAX | HWMON_P_CRIT),
+ HWMON_CHANNEL_INFO(energy, HWMON_E_INPUT),
+ HWMON_CHANNEL_INFO(curr, HWMON_C_CRIT),
+ NULL
+};
+
+static const struct hwmon_channel_info *hwm_gt_info[] = {
+ HWMON_CHANNEL_INFO(energy, HWMON_E_INPUT),
+ NULL
+};
+
+/* I1 is exposed as power_crit or as curr_crit depending on bit 31 */
+static int hwm_pcode_read_i1(struct drm_i915_private *i915, u32 *uval)
+{
+ return snb_pcode_read_p(&i915->uncore, PCODE_POWER_SETUP,
+ POWER_SETUP_SUBCOMMAND_READ_I1, 0, uval);
+}
+
+static int hwm_pcode_write_i1(struct drm_i915_private *i915, u32 uval)
+{
+ return snb_pcode_write_p(&i915->uncore, PCODE_POWER_SETUP,
+ POWER_SETUP_SUBCOMMAND_WRITE_I1, 0, uval);
+}
+
+static umode_t
+hwm_in_is_visible(const struct hwm_drvdata *ddat, u32 attr)
+{
+ struct drm_i915_private *i915 = ddat->uncore->i915;
+
+ switch (attr) {
+ case hwmon_in_input:
+ return IS_DG1(i915) || IS_DG2(i915) ? 0444 : 0;
+ default:
+ return 0;
+ }
+}
+
+static int
+hwm_in_read(struct hwm_drvdata *ddat, u32 attr, long *val)
+{
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ intel_wakeref_t wakeref;
+ u32 reg_value;
+
+ switch (attr) {
+ case hwmon_in_input:
+ with_intel_runtime_pm(ddat->uncore->rpm, wakeref)
+ reg_value = intel_uncore_read(ddat->uncore, hwmon->rg.gt_perf_status);
+ /* HW register value in units of 2.5 millivolt */
+ *val = DIV_ROUND_CLOSEST(REG_FIELD_GET(GEN12_VOLTAGE_MASK, reg_value) * 25, 10);
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static umode_t
+hwm_power_is_visible(const struct hwm_drvdata *ddat, u32 attr, int chan)
+{
+ struct drm_i915_private *i915 = ddat->uncore->i915;
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ u32 uval;
+
+ switch (attr) {
+ case hwmon_power_max:
+ return i915_mmio_reg_valid(hwmon->rg.pkg_rapl_limit) ? 0664 : 0;
+ case hwmon_power_rated_max:
+ return i915_mmio_reg_valid(hwmon->rg.pkg_power_sku) ? 0444 : 0;
+ case hwmon_power_crit:
+ return (hwm_pcode_read_i1(i915, &uval) ||
+ !(uval & POWER_SETUP_I1_WATTS)) ? 0 : 0644;
+ default:
+ return 0;
+ }
+}
+
+static int
+hwm_power_read(struct hwm_drvdata *ddat, u32 attr, int chan, long *val)
+{
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ int ret;
+ u32 uval;
+
+ switch (attr) {
+ case hwmon_power_max:
+ *val = hwm_field_read_and_scale(ddat,
+ hwmon->rg.pkg_rapl_limit,
+ PKG_PWR_LIM_1,
+ hwmon->scl_shift_power,
+ SF_POWER);
+ return 0;
+ case hwmon_power_rated_max:
+ *val = hwm_field_read_and_scale(ddat,
+ hwmon->rg.pkg_power_sku,
+ PKG_PKG_TDP,
+ hwmon->scl_shift_power,
+ SF_POWER);
+ return 0;
+ case hwmon_power_crit:
+ ret = hwm_pcode_read_i1(ddat->uncore->i915, &uval);
+ if (ret)
+ return ret;
+ if (!(uval & POWER_SETUP_I1_WATTS))
+ return -ENODEV;
+ *val = mul_u64_u32_shr(REG_FIELD_GET(POWER_SETUP_I1_DATA_MASK, uval),
+ SF_POWER, POWER_SETUP_I1_SHIFT);
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int
+hwm_power_write(struct hwm_drvdata *ddat, u32 attr, int chan, long val)
+{
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ u32 uval;
+
+ switch (attr) {
+ case hwmon_power_max:
+ hwm_field_scale_and_write(ddat,
+ hwmon->rg.pkg_rapl_limit,
+ hwmon->scl_shift_power,
+ SF_POWER, val);
+ return 0;
+ case hwmon_power_crit:
+ uval = DIV_ROUND_CLOSEST_ULL(val << POWER_SETUP_I1_SHIFT, SF_POWER);
+ return hwm_pcode_write_i1(ddat->uncore->i915, uval);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static umode_t
+hwm_energy_is_visible(const struct hwm_drvdata *ddat, u32 attr)
+{
+ struct i915_hwmon *hwmon = ddat->hwmon;
+ i915_reg_t rgaddr;
+
+ switch (attr) {
+ case hwmon_energy_input:
+ if (ddat->gt_n >= 0)
+ rgaddr = hwmon->rg.energy_status_tile;
+ else
+ rgaddr = hwmon->rg.energy_status_all;
+ return i915_mmio_reg_valid(rgaddr) ? 0444 : 0;
+ default:
+ return 0;
+ }
+}
+
+static int
+hwm_energy_read(struct hwm_drvdata *ddat, u32 attr, long *val)
+{
+ switch (attr) {
+ case hwmon_energy_input:
+ hwm_energy(ddat, val);
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static umode_t
+hwm_curr_is_visible(const struct hwm_drvdata *ddat, u32 attr)
+{
+ struct drm_i915_private *i915 = ddat->uncore->i915;
+ u32 uval;
+
+ switch (attr) {
+ case hwmon_curr_crit:
+ return (hwm_pcode_read_i1(i915, &uval) ||
+ (uval & POWER_SETUP_I1_WATTS)) ? 0 : 0644;
+ default:
+ return 0;
+ }
+}
+
+static int
+hwm_curr_read(struct hwm_drvdata *ddat, u32 attr, long *val)
+{
+ int ret;
+ u32 uval;
+
+ switch (attr) {
+ case hwmon_curr_crit:
+ ret = hwm_pcode_read_i1(ddat->uncore->i915, &uval);
+ if (ret)
+ return ret;
+ if (uval & POWER_SETUP_I1_WATTS)
+ return -ENODEV;
+ *val = mul_u64_u32_shr(REG_FIELD_GET(POWER_SETUP_I1_DATA_MASK, uval),
+ SF_CURR, POWER_SETUP_I1_SHIFT);
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int
+hwm_curr_write(struct hwm_drvdata *ddat, u32 attr, long val)
+{
+ u32 uval;
+
+ switch (attr) {
+ case hwmon_curr_crit:
+ uval = DIV_ROUND_CLOSEST_ULL(val << POWER_SETUP_I1_SHIFT, SF_CURR);
+ return hwm_pcode_write_i1(ddat->uncore->i915, uval);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static umode_t
+hwm_is_visible(const void *drvdata, enum hwmon_sensor_types type,
+ u32 attr, int channel)
+{
+ struct hwm_drvdata *ddat = (struct hwm_drvdata *)drvdata;
+
+ switch (type) {
+ case hwmon_in:
+ return hwm_in_is_visible(ddat, attr);
+ case hwmon_power:
+ return hwm_power_is_visible(ddat, attr, channel);
+ case hwmon_energy:
+ return hwm_energy_is_visible(ddat, attr);
+ case hwmon_curr:
+ return hwm_curr_is_visible(ddat, attr);
+ default:
+ return 0;
+ }
+}
+
+static int
+hwm_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
+ int channel, long *val)
+{
+ struct hwm_drvdata *ddat = dev_get_drvdata(dev);
+
+ switch (type) {
+ case hwmon_in:
+ return hwm_in_read(ddat, attr, val);
+ case hwmon_power:
+ return hwm_power_read(ddat, attr, channel, val);
+ case hwmon_energy:
+ return hwm_energy_read(ddat, attr, val);
+ case hwmon_curr:
+ return hwm_curr_read(ddat, attr, val);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int
+hwm_write(struct device *dev, enum hwmon_sensor_types type, u32 attr,
+ int channel, long val)
+{
+ struct hwm_drvdata *ddat = dev_get_drvdata(dev);
+
+ switch (type) {
+ case hwmon_power:
+ return hwm_power_write(ddat, attr, channel, val);
+ case hwmon_curr:
+ return hwm_curr_write(ddat, attr, val);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static const struct hwmon_ops hwm_ops = {
+ .is_visible = hwm_is_visible,
+ .read = hwm_read,
+ .write = hwm_write,
+};
+
+static const struct hwmon_chip_info hwm_chip_info = {
+ .ops = &hwm_ops,
+ .info = hwm_info,
+};
+
+static umode_t
+hwm_gt_is_visible(const void *drvdata, enum hwmon_sensor_types type,
+ u32 attr, int channel)
+{
+ struct hwm_drvdata *ddat = (struct hwm_drvdata *)drvdata;
+
+ switch (type) {
+ case hwmon_energy:
+ return hwm_energy_is_visible(ddat, attr);
+ default:
+ return 0;
+ }
+}
+
+static int
+hwm_gt_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
+ int channel, long *val)
+{
+ struct hwm_drvdata *ddat = dev_get_drvdata(dev);
+
+ switch (type) {
+ case hwmon_energy:
+ return hwm_energy_read(ddat, attr, val);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static const struct hwmon_ops hwm_gt_ops = {
+ .is_visible = hwm_gt_is_visible,
+ .read = hwm_gt_read,
+};
+
+static const struct hwmon_chip_info hwm_gt_chip_info = {
+ .ops = &hwm_gt_ops,
+ .info = hwm_gt_info,
+};
+
+static void
+hwm_get_preregistration_info(struct drm_i915_private *i915)
+{
+ struct i915_hwmon *hwmon = i915->hwmon;
+ struct intel_uncore *uncore = &i915->uncore;
+ struct hwm_drvdata *ddat = &hwmon->ddat;
+ intel_wakeref_t wakeref;
+ u32 val_sku_unit = 0;
+ struct intel_gt *gt;
+ long energy;
+ int i;
+
+ /* Available for all Gen12+/dGfx */
+ hwmon->rg.gt_perf_status = GEN12_RPSTAT1;
+
+ if (IS_DG1(i915) || IS_DG2(i915)) {
+ hwmon->rg.pkg_power_sku_unit = PCU_PACKAGE_POWER_SKU_UNIT;
+ hwmon->rg.pkg_power_sku = PCU_PACKAGE_POWER_SKU;
+ hwmon->rg.pkg_rapl_limit = PCU_PACKAGE_RAPL_LIMIT;
+ hwmon->rg.energy_status_all = PCU_PACKAGE_ENERGY_STATUS;
+ hwmon->rg.energy_status_tile = INVALID_MMIO_REG;
+ } else if (IS_XEHPSDV(i915)) {
+ hwmon->rg.pkg_power_sku_unit = GT0_PACKAGE_POWER_SKU_UNIT;
+ hwmon->rg.pkg_power_sku = INVALID_MMIO_REG;
+ hwmon->rg.pkg_rapl_limit = GT0_PACKAGE_RAPL_LIMIT;
+ hwmon->rg.energy_status_all = GT0_PLATFORM_ENERGY_STATUS;
+ hwmon->rg.energy_status_tile = GT0_PACKAGE_ENERGY_STATUS;
+ } else {
+ hwmon->rg.pkg_power_sku_unit = INVALID_MMIO_REG;
+ hwmon->rg.pkg_power_sku = INVALID_MMIO_REG;
+ hwmon->rg.pkg_rapl_limit = INVALID_MMIO_REG;
+ hwmon->rg.energy_status_all = INVALID_MMIO_REG;
+ hwmon->rg.energy_status_tile = INVALID_MMIO_REG;
+ }
+
+ with_intel_runtime_pm(uncore->rpm, wakeref) {
+ /*
+ * The contents of register hwmon->rg.pkg_power_sku_unit do not change,
+ * so read it once and store the shift values.
+ */
+ if (i915_mmio_reg_valid(hwmon->rg.pkg_power_sku_unit))
+ val_sku_unit = intel_uncore_read(uncore,
+ hwmon->rg.pkg_power_sku_unit);
+ }
+
+ hwmon->scl_shift_power = REG_FIELD_GET(PKG_PWR_UNIT, val_sku_unit);
+ hwmon->scl_shift_energy = REG_FIELD_GET(PKG_ENERGY_UNIT, val_sku_unit);
+ hwmon->scl_shift_time = REG_FIELD_GET(PKG_TIME_UNIT, val_sku_unit);
+
+ /*
+ * Initialize 'struct hwm_energy_info', i.e. set fields to the
+ * first value of the energy register read
+ */
+ if (i915_mmio_reg_valid(hwmon->rg.energy_status_all))
+ hwm_energy(ddat, &energy);
+ if (i915_mmio_reg_valid(hwmon->rg.energy_status_tile)) {
+ for_each_gt(gt, i915, i)
+ hwm_energy(&hwmon->ddat_gt[i], &energy);
+ }
+}
+
+void i915_hwmon_register(struct drm_i915_private *i915)
+{
+ struct device *dev = i915->drm.dev;
+ struct i915_hwmon *hwmon;
+ struct device *hwmon_dev;
+ struct hwm_drvdata *ddat;
+ struct hwm_drvdata *ddat_gt;
+ struct intel_gt *gt;
+ int i;
+
+ /* hwmon is available only for dGfx */
+ if (!IS_DGFX(i915))
+ return;
+
+ hwmon = devm_kzalloc(dev, sizeof(*hwmon), GFP_KERNEL);
+ if (!hwmon)
+ return;
+
+ i915->hwmon = hwmon;
+ mutex_init(&hwmon->hwmon_lock);
+ ddat = &hwmon->ddat;
+
+ ddat->hwmon = hwmon;
+ ddat->uncore = &i915->uncore;
+ snprintf(ddat->name, sizeof(ddat->name), "i915");
+ ddat->gt_n = -1;
+
+ for_each_gt(gt, i915, i) {
+ ddat_gt = hwmon->ddat_gt + i;
+
+ ddat_gt->hwmon = hwmon;
+ ddat_gt->uncore = gt->uncore;
+ snprintf(ddat_gt->name, sizeof(ddat_gt->name), "i915_gt%u", i);
+ ddat_gt->gt_n = i;
+ }
+
+ hwm_get_preregistration_info(i915);
+
+ /* hwmon_dev points to device hwmon<i> */
+ hwmon_dev = devm_hwmon_device_register_with_info(dev, ddat->name,
+ ddat,
+ &hwm_chip_info,
+ hwm_groups);
+ if (IS_ERR(hwmon_dev)) {
+ i915->hwmon = NULL;
+ return;
+ }
+
+ ddat->hwmon_dev = hwmon_dev;
+
+ for_each_gt(gt, i915, i) {
+ ddat_gt = hwmon->ddat_gt + i;
+ /*
+ * Create per-gt directories only if a per-gt attribute is
+ * visible. Currently this is only energy
+ */
+ if (!hwm_gt_is_visible(ddat_gt, hwmon_energy, hwmon_energy_input, 0))
+ continue;
+
+ hwmon_dev = devm_hwmon_device_register_with_info(dev, ddat_gt->name,
+ ddat_gt,
+ &hwm_gt_chip_info,
+ NULL);
+ if (!IS_ERR(hwmon_dev))
+ ddat_gt->hwmon_dev = hwmon_dev;
+ }
+}
+
+void i915_hwmon_unregister(struct drm_i915_private *i915)
+{
+ fetch_and_zero(&i915->hwmon);
+}
diff --git a/drivers/gpu/drm/i915/i915_hwmon.h b/drivers/gpu/drm/i915/i915_hwmon.h
new file mode 100644
index 0000000..7ca9cf2c
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_hwmon.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: MIT */
+
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#ifndef __I915_HWMON_H__
+#define __I915_HWMON_H__
+
+struct drm_i915_private;
+
+#if IS_REACHABLE(CONFIG_HWMON)
+void i915_hwmon_register(struct drm_i915_private *i915);
+void i915_hwmon_unregister(struct drm_i915_private *i915);
+#else
+static inline void i915_hwmon_register(struct drm_i915_private *i915) { };
+static inline void i915_hwmon_unregister(struct drm_i915_private *i915) { };
+#endif
+
+#endif /* __I915_HWMON_H__ */
diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c
index 86a42d9..edfe363 100644
--- a/drivers/gpu/drm/i915/i915_irq.c
+++ b/drivers/gpu/drm/i915/i915_irq.c
@@ -28,7 +28,6 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-#include <linux/circ_buf.h>
#include <linux/slab.h>
#include <linux/sysrq.h>
@@ -248,7 +247,7 @@ void gen3_irq_reset(struct intel_uncore *uncore, i915_reg_t imr,
intel_uncore_posting_read(uncore, iir);
}
-void gen2_irq_reset(struct intel_uncore *uncore)
+static void gen2_irq_reset(struct intel_uncore *uncore)
{
intel_uncore_write16(uncore, GEN2_IMR, 0xffff);
intel_uncore_posting_read16(uncore, GEN2_IMR);
@@ -309,8 +308,8 @@ void gen3_irq_init(struct intel_uncore *uncore,
intel_uncore_posting_read(uncore, imr);
}
-void gen2_irq_init(struct intel_uncore *uncore,
- u32 imr_val, u32 ier_val)
+static void gen2_irq_init(struct intel_uncore *uncore,
+ u32 imr_val, u32 ier_val)
{
gen2_assert_iir_is_zero(uncore);
@@ -325,15 +324,10 @@ i915_hotplug_interrupt_update_locked(struct drm_i915_private *dev_priv,
u32 mask,
u32 bits)
{
- u32 val;
-
lockdep_assert_held(&dev_priv->irq_lock);
drm_WARN_ON(&dev_priv->drm, bits & ~mask);
- val = intel_uncore_read(&dev_priv->uncore, PORT_HOTPLUG_EN);
- val &= ~mask;
- val |= bits;
- intel_uncore_write(&dev_priv->uncore, PORT_HOTPLUG_EN, val);
+ intel_uncore_rmw(&dev_priv->uncore, PORT_HOTPLUG_EN, mask, bits);
}
/**
@@ -1057,8 +1051,8 @@ static void ivb_parity_work(struct work_struct *work)
if (drm_WARN_ON(&dev_priv->drm, !dev_priv->l3_parity.which_slice))
goto out;
- misccpctl = intel_uncore_read(&dev_priv->uncore, GEN7_MISCCPCTL);
- intel_uncore_write(&dev_priv->uncore, GEN7_MISCCPCTL, misccpctl & ~GEN7_DOP_CLOCK_GATE_ENABLE);
+ misccpctl = intel_uncore_rmw(&dev_priv->uncore, GEN7_MISCCPCTL,
+ GEN7_DOP_CLOCK_GATE_ENABLE, 0);
intel_uncore_posting_read(&dev_priv->uncore, GEN7_MISCCPCTL);
while ((slice = ffs(dev_priv->l3_parity.which_slice)) != 0) {
@@ -1091,8 +1085,9 @@ static void ivb_parity_work(struct work_struct *work)
kobject_uevent_env(&dev_priv->drm.primary->kdev->kobj,
KOBJ_CHANGE, parity_event);
- DRM_DEBUG("Parity error: Slice = %d, Row = %d, Bank = %d, Sub bank = %d.\n",
- slice, row, bank, subbank);
+ drm_dbg(&dev_priv->drm,
+ "Parity error: Slice = %d, Row = %d, Bank = %d, Sub bank = %d.\n",
+ slice, row, bank, subbank);
kfree(parity_event[4]);
kfree(parity_event[3]);
@@ -1689,8 +1684,7 @@ static irqreturn_t valleyview_irq_handler(int irq, void *arg)
* bits this time around.
*/
intel_uncore_write(&dev_priv->uncore, VLV_MASTER_IER, 0);
- ier = intel_uncore_read(&dev_priv->uncore, VLV_IER);
- intel_uncore_write(&dev_priv->uncore, VLV_IER, 0);
+ ier = intel_uncore_rmw(&dev_priv->uncore, VLV_IER, ~0, 0);
if (gt_iir)
intel_uncore_write(&dev_priv->uncore, GTIIR, gt_iir);
@@ -1775,8 +1769,7 @@ static irqreturn_t cherryview_irq_handler(int irq, void *arg)
* bits this time around.
*/
intel_uncore_write(&dev_priv->uncore, GEN8_MASTER_IRQ, 0);
- ier = intel_uncore_read(&dev_priv->uncore, VLV_IER);
- intel_uncore_write(&dev_priv->uncore, VLV_IER, 0);
+ ier = intel_uncore_rmw(&dev_priv->uncore, VLV_IER, ~0, 0);
gen8_gt_irq_handler(to_gt(dev_priv), master_ctl);
@@ -1981,8 +1974,7 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir)
if (ddi_hotplug_trigger) {
u32 dig_hotplug_reg;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, SHOTPLUG_CTL_DDI);
- intel_uncore_write(&dev_priv->uncore, SHOTPLUG_CTL_DDI, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, SHOTPLUG_CTL_DDI, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
ddi_hotplug_trigger, dig_hotplug_reg,
@@ -1993,8 +1985,7 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir)
if (tc_hotplug_trigger) {
u32 dig_hotplug_reg;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, SHOTPLUG_CTL_TC);
- intel_uncore_write(&dev_priv->uncore, SHOTPLUG_CTL_TC, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, SHOTPLUG_CTL_TC, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
tc_hotplug_trigger, dig_hotplug_reg,
@@ -2019,8 +2010,7 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir)
if (hotplug_trigger) {
u32 dig_hotplug_reg;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, PCH_PORT_HOTPLUG);
- intel_uncore_write(&dev_priv->uncore, PCH_PORT_HOTPLUG, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, PCH_PORT_HOTPLUG, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
hotplug_trigger, dig_hotplug_reg,
@@ -2031,8 +2021,7 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir)
if (hotplug2_trigger) {
u32 dig_hotplug_reg;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, PCH_PORT_HOTPLUG2);
- intel_uncore_write(&dev_priv->uncore, PCH_PORT_HOTPLUG2, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, PCH_PORT_HOTPLUG2, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
hotplug2_trigger, dig_hotplug_reg,
@@ -2052,8 +2041,7 @@ static void ilk_hpd_irq_handler(struct drm_i915_private *dev_priv,
{
u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, DIGITAL_PORT_HOTPLUG_CNTRL);
- intel_uncore_write(&dev_priv->uncore, DIGITAL_PORT_HOTPLUG_CNTRL, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, DIGITAL_PORT_HOTPLUG_CNTRL, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
hotplug_trigger, dig_hotplug_reg,
@@ -2232,8 +2220,7 @@ static void bxt_hpd_irq_handler(struct drm_i915_private *dev_priv,
{
u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, PCH_PORT_HOTPLUG);
- intel_uncore_write(&dev_priv->uncore, PCH_PORT_HOTPLUG, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, PCH_PORT_HOTPLUG, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
hotplug_trigger, dig_hotplug_reg,
@@ -2252,8 +2239,7 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir)
if (trigger_tc) {
u32 dig_hotplug_reg;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, GEN11_TC_HOTPLUG_CTL);
- intel_uncore_write(&dev_priv->uncore, GEN11_TC_HOTPLUG_CTL, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, GEN11_TC_HOTPLUG_CTL, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
trigger_tc, dig_hotplug_reg,
@@ -2264,8 +2250,7 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir)
if (trigger_tbt) {
u32 dig_hotplug_reg;
- dig_hotplug_reg = intel_uncore_read(&dev_priv->uncore, GEN11_TBT_HOTPLUG_CTL);
- intel_uncore_write(&dev_priv->uncore, GEN11_TBT_HOTPLUG_CTL, dig_hotplug_reg);
+ dig_hotplug_reg = intel_uncore_rmw(&dev_priv->uncore, GEN11_TBT_HOTPLUG_CTL, 0, 0);
intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
trigger_tbt, dig_hotplug_reg,
@@ -2355,8 +2340,7 @@ gen8_de_misc_irq_handler(struct drm_i915_private *dev_priv, u32 iir)
else
iir_reg = EDP_PSR_IIR;
- psr_iir = intel_uncore_read(&dev_priv->uncore, iir_reg);
- intel_uncore_write(&dev_priv->uncore, iir_reg, psr_iir);
+ psr_iir = intel_uncore_rmw(&dev_priv->uncore, iir_reg, 0, 0);
if (psr_iir)
found = true;
@@ -2426,8 +2410,7 @@ static void gen11_dsi_te_interrupt_handler(struct drm_i915_private *dev_priv,
/* clear TE in dsi IIR */
port = (te_trigger & DSI1_TE) ? PORT_B : PORT_A;
- tmp = intel_uncore_read(&dev_priv->uncore, DSI_INTR_IDENT_REG(port));
- intel_uncore_write(&dev_priv->uncore, DSI_INTR_IDENT_REG(port), tmp);
+ tmp = intel_uncore_rmw(&dev_priv->uncore, DSI_INTR_IDENT_REG(port), 0, 0);
}
static u32 gen8_de_pipe_flip_done_mask(struct drm_i915_private *i915)
@@ -2791,7 +2774,8 @@ static irqreturn_t dg1_irq_handler(int irq, void *arg)
master_ctl = raw_reg_read(regs, GEN11_GFX_MSTR_IRQ);
raw_reg_write(regs, GEN11_GFX_MSTR_IRQ, master_ctl);
} else {
- DRM_ERROR("Tile not supported: 0x%08x\n", master_tile_ctl);
+ drm_err(&i915->drm, "Tile not supported: 0x%08x\n",
+ master_tile_ctl);
dg1_master_intr_enable(regs);
return IRQ_NONE;
}
@@ -2884,7 +2868,6 @@ static bool gen11_dsi_configure_te(struct intel_crtc *intel_crtc,
{
struct drm_i915_private *dev_priv = to_i915(intel_crtc->base.dev);
enum port port;
- u32 tmp;
if (!(intel_crtc->mode_flags &
(I915_MODE_FLAG_DSI_USE_TE1 | I915_MODE_FLAG_DSI_USE_TE0)))
@@ -2896,16 +2879,10 @@ static bool gen11_dsi_configure_te(struct intel_crtc *intel_crtc,
else
port = PORT_A;
- tmp = intel_uncore_read(&dev_priv->uncore, DSI_INTR_MASK_REG(port));
- if (enable)
- tmp &= ~DSI_TE_EVENT;
- else
- tmp |= DSI_TE_EVENT;
+ intel_uncore_rmw(&dev_priv->uncore, DSI_INTR_MASK_REG(port), DSI_TE_EVENT,
+ enable ? 0 : DSI_TE_EVENT);
- intel_uncore_write(&dev_priv->uncore, DSI_INTR_MASK_REG(port), tmp);
-
- tmp = intel_uncore_read(&dev_priv->uncore, DSI_INTR_IDENT_REG(port));
- intel_uncore_write(&dev_priv->uncore, DSI_INTR_IDENT_REG(port), tmp);
+ intel_uncore_rmw(&dev_priv->uncore, DSI_INTR_IDENT_REG(port), 0, 0);
return true;
}
@@ -3020,7 +2997,7 @@ static void vlv_display_irq_reset(struct drm_i915_private *dev_priv)
intel_uncore_write(uncore, DPINVGTT, DPINVGTT_STATUS_MASK_VLV);
i915_hotplug_interrupt_update_locked(dev_priv, 0xffffffff, 0);
- intel_uncore_write(uncore, PORT_HOTPLUG_STAT, intel_uncore_read(&dev_priv->uncore, PORT_HOTPLUG_STAT));
+ intel_uncore_rmw(uncore, PORT_HOTPLUG_STAT, 0, 0);
i9xx_pipestat_irq_reset(dev_priv);
@@ -3118,7 +3095,7 @@ static void gen8_irq_reset(struct drm_i915_private *dev_priv)
{
struct intel_uncore *uncore = &dev_priv->uncore;
- gen8_master_intr_disable(dev_priv->uncore.regs);
+ gen8_master_intr_disable(uncore->regs);
gen8_gt_irq_reset(to_gt(dev_priv));
gen8_display_irq_reset(dev_priv);
@@ -3250,7 +3227,7 @@ static void cherryview_irq_reset(struct drm_i915_private *dev_priv)
{
struct intel_uncore *uncore = &dev_priv->uncore;
- intel_uncore_write(&dev_priv->uncore, GEN8_MASTER_IRQ, 0);
+ intel_uncore_write(uncore, GEN8_MASTER_IRQ, 0);
intel_uncore_posting_read(&dev_priv->uncore, GEN8_MASTER_IRQ);
gen8_gt_irq_reset(to_gt(dev_priv));
@@ -3290,23 +3267,20 @@ static u32 ibx_hotplug_enables(struct drm_i915_private *i915,
static void ibx_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug;
-
/*
* Enable digital hotplug on the PCH, and configure the DP short pulse
* duration to 2ms (which is the minimum in the Display Port spec).
* The pulse duration bits are reserved on LPT+.
*/
- hotplug = intel_uncore_read(&dev_priv->uncore, PCH_PORT_HOTPLUG);
- hotplug &= ~(PORTA_HOTPLUG_ENABLE |
- PORTB_HOTPLUG_ENABLE |
- PORTC_HOTPLUG_ENABLE |
- PORTD_HOTPLUG_ENABLE |
- PORTB_PULSE_DURATION_MASK |
- PORTC_PULSE_DURATION_MASK |
- PORTD_PULSE_DURATION_MASK);
- hotplug |= intel_hpd_hotplug_enables(dev_priv, ibx_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, PCH_PORT_HOTPLUG, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, PCH_PORT_HOTPLUG,
+ PORTA_HOTPLUG_ENABLE |
+ PORTB_HOTPLUG_ENABLE |
+ PORTC_HOTPLUG_ENABLE |
+ PORTD_HOTPLUG_ENABLE |
+ PORTB_PULSE_DURATION_MASK |
+ PORTC_PULSE_DURATION_MASK |
+ PORTD_PULSE_DURATION_MASK,
+ intel_hpd_hotplug_enables(dev_priv, ibx_hotplug_enables));
}
static void ibx_hpd_irq_setup(struct drm_i915_private *dev_priv)
@@ -3353,30 +3327,24 @@ static u32 icp_tc_hotplug_enables(struct drm_i915_private *i915,
static void icp_ddi_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug;
-
- hotplug = intel_uncore_read(&dev_priv->uncore, SHOTPLUG_CTL_DDI);
- hotplug &= ~(SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_A) |
- SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_B) |
- SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_C) |
- SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_D));
- hotplug |= intel_hpd_hotplug_enables(dev_priv, icp_ddi_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, SHOTPLUG_CTL_DDI, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, SHOTPLUG_CTL_DDI,
+ SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_A) |
+ SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_B) |
+ SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_C) |
+ SHOTPLUG_CTL_DDI_HPD_ENABLE(HPD_PORT_D),
+ intel_hpd_hotplug_enables(dev_priv, icp_ddi_hotplug_enables));
}
static void icp_tc_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug;
-
- hotplug = intel_uncore_read(&dev_priv->uncore, SHOTPLUG_CTL_TC);
- hotplug &= ~(ICP_TC_HPD_ENABLE(HPD_PORT_TC1) |
- ICP_TC_HPD_ENABLE(HPD_PORT_TC2) |
- ICP_TC_HPD_ENABLE(HPD_PORT_TC3) |
- ICP_TC_HPD_ENABLE(HPD_PORT_TC4) |
- ICP_TC_HPD_ENABLE(HPD_PORT_TC5) |
- ICP_TC_HPD_ENABLE(HPD_PORT_TC6));
- hotplug |= intel_hpd_hotplug_enables(dev_priv, icp_tc_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, SHOTPLUG_CTL_TC, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, SHOTPLUG_CTL_TC,
+ ICP_TC_HPD_ENABLE(HPD_PORT_TC1) |
+ ICP_TC_HPD_ENABLE(HPD_PORT_TC2) |
+ ICP_TC_HPD_ENABLE(HPD_PORT_TC3) |
+ ICP_TC_HPD_ENABLE(HPD_PORT_TC4) |
+ ICP_TC_HPD_ENABLE(HPD_PORT_TC5) |
+ ICP_TC_HPD_ENABLE(HPD_PORT_TC6),
+ intel_hpd_hotplug_enables(dev_priv, icp_tc_hotplug_enables));
}
static void icp_hpd_irq_setup(struct drm_i915_private *dev_priv)
@@ -3411,62 +3379,54 @@ static u32 gen11_hotplug_enables(struct drm_i915_private *i915,
}
}
+static void dg1_hpd_invert(struct drm_i915_private *i915)
+{
+ u32 val = (INVERT_DDIA_HPD |
+ INVERT_DDIB_HPD |
+ INVERT_DDIC_HPD |
+ INVERT_DDID_HPD);
+ intel_uncore_rmw(&i915->uncore, SOUTH_CHICKEN1, 0, val);
+}
+
static void dg1_hpd_irq_setup(struct drm_i915_private *dev_priv)
{
- u32 val;
-
- val = intel_uncore_read(&dev_priv->uncore, SOUTH_CHICKEN1);
- val |= (INVERT_DDIA_HPD |
- INVERT_DDIB_HPD |
- INVERT_DDIC_HPD |
- INVERT_DDID_HPD);
- intel_uncore_write(&dev_priv->uncore, SOUTH_CHICKEN1, val);
-
+ dg1_hpd_invert(dev_priv);
icp_hpd_irq_setup(dev_priv);
}
static void gen11_tc_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug;
-
- hotplug = intel_uncore_read(&dev_priv->uncore, GEN11_TC_HOTPLUG_CTL);
- hotplug &= ~(GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC1) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC2) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC3) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC4) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC5) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC6));
- hotplug |= intel_hpd_hotplug_enables(dev_priv, gen11_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, GEN11_TC_HOTPLUG_CTL, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, GEN11_TC_HOTPLUG_CTL,
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC1) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC2) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC3) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC4) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC5) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC6),
+ intel_hpd_hotplug_enables(dev_priv, gen11_hotplug_enables));
}
static void gen11_tbt_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug;
-
- hotplug = intel_uncore_read(&dev_priv->uncore, GEN11_TBT_HOTPLUG_CTL);
- hotplug &= ~(GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC1) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC2) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC3) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC4) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC5) |
- GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC6));
- hotplug |= intel_hpd_hotplug_enables(dev_priv, gen11_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, GEN11_TBT_HOTPLUG_CTL, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, GEN11_TBT_HOTPLUG_CTL,
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC1) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC2) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC3) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC4) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC5) |
+ GEN11_HOTPLUG_CTL_ENABLE(HPD_PORT_TC6),
+ intel_hpd_hotplug_enables(dev_priv, gen11_hotplug_enables));
}
static void gen11_hpd_irq_setup(struct drm_i915_private *dev_priv)
{
u32 hotplug_irqs, enabled_irqs;
- u32 val;
enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->display.hotplug.hpd);
hotplug_irqs = intel_hpd_hotplug_irqs(dev_priv, dev_priv->display.hotplug.hpd);
- val = intel_uncore_read(&dev_priv->uncore, GEN11_DE_HPD_IMR);
- val &= ~hotplug_irqs;
- val |= ~enabled_irqs & hotplug_irqs;
- intel_uncore_write(&dev_priv->uncore, GEN11_DE_HPD_IMR, val);
+ intel_uncore_rmw(&dev_priv->uncore, GEN11_DE_HPD_IMR, hotplug_irqs,
+ ~enabled_irqs & hotplug_irqs);
intel_uncore_posting_read(&dev_priv->uncore, GEN11_DE_HPD_IMR);
gen11_tc_hpd_detection_setup(dev_priv);
@@ -3506,29 +3466,22 @@ static u32 spt_hotplug2_enables(struct drm_i915_private *i915,
static void spt_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 val, hotplug;
-
/* Display WA #1179 WaHardHangonHotPlug: cnp */
if (HAS_PCH_CNP(dev_priv)) {
- val = intel_uncore_read(&dev_priv->uncore, SOUTH_CHICKEN1);
- val &= ~CHASSIS_CLK_REQ_DURATION_MASK;
- val |= CHASSIS_CLK_REQ_DURATION(0xf);
- intel_uncore_write(&dev_priv->uncore, SOUTH_CHICKEN1, val);
+ intel_uncore_rmw(&dev_priv->uncore, SOUTH_CHICKEN1, CHASSIS_CLK_REQ_DURATION_MASK,
+ CHASSIS_CLK_REQ_DURATION(0xf));
}
/* Enable digital hotplug on the PCH */
- hotplug = intel_uncore_read(&dev_priv->uncore, PCH_PORT_HOTPLUG);
- hotplug &= ~(PORTA_HOTPLUG_ENABLE |
- PORTB_HOTPLUG_ENABLE |
- PORTC_HOTPLUG_ENABLE |
- PORTD_HOTPLUG_ENABLE);
- hotplug |= intel_hpd_hotplug_enables(dev_priv, spt_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, PCH_PORT_HOTPLUG, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, PCH_PORT_HOTPLUG,
+ PORTA_HOTPLUG_ENABLE |
+ PORTB_HOTPLUG_ENABLE |
+ PORTC_HOTPLUG_ENABLE |
+ PORTD_HOTPLUG_ENABLE,
+ intel_hpd_hotplug_enables(dev_priv, spt_hotplug_enables));
- hotplug = intel_uncore_read(&dev_priv->uncore, PCH_PORT_HOTPLUG2);
- hotplug &= ~PORTE_HOTPLUG_ENABLE;
- hotplug |= intel_hpd_hotplug_enables(dev_priv, spt_hotplug2_enables);
- intel_uncore_write(&dev_priv->uncore, PCH_PORT_HOTPLUG2, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, PCH_PORT_HOTPLUG2, PORTE_HOTPLUG_ENABLE,
+ intel_hpd_hotplug_enables(dev_priv, spt_hotplug2_enables));
}
static void spt_hpd_irq_setup(struct drm_i915_private *dev_priv)
@@ -3560,18 +3513,14 @@ static u32 ilk_hotplug_enables(struct drm_i915_private *i915,
static void ilk_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug;
-
/*
* Enable digital hotplug on the CPU, and configure the DP short pulse
* duration to 2ms (which is the minimum in the Display Port spec)
* The pulse duration bits are reserved on HSW+.
*/
- hotplug = intel_uncore_read(&dev_priv->uncore, DIGITAL_PORT_HOTPLUG_CNTRL);
- hotplug &= ~(DIGITAL_PORTA_HOTPLUG_ENABLE |
- DIGITAL_PORTA_PULSE_DURATION_MASK);
- hotplug |= intel_hpd_hotplug_enables(dev_priv, ilk_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, DIGITAL_PORT_HOTPLUG_CNTRL, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, DIGITAL_PORT_HOTPLUG_CNTRL,
+ DIGITAL_PORTA_HOTPLUG_ENABLE | DIGITAL_PORTA_PULSE_DURATION_MASK,
+ intel_hpd_hotplug_enables(dev_priv, ilk_hotplug_enables));
}
static void ilk_hpd_irq_setup(struct drm_i915_private *dev_priv)
@@ -3619,17 +3568,12 @@ static u32 bxt_hotplug_enables(struct drm_i915_private *i915,
static void bxt_hpd_detection_setup(struct drm_i915_private *dev_priv)
{
- u32 hotplug;
-
- hotplug = intel_uncore_read(&dev_priv->uncore, PCH_PORT_HOTPLUG);
- hotplug &= ~(PORTA_HOTPLUG_ENABLE |
- PORTB_HOTPLUG_ENABLE |
- PORTC_HOTPLUG_ENABLE |
- BXT_DDIA_HPD_INVERT |
- BXT_DDIB_HPD_INVERT |
- BXT_DDIC_HPD_INVERT);
- hotplug |= intel_hpd_hotplug_enables(dev_priv, bxt_hotplug_enables);
- intel_uncore_write(&dev_priv->uncore, PCH_PORT_HOTPLUG, hotplug);
+ intel_uncore_rmw(&dev_priv->uncore, PCH_PORT_HOTPLUG,
+ PORTA_HOTPLUG_ENABLE |
+ PORTB_HOTPLUG_ENABLE |
+ PORTC_HOTPLUG_ENABLE |
+ BXT_DDI_HPD_INVERT_MASK,
+ intel_hpd_hotplug_enables(dev_priv, bxt_hotplug_enables));
}
static void bxt_hpd_irq_setup(struct drm_i915_private *dev_priv)
@@ -3928,7 +3872,7 @@ static void i8xx_irq_reset(struct drm_i915_private *dev_priv)
i9xx_pipestat_irq_reset(dev_priv);
- GEN2_IRQ_RESET(uncore);
+ gen2_irq_reset(uncore);
dev_priv->irq_mask = ~0u;
}
@@ -3954,7 +3898,7 @@ static void i8xx_irq_postinstall(struct drm_i915_private *dev_priv)
I915_MASTER_ERROR_INTERRUPT |
I915_USER_INTERRUPT;
- GEN2_IRQ_INIT(uncore, dev_priv->irq_mask, enable_mask);
+ gen2_irq_init(uncore, dev_priv->irq_mask, enable_mask);
/* Interrupt setup is already guaranteed to be single-threaded, this is
* just to make the assert_spin_locked check happy. */
@@ -3997,7 +3941,7 @@ static void i8xx_error_irq_ack(struct drm_i915_private *i915,
static void i8xx_error_irq_handler(struct drm_i915_private *dev_priv,
u16 eir, u16 eir_stuck)
{
- DRM_DEBUG("Master Error: EIR 0x%04x\n", eir);
+ drm_dbg(&dev_priv->drm, "Master Error: EIR 0x%04x\n", eir);
if (eir_stuck)
drm_dbg(&dev_priv->drm, "EIR stuck: 0x%04x, masked\n",
@@ -4009,9 +3953,7 @@ static void i9xx_error_irq_ack(struct drm_i915_private *dev_priv,
{
u32 emr;
- *eir = intel_uncore_read(&dev_priv->uncore, EIR);
-
- intel_uncore_write(&dev_priv->uncore, EIR, *eir);
+ *eir = intel_uncore_rmw(&dev_priv->uncore, EIR, 0, 0);
*eir_stuck = intel_uncore_read(&dev_priv->uncore, EIR);
if (*eir_stuck == 0)
@@ -4027,15 +3969,14 @@ static void i9xx_error_irq_ack(struct drm_i915_private *dev_priv,
* (or by a GPU reset) so we mask any bit that
* remains set.
*/
- emr = intel_uncore_read(&dev_priv->uncore, EMR);
- intel_uncore_write(&dev_priv->uncore, EMR, 0xffffffff);
+ emr = intel_uncore_rmw(&dev_priv->uncore, EMR, ~0, 0xffffffff);
intel_uncore_write(&dev_priv->uncore, EMR, emr | *eir_stuck);
}
static void i9xx_error_irq_handler(struct drm_i915_private *dev_priv,
u32 eir, u32 eir_stuck)
{
- DRM_DEBUG("Master Error, EIR 0x%08x\n", eir);
+ drm_dbg(&dev_priv->drm, "Master Error, EIR 0x%08x\n", eir);
if (eir_stuck)
drm_dbg(&dev_priv->drm, "EIR stuck: 0x%08x, masked\n",
@@ -4095,7 +4036,7 @@ static void i915_irq_reset(struct drm_i915_private *dev_priv)
if (I915_HAS_HOTPLUG(dev_priv)) {
i915_hotplug_interrupt_update(dev_priv, 0xffffffff, 0);
- intel_uncore_write(&dev_priv->uncore, PORT_HOTPLUG_STAT, intel_uncore_read(&dev_priv->uncore, PORT_HOTPLUG_STAT));
+ intel_uncore_rmw(&dev_priv->uncore, PORT_HOTPLUG_STAT, 0, 0);
}
i9xx_pipestat_irq_reset(dev_priv);
@@ -4109,8 +4050,8 @@ static void i915_irq_postinstall(struct drm_i915_private *dev_priv)
struct intel_uncore *uncore = &dev_priv->uncore;
u32 enable_mask;
- intel_uncore_write(&dev_priv->uncore, EMR, ~(I915_ERROR_PAGE_TABLE |
- I915_ERROR_MEMORY_REFRESH));
+ intel_uncore_write(uncore, EMR, ~(I915_ERROR_PAGE_TABLE |
+ I915_ERROR_MEMORY_REFRESH));
/* Unmask the interrupts that we always want on. */
dev_priv->irq_mask =
@@ -4205,7 +4146,7 @@ static void i965_irq_reset(struct drm_i915_private *dev_priv)
struct intel_uncore *uncore = &dev_priv->uncore;
i915_hotplug_interrupt_update(dev_priv, 0xffffffff, 0);
- intel_uncore_write(&dev_priv->uncore, PORT_HOTPLUG_STAT, intel_uncore_read(&dev_priv->uncore, PORT_HOTPLUG_STAT));
+ intel_uncore_rmw(uncore, PORT_HOTPLUG_STAT, 0, 0);
i9xx_pipestat_irq_reset(dev_priv);
@@ -4232,7 +4173,7 @@ static void i965_irq_postinstall(struct drm_i915_private *dev_priv)
error_mask = ~(I915_ERROR_PAGE_TABLE |
I915_ERROR_MEMORY_REFRESH);
}
- intel_uncore_write(&dev_priv->uncore, EMR, error_mask);
+ intel_uncore_write(uncore, EMR, error_mask);
/* Unmask the interrupts that we always want on. */
dev_priv->irq_mask =
@@ -4383,7 +4324,6 @@ void intel_hpd_irq_setup(struct drm_i915_private *i915)
*/
void intel_irq_init(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = &dev_priv->drm;
int i;
INIT_WORK(&dev_priv->l3_parity.error_work, ivb_parity_work);
@@ -4399,9 +4339,9 @@ void intel_irq_init(struct drm_i915_private *dev_priv)
intel_hpd_init_pins(dev_priv);
- intel_hpd_init_work(dev_priv);
+ intel_hpd_init_early(dev_priv);
- dev->vblank_disable_immediate = true;
+ dev_priv->drm.vblank_disable_immediate = true;
/* Most platforms treat the display irq block as an always-on
* power domain. vlv/chv can disable it at runtime and need
@@ -4413,15 +4353,6 @@ void intel_irq_init(struct drm_i915_private *dev_priv)
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
dev_priv->display_irqs_enabled = false;
- dev_priv->display.hotplug.hpd_storm_threshold = HPD_STORM_DEFAULT_THRESHOLD;
- /* If we have MST support, we want to avoid doing short HPD IRQ storm
- * detection, as short HPD storms will occur as a natural part of
- * sideband messaging with MST.
- * On older platforms however, IRQ storms can occur with both long and
- * short pulses, as seen on some G4x systems.
- */
- dev_priv->display.hotplug.hpd_short_storm_enabled = !HAS_DP_MST(dev_priv);
-
if (HAS_GMCH(dev_priv)) {
if (I915_HAS_HOTPLUG(dev_priv))
dev_priv->display.funcs.hotplug = &i915_hpd_funcs;
diff --git a/drivers/gpu/drm/i915/i915_irq.h b/drivers/gpu/drm/i915/i915_irq.h
index 82639d9..9b004fc 100644
--- a/drivers/gpu/drm/i915/i915_irq.h
+++ b/drivers/gpu/drm/i915/i915_irq.h
@@ -90,12 +90,9 @@ void i965_disable_vblank(struct drm_crtc *crtc);
void ilk_disable_vblank(struct drm_crtc *crtc);
void bdw_disable_vblank(struct drm_crtc *crtc);
-void gen2_irq_reset(struct intel_uncore *uncore);
void gen3_irq_reset(struct intel_uncore *uncore, i915_reg_t imr,
i915_reg_t iir, i915_reg_t ier);
-void gen2_irq_init(struct intel_uncore *uncore,
- u32 imr_val, u32 ier_val);
void gen3_irq_init(struct intel_uncore *uncore,
i915_reg_t imr, u32 imr_val,
i915_reg_t ier, u32 ier_val,
@@ -111,9 +108,6 @@ void gen3_irq_init(struct intel_uncore *uncore,
#define GEN3_IRQ_RESET(uncore, type) \
gen3_irq_reset((uncore), type##IMR, type##IIR, type##IER)
-#define GEN2_IRQ_RESET(uncore) \
- gen2_irq_reset(uncore)
-
#define GEN8_IRQ_INIT_NDX(uncore, type, which, imr_val, ier_val) \
({ \
unsigned int which_ = which; \
@@ -129,7 +123,4 @@ void gen3_irq_init(struct intel_uncore *uncore,
type##IER, ier_val, \
type##IIR)
-#define GEN2_IRQ_INIT(uncore, imr_val, ier_val) \
- gen2_irq_init((uncore), imr_val, ier_val)
-
#endif /* __I915_IRQ_H__ */
diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c
index cd4487a..6da9784 100644
--- a/drivers/gpu/drm/i915/i915_pci.c
+++ b/drivers/gpu/drm/i915/i915_pci.c
@@ -1023,6 +1023,8 @@ static const struct intel_device_info adl_p_info = {
.has_logical_ring_contexts = 1, \
.has_logical_ring_elsq = 1, \
.has_mslice_steering = 1, \
+ .has_oa_bpc_reporting = 1, \
+ .has_oa_slice_contrib_limits = 1, \
.has_rc6 = 1, \
.has_reset_engine = 1, \
.has_rps = 1, \
@@ -1042,7 +1044,6 @@ static const struct intel_device_info xehpsdv_info = {
PLATFORM(INTEL_XEHPSDV),
NO_DISPLAY,
.has_64k_pages = 1,
- .needs_compact_pt = 1,
.has_media_ratio_mode = 1,
.__runtime.platform_engine_mask =
BIT(RCS0) | BIT(BCS0) |
@@ -1064,8 +1065,8 @@ static const struct intel_device_info xehpsdv_info = {
.has_64k_pages = 1, \
.has_guc_deprivilege = 1, \
.has_heci_pxp = 1, \
- .needs_compact_pt = 1, \
.has_media_ratio_mode = 1, \
+ .display.has_cdclk_squash = 1, \
.__runtime.platform_engine_mask = \
BIT(RCS0) | BIT(BCS0) | \
BIT(VECS0) | BIT(VECS1) | \
@@ -1077,7 +1078,6 @@ static const struct intel_device_info dg2_info = {
XE_LPD_FEATURES,
.__runtime.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) |
BIT(TRANSCODER_C) | BIT(TRANSCODER_D),
- .require_force_probe = 1,
};
static const struct intel_device_info ats_m_info = {
@@ -1144,6 +1144,9 @@ static const struct intel_device_info mtl_info = {
.display.has_modular_fia = 1,
.extra_gt_list = xelpmp_extra_gt,
.has_flat_ccs = 0,
+ .has_gmd_id = 1,
+ .has_guc_deprivilege = 1,
+ .has_mslice_steering = 0,
.has_snoop = 1,
.__runtime.memory_regions = REGION_SMEM | REGION_STOLEN_LMEM,
.__runtime.platform_engine_mask = BIT(RCS0) | BIT(BCS0) | BIT(CCS0),
@@ -1296,9 +1299,7 @@ bool i915_pci_resource_valid(struct pci_dev *pdev, int bar)
static bool intel_mmio_bar_valid(struct pci_dev *pdev, struct intel_device_info *intel_info)
{
- int gttmmaddr_bar = intel_info->__runtime.graphics.ip.ver == 2 ? GEN2_GTTMMADR_BAR : GTTMMADR_BAR;
-
- return i915_pci_resource_valid(pdev, gttmmaddr_bar);
+ return i915_pci_resource_valid(pdev, intel_mmio_bar(intel_info->__runtime.graphics.ip.ver));
}
static int i915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
diff --git a/drivers/gpu/drm/i915/i915_perf.c b/drivers/gpu/drm/i915/i915_perf.c
index 0defbb4..00e09bb 100644
--- a/drivers/gpu/drm/i915/i915_perf.c
+++ b/drivers/gpu/drm/i915/i915_perf.c
@@ -204,15 +204,18 @@
#include "gt/intel_gpu_commands.h"
#include "gt/intel_gt.h"
#include "gt/intel_gt_clock_utils.h"
+#include "gt/intel_gt_mcr.h"
#include "gt/intel_gt_regs.h"
#include "gt/intel_lrc.h"
#include "gt/intel_lrc_reg.h"
#include "gt/intel_ring.h"
+#include "gt/uc/intel_guc_slpc.h"
#include "i915_drv.h"
#include "i915_file_private.h"
#include "i915_perf.h"
#include "i915_perf_oa_regs.h"
+#include "i915_reg.h"
/* HW requires this to be a power of two, between 128k and 16M, though driver
* is currently generally designed assuming the largest 16M size is used such
@@ -286,6 +289,7 @@ static u32 i915_perf_stream_paranoid = true;
#define OAREPORT_REASON_CTX_SWITCH (1<<3)
#define OAREPORT_REASON_CLK_RATIO (1<<5)
+#define HAS_MI_SET_PREDICATE(i915) (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50))
/* For sysctl proc_dointvec_minmax of i915_oa_max_sample_rate
*
@@ -320,6 +324,8 @@ static const struct i915_oa_format oa_formats[I915_OA_FORMAT_MAX] = {
[I915_OA_FORMAT_A12] = { 0, 64 },
[I915_OA_FORMAT_A12_B8_C8] = { 2, 128 },
[I915_OA_FORMAT_A32u40_A4u32_B8_C8] = { 5, 256 },
+ [I915_OAR_FORMAT_A32u40_A4u32_B8_C8] = { 5, 256 },
+ [I915_OA_FORMAT_A24u40_A14u32_B8_C8] = { 5, 256 },
};
#define SAMPLE_OA_REPORT (1<<0)
@@ -462,7 +468,7 @@ static u32 gen7_oa_hw_tail_read(struct i915_perf_stream *stream)
static bool oa_buffer_check_unlocked(struct i915_perf_stream *stream)
{
u32 gtt_offset = i915_ggtt_offset(stream->oa_buffer.vma);
- int report_size = stream->oa_buffer.format_size;
+ int report_size = stream->oa_buffer.format->size;
unsigned long flags;
bool pollin;
u32 hw_tail;
@@ -525,9 +531,9 @@ static bool oa_buffer_check_unlocked(struct i915_perf_stream *stream)
if (OA_TAKEN(hw_tail, tail) > report_size &&
__ratelimit(&stream->perf->tail_pointer_race))
- DRM_NOTE("unlanded report(s) head=0x%x "
- "tail=0x%x hw_tail=0x%x\n",
- head, tail, hw_tail);
+ drm_notice(&stream->uncore->i915->drm,
+ "unlanded report(s) head=0x%x tail=0x%x hw_tail=0x%x\n",
+ head, tail, hw_tail);
stream->oa_buffer.tail = gtt_offset + tail;
stream->oa_buffer.aging_tail = gtt_offset + hw_tail;
@@ -599,7 +605,7 @@ static int append_oa_sample(struct i915_perf_stream *stream,
size_t *offset,
const u8 *report)
{
- int report_size = stream->oa_buffer.format_size;
+ int report_size = stream->oa_buffer.format->size;
struct drm_i915_perf_record_header header;
header.type = DRM_I915_PERF_RECORD_SAMPLE;
@@ -649,14 +655,13 @@ static int gen8_append_oa_reports(struct i915_perf_stream *stream,
size_t *offset)
{
struct intel_uncore *uncore = stream->uncore;
- int report_size = stream->oa_buffer.format_size;
+ int report_size = stream->oa_buffer.format->size;
u8 *oa_buf_base = stream->oa_buffer.vaddr;
u32 gtt_offset = i915_ggtt_offset(stream->oa_buffer.vma);
u32 mask = (OA_BUFFER_SIZE - 1);
size_t start_offset = *offset;
unsigned long flags;
u32 head, tail;
- u32 taken;
int ret = 0;
if (drm_WARN_ON(&uncore->i915->drm, !stream->enabled))
@@ -692,7 +697,7 @@ static int gen8_append_oa_reports(struct i915_perf_stream *stream,
for (/* none */;
- (taken = OA_TAKEN(tail, head));
+ OA_TAKEN(tail, head);
head = (head + report_size) & mask) {
u8 *report = oa_buf_base + head;
u32 *report32 = (void *)report;
@@ -774,7 +779,7 @@ static int gen8_append_oa_reports(struct i915_perf_stream *stream,
* switches since it's not-uncommon for periodic samples to
* identify a switch before any 'context switch' report.
*/
- if (!stream->perf->exclusive_stream->ctx ||
+ if (!stream->ctx ||
stream->specific_ctx_id == ctx_id ||
stream->oa_buffer.last_ctx_id == stream->specific_ctx_id ||
reason & OAREPORT_REASON_CTX_SWITCH) {
@@ -783,7 +788,7 @@ static int gen8_append_oa_reports(struct i915_perf_stream *stream,
* While filtering for a single context we avoid
* leaking the IDs of other contexts.
*/
- if (stream->perf->exclusive_stream->ctx &&
+ if (stream->ctx &&
stream->specific_ctx_id != ctx_id) {
report32[2] = INVALID_CTX_ID;
}
@@ -943,14 +948,13 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,
size_t *offset)
{
struct intel_uncore *uncore = stream->uncore;
- int report_size = stream->oa_buffer.format_size;
+ int report_size = stream->oa_buffer.format->size;
u8 *oa_buf_base = stream->oa_buffer.vaddr;
u32 gtt_offset = i915_ggtt_offset(stream->oa_buffer.vma);
u32 mask = (OA_BUFFER_SIZE - 1);
size_t start_offset = *offset;
unsigned long flags;
u32 head, tail;
- u32 taken;
int ret = 0;
if (drm_WARN_ON(&uncore->i915->drm, !stream->enabled))
@@ -984,7 +988,7 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,
for (/* none */;
- (taken = OA_TAKEN(tail, head));
+ OA_TAKEN(tail, head);
head = (head + report_size) & mask) {
u8 *report = oa_buf_base + head;
u32 *report32 = (void *)report;
@@ -1012,7 +1016,8 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,
*/
if (report32[0] == 0) {
if (__ratelimit(&stream->perf->spurious_report_rs))
- DRM_NOTE("Skipping spurious, invalid OA report\n");
+ drm_notice(&uncore->i915->drm,
+ "Skipping spurious, invalid OA report\n");
continue;
}
@@ -1233,6 +1238,196 @@ static struct intel_context *oa_pin_context(struct i915_perf_stream *stream)
return stream->pinned_ctx;
}
+static int
+__store_reg_to_mem(struct i915_request *rq, i915_reg_t reg, u32 ggtt_offset)
+{
+ u32 *cs, cmd;
+
+ cmd = MI_STORE_REGISTER_MEM | MI_SRM_LRM_GLOBAL_GTT;
+ if (GRAPHICS_VER(rq->engine->i915) >= 8)
+ cmd++;
+
+ cs = intel_ring_begin(rq, 4);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ *cs++ = cmd;
+ *cs++ = i915_mmio_reg_offset(reg);
+ *cs++ = ggtt_offset;
+ *cs++ = 0;
+
+ intel_ring_advance(rq, cs);
+
+ return 0;
+}
+
+static int
+__read_reg(struct intel_context *ce, i915_reg_t reg, u32 ggtt_offset)
+{
+ struct i915_request *rq;
+ int err;
+
+ rq = i915_request_create(ce);
+ if (IS_ERR(rq))
+ return PTR_ERR(rq);
+
+ i915_request_get(rq);
+
+ err = __store_reg_to_mem(rq, reg, ggtt_offset);
+
+ i915_request_add(rq);
+ if (!err && i915_request_wait(rq, 0, HZ / 2) < 0)
+ err = -ETIME;
+
+ i915_request_put(rq);
+
+ return err;
+}
+
+static int
+gen12_guc_sw_ctx_id(struct intel_context *ce, u32 *ctx_id)
+{
+ struct i915_vma *scratch;
+ u32 *val;
+ int err;
+
+ scratch = __vm_create_scratch_for_read_pinned(&ce->engine->gt->ggtt->vm, 4);
+ if (IS_ERR(scratch))
+ return PTR_ERR(scratch);
+
+ err = i915_vma_sync(scratch);
+ if (err)
+ goto err_scratch;
+
+ err = __read_reg(ce, RING_EXECLIST_STATUS_HI(ce->engine->mmio_base),
+ i915_ggtt_offset(scratch));
+ if (err)
+ goto err_scratch;
+
+ val = i915_gem_object_pin_map_unlocked(scratch->obj, I915_MAP_WB);
+ if (IS_ERR(val)) {
+ err = PTR_ERR(val);
+ goto err_scratch;
+ }
+
+ *ctx_id = *val;
+ i915_gem_object_unpin_map(scratch->obj);
+
+err_scratch:
+ i915_vma_unpin_and_release(&scratch, 0);
+ return err;
+}
+
+/*
+ * For execlist mode of submission, pick an unused context id
+ * 0 - (NUM_CONTEXT_TAG -1) are used by other contexts
+ * XXX_MAX_CONTEXT_HW_ID is used by idle context
+ *
+ * For GuC mode of submission read context id from the upper dword of the
+ * EXECLIST_STATUS register. Note that we read this value only once and expect
+ * that the value stays fixed for the entire OA use case. There are cases where
+ * GuC KMD implementation may deregister a context to reuse it's context id, but
+ * we prevent that from happening to the OA context by pinning it.
+ */
+static int gen12_get_render_context_id(struct i915_perf_stream *stream)
+{
+ u32 ctx_id, mask;
+ int ret;
+
+ if (intel_engine_uses_guc(stream->engine)) {
+ ret = gen12_guc_sw_ctx_id(stream->pinned_ctx, &ctx_id);
+ if (ret)
+ return ret;
+
+ mask = ((1U << GEN12_GUC_SW_CTX_ID_WIDTH) - 1) <<
+ (GEN12_GUC_SW_CTX_ID_SHIFT - 32);
+ } else if (GRAPHICS_VER_FULL(stream->engine->i915) >= IP_VER(12, 50)) {
+ ctx_id = (XEHP_MAX_CONTEXT_HW_ID - 1) <<
+ (XEHP_SW_CTX_ID_SHIFT - 32);
+
+ mask = ((1U << XEHP_SW_CTX_ID_WIDTH) - 1) <<
+ (XEHP_SW_CTX_ID_SHIFT - 32);
+ } else {
+ ctx_id = (GEN12_MAX_CONTEXT_HW_ID - 1) <<
+ (GEN11_SW_CTX_ID_SHIFT - 32);
+
+ mask = ((1U << GEN11_SW_CTX_ID_WIDTH) - 1) <<
+ (GEN11_SW_CTX_ID_SHIFT - 32);
+ }
+ stream->specific_ctx_id = ctx_id & mask;
+ stream->specific_ctx_id_mask = mask;
+
+ return 0;
+}
+
+static bool oa_find_reg_in_lri(u32 *state, u32 reg, u32 *offset, u32 end)
+{
+ u32 idx = *offset;
+ u32 len = min(MI_LRI_LEN(state[idx]) + idx, end);
+ bool found = false;
+
+ idx++;
+ for (; idx < len; idx += 2) {
+ if (state[idx] == reg) {
+ found = true;
+ break;
+ }
+ }
+
+ *offset = idx;
+ return found;
+}
+
+static u32 oa_context_image_offset(struct intel_context *ce, u32 reg)
+{
+ u32 offset, len = (ce->engine->context_size - PAGE_SIZE) / 4;
+ u32 *state = ce->lrc_reg_state;
+
+ for (offset = 0; offset < len; ) {
+ if (IS_MI_LRI_CMD(state[offset])) {
+ /*
+ * We expect reg-value pairs in MI_LRI command, so
+ * MI_LRI_LEN() should be even, if not, issue a warning.
+ */
+ drm_WARN_ON(&ce->engine->i915->drm,
+ MI_LRI_LEN(state[offset]) & 0x1);
+
+ if (oa_find_reg_in_lri(state, reg, &offset, len))
+ break;
+ } else {
+ offset++;
+ }
+ }
+
+ return offset < len ? offset : U32_MAX;
+}
+
+static int set_oa_ctx_ctrl_offset(struct intel_context *ce)
+{
+ i915_reg_t reg = GEN12_OACTXCONTROL(ce->engine->mmio_base);
+ struct i915_perf *perf = &ce->engine->i915->perf;
+ u32 offset = perf->ctx_oactxctrl_offset;
+
+ /* Do this only once. Failure is stored as offset of U32_MAX */
+ if (offset)
+ goto exit;
+
+ offset = oa_context_image_offset(ce, i915_mmio_reg_offset(reg));
+ perf->ctx_oactxctrl_offset = offset;
+
+ drm_dbg(&ce->engine->i915->drm,
+ "%s oa ctx control at 0x%08x dword offset\n",
+ ce->engine->name, offset);
+
+exit:
+ return offset && offset != U32_MAX ? 0 : -ENODEV;
+}
+
+static bool engine_supports_mi_query(struct intel_engine_cs *engine)
+{
+ return engine->class == RENDER_CLASS;
+}
+
/**
* oa_get_render_ctx_id - determine and hold ctx hw id
* @stream: An i915-perf stream opened for OA metrics
@@ -1246,11 +1441,27 @@ static struct intel_context *oa_pin_context(struct i915_perf_stream *stream)
static int oa_get_render_ctx_id(struct i915_perf_stream *stream)
{
struct intel_context *ce;
+ int ret = 0;
ce = oa_pin_context(stream);
if (IS_ERR(ce))
return PTR_ERR(ce);
+ if (engine_supports_mi_query(stream->engine)) {
+ /*
+ * We are enabling perf query here. If we don't find the context
+ * offset here, just return an error.
+ */
+ ret = set_oa_ctx_ctrl_offset(ce);
+ if (ret) {
+ intel_context_unpin(ce);
+ drm_err(&stream->perf->i915->drm,
+ "Enabling perf query failed for %s\n",
+ stream->engine->name);
+ return ret;
+ }
+ }
+
switch (GRAPHICS_VER(ce->engine->i915)) {
case 7: {
/*
@@ -1292,24 +1503,7 @@ static int oa_get_render_ctx_id(struct i915_perf_stream *stream)
case 11:
case 12:
- if (GRAPHICS_VER_FULL(ce->engine->i915) >= IP_VER(12, 50)) {
- stream->specific_ctx_id_mask =
- ((1U << XEHP_SW_CTX_ID_WIDTH) - 1) <<
- (XEHP_SW_CTX_ID_SHIFT - 32);
- stream->specific_ctx_id =
- (XEHP_MAX_CONTEXT_HW_ID - 1) <<
- (XEHP_SW_CTX_ID_SHIFT - 32);
- } else {
- stream->specific_ctx_id_mask =
- ((1U << GEN11_SW_CTX_ID_WIDTH) - 1) << (GEN11_SW_CTX_ID_SHIFT - 32);
- /*
- * Pick an unused context id
- * 0 - BITS_PER_LONG are used by other contexts
- * GEN12_MAX_CONTEXT_HW_ID (0x7ff) is used by idle context
- */
- stream->specific_ctx_id =
- (GEN12_MAX_CONTEXT_HW_ID - 1) << (GEN11_SW_CTX_ID_SHIFT - 32);
- }
+ ret = gen12_get_render_context_id(stream);
break;
default:
@@ -1323,7 +1517,7 @@ static int oa_get_render_ctx_id(struct i915_perf_stream *stream)
stream->specific_ctx_id,
stream->specific_ctx_id_mask);
- return 0;
+ return ret;
}
/**
@@ -1375,8 +1569,9 @@ free_noa_wait(struct i915_perf_stream *stream)
static void i915_oa_stream_destroy(struct i915_perf_stream *stream)
{
struct i915_perf *perf = stream->perf;
+ struct intel_gt *gt = stream->engine->gt;
- if (WARN_ON(stream != perf->exclusive_stream))
+ if (WARN_ON(stream != gt->perf.exclusive_stream))
return;
/*
@@ -1385,11 +1580,20 @@ static void i915_oa_stream_destroy(struct i915_perf_stream *stream)
*
* See i915_oa_init_reg_state() and lrc_configure_all_contexts()
*/
- WRITE_ONCE(perf->exclusive_stream, NULL);
+ WRITE_ONCE(gt->perf.exclusive_stream, NULL);
perf->ops.disable_metric_set(stream);
free_oa_buffer(stream);
+ /*
+ * Wa_16011777198:dg2: Unset the override of GUCRC mode to enable rc6.
+ */
+ if (intel_uc_uses_guc_rc(>->uc) &&
+ (IS_DG2_GRAPHICS_STEP(gt->i915, G10, STEP_A0, STEP_C0) ||
+ IS_DG2_GRAPHICS_STEP(gt->i915, G11, STEP_A0, STEP_B0)))
+ drm_WARN_ON(>->i915->drm,
+ intel_guc_slpc_unset_gucrc_mode(>->uc.guc.slpc));
+
intel_uncore_forcewake_put(stream->uncore, FORCEWAKE_ALL);
intel_engine_pm_put(stream->engine);
@@ -1400,8 +1604,9 @@ static void i915_oa_stream_destroy(struct i915_perf_stream *stream)
free_noa_wait(stream);
if (perf->spurious_report_rs.missed) {
- DRM_NOTE("%d spurious OA report notices suppressed due to ratelimiting\n",
- perf->spurious_report_rs.missed);
+ drm_notice(>->i915->drm,
+ "%d spurious OA report notices suppressed due to ratelimiting\n",
+ perf->spurious_report_rs.missed);
}
}
@@ -1563,6 +1768,7 @@ static void gen12_init_oa_buffer(struct i915_perf_stream *stream)
static int alloc_oa_buffer(struct i915_perf_stream *stream)
{
struct drm_i915_private *i915 = stream->perf->i915;
+ struct intel_gt *gt = stream->engine->gt;
struct drm_i915_gem_object *bo;
struct i915_vma *vma;
int ret;
@@ -1582,11 +1788,22 @@ static int alloc_oa_buffer(struct i915_perf_stream *stream)
i915_gem_object_set_cache_coherency(bo, I915_CACHE_LLC);
/* PreHSW required 512K alignment, HSW requires 16M */
- vma = i915_gem_object_ggtt_pin(bo, NULL, 0, SZ_16M, 0);
+ vma = i915_vma_instance(bo, >->ggtt->vm, NULL);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto err_unref;
}
+
+ /*
+ * PreHSW required 512K alignment.
+ * HSW and onwards, align to requested size of OA buffer.
+ */
+ ret = i915_vma_pin(vma, 0, SZ_16M, PIN_GLOBAL | PIN_HIGH);
+ if (ret) {
+ drm_err(>->i915->drm, "Failed to pin OA buffer %d\n", ret);
+ goto err_unref;
+ }
+
stream->oa_buffer.vma = vma;
stream->oa_buffer.vaddr =
@@ -1636,6 +1853,7 @@ static u32 *save_restore_register(struct i915_perf_stream *stream, u32 *cs,
static int alloc_noa_wait(struct i915_perf_stream *stream)
{
struct drm_i915_private *i915 = stream->perf->i915;
+ struct intel_gt *gt = stream->engine->gt;
struct drm_i915_gem_object *bo;
struct i915_vma *vma;
const u64 delay_ticks = 0xffffffffffffffff -
@@ -1654,6 +1872,9 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
DELTA_TARGET,
N_CS_GPR
};
+ i915_reg_t mi_predicate_result = HAS_MI_SET_PREDICATE(i915) ?
+ MI_PREDICATE_RESULT_2_ENGINE(base) :
+ MI_PREDICATE_RESULT_1(RENDER_RING_BASE);
bo = i915_gem_object_create_internal(i915, 4096);
if (IS_ERR(bo)) {
@@ -1673,12 +1894,16 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
* multiple OA config BOs will have a jump to this address and it
* needs to be fixed during the lifetime of the i915/perf stream.
*/
- vma = i915_gem_object_ggtt_pin_ww(bo, &ww, NULL, 0, 0, PIN_HIGH);
+ vma = i915_vma_instance(bo, >->ggtt->vm, NULL);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto out_ww;
}
+ ret = i915_vma_pin_ww(vma, &ww, 0, 0, PIN_GLOBAL | PIN_HIGH);
+ if (ret)
+ goto out_ww;
+
batch = cs = i915_gem_object_pin_map(bo, I915_MAP_WB);
if (IS_ERR(batch)) {
ret = PTR_ERR(batch);
@@ -1691,7 +1916,7 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
stream, cs, true /* save */, CS_GPR(i),
INTEL_GT_SCRATCH_FIELD_PERF_CS_GPR + 8 * i, 2);
cs = save_restore_register(
- stream, cs, true /* save */, MI_PREDICATE_RESULT_1(RENDER_RING_BASE),
+ stream, cs, true /* save */, mi_predicate_result,
INTEL_GT_SCRATCH_FIELD_PERF_PREDICATE_RESULT_1, 1);
/* First timestamp snapshot location. */
@@ -1745,7 +1970,10 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
*/
*cs++ = MI_LOAD_REGISTER_REG | (3 - 2);
*cs++ = i915_mmio_reg_offset(CS_GPR(JUMP_PREDICATE));
- *cs++ = i915_mmio_reg_offset(MI_PREDICATE_RESULT_1(RENDER_RING_BASE));
+ *cs++ = i915_mmio_reg_offset(mi_predicate_result);
+
+ if (HAS_MI_SET_PREDICATE(i915))
+ *cs++ = MI_SET_PREDICATE | 1;
/* Restart from the beginning if we had timestamps roll over. */
*cs++ = (GRAPHICS_VER(i915) < 8 ?
@@ -1755,6 +1983,9 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
*cs++ = i915_ggtt_offset(vma) + (ts0 - batch) * 4;
*cs++ = 0;
+ if (HAS_MI_SET_PREDICATE(i915))
+ *cs++ = MI_SET_PREDICATE;
+
/*
* Now add the diff between to previous timestamps and add it to :
* (((1 * << 64) - 1) - delay_ns)
@@ -1782,7 +2013,10 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
*/
*cs++ = MI_LOAD_REGISTER_REG | (3 - 2);
*cs++ = i915_mmio_reg_offset(CS_GPR(JUMP_PREDICATE));
- *cs++ = i915_mmio_reg_offset(MI_PREDICATE_RESULT_1(RENDER_RING_BASE));
+ *cs++ = i915_mmio_reg_offset(mi_predicate_result);
+
+ if (HAS_MI_SET_PREDICATE(i915))
+ *cs++ = MI_SET_PREDICATE | 1;
/* Predicate the jump. */
*cs++ = (GRAPHICS_VER(i915) < 8 ?
@@ -1792,13 +2026,16 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
*cs++ = i915_ggtt_offset(vma) + (jump - batch) * 4;
*cs++ = 0;
+ if (HAS_MI_SET_PREDICATE(i915))
+ *cs++ = MI_SET_PREDICATE;
+
/* Restore registers. */
for (i = 0; i < N_CS_GPR; i++)
cs = save_restore_register(
stream, cs, false /* restore */, CS_GPR(i),
INTEL_GT_SCRATCH_FIELD_PERF_CS_GPR + 8 * i, 2);
cs = save_restore_register(
- stream, cs, false /* restore */, MI_PREDICATE_RESULT_1(RENDER_RING_BASE),
+ stream, cs, false /* restore */, mi_predicate_result,
INTEL_GT_SCRATCH_FIELD_PERF_PREDICATE_RESULT_1, 1);
/* And return to the ring. */
@@ -2017,9 +2254,7 @@ emit_oa_config(struct i915_perf_stream *stream,
goto err_add_request;
}
- err = i915_request_await_object(rq, vma->obj, 0);
- if (!err)
- err = i915_vma_move_to_active(vma, rq, 0);
+ err = i915_vma_move_to_active(vma, rq, 0);
if (err)
goto err_add_request;
@@ -2283,11 +2518,12 @@ static int gen12_configure_oar_context(struct i915_perf_stream *stream,
{
int err;
struct intel_context *ce = stream->pinned_ctx;
- u32 format = stream->oa_buffer.format;
+ u32 format = stream->oa_buffer.format->format;
+ u32 offset = stream->perf->ctx_oactxctrl_offset;
struct flex regs_context[] = {
{
GEN8_OACTXCONTROL,
- stream->perf->ctx_oactxctrl_offset + 1,
+ offset + 1,
active ? GEN8_OA_COUNTER_RESUME : 0,
},
};
@@ -2312,12 +2548,13 @@ static int gen12_configure_oar_context(struct i915_perf_stream *stream,
},
};
- /* Modify the context image of pinned context with regs_context*/
+ /* Modify the context image of pinned context with regs_context */
err = intel_context_lock_pinned(ce);
if (err)
return err;
- err = gen8_modify_context(ce, regs_context, ARRAY_SIZE(regs_context));
+ err = gen8_modify_context(ce, regs_context,
+ ARRAY_SIZE(regs_context));
intel_context_unlock_pinned(ce);
if (err)
return err;
@@ -2359,10 +2596,11 @@ oa_configure_all_contexts(struct i915_perf_stream *stream,
{
struct drm_i915_private *i915 = stream->perf->i915;
struct intel_engine_cs *engine;
+ struct intel_gt *gt = stream->engine->gt;
struct i915_gem_context *ctx, *cn;
int err;
- lockdep_assert_held(&stream->perf->lock);
+ lockdep_assert_held(>->perf.lock);
/*
* The OA register config is setup through the context image. This image
@@ -2442,6 +2680,7 @@ lrc_configure_all_contexts(struct i915_perf_stream *stream,
const struct i915_oa_config *oa_config,
struct i915_active *active)
{
+ u32 ctx_oactxctrl = stream->perf->ctx_oactxctrl_offset;
/* The MMIO offsets for Flex EU registers aren't contiguous */
const u32 ctx_flexeu0 = stream->perf->ctx_flexeu0_offset;
#define ctx_flexeuN(N) (ctx_flexeu0 + 2 * (N) + 1)
@@ -2452,7 +2691,7 @@ lrc_configure_all_contexts(struct i915_perf_stream *stream,
},
{
GEN8_OACTXCONTROL,
- stream->perf->ctx_oactxctrl_offset + 1,
+ ctx_oactxctrl + 1,
},
{ EU_PERF_CNTL0, ctx_flexeuN(0) },
{ EU_PERF_CNTL1, ctx_flexeuN(1) },
@@ -2540,12 +2779,26 @@ static int
gen12_enable_metric_set(struct i915_perf_stream *stream,
struct i915_active *active)
{
+ struct drm_i915_private *i915 = stream->perf->i915;
struct intel_uncore *uncore = stream->uncore;
struct i915_oa_config *oa_config = stream->oa_config;
bool periodic = stream->periodic;
u32 period_exponent = stream->period_exponent;
+ u32 sqcnt1;
int ret;
+ /*
+ * Wa_1508761755:xehpsdv, dg2
+ * EU NOA signals behave incorrectly if EU clock gating is enabled.
+ * Disable thread stall DOP gating and EU DOP gating.
+ */
+ if (IS_XEHPSDV(i915) || IS_DG2(i915)) {
+ intel_gt_mcr_multicast_write(uncore->gt, GEN8_ROW_CHICKEN,
+ _MASKED_BIT_ENABLE(STALL_DOP_GATING_DISABLE));
+ intel_uncore_write(uncore, GEN7_ROW_CHICKEN2,
+ _MASKED_BIT_ENABLE(GEN12_DISABLE_DOP_GATING));
+ }
+
intel_uncore_write(uncore, GEN12_OAG_OA_DEBUG,
/* Disable clk ratio reports, like previous Gens. */
_MASKED_BIT_ENABLE(GEN12_OAG_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS |
@@ -2563,6 +2816,16 @@ gen12_enable_metric_set(struct i915_perf_stream *stream,
: 0);
/*
+ * Initialize Super Queue Internal Cnt Register
+ * Set PMON Enable in order to collect valid metrics.
+ * Enable byets per clock reporting in OA for XEHPSDV onward.
+ */
+ sqcnt1 = GEN12_SQCNT1_PMON_ENABLE |
+ (HAS_OA_BPC_REPORTING(i915) ? GEN12_SQCNT1_OABPC : 0);
+
+ intel_uncore_rmw(uncore, GEN12_SQCNT1, 0, sqcnt1);
+
+ /*
* Update all contexts prior writing the mux configurations as we need
* to make sure all slices/subslices are ON before writing to NOA
* registers.
@@ -2611,6 +2874,19 @@ static void gen11_disable_metric_set(struct i915_perf_stream *stream)
static void gen12_disable_metric_set(struct i915_perf_stream *stream)
{
struct intel_uncore *uncore = stream->uncore;
+ struct drm_i915_private *i915 = stream->perf->i915;
+ u32 sqcnt1;
+
+ /*
+ * Wa_1508761755:xehpsdv, dg2
+ * Enable thread stall DOP gating and EU DOP gating.
+ */
+ if (IS_XEHPSDV(i915) || IS_DG2(i915)) {
+ intel_gt_mcr_multicast_write(uncore->gt, GEN8_ROW_CHICKEN,
+ _MASKED_BIT_DISABLE(STALL_DOP_GATING_DISABLE));
+ intel_uncore_write(uncore, GEN7_ROW_CHICKEN2,
+ _MASKED_BIT_DISABLE(GEN12_DISABLE_DOP_GATING));
+ }
/* Reset all contexts' slices/subslices configurations. */
gen12_configure_all_contexts(stream, NULL, NULL);
@@ -2621,6 +2897,12 @@ static void gen12_disable_metric_set(struct i915_perf_stream *stream)
/* Make sure we disable noa to save power. */
intel_uncore_rmw(uncore, RPM_CONFIG1, GEN10_GT_NOA_ENABLE, 0);
+
+ sqcnt1 = GEN12_SQCNT1_PMON_ENABLE |
+ (HAS_OA_BPC_REPORTING(i915) ? GEN12_SQCNT1_OABPC : 0);
+
+ /* Reset PMON Enable to save power. */
+ intel_uncore_rmw(uncore, GEN12_SQCNT1, sqcnt1, 0);
}
static void gen7_oa_enable(struct i915_perf_stream *stream)
@@ -2630,7 +2912,7 @@ static void gen7_oa_enable(struct i915_perf_stream *stream)
u32 ctx_id = stream->specific_ctx_id;
bool periodic = stream->periodic;
u32 period_exponent = stream->period_exponent;
- u32 report_format = stream->oa_buffer.format;
+ u32 report_format = stream->oa_buffer.format->format;
/*
* Reset buf pointers so we don't forward reports from before now.
@@ -2656,7 +2938,7 @@ static void gen7_oa_enable(struct i915_perf_stream *stream)
static void gen8_oa_enable(struct i915_perf_stream *stream)
{
struct intel_uncore *uncore = stream->uncore;
- u32 report_format = stream->oa_buffer.format;
+ u32 report_format = stream->oa_buffer.format->format;
/*
* Reset buf pointers so we don't forward reports from before now.
@@ -2682,7 +2964,7 @@ static void gen8_oa_enable(struct i915_perf_stream *stream)
static void gen12_oa_enable(struct i915_perf_stream *stream)
{
struct intel_uncore *uncore = stream->uncore;
- u32 report_format = stream->oa_buffer.format;
+ u32 report_format = stream->oa_buffer.format->format;
/*
* If we don't want OA reports from the OA buffer, then we don't even
@@ -2838,6 +3120,30 @@ get_sseu_config(struct intel_sseu *out_sseu,
return i915_gem_user_to_context_sseu(engine->gt, drm_sseu, out_sseu);
}
+/*
+ * OA timestamp frequency = CS timestamp frequency in most platforms. On some
+ * platforms OA unit ignores the CTC_SHIFT and the 2 timestamps differ. In such
+ * cases, return the adjusted CS timestamp frequency to the user.
+ */
+u32 i915_perf_oa_timestamp_frequency(struct drm_i915_private *i915)
+{
+ /* Wa_18013179988:dg2 */
+ if (IS_DG2(i915)) {
+ intel_wakeref_t wakeref;
+ u32 reg, shift;
+
+ with_intel_runtime_pm(to_gt(i915)->uncore->rpm, wakeref)
+ reg = intel_uncore_read(to_gt(i915)->uncore, RPM_CONFIG0);
+
+ shift = REG_FIELD_GET(GEN10_RPM_CONFIG0_CTC_SHIFT_PARAMETER_MASK,
+ reg);
+
+ return to_gt(i915)->clock_frequency << (3 - shift);
+ }
+
+ return to_gt(i915)->clock_frequency;
+}
+
/**
* i915_oa_stream_init - validate combined props for OA stream and init
* @stream: An i915 perf stream
@@ -2862,7 +3168,7 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
{
struct drm_i915_private *i915 = stream->perf->i915;
struct i915_perf *perf = stream->perf;
- int format_size;
+ struct intel_gt *gt;
int ret;
if (!props->engine) {
@@ -2870,6 +3176,7 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
"OA engine not specified\n");
return -EINVAL;
}
+ gt = props->engine->gt;
/*
* If the sysfs metrics/ directory wasn't registered for some
@@ -2900,7 +3207,7 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
* counter reports and marshal to the appropriate client
* we currently only allow exclusive access
*/
- if (perf->exclusive_stream) {
+ if (gt->perf.exclusive_stream) {
drm_dbg(&stream->perf->i915->drm,
"OA unit already in use\n");
return -EBUSY;
@@ -2917,19 +3224,14 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
stream->sample_size = sizeof(struct drm_i915_perf_record_header);
- format_size = perf->oa_formats[props->oa_format].size;
-
- stream->sample_flags = props->sample_flags;
- stream->sample_size += format_size;
-
- stream->oa_buffer.format_size = format_size;
- if (drm_WARN_ON(&i915->drm, stream->oa_buffer.format_size == 0))
+ stream->oa_buffer.format = &perf->oa_formats[props->oa_format];
+ if (drm_WARN_ON(&i915->drm, stream->oa_buffer.format->size == 0))
return -EINVAL;
- stream->hold_preemption = props->hold_preemption;
+ stream->sample_flags = props->sample_flags;
+ stream->sample_size += stream->oa_buffer.format->size;
- stream->oa_buffer.format =
- perf->oa_formats[props->oa_format].format;
+ stream->hold_preemption = props->hold_preemption;
stream->periodic = props->oa_periodic;
if (stream->periodic)
@@ -2974,14 +3276,31 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
intel_engine_pm_get(stream->engine);
intel_uncore_forcewake_get(stream->uncore, FORCEWAKE_ALL);
+ /*
+ * Wa_16011777198:dg2: GuC resets render as part of the Wa. This causes
+ * OA to lose the configuration state. Prevent this by overriding GUCRC
+ * mode.
+ */
+ if (intel_uc_uses_guc_rc(>->uc) &&
+ (IS_DG2_GRAPHICS_STEP(gt->i915, G10, STEP_A0, STEP_C0) ||
+ IS_DG2_GRAPHICS_STEP(gt->i915, G11, STEP_A0, STEP_B0))) {
+ ret = intel_guc_slpc_override_gucrc_mode(>->uc.guc.slpc,
+ SLPC_GUCRC_MODE_GUCRC_NO_RC6);
+ if (ret) {
+ drm_dbg(&stream->perf->i915->drm,
+ "Unable to override gucrc mode\n");
+ goto err_config;
+ }
+ }
+
ret = alloc_oa_buffer(stream);
if (ret)
goto err_oa_buf_alloc;
stream->ops = &i915_oa_stream_ops;
- perf->sseu = props->sseu;
- WRITE_ONCE(perf->exclusive_stream, stream);
+ stream->engine->gt->perf.sseu = props->sseu;
+ WRITE_ONCE(gt->perf.exclusive_stream, stream);
ret = i915_perf_stream_enable_sync(stream);
if (ret) {
@@ -2999,11 +3318,12 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
stream->poll_check_timer.function = oa_poll_check_timer_cb;
init_waitqueue_head(&stream->poll_wq);
spin_lock_init(&stream->oa_buffer.ptr_lock);
+ mutex_init(&stream->lock);
return 0;
err_enable:
- WRITE_ONCE(perf->exclusive_stream, NULL);
+ WRITE_ONCE(gt->perf.exclusive_stream, NULL);
perf->ops.disable_metric_set(stream);
free_oa_buffer(stream);
@@ -3033,7 +3353,7 @@ void i915_oa_init_reg_state(const struct intel_context *ce,
return;
/* perf.exclusive_stream serialised by lrc_configure_all_contexts() */
- stream = READ_ONCE(engine->i915->perf.exclusive_stream);
+ stream = READ_ONCE(engine->gt->perf.exclusive_stream);
if (stream && GRAPHICS_VER(stream->perf->i915) < 12)
gen8_update_reg_state_unlocked(ce, stream);
}
@@ -3062,7 +3382,6 @@ static ssize_t i915_perf_read(struct file *file,
loff_t *ppos)
{
struct i915_perf_stream *stream = file->private_data;
- struct i915_perf *perf = stream->perf;
size_t offset = 0;
int ret;
@@ -3086,14 +3405,14 @@ static ssize_t i915_perf_read(struct file *file,
if (ret)
return ret;
- mutex_lock(&perf->lock);
+ mutex_lock(&stream->lock);
ret = stream->ops->read(stream, buf, count, &offset);
- mutex_unlock(&perf->lock);
+ mutex_unlock(&stream->lock);
} while (!offset && !ret);
} else {
- mutex_lock(&perf->lock);
+ mutex_lock(&stream->lock);
ret = stream->ops->read(stream, buf, count, &offset);
- mutex_unlock(&perf->lock);
+ mutex_unlock(&stream->lock);
}
/* We allow the poll checking to sometimes report false positive EPOLLIN
@@ -3140,9 +3459,6 @@ static enum hrtimer_restart oa_poll_check_timer_cb(struct hrtimer *hrtimer)
* &i915_perf_stream_ops->poll_wait to call poll_wait() with a wait queue that
* will be woken for new stream data.
*
- * Note: The &perf->lock mutex has been taken to serialize
- * with any non-file-operation driver hooks.
- *
* Returns: any poll events that are ready without sleeping
*/
static __poll_t i915_perf_poll_locked(struct i915_perf_stream *stream,
@@ -3181,12 +3497,11 @@ static __poll_t i915_perf_poll_locked(struct i915_perf_stream *stream,
static __poll_t i915_perf_poll(struct file *file, poll_table *wait)
{
struct i915_perf_stream *stream = file->private_data;
- struct i915_perf *perf = stream->perf;
__poll_t ret;
- mutex_lock(&perf->lock);
+ mutex_lock(&stream->lock);
ret = i915_perf_poll_locked(stream, file, wait);
- mutex_unlock(&perf->lock);
+ mutex_unlock(&stream->lock);
return ret;
}
@@ -3285,9 +3600,6 @@ static long i915_perf_config_locked(struct i915_perf_stream *stream,
* @cmd: the ioctl request
* @arg: the ioctl data
*
- * Note: The &perf->lock mutex has been taken to serialize
- * with any non-file-operation driver hooks.
- *
* Returns: zero on success or a negative error code. Returns -EINVAL for
* an unknown ioctl request.
*/
@@ -3325,12 +3637,11 @@ static long i915_perf_ioctl(struct file *file,
unsigned long arg)
{
struct i915_perf_stream *stream = file->private_data;
- struct i915_perf *perf = stream->perf;
long ret;
- mutex_lock(&perf->lock);
+ mutex_lock(&stream->lock);
ret = i915_perf_ioctl_locked(stream, cmd, arg);
- mutex_unlock(&perf->lock);
+ mutex_unlock(&stream->lock);
return ret;
}
@@ -3342,7 +3653,7 @@ static long i915_perf_ioctl(struct file *file,
* Frees all resources associated with the given i915 perf @stream, disabling
* any associated data capture in the process.
*
- * Note: The &perf->lock mutex has been taken to serialize
+ * Note: The >->perf.lock mutex has been taken to serialize
* with any non-file-operation driver hooks.
*/
static void i915_perf_destroy_locked(struct i915_perf_stream *stream)
@@ -3374,10 +3685,16 @@ static int i915_perf_release(struct inode *inode, struct file *file)
{
struct i915_perf_stream *stream = file->private_data;
struct i915_perf *perf = stream->perf;
+ struct intel_gt *gt = stream->engine->gt;
- mutex_lock(&perf->lock);
+ /*
+ * Within this call, we know that the fd is being closed and we have no
+ * other user of stream->lock. Use the perf lock to destroy the stream
+ * here.
+ */
+ mutex_lock(>->perf.lock);
i915_perf_destroy_locked(stream);
- mutex_unlock(&perf->lock);
+ mutex_unlock(>->perf.lock);
/* Release the reference the perf stream kept on the driver. */
drm_dev_put(&perf->i915->drm);
@@ -3410,7 +3727,7 @@ static const struct file_operations fops = {
* See i915_perf_ioctl_open() for interface details.
*
* Implements further stream config validation and stream initialization on
- * behalf of i915_perf_open_ioctl() with the &perf->lock mutex
+ * behalf of i915_perf_open_ioctl() with the >->perf.lock mutex
* taken to serialize with any non-file-operation driver hooks.
*
* Note: at this point the @props have only been validated in isolation and
@@ -3565,8 +3882,10 @@ i915_perf_open_ioctl_locked(struct i915_perf *perf,
static u64 oa_exponent_to_ns(struct i915_perf *perf, int exponent)
{
- return intel_gt_clock_interval_to_ns(to_gt(perf->i915),
- 2ULL << exponent);
+ u64 nom = (2ULL << exponent) * NSEC_PER_SEC;
+ u32 den = i915_perf_oa_timestamp_frequency(perf->i915);
+
+ return div_u64(nom + den - 1, den);
}
static __always_inline bool
@@ -3794,7 +4113,7 @@ static int read_properties_unlocked(struct i915_perf *perf,
* mutex to avoid an awkward lockdep with mmap_lock.
*
* Most of the implementation details are handled by
- * i915_perf_open_ioctl_locked() after taking the &perf->lock
+ * i915_perf_open_ioctl_locked() after taking the >->perf.lock
* mutex for serializing with any non-file-operation driver hooks.
*
* Return: A newly opened i915 Perf stream file descriptor or negative
@@ -3805,6 +4124,7 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,
{
struct i915_perf *perf = &to_i915(dev)->perf;
struct drm_i915_perf_open_param *param = data;
+ struct intel_gt *gt;
struct perf_open_properties props;
u32 known_open_flags;
int ret;
@@ -3831,9 +4151,11 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,
if (ret)
return ret;
- mutex_lock(&perf->lock);
+ gt = props.engine->gt;
+
+ mutex_lock(>->perf.lock);
ret = i915_perf_open_ioctl_locked(perf, param, &props, file);
- mutex_unlock(&perf->lock);
+ mutex_unlock(>->perf.lock);
return ret;
}
@@ -3849,6 +4171,7 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,
void i915_perf_register(struct drm_i915_private *i915)
{
struct i915_perf *perf = &i915->perf;
+ struct intel_gt *gt = to_gt(i915);
if (!perf->i915)
return;
@@ -3857,13 +4180,13 @@ void i915_perf_register(struct drm_i915_private *i915)
* i915_perf_open_ioctl(); considering that we register after
* being exposed to userspace.
*/
- mutex_lock(&perf->lock);
+ mutex_lock(>->perf.lock);
perf->metrics_kobj =
kobject_create_and_add("metrics",
&i915->drm.primary->kdev->kobj);
- mutex_unlock(&perf->lock);
+ mutex_unlock(>->perf.lock);
}
/**
@@ -3939,6 +4262,11 @@ static const struct i915_range gen12_oa_b_counters[] = {
{}
};
+static const struct i915_range xehp_oa_b_counters[] = {
+ { .start = 0xdc48, .end = 0xdc48 }, /* OAA_ENABLE_REG */
+ { .start = 0xdd00, .end = 0xdd48 }, /* OAG_LCE0_0 - OAA_LENABLE_REG */
+};
+
static const struct i915_range gen7_oa_mux_regs[] = {
{ .start = 0x91b8, .end = 0x91cc }, /* OA_PERFCNT[1-2], OA_PERFMATRIX */
{ .start = 0x9800, .end = 0x9888 }, /* MICRO_BP0_0 - NOA_WRITE */
@@ -4013,6 +4341,12 @@ static bool gen12_is_valid_b_counter_addr(struct i915_perf *perf, u32 addr)
return reg_in_range_table(addr, gen12_oa_b_counters);
}
+static bool xehp_is_valid_b_counter_addr(struct i915_perf *perf, u32 addr)
+{
+ return reg_in_range_table(addr, xehp_oa_b_counters) ||
+ reg_in_range_table(addr, gen12_oa_b_counters);
+}
+
static bool gen12_is_valid_mux_addr(struct i915_perf *perf, u32 addr)
{
return reg_in_range_table(addr, gen12_oa_mux_regs);
@@ -4411,11 +4745,47 @@ static void oa_init_supported_formats(struct i915_perf *perf)
oa_format_add(perf, I915_OA_FORMAT_C4_B8);
break;
+ case INTEL_DG2:
+ oa_format_add(perf, I915_OAR_FORMAT_A32u40_A4u32_B8_C8);
+ oa_format_add(perf, I915_OA_FORMAT_A24u40_A14u32_B8_C8);
+ break;
+
default:
MISSING_CASE(platform);
}
}
+static void i915_perf_init_info(struct drm_i915_private *i915)
+{
+ struct i915_perf *perf = &i915->perf;
+
+ switch (GRAPHICS_VER(i915)) {
+ case 8:
+ perf->ctx_oactxctrl_offset = 0x120;
+ perf->ctx_flexeu0_offset = 0x2ce;
+ perf->gen8_valid_ctx_bit = BIT(25);
+ break;
+ case 9:
+ perf->ctx_oactxctrl_offset = 0x128;
+ perf->ctx_flexeu0_offset = 0x3de;
+ perf->gen8_valid_ctx_bit = BIT(16);
+ break;
+ case 11:
+ perf->ctx_oactxctrl_offset = 0x124;
+ perf->ctx_flexeu0_offset = 0x78e;
+ perf->gen8_valid_ctx_bit = BIT(16);
+ break;
+ case 12:
+ /*
+ * Calculate offset at runtime in oa_pin_context for gen12 and
+ * cache the value in perf->ctx_oactxctrl_offset.
+ */
+ break;
+ default:
+ MISSING_CASE(GRAPHICS_VER(i915));
+ }
+}
+
/**
* i915_perf_init - initialize i915-perf state on module bind
* @i915: i915 device instance
@@ -4429,12 +4799,6 @@ void i915_perf_init(struct drm_i915_private *i915)
{
struct i915_perf *perf = &i915->perf;
- /* XXX const struct i915_perf_ops! */
-
- /* i915_perf is not enabled for DG2 yet */
- if (IS_DG2(i915))
- return;
-
perf->oa_formats = oa_formats;
if (IS_HASWELL(i915)) {
perf->ops.is_valid_b_counter_reg = gen7_is_valid_b_counter_addr;
@@ -4454,6 +4818,7 @@ void i915_perf_init(struct drm_i915_private *i915)
* execlist mode by default.
*/
perf->ops.read = gen8_oa_read;
+ i915_perf_init_info(i915);
if (IS_GRAPHICS_VER(i915, 8, 9)) {
perf->ops.is_valid_b_counter_reg =
@@ -4473,18 +4838,6 @@ void i915_perf_init(struct drm_i915_private *i915)
perf->ops.enable_metric_set = gen8_enable_metric_set;
perf->ops.disable_metric_set = gen8_disable_metric_set;
perf->ops.oa_hw_tail_read = gen8_oa_hw_tail_read;
-
- if (GRAPHICS_VER(i915) == 8) {
- perf->ctx_oactxctrl_offset = 0x120;
- perf->ctx_flexeu0_offset = 0x2ce;
-
- perf->gen8_valid_ctx_bit = BIT(25);
- } else {
- perf->ctx_oactxctrl_offset = 0x128;
- perf->ctx_flexeu0_offset = 0x3de;
-
- perf->gen8_valid_ctx_bit = BIT(16);
- }
} else if (GRAPHICS_VER(i915) == 11) {
perf->ops.is_valid_b_counter_reg =
gen7_is_valid_b_counter_addr;
@@ -4498,13 +4851,10 @@ void i915_perf_init(struct drm_i915_private *i915)
perf->ops.enable_metric_set = gen8_enable_metric_set;
perf->ops.disable_metric_set = gen11_disable_metric_set;
perf->ops.oa_hw_tail_read = gen8_oa_hw_tail_read;
-
- perf->ctx_oactxctrl_offset = 0x124;
- perf->ctx_flexeu0_offset = 0x78e;
-
- perf->gen8_valid_ctx_bit = BIT(16);
} else if (GRAPHICS_VER(i915) == 12) {
perf->ops.is_valid_b_counter_reg =
+ HAS_OA_SLICE_CONTRIB_LIMITS(i915) ?
+ xehp_is_valid_b_counter_addr :
gen12_is_valid_b_counter_addr;
perf->ops.is_valid_mux_reg =
gen12_is_valid_mux_addr;
@@ -4516,14 +4866,15 @@ void i915_perf_init(struct drm_i915_private *i915)
perf->ops.enable_metric_set = gen12_enable_metric_set;
perf->ops.disable_metric_set = gen12_disable_metric_set;
perf->ops.oa_hw_tail_read = gen12_oa_hw_tail_read;
-
- perf->ctx_flexeu0_offset = 0;
- perf->ctx_oactxctrl_offset = 0x144;
}
}
if (perf->ops.enable_metric_set) {
- mutex_init(&perf->lock);
+ struct intel_gt *gt;
+ int i;
+
+ for_each_gt(gt, i915, i)
+ mutex_init(>->perf.lock);
/* Choose a representative limit */
oa_sample_rate_hard_limit = to_gt(i915)->clock_frequency / 2;
diff --git a/drivers/gpu/drm/i915/i915_perf.h b/drivers/gpu/drm/i915/i915_perf.h
index 1d1329e..f96e09a4a 100644
--- a/drivers/gpu/drm/i915/i915_perf.h
+++ b/drivers/gpu/drm/i915/i915_perf.h
@@ -57,4 +57,6 @@ static inline void i915_oa_config_put(struct i915_oa_config *oa_config)
kref_put(&oa_config->ref, i915_oa_config_release);
}
+u32 i915_perf_oa_timestamp_frequency(struct drm_i915_private *i915);
+
#endif /* __I915_PERF_H__ */
diff --git a/drivers/gpu/drm/i915/i915_perf_oa_regs.h b/drivers/gpu/drm/i915/i915_perf_oa_regs.h
index f31c9f1..381d941 100644
--- a/drivers/gpu/drm/i915/i915_perf_oa_regs.h
+++ b/drivers/gpu/drm/i915/i915_perf_oa_regs.h
@@ -97,7 +97,7 @@
#define GEN12_OAR_OACONTROL_COUNTER_FORMAT_SHIFT 1
#define GEN12_OAR_OACONTROL_COUNTER_ENABLE (1 << 0)
-#define GEN12_OACTXCONTROL _MMIO(0x2360)
+#define GEN12_OACTXCONTROL(base) _MMIO((base) + 0x360)
#define GEN12_OAR_OASTATUS _MMIO(0x2968)
/* Gen12 OAG unit */
@@ -134,4 +134,8 @@
#define GDT_CHICKEN_BITS _MMIO(0x9840)
#define GT_NOA_ENABLE 0x00000080
+#define GEN12_SQCNT1 _MMIO(0x8718)
+#define GEN12_SQCNT1_PMON_ENABLE REG_BIT(30)
+#define GEN12_SQCNT1_OABPC REG_BIT(29)
+
#endif /* __INTEL_PERF_OA_REGS__ */
diff --git a/drivers/gpu/drm/i915/i915_perf_types.h b/drivers/gpu/drm/i915/i915_perf_types.h
index 05cb9a3..ca150b7 100644
--- a/drivers/gpu/drm/i915/i915_perf_types.h
+++ b/drivers/gpu/drm/i915/i915_perf_types.h
@@ -147,6 +147,11 @@ struct i915_perf_stream {
struct intel_engine_cs *engine;
/**
+ * @lock: Lock associated with operations on stream
+ */
+ struct mutex lock;
+
+ /**
* @sample_flags: Flags representing the `DRM_I915_PERF_PROP_SAMPLE_*`
* properties given when opening a stream, representing the contents
* of a single sample as read() by userspace.
@@ -245,11 +250,10 @@ struct i915_perf_stream {
* @oa_buffer: State of the OA buffer.
*/
struct {
+ const struct i915_oa_format *format;
struct i915_vma *vma;
u8 *vaddr;
u32 last_ctx_id;
- int format;
- int format_size;
int size_exponent;
/**
@@ -380,6 +384,26 @@ struct i915_oa_ops {
u32 (*oa_hw_tail_read)(struct i915_perf_stream *stream);
};
+struct i915_perf_gt {
+ /*
+ * Lock associated with anything below within this structure.
+ */
+ struct mutex lock;
+
+ /**
+ * @sseu: sseu configuration selected to run while perf is active,
+ * applies to all contexts.
+ */
+ struct intel_sseu sseu;
+
+ /*
+ * @exclusive_stream: The stream currently using the OA unit. This is
+ * sometimes accessed outside a syscall associated to its file
+ * descriptor.
+ */
+ struct i915_perf_stream *exclusive_stream;
+};
+
struct i915_perf {
struct drm_i915_private *i915;
@@ -397,25 +421,6 @@ struct i915_perf {
*/
struct idr metrics_idr;
- /*
- * Lock associated with anything below within this structure
- * except exclusive_stream.
- */
- struct mutex lock;
-
- /*
- * The stream currently using the OA unit. If accessed
- * outside a syscall associated to its file
- * descriptor.
- */
- struct i915_perf_stream *exclusive_stream;
-
- /**
- * @sseu: sseu configuration selected to run while perf is active,
- * applies to all contexts.
- */
- struct intel_sseu sseu;
-
/**
* For rate limiting any notifications of spurious
* invalid OA reports
diff --git a/drivers/gpu/drm/i915/i915_pmu.c b/drivers/gpu/drm/i915/i915_pmu.c
index 958b371..52531ab 100644
--- a/drivers/gpu/drm/i915/i915_pmu.c
+++ b/drivers/gpu/drm/i915/i915_pmu.c
@@ -148,13 +148,13 @@ static u64 __get_rc6(struct intel_gt *gt)
struct drm_i915_private *i915 = gt->i915;
u64 val;
- val = intel_rc6_residency_ns(>->rc6, GEN6_GT_GFX_RC6);
+ val = intel_rc6_residency_ns(>->rc6, INTEL_RC6_RES_RC6);
if (HAS_RC6p(i915))
- val += intel_rc6_residency_ns(>->rc6, GEN6_GT_GFX_RC6p);
+ val += intel_rc6_residency_ns(>->rc6, INTEL_RC6_RES_RC6p);
if (HAS_RC6pp(i915))
- val += intel_rc6_residency_ns(>->rc6, GEN6_GT_GFX_RC6pp);
+ val += intel_rc6_residency_ns(>->rc6, INTEL_RC6_RES_RC6pp);
return val;
}
@@ -371,7 +371,6 @@ static void
frequency_sample(struct intel_gt *gt, unsigned int period_ns)
{
struct drm_i915_private *i915 = gt->i915;
- struct intel_uncore *uncore = gt->uncore;
struct i915_pmu *pmu = &i915->pmu;
struct intel_rps *rps = >->rps;
@@ -394,7 +393,7 @@ frequency_sample(struct intel_gt *gt, unsigned int period_ns)
* case we assume the system is running at the intended
* frequency. Fortunately, the read should rarely fail!
*/
- val = intel_uncore_read_fw(uncore, GEN6_RPSTAT1);
+ val = intel_rps_read_rpstat_fw(rps);
if (val)
val = intel_rps_get_cagf(rps, val);
else
diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c
index 6ec9c9f..00871ef 100644
--- a/drivers/gpu/drm/i915/i915_query.c
+++ b/drivers/gpu/drm/i915/i915_query.c
@@ -250,8 +250,9 @@ static int query_perf_config_data(struct drm_i915_private *i915,
return total_size;
if (query_item->length < total_size) {
- DRM_DEBUG("Invalid query config data item size=%u expected=%u\n",
- query_item->length, total_size);
+ drm_dbg(&i915->drm,
+ "Invalid query config data item size=%u expected=%u\n",
+ query_item->length, total_size);
return -EINVAL;
}
@@ -418,9 +419,10 @@ static int query_perf_config_list(struct drm_i915_private *i915,
} while (n_configs > alloc);
if (query_item->length < sizeof_perf_config_list(n_configs)) {
- DRM_DEBUG("Invalid query config list item size=%u expected=%zu\n",
- query_item->length,
- sizeof_perf_config_list(n_configs));
+ drm_dbg(&i915->drm,
+ "Invalid query config list item size=%u expected=%zu\n",
+ query_item->length,
+ sizeof_perf_config_list(n_configs));
kfree(oa_config_ids);
return -EINVAL;
}
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index da35bb2..8e1892d 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -26,6 +26,7 @@
#define _I915_REG_H_
#include "i915_reg_defs.h"
+#include "display/intel_display_reg_defs.h"
/**
* DOC: The i915 register macro definition style guide
@@ -115,75 +116,6 @@
* #define GEN8_BAR _MMIO(0xb888)
*/
-#define DISPLAY_MMIO_BASE(dev_priv) (INTEL_INFO(dev_priv)->display.mmio_offset)
-
-/*
- * Given the first two numbers __a and __b of arbitrarily many evenly spaced
- * numbers, pick the 0-based __index'th value.
- *
- * Always prefer this over _PICK() if the numbers are evenly spaced.
- */
-#define _PICK_EVEN(__index, __a, __b) ((__a) + (__index) * ((__b) - (__a)))
-
-/*
- * Given the arbitrary numbers in varargs, pick the 0-based __index'th number.
- *
- * Always prefer _PICK_EVEN() over this if the numbers are evenly spaced.
- */
-#define _PICK(__index, ...) (((const u32 []){ __VA_ARGS__ })[__index])
-
-/*
- * Named helper wrappers around _PICK_EVEN() and _PICK().
- */
-#define _PIPE(pipe, a, b) _PICK_EVEN(pipe, a, b)
-#define _PLANE(plane, a, b) _PICK_EVEN(plane, a, b)
-#define _TRANS(tran, a, b) _PICK_EVEN(tran, a, b)
-#define _PORT(port, a, b) _PICK_EVEN(port, a, b)
-#define _PLL(pll, a, b) _PICK_EVEN(pll, a, b)
-#define _PHY(phy, a, b) _PICK_EVEN(phy, a, b)
-
-#define _MMIO_PIPE(pipe, a, b) _MMIO(_PIPE(pipe, a, b))
-#define _MMIO_PLANE(plane, a, b) _MMIO(_PLANE(plane, a, b))
-#define _MMIO_TRANS(tran, a, b) _MMIO(_TRANS(tran, a, b))
-#define _MMIO_PORT(port, a, b) _MMIO(_PORT(port, a, b))
-#define _MMIO_PLL(pll, a, b) _MMIO(_PLL(pll, a, b))
-#define _MMIO_PHY(phy, a, b) _MMIO(_PHY(phy, a, b))
-
-#define _PHY3(phy, ...) _PICK(phy, __VA_ARGS__)
-
-#define _MMIO_PIPE3(pipe, a, b, c) _MMIO(_PICK(pipe, a, b, c))
-#define _MMIO_PORT3(pipe, a, b, c) _MMIO(_PICK(pipe, a, b, c))
-#define _MMIO_PHY3(phy, a, b, c) _MMIO(_PHY3(phy, a, b, c))
-#define _MMIO_PLL3(pll, ...) _MMIO(_PICK(pll, __VA_ARGS__))
-
-
-/*
- * Device info offset array based helpers for groups of registers with unevenly
- * spaced base offsets.
- */
-#define _MMIO_PIPE2(pipe, reg) _MMIO(INTEL_INFO(dev_priv)->display.pipe_offsets[(pipe)] - \
- INTEL_INFO(dev_priv)->display.pipe_offsets[PIPE_A] + \
- DISPLAY_MMIO_BASE(dev_priv) + (reg))
-#define _MMIO_TRANS2(tran, reg) _MMIO(INTEL_INFO(dev_priv)->display.trans_offsets[(tran)] - \
- INTEL_INFO(dev_priv)->display.trans_offsets[TRANSCODER_A] + \
- DISPLAY_MMIO_BASE(dev_priv) + (reg))
-#define _MMIO_CURSOR2(pipe, reg) _MMIO(INTEL_INFO(dev_priv)->display.cursor_offsets[(pipe)] - \
- INTEL_INFO(dev_priv)->display.cursor_offsets[PIPE_A] + \
- DISPLAY_MMIO_BASE(dev_priv) + (reg))
-
-#define __MASKED_FIELD(mask, value) ((mask) << 16 | (value))
-#define _MASKED_FIELD(mask, value) ({ \
- if (__builtin_constant_p(mask)) \
- BUILD_BUG_ON_MSG(((mask) & 0xffff0000), "Incorrect mask"); \
- if (__builtin_constant_p(value)) \
- BUILD_BUG_ON_MSG((value) & 0xffff0000, "Incorrect value"); \
- if (__builtin_constant_p(mask) && __builtin_constant_p(value)) \
- BUILD_BUG_ON_MSG((value) & ~(mask), \
- "Incorrect value for mask"); \
- __MASKED_FIELD(mask, value); })
-#define _MASKED_BIT_ENABLE(a) ({ typeof(a) _a = (a); _MASKED_FIELD(_a, _a); })
-#define _MASKED_BIT_DISABLE(a) (_MASKED_FIELD((a), 0))
-
#define GU_CNTL _MMIO(0x101010)
#define LMEM_INIT REG_BIT(7)
@@ -970,6 +902,7 @@
#define GEN11_VEBOX2_RING_BASE 0x1d8000
#define XEHP_VEBOX3_RING_BASE 0x1e8000
#define XEHP_VEBOX4_RING_BASE 0x1f8000
+#define MTL_GSC_RING_BASE 0x11a000
#define GEN12_COMPUTE0_RING_BASE 0x1a000
#define GEN12_COMPUTE1_RING_BASE 0x1c000
#define GEN12_COMPUTE2_RING_BASE 0x1e000
@@ -1147,11 +1080,6 @@
#define MBUS_JOIN_PIPE_SELECT(pipe) REG_FIELD_PREP(MBUS_JOIN_PIPE_SELECT_MASK, pipe)
#define MBUS_JOIN_PIPE_SELECT_NONE MBUS_JOIN_PIPE_SELECT(7)
-#define HDPORT_STATE _MMIO(0x45050)
-#define HDPORT_DPLL_USED_MASK REG_GENMASK(15, 12)
-#define HDPORT_DDI_USED(phy) REG_BIT(2 * (phy) + 1)
-#define HDPORT_ENABLED REG_BIT(0)
-
/* Make render/texture TLB fetches lower priorty than associated data
* fetches. This is not turned on by default
*/
@@ -1781,9 +1709,10 @@
#define _PALETTE_A 0xa000
#define _PALETTE_B 0xa800
#define _CHV_PALETTE_C 0xc000
-#define PALETTE_RED_MASK REG_GENMASK(23, 16)
-#define PALETTE_GREEN_MASK REG_GENMASK(15, 8)
-#define PALETTE_BLUE_MASK REG_GENMASK(7, 0)
+/* 8bit mode / i965+ 10.6 interpolated mode ldw/udw */
+#define PALETTE_RED_MASK REG_GENMASK(23, 16)
+#define PALETTE_GREEN_MASK REG_GENMASK(15, 8)
+#define PALETTE_BLUE_MASK REG_GENMASK(7, 0)
#define PALETTE(pipe, i) _MMIO(DISPLAY_MMIO_BASE(dev_priv) + \
_PICK((pipe), _PALETTE_A, \
_PALETTE_B, _CHV_PALETTE_C) + \
@@ -1796,6 +1725,15 @@
#define XEHPSDV_RP_STATE_CAP _MMIO(0x250014)
#define PVC_RP_STATE_CAP _MMIO(0x281014)
+#define MTL_RP_STATE_CAP _MMIO(0x138000)
+#define MTL_MEDIAP_STATE_CAP _MMIO(0x138020)
+#define MTL_RP0_CAP_MASK REG_GENMASK(8, 0)
+#define MTL_RPN_CAP_MASK REG_GENMASK(24, 16)
+
+#define MTL_GT_RPE_FREQUENCY _MMIO(0x13800c)
+#define MTL_MPE_FREQUENCY _MMIO(0x13802c)
+#define MTL_RPE_MASK REG_GENMASK(8, 0)
+
#define GT0_PERF_LIMIT_REASONS _MMIO(0x1381a8)
#define GT0_PERF_LIMIT_REASONS_MASK 0xde3
#define PROCHOT_MASK REG_BIT(0)
@@ -1806,6 +1744,8 @@
#define POWER_LIMIT_4_MASK REG_BIT(8)
#define POWER_LIMIT_1_MASK REG_BIT(10)
#define POWER_LIMIT_2_MASK REG_BIT(11)
+#define GT0_PERF_LIMIT_REASONS_LOG_MASK REG_GENMASK(31, 16)
+#define MTL_MEDIA_PERF_LIMIT_REASONS _MMIO(0x138030)
#define CHV_CLK_CTL1 _MMIO(0x101100)
#define VLV_CLK_CTL2 _MMIO(0x101104)
@@ -3747,9 +3687,10 @@
/* Skylake+ pipe bottom (background) color */
#define _SKL_BOTTOM_COLOR_A 0x70034
+#define _SKL_BOTTOM_COLOR_B 0x71034
#define SKL_BOTTOM_COLOR_GAMMA_ENABLE REG_BIT(31)
#define SKL_BOTTOM_COLOR_CSC_ENABLE REG_BIT(30)
-#define SKL_BOTTOM_COLOR(pipe) _MMIO_PIPE2(pipe, _SKL_BOTTOM_COLOR_A)
+#define SKL_BOTTOM_COLOR(pipe) _MMIO_PIPE(pipe, _SKL_BOTTOM_COLOR_A, _SKL_BOTTOM_COLOR_B)
#define _ICL_PIPE_A_STATUS 0x70058
#define ICL_PIPESTATUS(pipe) _MMIO_PIPE2(pipe, _ICL_PIPE_A_STATUS)
@@ -5367,17 +5308,24 @@
/* legacy palette */
#define _LGC_PALETTE_A 0x4a000
#define _LGC_PALETTE_B 0x4a800
-#define LGC_PALETTE_RED_MASK REG_GENMASK(23, 16)
-#define LGC_PALETTE_GREEN_MASK REG_GENMASK(15, 8)
-#define LGC_PALETTE_BLUE_MASK REG_GENMASK(7, 0)
+/* see PALETTE_* for the bits */
#define LGC_PALETTE(pipe, i) _MMIO(_PIPE(pipe, _LGC_PALETTE_A, _LGC_PALETTE_B) + (i) * 4)
/* ilk/snb precision palette */
#define _PREC_PALETTE_A 0x4b000
#define _PREC_PALETTE_B 0x4c000
-#define PREC_PALETTE_RED_MASK REG_GENMASK(29, 20)
-#define PREC_PALETTE_GREEN_MASK REG_GENMASK(19, 10)
-#define PREC_PALETTE_BLUE_MASK REG_GENMASK(9, 0)
+/* 10bit mode */
+#define PREC_PALETTE_10_RED_MASK REG_GENMASK(29, 20)
+#define PREC_PALETTE_10_GREEN_MASK REG_GENMASK(19, 10)
+#define PREC_PALETTE_10_BLUE_MASK REG_GENMASK(9, 0)
+/* 12.4 interpolated mode ldw */
+#define PREC_PALETTE_12P4_RED_LDW_MASK REG_GENMASK(29, 24)
+#define PREC_PALETTE_12P4_GREEN_LDW_MASK REG_GENMASK(19, 14)
+#define PREC_PALETTE_12P4_BLUE_LDW_MASK REG_GENMASK(9, 4)
+/* 12.4 interpolated mode udw */
+#define PREC_PALETTE_12P4_RED_UDW_MASK REG_GENMASK(29, 20)
+#define PREC_PALETTE_12P4_GREEN_UDW_MASK REG_GENMASK(19, 10)
+#define PREC_PALETTE_12P4_BLUE_UDW_MASK REG_GENMASK(9, 0)
#define PREC_PALETTE(pipe, i) _MMIO(_PIPE(pipe, _PREC_PALETTE_A, _PREC_PALETTE_B) + (i) * 4)
#define _PREC_PIPEAGCMAX 0x4d000
@@ -5847,6 +5795,11 @@
#define ICL_DSSM_CDCLK_PLL_REFCLK_19_2MHz (1 << 29)
#define ICL_DSSM_CDCLK_PLL_REFCLK_38_4MHz (2 << 29)
+#define GMD_ID_DISPLAY _MMIO(0x510a0)
+#define GMD_ID_ARCH_MASK REG_GENMASK(31, 22)
+#define GMD_ID_RELEASE_MASK REG_GENMASK(21, 14)
+#define GMD_ID_STEP REG_GENMASK(5, 0)
+
/*GEN11 chicken */
#define _PIPEA_CHICKEN 0x70038
#define _PIPEB_CHICKEN 0x71038
@@ -6647,6 +6600,12 @@
#define DG1_PCODE_STATUS 0x7E
#define DG1_UNCORE_GET_INIT_STATUS 0x0
#define DG1_UNCORE_INIT_STATUS_COMPLETE 0x1
+#define PCODE_POWER_SETUP 0x7C
+#define POWER_SETUP_SUBCOMMAND_READ_I1 0x4
+#define POWER_SETUP_SUBCOMMAND_WRITE_I1 0x5
+#define POWER_SETUP_I1_WATTS REG_BIT(31)
+#define POWER_SETUP_I1_SHIFT 6 /* 10.6 fixed point format */
+#define POWER_SETUP_I1_DATA_MASK REG_GENMASK(15, 0)
#define GEN12_PCODE_READ_SAGV_BLOCK_TIME_US 0x23
#define XEHP_PCODE_FREQUENCY_CONFIG 0x6e /* xehpsdv, pvc */
/* XEHP_PCODE_FREQUENCY_CONFIG sub-commands (param1) */
@@ -6656,6 +6615,15 @@
/* XEHP_PCODE_FREQUENCY_CONFIG param2 */
#define PCODE_MBOX_DOMAIN_NONE 0x0
#define PCODE_MBOX_DOMAIN_MEDIAFF 0x3
+
+/* Wa_14017210380: mtl */
+#define PCODE_MBOX_GT_STATE 0x50
+/* sub-commands (param1) */
+#define PCODE_MBOX_GT_STATE_MEDIA_BUSY 0x1
+#define PCODE_MBOX_GT_STATE_MEDIA_NOT_BUSY 0x2
+/* param2 */
+#define PCODE_MBOX_GT_STATE_DOMAIN_MEDIA 0x1
+
#define GEN6_PCODE_DATA _MMIO(0x138128)
#define GEN6_PCODE_FREQ_IA_RATIO_SHIFT 8
#define GEN6_PCODE_FREQ_RING_RATIO_SHIFT 16
@@ -7413,185 +7381,6 @@ enum skl_power_gate {
_ADLS_DPLL4_CFGCR1, \
_ADLS_DPLL3_CFGCR1)
-#define _DKL_PHY1_BASE 0x168000
-#define _DKL_PHY2_BASE 0x169000
-#define _DKL_PHY3_BASE 0x16A000
-#define _DKL_PHY4_BASE 0x16B000
-#define _DKL_PHY5_BASE 0x16C000
-#define _DKL_PHY6_BASE 0x16D000
-
-#define DKL_REG_TC_PORT(__reg) \
- (TC_PORT_1 + ((__reg).reg - _DKL_PHY1_BASE) / (_DKL_PHY2_BASE - _DKL_PHY1_BASE))
-
-/* DEKEL PHY MMIO Address = Phy base + (internal address & ~index_mask) */
-#define _DKL_PCS_DW5 0x14
-#define DKL_PCS_DW5(tc_port) _MMIO(_PORT(tc_port, _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_PCS_DW5)
-#define DKL_PCS_DW5_CORE_SOFTRESET REG_BIT(11)
-
-#define _DKL_PLL_DIV0 0x200
-#define DKL_PLL_DIV0_AFC_STARTUP_MASK REG_GENMASK(27, 25)
-#define DKL_PLL_DIV0_AFC_STARTUP(val) REG_FIELD_PREP(DKL_PLL_DIV0_AFC_STARTUP_MASK, (val))
-#define DKL_PLL_DIV0_INTEG_COEFF(x) ((x) << 16)
-#define DKL_PLL_DIV0_INTEG_COEFF_MASK (0x1F << 16)
-#define DKL_PLL_DIV0_PROP_COEFF(x) ((x) << 12)
-#define DKL_PLL_DIV0_PROP_COEFF_MASK (0xF << 12)
-#define DKL_PLL_DIV0_FBPREDIV_SHIFT (8)
-#define DKL_PLL_DIV0_FBPREDIV(x) ((x) << DKL_PLL_DIV0_FBPREDIV_SHIFT)
-#define DKL_PLL_DIV0_FBPREDIV_MASK (0xF << DKL_PLL_DIV0_FBPREDIV_SHIFT)
-#define DKL_PLL_DIV0_FBDIV_INT(x) ((x) << 0)
-#define DKL_PLL_DIV0_FBDIV_INT_MASK (0xFF << 0)
-#define DKL_PLL_DIV0_MASK (DKL_PLL_DIV0_INTEG_COEFF_MASK | \
- DKL_PLL_DIV0_PROP_COEFF_MASK | \
- DKL_PLL_DIV0_FBPREDIV_MASK | \
- DKL_PLL_DIV0_FBDIV_INT_MASK)
-#define DKL_PLL_DIV0(tc_port) _MMIO(_PORT(tc_port, _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_PLL_DIV0)
-
-#define _DKL_PLL_DIV1 0x204
-#define DKL_PLL_DIV1_IREF_TRIM(x) ((x) << 16)
-#define DKL_PLL_DIV1_IREF_TRIM_MASK (0x1F << 16)
-#define DKL_PLL_DIV1_TDC_TARGET_CNT(x) ((x) << 0)
-#define DKL_PLL_DIV1_TDC_TARGET_CNT_MASK (0xFF << 0)
-#define DKL_PLL_DIV1(tc_port) _MMIO(_PORT(tc_port, _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_PLL_DIV1)
-
-#define _DKL_PLL_SSC 0x210
-#define DKL_PLL_SSC_IREF_NDIV_RATIO(x) ((x) << 29)
-#define DKL_PLL_SSC_IREF_NDIV_RATIO_MASK (0x7 << 29)
-#define DKL_PLL_SSC_STEP_LEN(x) ((x) << 16)
-#define DKL_PLL_SSC_STEP_LEN_MASK (0xFF << 16)
-#define DKL_PLL_SSC_STEP_NUM(x) ((x) << 11)
-#define DKL_PLL_SSC_STEP_NUM_MASK (0x7 << 11)
-#define DKL_PLL_SSC_EN (1 << 9)
-#define DKL_PLL_SSC(tc_port) _MMIO(_PORT(tc_port, _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_PLL_SSC)
-
-#define _DKL_PLL_BIAS 0x214
-#define DKL_PLL_BIAS_FRAC_EN_H (1 << 30)
-#define DKL_PLL_BIAS_FBDIV_SHIFT (8)
-#define DKL_PLL_BIAS_FBDIV_FRAC(x) ((x) << DKL_PLL_BIAS_FBDIV_SHIFT)
-#define DKL_PLL_BIAS_FBDIV_FRAC_MASK (0x3FFFFF << DKL_PLL_BIAS_FBDIV_SHIFT)
-#define DKL_PLL_BIAS(tc_port) _MMIO(_PORT(tc_port, _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_PLL_BIAS)
-
-#define _DKL_PLL_TDC_COLDST_BIAS 0x218
-#define DKL_PLL_TDC_SSC_STEP_SIZE(x) ((x) << 8)
-#define DKL_PLL_TDC_SSC_STEP_SIZE_MASK (0xFF << 8)
-#define DKL_PLL_TDC_FEED_FWD_GAIN(x) ((x) << 0)
-#define DKL_PLL_TDC_FEED_FWD_GAIN_MASK (0xFF << 0)
-#define DKL_PLL_TDC_COLDST_BIAS(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_PLL_TDC_COLDST_BIAS)
-
-#define _DKL_REFCLKIN_CTL 0x12C
-/* Bits are the same as MG_REFCLKIN_CTL */
-#define DKL_REFCLKIN_CTL(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_REFCLKIN_CTL)
-
-#define _DKL_CLKTOP2_HSCLKCTL 0xD4
-/* Bits are the same as MG_CLKTOP2_HSCLKCTL */
-#define DKL_CLKTOP2_HSCLKCTL(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_CLKTOP2_HSCLKCTL)
-
-#define _DKL_CLKTOP2_CORECLKCTL1 0xD8
-/* Bits are the same as MG_CLKTOP2_CORECLKCTL1 */
-#define DKL_CLKTOP2_CORECLKCTL1(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_CLKTOP2_CORECLKCTL1)
-
-#define _DKL_TX_DPCNTL0 0x2C0
-#define DKL_TX_PRESHOOT_COEFF(x) ((x) << 13)
-#define DKL_TX_PRESHOOT_COEFF_MASK (0x1f << 13)
-#define DKL_TX_DE_EMPHASIS_COEFF(x) ((x) << 8)
-#define DKL_TX_DE_EMPAHSIS_COEFF_MASK (0x1f << 8)
-#define DKL_TX_VSWING_CONTROL(x) ((x) << 0)
-#define DKL_TX_VSWING_CONTROL_MASK (0x7 << 0)
-#define DKL_TX_DPCNTL0(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_TX_DPCNTL0)
-
-#define _DKL_TX_DPCNTL1 0x2C4
-/* Bits are the same as DKL_TX_DPCNTRL0 */
-#define DKL_TX_DPCNTL1(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_TX_DPCNTL1)
-
-#define _DKL_TX_DPCNTL2 0x2C8
-#define DKL_TX_DP20BITMODE REG_BIT(2)
-#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX1_MASK REG_GENMASK(4, 3)
-#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX1(val) REG_FIELD_PREP(DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX1_MASK, (val))
-#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2_MASK REG_GENMASK(6, 5)
-#define DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2(val) REG_FIELD_PREP(DKL_TX_DPCNTL2_CFG_LOADGENSELECT_TX2_MASK, (val))
-#define DKL_TX_DPCNTL2(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_TX_DPCNTL2)
-
-#define _DKL_TX_FW_CALIB 0x2F8
-#define DKL_TX_CFG_DISABLE_WAIT_INIT (1 << 7)
-#define DKL_TX_FW_CALIB(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_TX_FW_CALIB)
-
-#define _DKL_TX_PMD_LANE_SUS 0xD00
-#define DKL_TX_PMD_LANE_SUS(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_TX_PMD_LANE_SUS)
-
-#define _DKL_TX_DW17 0xDC4
-#define DKL_TX_DW17(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_TX_DW17)
-
-#define _DKL_TX_DW18 0xDC8
-#define DKL_TX_DW18(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_TX_DW18)
-
-#define _DKL_DP_MODE 0xA0
-#define DKL_DP_MODE(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_DP_MODE)
-
-#define _DKL_CMN_UC_DW27 0x36C
-#define DKL_CMN_UC_DW27_UC_HEALTH (0x1 << 15)
-#define DKL_CMN_UC_DW_27(tc_port) _MMIO(_PORT(tc_port, \
- _DKL_PHY1_BASE, \
- _DKL_PHY2_BASE) + \
- _DKL_CMN_UC_DW27)
-
-/*
- * Each Dekel PHY is addressed through a 4KB aperture. Each PHY has more than
- * 4KB of register space, so a separate index is programmed in HIP_INDEX_REG0
- * or HIP_INDEX_REG1, based on the port number, to set the upper 2 address
- * bits that point the 4KB window into the full PHY register space.
- */
-#define _HIP_INDEX_REG0 0x1010A0
-#define _HIP_INDEX_REG1 0x1010A4
-#define HIP_INDEX_REG(tc_port) _MMIO((tc_port) < 4 ? _HIP_INDEX_REG0 \
- : _HIP_INDEX_REG1)
-#define _HIP_INDEX_SHIFT(tc_port) (8 * ((tc_port) % 4))
-#define HIP_INDEX_VAL(tc_port, val) ((val) << _HIP_INDEX_SHIFT(tc_port))
-
/* BXT display engine PLL */
#define BXT_DE_PLL_CTL _MMIO(0x6d000)
#define BXT_DE_PLL_RATIO(x) (x) /* {60,65,100} * 19.2MHz */
@@ -7775,12 +7564,10 @@ enum skl_power_gate {
#define _PAL_PREC_DATA_A 0x4A404
#define _PAL_PREC_DATA_B 0x4AC04
#define _PAL_PREC_DATA_C 0x4B404
+/* see PREC_PALETTE_* for the bits */
#define _PAL_PREC_GC_MAX_A 0x4A410
#define _PAL_PREC_GC_MAX_B 0x4AC10
#define _PAL_PREC_GC_MAX_C 0x4B410
-#define PREC_PAL_DATA_RED_MASK REG_GENMASK(29, 20)
-#define PREC_PAL_DATA_GREEN_MASK REG_GENMASK(19, 10)
-#define PREC_PAL_DATA_BLUE_MASK REG_GENMASK(9, 0)
#define _PAL_PREC_EXT_GC_MAX_A 0x4A420
#define _PAL_PREC_EXT_GC_MAX_B 0x4AC20
#define _PAL_PREC_EXT_GC_MAX_C 0x4B420
@@ -7813,12 +7600,7 @@ enum skl_power_gate {
#define _PAL_PREC_MULTI_SEG_DATA_A 0x4A40C
#define _PAL_PREC_MULTI_SEG_DATA_B 0x4AC0C
-#define PAL_PREC_MULTI_SEG_RED_LDW_MASK REG_GENMASK(29, 24)
-#define PAL_PREC_MULTI_SEG_RED_UDW_MASK REG_GENMASK(29, 20)
-#define PAL_PREC_MULTI_SEG_GREEN_LDW_MASK REG_GENMASK(19, 14)
-#define PAL_PREC_MULTI_SEG_GREEN_UDW_MASK REG_GENMASK(19, 10)
-#define PAL_PREC_MULTI_SEG_BLUE_LDW_MASK REG_GENMASK(9, 4)
-#define PAL_PREC_MULTI_SEG_BLUE_UDW_MASK REG_GENMASK(9, 0)
+/* see PREC_PALETTE_12P4_* for the bits */
#define PREC_PAL_MULTI_SEG_INDEX(pipe) _MMIO_PIPE(pipe, \
_PAL_PREC_MULTI_SEG_INDEX_A, \
@@ -7879,13 +7661,17 @@ enum skl_power_gate {
#define _CGM_PIPE_A_CSC_COEFF67 (VLV_DISPLAY_BASE + 0x6790C)
#define _CGM_PIPE_A_CSC_COEFF8 (VLV_DISPLAY_BASE + 0x67910)
#define _CGM_PIPE_A_DEGAMMA (VLV_DISPLAY_BASE + 0x66000)
-#define CGM_PIPE_DEGAMMA_RED_MASK REG_GENMASK(13, 0)
-#define CGM_PIPE_DEGAMMA_GREEN_MASK REG_GENMASK(29, 16)
-#define CGM_PIPE_DEGAMMA_BLUE_MASK REG_GENMASK(13, 0)
+/* cgm degamma ldw */
+#define CGM_PIPE_DEGAMMA_GREEN_LDW_MASK REG_GENMASK(29, 16)
+#define CGM_PIPE_DEGAMMA_BLUE_LDW_MASK REG_GENMASK(13, 0)
+/* cgm degamma udw */
+#define CGM_PIPE_DEGAMMA_RED_UDW_MASK REG_GENMASK(13, 0)
#define _CGM_PIPE_A_GAMMA (VLV_DISPLAY_BASE + 0x67000)
-#define CGM_PIPE_GAMMA_RED_MASK REG_GENMASK(9, 0)
-#define CGM_PIPE_GAMMA_GREEN_MASK REG_GENMASK(25, 16)
-#define CGM_PIPE_GAMMA_BLUE_MASK REG_GENMASK(9, 0)
+/* cgm gamma ldw */
+#define CGM_PIPE_GAMMA_GREEN_LDW_MASK REG_GENMASK(25, 16)
+#define CGM_PIPE_GAMMA_BLUE_LDW_MASK REG_GENMASK(9, 0)
+/* cgm gamma udw */
+#define CGM_PIPE_GAMMA_RED_UDW_MASK REG_GENMASK(9, 0)
#define _CGM_PIPE_A_MODE (VLV_DISPLAY_BASE + 0x67A00)
#define CGM_PIPE_MODE_GAMMA (1 << 2)
#define CGM_PIPE_MODE_CSC (1 << 1)
@@ -7962,8 +7748,13 @@ enum skl_power_gate {
_ICL_PIPE_DSS_CTL2_PB, \
_ICL_PIPE_DSS_CTL2_PC)
+#define GGC _MMIO(0x108040)
+#define GMS_MASK REG_GENMASK(15, 8)
+#define GGMS_MASK REG_GENMASK(7, 6)
+
#define GEN12_GSMBASE _MMIO(0x108100)
#define GEN12_DSMBASE _MMIO(0x1080C0)
+#define GEN12_BDSM_MASK REG_GENMASK64(63, 20)
#define XEHP_CLOCK_GATE_DIS _MMIO(0x101014)
#define SGSI_SIDECLK_DIS REG_BIT(17)
@@ -8344,6 +8135,11 @@ enum skl_power_gate {
#define GEN12_CULLBIT2 _MMIO(0x7030)
#define GEN12_STATE_ACK_DEBUG _MMIO(0x20BC)
+#define _MTL_CLKGATE_DIS_TRANS_A 0x604E8
+#define _MTL_CLKGATE_DIS_TRANS_B 0x614E8
+#define MTL_CLKGATE_DIS_TRANS(trans) _MMIO_TRANS2(trans, _MTL_CLKGATE_DIS_TRANS_A)
+#define MTL_CLKGATE_DIS_TRANS_DMASC_GATING_DIS REG_BIT(7)
+
#define MTL_LATENCY_LP0_LP1 _MMIO(0x45780)
#define MTL_LATENCY_LP2_LP3 _MMIO(0x45784)
#define MTL_LATENCY_LP4_LP5 _MMIO(0x45788)
@@ -8367,4 +8163,6 @@ enum skl_power_gate {
#define MTL_TRAS_MASK REG_GENMASK(16, 8)
#define MTL_TRDPRE_MASK REG_GENMASK(7, 0)
+#define MTL_MEDIA_GSI_BASE 0x380000
+
#endif /* _I915_REG_H_ */
diff --git a/drivers/gpu/drm/i915/i915_reg_defs.h b/drivers/gpu/drm/i915/i915_reg_defs.h
index 8f486f7..be43580 100644
--- a/drivers/gpu/drm/i915/i915_reg_defs.h
+++ b/drivers/gpu/drm/i915/i915_reg_defs.h
@@ -98,29 +98,54 @@
*/
#define REG_FIELD_GET64(__mask, __val) ((u64)FIELD_GET(__mask, __val))
+#define __MASKED_FIELD(mask, value) ((mask) << 16 | (value))
+#define _MASKED_FIELD(mask, value) ({ \
+ if (__builtin_constant_p(mask)) \
+ BUILD_BUG_ON_MSG(((mask) & 0xffff0000), "Incorrect mask"); \
+ if (__builtin_constant_p(value)) \
+ BUILD_BUG_ON_MSG((value) & 0xffff0000, "Incorrect value"); \
+ if (__builtin_constant_p(mask) && __builtin_constant_p(value)) \
+ BUILD_BUG_ON_MSG((value) & ~(mask), \
+ "Incorrect value for mask"); \
+ __MASKED_FIELD(mask, value); })
+#define _MASKED_BIT_ENABLE(a) ({ typeof(a) _a = (a); _MASKED_FIELD(_a, _a); })
+#define _MASKED_BIT_DISABLE(a) (_MASKED_FIELD((a), 0))
+
+/*
+ * Given the first two numbers __a and __b of arbitrarily many evenly spaced
+ * numbers, pick the 0-based __index'th value.
+ *
+ * Always prefer this over _PICK() if the numbers are evenly spaced.
+ */
+#define _PICK_EVEN(__index, __a, __b) ((__a) + (__index) * ((__b) - (__a)))
+
+/*
+ * Given the arbitrary numbers in varargs, pick the 0-based __index'th number.
+ *
+ * Always prefer _PICK_EVEN() over this if the numbers are evenly spaced.
+ */
+#define _PICK(__index, ...) (((const u32 []){ __VA_ARGS__ })[__index])
+
typedef struct {
u32 reg;
} i915_reg_t;
#define _MMIO(r) ((const i915_reg_t){ .reg = (r) })
+typedef struct {
+ u32 reg;
+} i915_mcr_reg_t;
+
#define INVALID_MMIO_REG _MMIO(0)
-static __always_inline u32 i915_mmio_reg_offset(i915_reg_t reg)
-{
- return reg.reg;
-}
-
-static inline bool i915_mmio_reg_equal(i915_reg_t a, i915_reg_t b)
-{
- return i915_mmio_reg_offset(a) == i915_mmio_reg_offset(b);
-}
-
-static inline bool i915_mmio_reg_valid(i915_reg_t reg)
-{
- return !i915_mmio_reg_equal(reg, INVALID_MMIO_REG);
-}
-
-#define VLV_DISPLAY_BASE 0x180000
+/*
+ * These macros can be used on either i915_reg_t or i915_mcr_reg_t since they're
+ * simply operations on the register's offset and don't care about the MCR vs
+ * non-MCR nature of the register.
+ */
+#define i915_mmio_reg_offset(r) \
+ _Generic((r), i915_reg_t: (r).reg, i915_mcr_reg_t: (r).reg)
+#define i915_mmio_reg_equal(a, b) (i915_mmio_reg_offset(a) == i915_mmio_reg_offset(b))
+#define i915_mmio_reg_valid(r) (!i915_mmio_reg_equal(r, INVALID_MMIO_REG))
#endif /* __I915_REG_DEFS__ */
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 62fad16..f949a94 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -1621,6 +1621,20 @@ i915_request_await_object(struct i915_request *to,
return ret;
}
+static void i915_request_await_huc(struct i915_request *rq)
+{
+ struct intel_huc *huc = &rq->context->engine->gt->uc.huc;
+
+ /* don't stall kernel submissions! */
+ if (!rcu_access_pointer(rq->context->gem_context))
+ return;
+
+ if (intel_huc_wait_required(huc))
+ i915_sw_fence_await_sw_fence(&rq->submit,
+ &huc->delayed_load.fence,
+ &rq->hucq);
+}
+
static struct i915_request *
__i915_request_ensure_parallel_ordering(struct i915_request *rq,
struct intel_timeline *timeline)
@@ -1703,6 +1717,16 @@ __i915_request_add_to_timeline(struct i915_request *rq)
struct i915_request *prev;
/*
+ * Media workloads may require HuC, so stall them until HuC loading is
+ * complete. Note that HuC not being loaded when a user submission
+ * arrives can only happen when HuC is loaded via GSC and in that case
+ * we still expect the window between us starting to accept submissions
+ * and HuC loading completion to be small (a few hundred ms).
+ */
+ if (rq->engine->class == VIDEO_DECODE_CLASS)
+ i915_request_await_huc(rq);
+
+ /*
* Dependency tracking and request ordering along the timeline
* is special cased so that we can eliminate redundant ordering
* operations while building the request (we know that the timeline
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 47041ec..f5e1bb5e8 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -348,6 +348,11 @@ struct i915_request {
#define GUC_PRIO_FINI 0xfe
u8 guc_prio;
+ /**
+ * @hucq: wait queue entry used to wait on the HuC load to complete
+ */
+ wait_queue_entry_t hucq;
+
I915_SELFTEST_DECLARE(struct {
struct list_head link;
unsigned long delay;
diff --git a/drivers/gpu/drm/i915/i915_scatterlist.c b/drivers/gpu/drm/i915/i915_scatterlist.c
index dcc0818..114e5e3 100644
--- a/drivers/gpu/drm/i915/i915_scatterlist.c
+++ b/drivers/gpu/drm/i915/i915_scatterlist.c
@@ -158,7 +158,7 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
u32 page_alignment)
{
struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
- const u64 size = res->num_pages << PAGE_SHIFT;
+ const u64 size = res->size;
const u32 max_segment = round_down(UINT_MAX, page_alignment);
struct drm_buddy *mm = bman_res->mm;
struct list_head *blocks = &bman_res->blocks;
@@ -177,7 +177,7 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
i915_refct_sgt_init(rsgt, size);
st = &rsgt->table;
- if (sg_alloc_table(st, res->num_pages, GFP_KERNEL)) {
+ if (sg_alloc_table(st, PFN_UP(res->size), GFP_KERNEL)) {
i915_refct_sgt_put(rsgt);
return ERR_PTR(-ENOMEM);
}
diff --git a/drivers/gpu/drm/i915/i915_selftest.h b/drivers/gpu/drm/i915/i915_selftest.h
index f54de04..bdf3e22 100644
--- a/drivers/gpu/drm/i915/i915_selftest.h
+++ b/drivers/gpu/drm/i915/i915_selftest.h
@@ -92,12 +92,14 @@ int __i915_subtests(const char *caller,
T, ARRAY_SIZE(T), data)
#define i915_live_subtests(T, data) ({ \
typecheck(struct drm_i915_private *, data); \
+ (data)->gt[0]->uc.guc.submission_state.sched_disable_delay_ms = 0; \
__i915_subtests(__func__, \
__i915_live_setup, __i915_live_teardown, \
T, ARRAY_SIZE(T), data); \
})
#define intel_gt_live_subtests(T, data) ({ \
typecheck(struct intel_gt *, data); \
+ (data)->uc.guc.submission_state.sched_disable_delay_ms = 0; \
__i915_subtests(__func__, \
__intel_gt_live_setup, __intel_gt_live_teardown, \
T, ARRAY_SIZE(T), data); \
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.c b/drivers/gpu/drm/i915/i915_sw_fence.c
index 6fc0d1b..cc2a882 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.c
+++ b/drivers/gpu/drm/i915/i915_sw_fence.c
@@ -571,7 +571,6 @@ int __i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence,
int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
struct dma_resv *resv,
- const struct dma_fence_ops *exclude,
bool write,
unsigned long timeout,
gfp_t gfp)
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.h b/drivers/gpu/drm/i915/i915_sw_fence.h
index 619fc5a..f752bfc 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.h
+++ b/drivers/gpu/drm/i915/i915_sw_fence.h
@@ -91,7 +91,6 @@ int i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence,
int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
struct dma_resv *resv,
- const struct dma_fence_ops *exclude,
bool write,
unsigned long timeout,
gfp_t gfp);
diff --git a/drivers/gpu/drm/i915/i915_sysfs.c b/drivers/gpu/drm/i915/i915_sysfs.c
index 1e27502..595e8b5 100644
--- a/drivers/gpu/drm/i915/i915_sysfs.c
+++ b/drivers/gpu/drm/i915/i915_sysfs.c
@@ -218,7 +218,8 @@ static const struct bin_attribute error_state_attr = {
static void i915_setup_error_capture(struct device *kdev)
{
if (sysfs_create_bin_file(&kdev->kobj, &error_state_attr))
- DRM_ERROR("error_state sysfs setup failed\n");
+ drm_err(&kdev_minor_to_i915(kdev)->drm,
+ "error_state sysfs setup failed\n");
}
static void i915_teardown_error_capture(struct device *kdev)
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index 37b5c9e..f6f9228 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -15,7 +15,6 @@
#include "gt/intel_engine.h"
#include "i915_drv.h"
-#include "i915_irq.h"
/* object tracking */
@@ -671,21 +670,6 @@ TRACE_EVENT_CONDITION(i915_reg_rw,
(u32)(__entry->val >> 32))
);
-TRACE_EVENT(intel_gpu_freq_change,
- TP_PROTO(u32 freq),
- TP_ARGS(freq),
-
- TP_STRUCT__entry(
- __field(u32, freq)
- ),
-
- TP_fast_assign(
- __entry->freq = freq;
- ),
-
- TP_printk("new_freq=%u", __entry->freq)
-);
-
/**
* DOC: i915_ppgtt_create and i915_ppgtt_release tracepoints
*
diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
index e19452f..7e61147 100644
--- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
+++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
@@ -62,8 +62,8 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
if (place->fpfn || lpfn != man->size)
bman_res->flags |= DRM_BUDDY_RANGE_ALLOCATION;
- GEM_BUG_ON(!bman_res->base.num_pages);
- size = bman_res->base.num_pages << PAGE_SHIFT;
+ GEM_BUG_ON(!bman_res->base.size);
+ size = bman_res->base.size;
min_page_size = bman->default_page_size;
if (bo->page_alignment)
@@ -72,7 +72,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
GEM_BUG_ON(min_page_size < mm->chunk_size);
GEM_BUG_ON(!IS_ALIGNED(size, min_page_size));
- if (place->fpfn + bman_res->base.num_pages != place->lpfn &&
+ if (place->fpfn + PFN_UP(bman_res->base.size) != place->lpfn &&
place->flags & TTM_PL_FLAG_CONTIGUOUS) {
unsigned long pages;
@@ -108,7 +108,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
goto err_free_blocks;
if (place->flags & TTM_PL_FLAG_CONTIGUOUS) {
- u64 original_size = (u64)bman_res->base.num_pages << PAGE_SHIFT;
+ u64 original_size = (u64)bman_res->base.size;
drm_buddy_block_trim(mm,
original_size,
@@ -116,7 +116,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
}
if (lpfn <= bman->visible_size) {
- bman_res->used_visible_size = bman_res->base.num_pages;
+ bman_res->used_visible_size = PFN_UP(bman_res->base.size);
} else {
struct drm_buddy_block *block;
@@ -228,7 +228,7 @@ static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man,
if (!place->fpfn &&
place->lpfn == i915_ttm_buddy_man_visible_size(man))
- return bman_res->used_visible_size == res->num_pages;
+ return bman_res->used_visible_size == PFN_UP(res->size);
/* Check each drm buddy block individually */
list_for_each_entry(block, &bman_res->blocks, link) {
diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index f17c09ea..703fee6 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -73,14 +73,16 @@ static void vma_print_allocator(struct i915_vma *vma, const char *reason)
char buf[512];
if (!vma->node.stack) {
- DRM_DEBUG_DRIVER("vma.node [%08llx + %08llx] %s: unknown owner\n",
- vma->node.start, vma->node.size, reason);
+ drm_dbg(&to_i915(vma->obj->base.dev)->drm,
+ "vma.node [%08llx + %08llx] %s: unknown owner\n",
+ vma->node.start, vma->node.size, reason);
return;
}
stack_depot_snprint(vma->node.stack, buf, sizeof(buf), 0);
- DRM_DEBUG_DRIVER("vma.node [%08llx + %08llx] %s: inserted at %s\n",
- vma->node.start, vma->node.size, reason, buf);
+ drm_dbg(&to_i915(vma->obj->base.dev)->drm,
+ "vma.node [%08llx + %08llx] %s: inserted at %s\n",
+ vma->node.start, vma->node.size, reason, buf);
}
#else
@@ -776,21 +778,15 @@ i915_vma_insert(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
GEM_BUG_ON(!IS_ALIGNED(end, I915_GTT_PAGE_SIZE));
alignment = max(alignment, i915_vm_obj_min_alignment(vma->vm, vma->obj));
- /*
- * for compact-pt we round up the reservation to prevent
- * any smaller pages being used within the same PDE
- */
- if (NEEDS_COMPACT_PT(vma->vm->i915))
- size = round_up(size, alignment);
/* If binding the object/GGTT view requires more space than the entire
* aperture has, reject it early before evicting everything in a vain
* attempt to find space.
*/
if (size > end) {
- DRM_DEBUG("Attempting to bind an object larger than the aperture: request=%llu > %s aperture=%llu\n",
- size, flags & PIN_MAPPABLE ? "mappable" : "total",
- end);
+ drm_dbg(&to_i915(vma->obj->base.dev)->drm,
+ "Attempting to bind an object larger than the aperture: request=%llu > %s aperture=%llu\n",
+ size, flags & PIN_MAPPABLE ? "mappable" : "total", end);
return -ENOSPC;
}
@@ -820,7 +816,8 @@ i915_vma_insert(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
* forseeable future. See also i915_ggtt_offset().
*/
if (upper_32_bits(end - 1) &&
- vma->page_sizes.sg > I915_GTT_PAGE_SIZE) {
+ vma->page_sizes.sg > I915_GTT_PAGE_SIZE &&
+ !HAS_64K_PAGES(vma->vm->i915)) {
/*
* We can't mix 64K and 4K PTEs in the same page-table
* (2M block), and so to avoid the ugliness and
@@ -1847,6 +1844,11 @@ int _i915_vma_move_to_active(struct i915_vma *vma,
GEM_BUG_ON(!vma->pages);
+ if (!(flags & __EXEC_OBJECT_NO_REQUEST_AWAIT)) {
+ err = i915_request_await_object(rq, vma->obj, flags & EXEC_OBJECT_WRITE);
+ if (unlikely(err))
+ return err;
+ }
err = __i915_vma_move_to_active(vma, rq);
if (unlikely(err))
return err;
diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
index aecd9c6..0757977 100644
--- a/drivers/gpu/drm/i915/i915_vma.h
+++ b/drivers/gpu/drm/i915/i915_vma.h
@@ -55,6 +55,7 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)
/* do not reserve memory to prevent deadlocks */
#define __EXEC_OBJECT_NO_RESERVE BIT(31)
+#define __EXEC_OBJECT_NO_REQUEST_AWAIT BIT(30)
int __must_check _i915_vma_move_to_active(struct i915_vma *vma,
struct i915_request *rq,
diff --git a/drivers/gpu/drm/i915/intel_device_info.c b/drivers/gpu/drm/i915/intel_device_info.c
index 20575eb..849baf6 100644
--- a/drivers/gpu/drm/i915/intel_device_info.c
+++ b/drivers/gpu/drm/i915/intel_device_info.c
@@ -29,9 +29,11 @@
#include "display/intel_cdclk.h"
#include "display/intel_de.h"
-#include "intel_device_info.h"
+#include "gt/intel_gt_regs.h"
#include "i915_drv.h"
+#include "i915_reg.h"
#include "i915_utils.h"
+#include "intel_device_info.h"
#define PLATFORM_NAME(x) [INTEL_##x] = #x
static const char * const platform_names[] = {
@@ -231,7 +233,7 @@ static bool find_devid(u16 id, const u16 *p, unsigned int num)
return false;
}
-void intel_device_info_subplatform_init(struct drm_i915_private *i915)
+static void intel_device_info_subplatform_init(struct drm_i915_private *i915)
{
const struct intel_device_info *info = INTEL_INFO(i915);
const struct intel_runtime_info *rinfo = RUNTIME_INFO(i915);
@@ -288,6 +290,78 @@ void intel_device_info_subplatform_init(struct drm_i915_private *i915)
RUNTIME_INFO(i915)->platform_mask[pi] |= mask;
}
+static void ip_ver_read(struct drm_i915_private *i915, u32 offset, struct intel_ip_version *ip)
+{
+ struct pci_dev *pdev = to_pci_dev(i915->drm.dev);
+ void __iomem *addr;
+ u32 val;
+ u8 expected_ver = ip->ver;
+ u8 expected_rel = ip->rel;
+
+ addr = pci_iomap_range(pdev, 0, offset, sizeof(u32));
+ if (drm_WARN_ON(&i915->drm, !addr))
+ return;
+
+ val = ioread32(addr);
+ pci_iounmap(pdev, addr);
+
+ ip->ver = REG_FIELD_GET(GMD_ID_ARCH_MASK, val);
+ ip->rel = REG_FIELD_GET(GMD_ID_RELEASE_MASK, val);
+ ip->step = REG_FIELD_GET(GMD_ID_STEP, val);
+
+ /* Sanity check against expected versions from device info */
+ if (IP_VER(ip->ver, ip->rel) < IP_VER(expected_ver, expected_rel))
+ drm_dbg(&i915->drm,
+ "Hardware reports GMD IP version %u.%u (REG[0x%x] = 0x%08x) but minimum expected is %u.%u\n",
+ ip->ver, ip->rel, offset, val, expected_ver, expected_rel);
+}
+
+/*
+ * Setup the graphics version for the current device. This must be done before
+ * any code that performs checks on GRAPHICS_VER or DISPLAY_VER, so this
+ * function should be called very early in the driver initialization sequence.
+ *
+ * Regular MMIO access is not yet setup at the point this function is called so
+ * we peek at the appropriate MMIO offset directly. The GMD_ID register is
+ * part of an 'always on' power well by design, so we don't need to worry about
+ * forcewake while reading it.
+ */
+static void intel_ipver_early_init(struct drm_i915_private *i915)
+{
+ struct intel_runtime_info *runtime = RUNTIME_INFO(i915);
+
+ if (!HAS_GMD_ID(i915)) {
+ drm_WARN_ON(&i915->drm, RUNTIME_INFO(i915)->graphics.ip.ver > 12);
+ /*
+ * On older platforms, graphics and media share the same ip
+ * version and release.
+ */
+ RUNTIME_INFO(i915)->media.ip =
+ RUNTIME_INFO(i915)->graphics.ip;
+ return;
+ }
+
+ ip_ver_read(i915, i915_mmio_reg_offset(GMD_ID_GRAPHICS),
+ &runtime->graphics.ip);
+ ip_ver_read(i915, i915_mmio_reg_offset(GMD_ID_DISPLAY),
+ &runtime->display.ip);
+ ip_ver_read(i915, i915_mmio_reg_offset(GMD_ID_MEDIA),
+ &runtime->media.ip);
+}
+
+/**
+ * intel_device_info_runtime_init_early - initialize early runtime info
+ * @i915: the i915 device
+ *
+ * Determine early intel_device_info fields at runtime. This function needs
+ * to be called before the MMIO has been setup.
+ */
+void intel_device_info_runtime_init_early(struct drm_i915_private *i915)
+{
+ intel_ipver_early_init(i915);
+ intel_device_info_subplatform_init(i915);
+}
+
/**
* intel_device_info_runtime_init - initialize runtime info
* @dev_priv: the i915 device
@@ -415,7 +489,7 @@ void intel_device_info_runtime_init(struct drm_i915_private *dev_priv)
if (DISPLAY_VER(dev_priv) >= 11 && (dfsm & ICL_DFSM_DMC_DISABLE))
runtime->has_dmc = 0;
- if (DISPLAY_VER(dev_priv) >= 10 &&
+ if (IS_DISPLAY_VER(dev_priv, 10, 12) &&
(dfsm & GLK_DFSM_DISPLAY_DSC_DISABLE))
runtime->has_dsc = 0;
}
@@ -442,6 +516,11 @@ void intel_device_info_runtime_init(struct drm_i915_private *dev_priv)
runtime->has_dmc = false;
runtime->has_dsc = false;
}
+
+ /* Disable nuclear pageflip by default on pre-g4x */
+ if (!dev_priv->params.nuclear_pageflip &&
+ DISPLAY_VER(dev_priv) < 5 && !IS_G4X(dev_priv))
+ dev_priv->drm.driver_features &= ~DRIVER_ATOMIC;
}
void intel_driver_caps_print(const struct intel_driver_caps *caps,
diff --git a/drivers/gpu/drm/i915/intel_device_info.h b/drivers/gpu/drm/i915/intel_device_info.h
index d638235..d588e5f 100644
--- a/drivers/gpu/drm/i915/intel_device_info.h
+++ b/drivers/gpu/drm/i915/intel_device_info.h
@@ -146,13 +146,13 @@ enum intel_ppgtt_type {
/* Keep has_* in alphabetical order */ \
func(has_64bit_reloc); \
func(has_64k_pages); \
- func(needs_compact_pt); \
func(gpu_reset_clobbers_display); \
func(has_reset_engine); \
func(has_3d_pipeline); \
func(has_4tile); \
func(has_flat_ccs); \
func(has_global_mocs); \
+ func(has_gmd_id); \
func(has_gt_uc); \
func(has_heci_pxp); \
func(has_heci_gscfi); \
@@ -164,6 +164,8 @@ enum intel_ppgtt_type {
func(has_logical_ring_elsq); \
func(has_media_ratio_mode); \
func(has_mslice_steering); \
+ func(has_oa_bpc_reporting); \
+ func(has_oa_slice_contrib_limits); \
func(has_one_eu_per_fuse_bit); \
func(has_pxp); \
func(has_rc6); \
@@ -180,6 +182,7 @@ enum intel_ppgtt_type {
/* Keep in alphabetical order */ \
func(cursor_needs_physical); \
func(has_cdclk_crawl); \
+ func(has_cdclk_squash); \
func(has_ddi); \
func(has_dp_mst); \
func(has_dsb); \
@@ -195,20 +198,25 @@ enum intel_ppgtt_type {
func(overlay_needs_physical); \
func(supports_tv);
-struct ip_version {
+struct intel_ip_version {
u8 ver;
u8 rel;
+ u8 step;
};
struct intel_runtime_info {
+ /*
+ * Single "graphics" IP version that represents
+ * render, compute and copy behavior.
+ */
struct {
- struct ip_version ip;
+ struct intel_ip_version ip;
} graphics;
struct {
- struct ip_version ip;
+ struct intel_ip_version ip;
} media;
struct {
- struct ip_version ip;
+ struct intel_ip_version ip;
} display;
/*
@@ -308,7 +316,7 @@ struct intel_driver_caps {
const char *intel_platform_name(enum intel_platform platform);
-void intel_device_info_subplatform_init(struct drm_i915_private *dev_priv);
+void intel_device_info_runtime_init_early(struct drm_i915_private *dev_priv);
void intel_device_info_runtime_init(struct drm_i915_private *dev_priv);
void intel_device_info_print(const struct intel_device_info *info,
diff --git a/drivers/gpu/drm/i915/intel_gvt_mmio_table.c b/drivers/gpu/drm/i915/intel_gvt_mmio_table.c
index 8279dc5..ce6b3c3 100644
--- a/drivers/gpu/drm/i915/intel_gvt_mmio_table.c
+++ b/drivers/gpu/drm/i915/intel_gvt_mmio_table.c
@@ -6,6 +6,7 @@
#include "display/intel_audio_regs.h"
#include "display/intel_backlight_regs.h"
#include "display/intel_dmc_regs.h"
+#include "display/intel_dpio_phy.h"
#include "display/vlv_dsi_pll_regs.h"
#include "gt/intel_gt_regs.h"
#include "gvt/gvt.h"
@@ -102,7 +103,7 @@ static int iterate_generic_mmio(struct intel_gvt_mmio_table_iter *iter)
MMIO_D(_MMIO(0x2438));
MMIO_D(_MMIO(0x243c));
MMIO_D(_MMIO(0x7018));
- MMIO_D(HALF_SLICE_CHICKEN3);
+ MMIO_D(HSW_HALF_SLICE_CHICKEN3);
MMIO_D(GEN7_HALF_SLICE_CHICKEN1);
/* display */
MMIO_F(_MMIO(0x60220), 0x20);
diff --git a/drivers/gpu/drm/i915/intel_mchbar_regs.h b/drivers/gpu/drm/i915/intel_mchbar_regs.h
index ffc702b..f93e9af 100644
--- a/drivers/gpu/drm/i915/intel_mchbar_regs.h
+++ b/drivers/gpu/drm/i915/intel_mchbar_regs.h
@@ -189,6 +189,21 @@
#define DG1_QCLK_RATIO_MASK REG_GENMASK(9, 2)
#define DG1_QCLK_REFERENCE REG_BIT(10)
+/*
+ * *_PACKAGE_POWER_SKU - SKU power and timing parameters.
+ */
+#define PCU_PACKAGE_POWER_SKU _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x5930)
+#define PKG_PKG_TDP GENMASK_ULL(14, 0)
+#define PKG_MAX_WIN GENMASK_ULL(54, 48)
+#define PKG_MAX_WIN_X GENMASK_ULL(54, 53)
+#define PKG_MAX_WIN_Y GENMASK_ULL(52, 48)
+
+#define PCU_PACKAGE_POWER_SKU_UNIT _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x5938)
+#define PKG_PWR_UNIT REG_GENMASK(3, 0)
+#define PKG_ENERGY_UNIT REG_GENMASK(12, 8)
+#define PKG_TIME_UNIT REG_GENMASK(19, 16)
+#define PCU_PACKAGE_ENERGY_STATUS _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x593c)
+
#define GEN6_GT_PERF_STATUS _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x5948)
#define GEN6_RP_STATE_LIMITS _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x5994)
#define GEN6_RP_STATE_CAP _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x5998)
@@ -198,6 +213,12 @@
#define GEN10_FREQ_INFO_REC _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x5ef0)
#define RPE_MASK REG_GENMASK(15, 8)
+#define PCU_PACKAGE_RAPL_LIMIT _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x59a0)
+#define PKG_PWR_LIM_1 REG_GENMASK(14, 0)
+#define PKG_PWR_LIM_1_EN REG_BIT(15)
+#define PKG_PWR_LIM_1_TIME REG_GENMASK(23, 17)
+#define PKG_PWR_LIM_1_TIME_X REG_GENMASK(23, 22)
+#define PKG_PWR_LIM_1_TIME_Y REG_GENMASK(21, 17)
/* snb MCH registers for priority tuning */
#define MCH_SSKPD _MMIO(MCHBAR_MIRROR_BASE_SNB + 0x5d10)
diff --git a/drivers/gpu/drm/i915/intel_pci_config.h b/drivers/gpu/drm/i915/intel_pci_config.h
index 4977a52..23b8e51 100644
--- a/drivers/gpu/drm/i915/intel_pci_config.h
+++ b/drivers/gpu/drm/i915/intel_pci_config.h
@@ -7,11 +7,29 @@
#define __INTEL_PCI_CONFIG_H__
/* PCI BARs */
-#define GTTMMADR_BAR 0
-#define GEN2_GTTMMADR_BAR 1
-#define GFXMEM_BAR 2
-#define GTT_APERTURE_BAR GFXMEM_BAR
-#define GEN12_LMEM_BAR GFXMEM_BAR
+#define GEN2_GMADR_BAR 0
+#define GEN2_MMADR_BAR 1 /* MMIO+GTT, despite the name */
+#define GEN2_IO_BAR 2 /* 85x/865 */
+
+#define GEN3_MMADR_BAR 0 /* MMIO only */
+#define GEN3_IO_BAR 1
+#define GEN3_GMADR_BAR 2
+#define GEN3_GTTADR_BAR 3 /* GTT only */
+
+#define GEN4_GTTMMADR_BAR 0 /* MMIO+GTT */
+#define GEN4_GMADR_BAR 2
+#define GEN4_IO_BAR 4
+
+#define GEN12_LMEM_BAR 2
+
+static inline int intel_mmio_bar(int graphics_ver)
+{
+ switch (graphics_ver) {
+ case 2: return GEN2_MMADR_BAR;
+ case 3: return GEN3_MMADR_BAR;
+ default: return GEN4_GTTMMADR_BAR;
+ }
+}
/* BSM in include/drm/i915_drm.h */
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index 8f86f56..73c88b1 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -30,6 +30,8 @@
#include "display/skl_watermark.h"
#include "gt/intel_engine_regs.h"
+#include "gt/intel_gt.h"
+#include "gt/intel_gt_mcr.h"
#include "gt/intel_gt_regs.h"
#include "i915_drv.h"
@@ -58,25 +60,20 @@ static void gen9_init_clock_gating(struct drm_i915_private *dev_priv)
* Must match Sampler, Pixel Back End, and Media. See
* WaCompressedResourceSamplerPbeMediaNewHashMode.
*/
- intel_uncore_write(&dev_priv->uncore, CHICKEN_PAR1_1,
- intel_uncore_read(&dev_priv->uncore, CHICKEN_PAR1_1) |
- SKL_DE_COMPRESSED_HASH_MODE);
+ intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PAR1_1, 0, SKL_DE_COMPRESSED_HASH_MODE);
}
/* See Bspec note for PSR2_CTL bit 31, Wa#828:skl,bxt,kbl,cfl */
- intel_uncore_write(&dev_priv->uncore, CHICKEN_PAR1_1,
- intel_uncore_read(&dev_priv->uncore, CHICKEN_PAR1_1) | SKL_EDP_PSR_FIX_RDWRAP);
+ intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PAR1_1, 0, SKL_EDP_PSR_FIX_RDWRAP);
/* WaEnableChickenDCPR:skl,bxt,kbl,glk,cfl */
- intel_uncore_write(&dev_priv->uncore, GEN8_CHICKEN_DCPR_1,
- intel_uncore_read(&dev_priv->uncore, GEN8_CHICKEN_DCPR_1) | MASK_WAKEMEM);
+ intel_uncore_rmw(&dev_priv->uncore, GEN8_CHICKEN_DCPR_1, 0, MASK_WAKEMEM);
/*
* WaFbcWakeMemOn:skl,bxt,kbl,glk,cfl
* Display WA #0859: skl,bxt,kbl,glk,cfl
*/
- intel_uncore_write(&dev_priv->uncore, DISP_ARB_CTL, intel_uncore_read(&dev_priv->uncore, DISP_ARB_CTL) |
- DISP_FBC_MEMORY_WAKE);
+ intel_uncore_rmw(&dev_priv->uncore, DISP_ARB_CTL, 0, DISP_FBC_MEMORY_WAKE);
}
static void bxt_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -84,15 +81,13 @@ static void bxt_init_clock_gating(struct drm_i915_private *dev_priv)
gen9_init_clock_gating(dev_priv);
/* WaDisableSDEUnitClockGating:bxt */
- intel_uncore_write(&dev_priv->uncore, GEN8_UCGCTL6, intel_uncore_read(&dev_priv->uncore, GEN8_UCGCTL6) |
- GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN8_UCGCTL6, 0, GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
/*
* FIXME:
* GEN8_HDCUNIT_CLOCK_GATE_DISABLE_HDCREQ applies on 3x6 GT SKUs only.
*/
- intel_uncore_write(&dev_priv->uncore, GEN8_UCGCTL6, intel_uncore_read(&dev_priv->uncore, GEN8_UCGCTL6) |
- GEN8_HDCUNIT_CLOCK_GATE_DISABLE_HDCREQ);
+ intel_uncore_rmw(&dev_priv->uncore, GEN8_UCGCTL6, 0, GEN8_HDCUNIT_CLOCK_GATE_DISABLE_HDCREQ);
/*
* Wa: Backlight PWM may stop in the asserted state, causing backlight
@@ -113,16 +108,13 @@ static void bxt_init_clock_gating(struct drm_i915_private *dev_priv)
* WaFbcTurnOffFbcWatermark:bxt
* Display WA #0562: bxt
*/
- intel_uncore_write(&dev_priv->uncore, DISP_ARB_CTL, intel_uncore_read(&dev_priv->uncore, DISP_ARB_CTL) |
- DISP_FBC_WM_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, DISP_ARB_CTL, 0, DISP_FBC_WM_DIS);
/*
* WaFbcHighMemBwCorruptionAvoidance:bxt
* Display WA #0883: bxt
*/
- intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
- intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A)) |
- DPFC_DISABLE_DUMMY0);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A), 0, DPFC_DISABLE_DUMMY0);
}
static void glk_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -895,19 +887,14 @@ static void pnv_update_wm(struct drm_i915_private *dev_priv)
wm = intel_calculate_wm(pixel_rate, &pnv_cursor_wm,
pnv_display_wm.fifo_size,
4, latency->cursor_sr);
- reg = intel_uncore_read(&dev_priv->uncore, DSPFW3);
- reg &= ~DSPFW_CURSOR_SR_MASK;
- reg |= FW_WM(wm, CURSOR_SR);
- intel_uncore_write(&dev_priv->uncore, DSPFW3, reg);
+ intel_uncore_rmw(&dev_priv->uncore, DSPFW3, DSPFW_CURSOR_SR_MASK,
+ FW_WM(wm, CURSOR_SR));
/* Display HPLL off SR */
wm = intel_calculate_wm(pixel_rate, &pnv_display_hplloff_wm,
pnv_display_hplloff_wm.fifo_size,
cpp, latency->display_hpll_disable);
- reg = intel_uncore_read(&dev_priv->uncore, DSPFW3);
- reg &= ~DSPFW_HPLL_SR_MASK;
- reg |= FW_WM(wm, HPLL_SR);
- intel_uncore_write(&dev_priv->uncore, DSPFW3, reg);
+ intel_uncore_rmw(&dev_priv->uncore, DSPFW3, DSPFW_HPLL_SR_MASK, FW_WM(wm, HPLL_SR));
/* cursor HPLL off SR */
wm = intel_calculate_wm(pixel_rate, &pnv_cursor_hplloff_wm,
@@ -1337,34 +1324,14 @@ static bool g4x_compute_fbc_en(const struct g4x_wm_state *wm_state,
return true;
}
-static int g4x_compute_pipe_wm(struct intel_atomic_state *state,
- struct intel_crtc *crtc)
+static int _g4x_compute_pipe_wm(struct intel_crtc_state *crtc_state)
{
- struct intel_crtc_state *crtc_state =
- intel_atomic_get_new_crtc_state(state, crtc);
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct g4x_wm_state *wm_state = &crtc_state->wm.g4x.optimal;
u8 active_planes = crtc_state->active_planes & ~BIT(PLANE_CURSOR);
const struct g4x_pipe_wm *raw;
- const struct intel_plane_state *old_plane_state;
- const struct intel_plane_state *new_plane_state;
- struct intel_plane *plane;
enum plane_id plane_id;
- int i, level;
- unsigned int dirty = 0;
-
- for_each_oldnew_intel_plane_in_state(state, plane,
- old_plane_state,
- new_plane_state, i) {
- if (new_plane_state->hw.crtc != &crtc->base &&
- old_plane_state->hw.crtc != &crtc->base)
- continue;
-
- if (g4x_raw_plane_wm_compute(crtc_state, new_plane_state))
- dirty |= BIT(plane->id);
- }
-
- if (!dirty)
- return 0;
+ int level;
level = G4X_WM_LEVEL_NORMAL;
if (!g4x_raw_crtc_wm_is_valid(crtc_state, level))
@@ -1417,6 +1384,34 @@ static int g4x_compute_pipe_wm(struct intel_atomic_state *state,
return 0;
}
+static int g4x_compute_pipe_wm(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct intel_crtc_state *crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ const struct intel_plane_state *old_plane_state;
+ const struct intel_plane_state *new_plane_state;
+ struct intel_plane *plane;
+ unsigned int dirty = 0;
+ int i;
+
+ for_each_oldnew_intel_plane_in_state(state, plane,
+ old_plane_state,
+ new_plane_state, i) {
+ if (new_plane_state->hw.crtc != &crtc->base &&
+ old_plane_state->hw.crtc != &crtc->base)
+ continue;
+
+ if (g4x_raw_plane_wm_compute(crtc_state, new_plane_state))
+ dirty |= BIT(plane->id);
+ }
+
+ if (!dirty)
+ return 0;
+
+ return _g4x_compute_pipe_wm(crtc_state);
+}
+
static int g4x_compute_intermediate_wm(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
@@ -1431,7 +1426,7 @@ static int g4x_compute_intermediate_wm(struct intel_atomic_state *state,
enum plane_id plane_id;
if (!new_crtc_state->hw.active ||
- drm_atomic_crtc_needs_modeset(&new_crtc_state->uapi)) {
+ intel_crtc_needs_modeset(new_crtc_state)) {
*intermediate = *optimal;
intermediate->cxsr = false;
@@ -1857,64 +1852,17 @@ static bool vlv_raw_crtc_wm_is_valid(const struct intel_crtc_state *crtc_state,
vlv_raw_plane_wm_is_valid(crtc_state, PLANE_CURSOR, level);
}
-static int vlv_compute_pipe_wm(struct intel_atomic_state *state,
- struct intel_crtc *crtc)
+static int _vlv_compute_pipe_wm(struct intel_crtc_state *crtc_state)
{
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
- struct intel_crtc_state *crtc_state =
- intel_atomic_get_new_crtc_state(state, crtc);
struct vlv_wm_state *wm_state = &crtc_state->wm.vlv.optimal;
const struct vlv_fifo_state *fifo_state =
&crtc_state->wm.vlv.fifo_state;
u8 active_planes = crtc_state->active_planes & ~BIT(PLANE_CURSOR);
int num_active_planes = hweight8(active_planes);
- bool needs_modeset = drm_atomic_crtc_needs_modeset(&crtc_state->uapi);
- const struct intel_plane_state *old_plane_state;
- const struct intel_plane_state *new_plane_state;
- struct intel_plane *plane;
enum plane_id plane_id;
- int level, ret, i;
- unsigned int dirty = 0;
-
- for_each_oldnew_intel_plane_in_state(state, plane,
- old_plane_state,
- new_plane_state, i) {
- if (new_plane_state->hw.crtc != &crtc->base &&
- old_plane_state->hw.crtc != &crtc->base)
- continue;
-
- if (vlv_raw_plane_wm_compute(crtc_state, new_plane_state))
- dirty |= BIT(plane->id);
- }
-
- /*
- * DSPARB registers may have been reset due to the
- * power well being turned off. Make sure we restore
- * them to a consistent state even if no primary/sprite
- * planes are initially active.
- */
- if (needs_modeset)
- crtc_state->fifo_changed = true;
-
- if (!dirty)
- return 0;
-
- /* cursor changes don't warrant a FIFO recompute */
- if (dirty & ~BIT(PLANE_CURSOR)) {
- const struct intel_crtc_state *old_crtc_state =
- intel_atomic_get_old_crtc_state(state, crtc);
- const struct vlv_fifo_state *old_fifo_state =
- &old_crtc_state->wm.vlv.fifo_state;
-
- ret = vlv_compute_fifo(crtc_state);
- if (ret)
- return ret;
-
- if (needs_modeset ||
- memcmp(old_fifo_state, fifo_state,
- sizeof(*fifo_state)) != 0)
- crtc_state->fifo_changed = true;
- }
+ int level;
/* initially allow all levels */
wm_state->num_levels = intel_wm_num_levels(dev_priv);
@@ -1961,6 +1909,66 @@ static int vlv_compute_pipe_wm(struct intel_atomic_state *state,
return 0;
}
+static int vlv_compute_pipe_wm(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct intel_crtc_state *crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ const struct intel_plane_state *old_plane_state;
+ const struct intel_plane_state *new_plane_state;
+ struct intel_plane *plane;
+ unsigned int dirty = 0;
+ int i;
+
+ for_each_oldnew_intel_plane_in_state(state, plane,
+ old_plane_state,
+ new_plane_state, i) {
+ if (new_plane_state->hw.crtc != &crtc->base &&
+ old_plane_state->hw.crtc != &crtc->base)
+ continue;
+
+ if (vlv_raw_plane_wm_compute(crtc_state, new_plane_state))
+ dirty |= BIT(plane->id);
+ }
+
+ /*
+ * DSPARB registers may have been reset due to the
+ * power well being turned off. Make sure we restore
+ * them to a consistent state even if no primary/sprite
+ * planes are initially active. We also force a FIFO
+ * recomputation so that we are sure to sanitize the
+ * FIFO setting we took over from the BIOS even if there
+ * are no active planes on the crtc.
+ */
+ if (intel_crtc_needs_modeset(crtc_state))
+ dirty = ~0;
+
+ if (!dirty)
+ return 0;
+
+ /* cursor changes don't warrant a FIFO recompute */
+ if (dirty & ~BIT(PLANE_CURSOR)) {
+ const struct intel_crtc_state *old_crtc_state =
+ intel_atomic_get_old_crtc_state(state, crtc);
+ const struct vlv_fifo_state *old_fifo_state =
+ &old_crtc_state->wm.vlv.fifo_state;
+ const struct vlv_fifo_state *new_fifo_state =
+ &crtc_state->wm.vlv.fifo_state;
+ int ret;
+
+ ret = vlv_compute_fifo(crtc_state);
+ if (ret)
+ return ret;
+
+ if (intel_crtc_needs_modeset(crtc_state) ||
+ memcmp(old_fifo_state, new_fifo_state,
+ sizeof(*new_fifo_state)) != 0)
+ crtc_state->fifo_changed = true;
+ }
+
+ return _vlv_compute_pipe_wm(crtc_state);
+}
+
#define VLV_FIFO(plane, value) \
(((value) << DSPARB_ ## plane ## _SHIFT_VLV) & DSPARB_ ## plane ## _MASK_VLV)
@@ -2075,7 +2083,7 @@ static int vlv_compute_intermediate_wm(struct intel_atomic_state *state,
int level;
if (!new_crtc_state->hw.active ||
- drm_atomic_crtc_needs_modeset(&new_crtc_state->uapi)) {
+ intel_crtc_needs_modeset(new_crtc_state)) {
*intermediate = *optimal;
intermediate->cxsr = false;
@@ -3133,7 +3141,7 @@ static int ilk_compute_intermediate_wm(struct intel_atomic_state *state,
*/
*a = new_crtc_state->wm.ilk.optimal;
if (!new_crtc_state->hw.active ||
- drm_atomic_crtc_needs_modeset(&new_crtc_state->uapi) ||
+ intel_crtc_needs_modeset(new_crtc_state) ||
state->skip_intermediate_wm)
return 0;
@@ -3458,7 +3466,6 @@ static void ilk_write_wm_values(struct drm_i915_private *dev_priv,
{
struct ilk_wm_values *previous = &dev_priv->display.wm.hw;
unsigned int dirty;
- u32 val;
dirty = ilk_compute_wm_dirty(dev_priv, previous, results);
if (!dirty)
@@ -3474,31 +3481,19 @@ static void ilk_write_wm_values(struct drm_i915_private *dev_priv,
intel_uncore_write(&dev_priv->uncore, WM0_PIPE_ILK(PIPE_C), results->wm_pipe[2]);
if (dirty & WM_DIRTY_DDB) {
- if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
- val = intel_uncore_read(&dev_priv->uncore, WM_MISC);
- if (results->partitioning == INTEL_DDB_PART_1_2)
- val &= ~WM_MISC_DATA_PARTITION_5_6;
- else
- val |= WM_MISC_DATA_PARTITION_5_6;
- intel_uncore_write(&dev_priv->uncore, WM_MISC, val);
- } else {
- val = intel_uncore_read(&dev_priv->uncore, DISP_ARB_CTL2);
- if (results->partitioning == INTEL_DDB_PART_1_2)
- val &= ~DISP_DATA_PARTITION_5_6;
- else
- val |= DISP_DATA_PARTITION_5_6;
- intel_uncore_write(&dev_priv->uncore, DISP_ARB_CTL2, val);
- }
+ if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
+ intel_uncore_rmw(&dev_priv->uncore, WM_MISC, WM_MISC_DATA_PARTITION_5_6,
+ results->partitioning == INTEL_DDB_PART_1_2 ? 0 :
+ WM_MISC_DATA_PARTITION_5_6);
+ else
+ intel_uncore_rmw(&dev_priv->uncore, DISP_ARB_CTL2, DISP_DATA_PARTITION_5_6,
+ results->partitioning == INTEL_DDB_PART_1_2 ? 0 :
+ DISP_DATA_PARTITION_5_6);
}
- if (dirty & WM_DIRTY_FBC) {
- val = intel_uncore_read(&dev_priv->uncore, DISP_ARB_CTL);
- if (results->enable_fbc_wm)
- val &= ~DISP_FBC_WM_DIS;
- else
- val |= DISP_FBC_WM_DIS;
- intel_uncore_write(&dev_priv->uncore, DISP_ARB_CTL, val);
- }
+ if (dirty & WM_DIRTY_FBC)
+ intel_uncore_rmw(&dev_priv->uncore, DISP_ARB_CTL, DISP_FBC_WM_DIS,
+ results->enable_fbc_wm ? 0 : DISP_FBC_WM_DIS);
if (dirty & WM_DIRTY_LP(1) &&
previous->wm_lp_spr[0] != results->wm_lp_spr[0])
@@ -3824,6 +3819,8 @@ void g4x_wm_get_hw_state(struct drm_i915_private *dev_priv)
plane_id, USHRT_MAX);
g4x_raw_fbc_wm_set(crtc_state, level, USHRT_MAX);
+ g4x_invalidate_wms(crtc, active, level);
+
crtc_state->wm.g4x.optimal = *active;
crtc_state->wm.g4x.intermediate = *active;
@@ -3860,37 +3857,30 @@ void g4x_wm_sanitize(struct drm_i915_private *dev_priv)
to_intel_crtc_state(crtc->base.state);
struct intel_plane_state *plane_state =
to_intel_plane_state(plane->base.state);
- struct g4x_wm_state *wm_state = &crtc_state->wm.g4x.optimal;
enum plane_id plane_id = plane->id;
- int level;
+ int level, num_levels = intel_wm_num_levels(dev_priv);
if (plane_state->uapi.visible)
continue;
- for (level = 0; level < 3; level++) {
+ for (level = 0; level < num_levels; level++) {
struct g4x_pipe_wm *raw =
&crtc_state->wm.g4x.raw[level];
raw->plane[plane_id] = 0;
- wm_state->wm.plane[plane_id] = 0;
- }
- if (plane_id == PLANE_PRIMARY) {
- for (level = 0; level < 3; level++) {
- struct g4x_pipe_wm *raw =
- &crtc_state->wm.g4x.raw[level];
+ if (plane_id == PLANE_PRIMARY)
raw->fbc = 0;
- }
-
- wm_state->sr.fbc = 0;
- wm_state->hpll.fbc = 0;
- wm_state->fbc_en = false;
}
}
for_each_intel_crtc(&dev_priv->drm, crtc) {
struct intel_crtc_state *crtc_state =
to_intel_crtc_state(crtc->base.state);
+ int ret;
+
+ ret = _g4x_compute_pipe_wm(crtc_state);
+ drm_WARN_ON(&dev_priv->drm, ret);
crtc_state->wm.g4x.intermediate =
crtc_state->wm.g4x.optimal;
@@ -4016,30 +4006,27 @@ void vlv_wm_sanitize(struct drm_i915_private *dev_priv)
to_intel_crtc_state(crtc->base.state);
struct intel_plane_state *plane_state =
to_intel_plane_state(plane->base.state);
- struct vlv_wm_state *wm_state = &crtc_state->wm.vlv.optimal;
- const struct vlv_fifo_state *fifo_state =
- &crtc_state->wm.vlv.fifo_state;
enum plane_id plane_id = plane->id;
- int level;
+ int level, num_levels = intel_wm_num_levels(dev_priv);
if (plane_state->uapi.visible)
continue;
- for (level = 0; level < wm_state->num_levels; level++) {
+ for (level = 0; level < num_levels; level++) {
struct g4x_pipe_wm *raw =
&crtc_state->wm.vlv.raw[level];
raw->plane[plane_id] = 0;
-
- wm_state->wm[level].plane[plane_id] =
- vlv_invert_wm_value(raw->plane[plane_id],
- fifo_state->plane[plane_id]);
}
}
for_each_intel_crtc(&dev_priv->drm, crtc) {
struct intel_crtc_state *crtc_state =
to_intel_crtc_state(crtc->base.state);
+ int ret;
+
+ ret = _vlv_compute_pipe_wm(crtc_state);
+ drm_WARN_ON(&dev_priv->drm, ret);
crtc_state->wm.vlv.intermediate =
crtc_state->wm.vlv.optimal;
@@ -4057,9 +4044,9 @@ void vlv_wm_sanitize(struct drm_i915_private *dev_priv)
*/
static void ilk_init_lp_watermarks(struct drm_i915_private *dev_priv)
{
- intel_uncore_write(&dev_priv->uncore, WM3_LP_ILK, intel_uncore_read(&dev_priv->uncore, WM3_LP_ILK) & ~WM_LP_ENABLE);
- intel_uncore_write(&dev_priv->uncore, WM2_LP_ILK, intel_uncore_read(&dev_priv->uncore, WM2_LP_ILK) & ~WM_LP_ENABLE);
- intel_uncore_write(&dev_priv->uncore, WM1_LP_ILK, intel_uncore_read(&dev_priv->uncore, WM1_LP_ILK) & ~WM_LP_ENABLE);
+ intel_uncore_rmw(&dev_priv->uncore, WM3_LP_ILK, WM_LP_ENABLE, 0);
+ intel_uncore_rmw(&dev_priv->uncore, WM2_LP_ILK, WM_LP_ENABLE, 0);
+ intel_uncore_rmw(&dev_priv->uncore, WM1_LP_ILK, WM_LP_ENABLE, 0);
/*
* Don't touch WM_LP_SPRITE_ENABLE here.
@@ -4113,11 +4100,9 @@ static void g4x_disable_trickle_feed(struct drm_i915_private *dev_priv)
enum pipe pipe;
for_each_pipe(dev_priv, pipe) {
- intel_uncore_write(&dev_priv->uncore, DSPCNTR(pipe),
- intel_uncore_read(&dev_priv->uncore, DSPCNTR(pipe)) |
- DISP_TRICKLE_FEED_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, DSPCNTR(pipe), 0, DISP_TRICKLE_FEED_DISABLE);
- intel_uncore_write(&dev_priv->uncore, DSPSURF(pipe), intel_uncore_read(&dev_priv->uncore, DSPSURF(pipe)));
+ intel_uncore_rmw(&dev_priv->uncore, DSPSURF(pipe), 0, 0);
intel_uncore_posting_read(&dev_priv->uncore, DSPSURF(pipe));
}
}
@@ -4164,19 +4149,13 @@ static void ilk_init_clock_gating(struct drm_i915_private *dev_priv)
*/
if (IS_IRONLAKE_M(dev_priv)) {
/* WaFbcAsynchFlipDisableFbcQueue:ilk */
- intel_uncore_write(&dev_priv->uncore, ILK_DISPLAY_CHICKEN1,
- intel_uncore_read(&dev_priv->uncore, ILK_DISPLAY_CHICKEN1) |
- ILK_FBCQ_DIS);
- intel_uncore_write(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2,
- intel_uncore_read(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2) |
- ILK_DPARB_GATE);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DISPLAY_CHICKEN1, 0, ILK_FBCQ_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2, 0, ILK_DPARB_GATE);
}
intel_uncore_write(&dev_priv->uncore, ILK_DSPCLK_GATE_D, dspclk_gate);
- intel_uncore_write(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2,
- intel_uncore_read(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2) |
- ILK_ELPIN_409_SELECT);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2, 0, ILK_ELPIN_409_SELECT);
g4x_disable_trickle_feed(dev_priv);
@@ -4196,8 +4175,7 @@ static void cpt_init_clock_gating(struct drm_i915_private *dev_priv)
intel_uncore_write(&dev_priv->uncore, SOUTH_DSPCLK_GATE_D, PCH_DPLSUNIT_CLOCK_GATE_DISABLE |
PCH_DPLUNIT_CLOCK_GATE_DISABLE |
PCH_CPUNIT_CLOCK_GATE_DISABLE);
- intel_uncore_write(&dev_priv->uncore, SOUTH_CHICKEN2, intel_uncore_read(&dev_priv->uncore, SOUTH_CHICKEN2) |
- DPLS_EDP_PPS_FIX_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, SOUTH_CHICKEN2, 0, DPLS_EDP_PPS_FIX_DIS);
/* The below fixes the weird display corruption, a few pixels shifted
* downward, on (only) LVDS of some HP laptops with IVY.
*/
@@ -4235,9 +4213,7 @@ static void gen6_init_clock_gating(struct drm_i915_private *dev_priv)
intel_uncore_write(&dev_priv->uncore, ILK_DSPCLK_GATE_D, dspclk_gate);
- intel_uncore_write(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2,
- intel_uncore_read(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2) |
- ILK_ELPIN_409_SELECT);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DISPLAY_CHICKEN2, 0, ILK_ELPIN_409_SELECT);
intel_uncore_write(&dev_priv->uncore, GEN6_UCGCTL1,
intel_uncore_read(&dev_priv->uncore, GEN6_UCGCTL1) |
@@ -4297,14 +4273,12 @@ static void lpt_init_clock_gating(struct drm_i915_private *dev_priv)
* disabled when not needed anymore in order to save power.
*/
if (HAS_PCH_LPT_LP(dev_priv))
- intel_uncore_write(&dev_priv->uncore, SOUTH_DSPCLK_GATE_D,
- intel_uncore_read(&dev_priv->uncore, SOUTH_DSPCLK_GATE_D) |
- PCH_LP_PARTITION_LEVEL_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, SOUTH_DSPCLK_GATE_D,
+ 0, PCH_LP_PARTITION_LEVEL_DISABLE);
/* WADPOClockGatingDisable:hsw */
- intel_uncore_write(&dev_priv->uncore, TRANS_CHICKEN1(PIPE_A),
- intel_uncore_read(&dev_priv->uncore, TRANS_CHICKEN1(PIPE_A)) |
- TRANS_CHICKEN1_DP0UNIT_GC_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, TRANS_CHICKEN1(PIPE_A),
+ 0, TRANS_CHICKEN1_DP0UNIT_GC_DISABLE);
}
static void lpt_suspend_hw(struct drm_i915_private *dev_priv)
@@ -4325,22 +4299,22 @@ static void gen8_set_l3sqc_credits(struct drm_i915_private *dev_priv,
u32 val;
/* WaTempDisableDOPClkGating:bdw */
- misccpctl = intel_uncore_read(&dev_priv->uncore, GEN7_MISCCPCTL);
- intel_uncore_write(&dev_priv->uncore, GEN7_MISCCPCTL, misccpctl & ~GEN7_DOP_CLOCK_GATE_ENABLE);
+ misccpctl = intel_gt_mcr_multicast_rmw(to_gt(dev_priv), GEN8_MISCCPCTL,
+ GEN8_DOP_CLOCK_GATE_ENABLE, 0);
- val = intel_uncore_read(&dev_priv->uncore, GEN8_L3SQCREG1);
+ val = intel_gt_mcr_read_any(to_gt(dev_priv), GEN8_L3SQCREG1);
val &= ~L3_PRIO_CREDITS_MASK;
val |= L3_GENERAL_PRIO_CREDITS(general_prio_credits);
val |= L3_HIGH_PRIO_CREDITS(high_prio_credits);
- intel_uncore_write(&dev_priv->uncore, GEN8_L3SQCREG1, val);
+ intel_gt_mcr_multicast_write(to_gt(dev_priv), GEN8_L3SQCREG1, val);
/*
* Wait at least 100 clocks before re-enabling clock gating.
* See the definition of L3SQCREG1 in BSpec.
*/
- intel_uncore_posting_read(&dev_priv->uncore, GEN8_L3SQCREG1);
+ intel_gt_mcr_read_any(to_gt(dev_priv), GEN8_L3SQCREG1);
udelay(1);
- intel_uncore_write(&dev_priv->uncore, GEN7_MISCCPCTL, misccpctl);
+ intel_gt_mcr_multicast_write(to_gt(dev_priv), GEN8_MISCCPCTL, misccpctl);
}
static void icl_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -4363,8 +4337,7 @@ static void gen12lp_init_clock_gating(struct drm_i915_private *dev_priv)
/* Wa_1409825376:tgl (pre-prod)*/
if (IS_TGL_DISPLAY_STEP(dev_priv, STEP_A0, STEP_C0))
- intel_uncore_write(&dev_priv->uncore, GEN9_CLKGATE_DIS_3, intel_uncore_read(&dev_priv->uncore, GEN9_CLKGATE_DIS_3) |
- TGL_VRH_GATING_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, GEN9_CLKGATE_DIS_3, 0, TGL_VRH_GATING_DIS);
/* Wa_14013723622:tgl,rkl,dg1,adl-s */
if (DISPLAY_VER(dev_priv) == 12)
@@ -4389,8 +4362,7 @@ static void dg1_init_clock_gating(struct drm_i915_private *dev_priv)
/* Wa_1409836686:dg1[a0] */
if (IS_DG1_GRAPHICS_STEP(dev_priv, STEP_A0, STEP_B0))
- intel_uncore_write(&dev_priv->uncore, GEN9_CLKGATE_DIS_3, intel_uncore_read(&dev_priv->uncore, GEN9_CLKGATE_DIS_3) |
- DPT_GATING_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, GEN9_CLKGATE_DIS_3, 0, DPT_GATING_DIS);
}
static void xehpsdv_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -4432,8 +4404,7 @@ static void cnp_init_clock_gating(struct drm_i915_private *dev_priv)
return;
/* Display WA #1181 WaSouthDisplayDisablePWMCGEGating: cnp */
- intel_uncore_write(&dev_priv->uncore, SOUTH_DSPCLK_GATE_D, intel_uncore_read(&dev_priv->uncore, SOUTH_DSPCLK_GATE_D) |
- CNP_PWM_CGE_GATING_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, SOUTH_DSPCLK_GATE_D, 0, CNP_PWM_CGE_GATING_DISABLE);
}
static void cfl_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -4442,23 +4413,20 @@ static void cfl_init_clock_gating(struct drm_i915_private *dev_priv)
gen9_init_clock_gating(dev_priv);
/* WAC6entrylatency:cfl */
- intel_uncore_write(&dev_priv->uncore, FBC_LLC_READ_CTRL, intel_uncore_read(&dev_priv->uncore, FBC_LLC_READ_CTRL) |
- FBC_LLC_FULLY_OPEN);
+ intel_uncore_rmw(&dev_priv->uncore, FBC_LLC_READ_CTRL, 0, FBC_LLC_FULLY_OPEN);
/*
* WaFbcTurnOffFbcWatermark:cfl
* Display WA #0562: cfl
*/
- intel_uncore_write(&dev_priv->uncore, DISP_ARB_CTL, intel_uncore_read(&dev_priv->uncore, DISP_ARB_CTL) |
- DISP_FBC_WM_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, DISP_ARB_CTL, 0, DISP_FBC_WM_DIS);
/*
* WaFbcNukeOnHostModify:cfl
* Display WA #0873: cfl
*/
- intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
- intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A)) |
- DPFC_NUKE_ON_ANY_MODIFICATION);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
+ 0, DPFC_NUKE_ON_ANY_MODIFICATION);
}
static void kbl_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -4466,33 +4434,30 @@ static void kbl_init_clock_gating(struct drm_i915_private *dev_priv)
gen9_init_clock_gating(dev_priv);
/* WAC6entrylatency:kbl */
- intel_uncore_write(&dev_priv->uncore, FBC_LLC_READ_CTRL, intel_uncore_read(&dev_priv->uncore, FBC_LLC_READ_CTRL) |
- FBC_LLC_FULLY_OPEN);
+ intel_uncore_rmw(&dev_priv->uncore, FBC_LLC_READ_CTRL, 0, FBC_LLC_FULLY_OPEN);
/* WaDisableSDEUnitClockGating:kbl */
if (IS_KBL_GRAPHICS_STEP(dev_priv, 0, STEP_C0))
- intel_uncore_write(&dev_priv->uncore, GEN8_UCGCTL6, intel_uncore_read(&dev_priv->uncore, GEN8_UCGCTL6) |
- GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN8_UCGCTL6,
+ 0, GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
/* WaDisableGamClockGating:kbl */
if (IS_KBL_GRAPHICS_STEP(dev_priv, 0, STEP_C0))
- intel_uncore_write(&dev_priv->uncore, GEN6_UCGCTL1, intel_uncore_read(&dev_priv->uncore, GEN6_UCGCTL1) |
- GEN6_GAMUNIT_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN6_UCGCTL1,
+ 0, GEN6_GAMUNIT_CLOCK_GATE_DISABLE);
/*
* WaFbcTurnOffFbcWatermark:kbl
* Display WA #0562: kbl
*/
- intel_uncore_write(&dev_priv->uncore, DISP_ARB_CTL, intel_uncore_read(&dev_priv->uncore, DISP_ARB_CTL) |
- DISP_FBC_WM_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, DISP_ARB_CTL, 0, DISP_FBC_WM_DIS);
/*
* WaFbcNukeOnHostModify:kbl
* Display WA #0873: kbl
*/
- intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
- intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A)) |
- DPFC_NUKE_ON_ANY_MODIFICATION);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
+ 0, DPFC_NUKE_ON_ANY_MODIFICATION);
}
static void skl_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -4500,35 +4465,30 @@ static void skl_init_clock_gating(struct drm_i915_private *dev_priv)
gen9_init_clock_gating(dev_priv);
/* WaDisableDopClockGating:skl */
- intel_uncore_write(&dev_priv->uncore, GEN7_MISCCPCTL, intel_uncore_read(&dev_priv->uncore, GEN7_MISCCPCTL) &
- ~GEN7_DOP_CLOCK_GATE_ENABLE);
+ intel_gt_mcr_multicast_rmw(to_gt(dev_priv), GEN8_MISCCPCTL,
+ GEN8_DOP_CLOCK_GATE_ENABLE, 0);
/* WAC6entrylatency:skl */
- intel_uncore_write(&dev_priv->uncore, FBC_LLC_READ_CTRL, intel_uncore_read(&dev_priv->uncore, FBC_LLC_READ_CTRL) |
- FBC_LLC_FULLY_OPEN);
+ intel_uncore_rmw(&dev_priv->uncore, FBC_LLC_READ_CTRL, 0, FBC_LLC_FULLY_OPEN);
/*
* WaFbcTurnOffFbcWatermark:skl
* Display WA #0562: skl
*/
- intel_uncore_write(&dev_priv->uncore, DISP_ARB_CTL, intel_uncore_read(&dev_priv->uncore, DISP_ARB_CTL) |
- DISP_FBC_WM_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, DISP_ARB_CTL, 0, DISP_FBC_WM_DIS);
/*
* WaFbcNukeOnHostModify:skl
* Display WA #0873: skl
*/
- intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
- intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A)) |
- DPFC_NUKE_ON_ANY_MODIFICATION);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
+ 0, DPFC_NUKE_ON_ANY_MODIFICATION);
/*
* WaFbcHighMemBwCorruptionAvoidance:skl
* Display WA #0883: skl
*/
- intel_uncore_write(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A),
- intel_uncore_read(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A)) |
- DPFC_DISABLE_DUMMY0);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DPFC_CHICKEN(INTEL_FBC_A), 0, DPFC_DISABLE_DUMMY0);
}
static void bdw_init_clock_gating(struct drm_i915_private *dev_priv)
@@ -4536,43 +4496,37 @@ static void bdw_init_clock_gating(struct drm_i915_private *dev_priv)
enum pipe pipe;
/* WaFbcAsynchFlipDisableFbcQueue:hsw,bdw */
- intel_uncore_write(&dev_priv->uncore, CHICKEN_PIPESL_1(PIPE_A),
- intel_uncore_read(&dev_priv->uncore, CHICKEN_PIPESL_1(PIPE_A)) |
- HSW_FBCQ_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PIPESL_1(PIPE_A), 0, HSW_FBCQ_DIS);
/* WaSwitchSolVfFArbitrationPriority:bdw */
- intel_uncore_write(&dev_priv->uncore, GAM_ECOCHK, intel_uncore_read(&dev_priv->uncore, GAM_ECOCHK) | HSW_ECOCHK_ARB_PRIO_SOL);
+ intel_uncore_rmw(&dev_priv->uncore, GAM_ECOCHK, 0, HSW_ECOCHK_ARB_PRIO_SOL);
/* WaPsrDPAMaskVBlankInSRD:bdw */
- intel_uncore_write(&dev_priv->uncore, CHICKEN_PAR1_1,
- intel_uncore_read(&dev_priv->uncore, CHICKEN_PAR1_1) | DPA_MASK_VBLANK_SRD);
+ intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PAR1_1, 0, DPA_MASK_VBLANK_SRD);
for_each_pipe(dev_priv, pipe) {
/* WaPsrDPRSUnmaskVBlankInSRD:bdw */
- intel_uncore_write(&dev_priv->uncore, CHICKEN_PIPESL_1(pipe),
- intel_uncore_read(&dev_priv->uncore, CHICKEN_PIPESL_1(pipe)) |
- BDW_DPRS_MASK_VBLANK_SRD);
+ intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PIPESL_1(pipe),
+ 0, BDW_DPRS_MASK_VBLANK_SRD);
}
/* WaVSRefCountFullforceMissDisable:bdw */
/* WaDSRefCountFullforceMissDisable:bdw */
- intel_uncore_write(&dev_priv->uncore, GEN7_FF_THREAD_MODE,
- intel_uncore_read(&dev_priv->uncore, GEN7_FF_THREAD_MODE) &
- ~(GEN8_FF_DS_REF_CNT_FFME | GEN7_FF_VS_REF_CNT_FFME));
+ intel_uncore_rmw(&dev_priv->uncore, GEN7_FF_THREAD_MODE,
+ GEN8_FF_DS_REF_CNT_FFME | GEN7_FF_VS_REF_CNT_FFME, 0);
intel_uncore_write(&dev_priv->uncore, RING_PSMI_CTL(RENDER_RING_BASE),
_MASKED_BIT_ENABLE(GEN8_RC_SEMA_IDLE_MSG_DISABLE));
/* WaDisableSDEUnitClockGating:bdw */
- intel_uncore_write(&dev_priv->uncore, GEN8_UCGCTL6, intel_uncore_read(&dev_priv->uncore, GEN8_UCGCTL6) |
- GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN8_UCGCTL6, 0, GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
/* WaProgramL3SqcReg1Default:bdw */
gen8_set_l3sqc_credits(dev_priv, 30, 2);
/* WaKVMNotificationOnConfigChange:bdw */
- intel_uncore_write(&dev_priv->uncore, CHICKEN_PAR2_1, intel_uncore_read(&dev_priv->uncore, CHICKEN_PAR2_1)
- | KVM_CONFIG_CHANGE_NOTIFICATION_SELECT);
+ intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PAR2_1,
+ 0, KVM_CONFIG_CHANGE_NOTIFICATION_SELECT);
lpt_init_clock_gating(dev_priv);
@@ -4581,38 +4535,30 @@ static void bdw_init_clock_gating(struct drm_i915_private *dev_priv)
* Also see the CHICKEN2 write in bdw_init_workarounds() to disable DOP
* clock gating.
*/
- intel_uncore_write(&dev_priv->uncore, GEN6_UCGCTL1,
- intel_uncore_read(&dev_priv->uncore, GEN6_UCGCTL1) | GEN6_EU_TCUNIT_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN6_UCGCTL1, 0, GEN6_EU_TCUNIT_CLOCK_GATE_DISABLE);
}
static void hsw_init_clock_gating(struct drm_i915_private *dev_priv)
{
/* WaFbcAsynchFlipDisableFbcQueue:hsw,bdw */
- intel_uncore_write(&dev_priv->uncore, CHICKEN_PIPESL_1(PIPE_A),
- intel_uncore_read(&dev_priv->uncore, CHICKEN_PIPESL_1(PIPE_A)) |
- HSW_FBCQ_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, CHICKEN_PIPESL_1(PIPE_A), 0, HSW_FBCQ_DIS);
/* This is required by WaCatErrorRejectionIssue:hsw */
- intel_uncore_write(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG,
- intel_uncore_read(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG) |
- GEN7_SQ_CHICKEN_MBCUNIT_SQINTMOB);
+ intel_uncore_rmw(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG,
+ 0, GEN7_SQ_CHICKEN_MBCUNIT_SQINTMOB);
/* WaSwitchSolVfFArbitrationPriority:hsw */
- intel_uncore_write(&dev_priv->uncore, GAM_ECOCHK, intel_uncore_read(&dev_priv->uncore, GAM_ECOCHK) | HSW_ECOCHK_ARB_PRIO_SOL);
+ intel_uncore_rmw(&dev_priv->uncore, GAM_ECOCHK, 0, HSW_ECOCHK_ARB_PRIO_SOL);
lpt_init_clock_gating(dev_priv);
}
static void ivb_init_clock_gating(struct drm_i915_private *dev_priv)
{
- u32 snpcr;
-
intel_uncore_write(&dev_priv->uncore, ILK_DSPCLK_GATE_D, ILK_VRHUNIT_CLOCK_GATE_DISABLE);
/* WaFbcAsynchFlipDisableFbcQueue:ivb */
- intel_uncore_write(&dev_priv->uncore, ILK_DISPLAY_CHICKEN1,
- intel_uncore_read(&dev_priv->uncore, ILK_DISPLAY_CHICKEN1) |
- ILK_FBCQ_DIS);
+ intel_uncore_rmw(&dev_priv->uncore, ILK_DISPLAY_CHICKEN1, 0, ILK_FBCQ_DIS);
/* WaDisableBackToBackFlipFix:ivb */
intel_uncore_write(&dev_priv->uncore, IVB_CHICKEN3,
@@ -4638,16 +4584,13 @@ static void ivb_init_clock_gating(struct drm_i915_private *dev_priv)
GEN6_RCZUNIT_CLOCK_GATE_DISABLE);
/* This is required by WaCatErrorRejectionIssue:ivb */
- intel_uncore_write(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG,
- intel_uncore_read(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG) |
- GEN7_SQ_CHICKEN_MBCUNIT_SQINTMOB);
+ intel_uncore_rmw(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG,
+ 0, GEN7_SQ_CHICKEN_MBCUNIT_SQINTMOB);
g4x_disable_trickle_feed(dev_priv);
- snpcr = intel_uncore_read(&dev_priv->uncore, GEN6_MBCUNIT_SNPCR);
- snpcr &= ~GEN6_MBC_SNPCR_MASK;
- snpcr |= GEN6_MBC_SNPCR_MED;
- intel_uncore_write(&dev_priv->uncore, GEN6_MBCUNIT_SNPCR, snpcr);
+ intel_uncore_rmw(&dev_priv->uncore, GEN6_MBCUNIT_SNPCR, GEN6_MBC_SNPCR_MASK,
+ GEN6_MBC_SNPCR_MED);
if (!HAS_PCH_NOP(dev_priv))
cpt_init_clock_gating(dev_priv);
@@ -4667,9 +4610,8 @@ static void vlv_init_clock_gating(struct drm_i915_private *dev_priv)
_MASKED_BIT_ENABLE(DOP_CLOCK_GATING_DISABLE));
/* This is required by WaCatErrorRejectionIssue:vlv */
- intel_uncore_write(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG,
- intel_uncore_read(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG) |
- GEN7_SQ_CHICKEN_MBCUNIT_SQINTMOB);
+ intel_uncore_rmw(&dev_priv->uncore, GEN7_SQ_CHICKEN_MBCUNIT_CONFIG,
+ 0, GEN7_SQ_CHICKEN_MBCUNIT_SQINTMOB);
/*
* According to the spec, bit 13 (RCZUNIT) must be set on IVB.
@@ -4681,8 +4623,7 @@ static void vlv_init_clock_gating(struct drm_i915_private *dev_priv)
/* WaDisableL3Bank2xClockGate:vlv
* Disabling L3 clock gating- MMIO 940c[25] = 1
* Set bit 25, to disable L3_BANK_2x_CLK_GATING */
- intel_uncore_write(&dev_priv->uncore, GEN7_UCGCTL4,
- intel_uncore_read(&dev_priv->uncore, GEN7_UCGCTL4) | GEN7_L3BANK2X_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN7_UCGCTL4, 0, GEN7_L3BANK2X_CLOCK_GATE_DISABLE);
/*
* WaDisableVLVClockGating_VBIIssue:vlv
@@ -4696,21 +4637,18 @@ static void chv_init_clock_gating(struct drm_i915_private *dev_priv)
{
/* WaVSRefCountFullforceMissDisable:chv */
/* WaDSRefCountFullforceMissDisable:chv */
- intel_uncore_write(&dev_priv->uncore, GEN7_FF_THREAD_MODE,
- intel_uncore_read(&dev_priv->uncore, GEN7_FF_THREAD_MODE) &
- ~(GEN8_FF_DS_REF_CNT_FFME | GEN7_FF_VS_REF_CNT_FFME));
+ intel_uncore_rmw(&dev_priv->uncore, GEN7_FF_THREAD_MODE,
+ GEN8_FF_DS_REF_CNT_FFME | GEN7_FF_VS_REF_CNT_FFME, 0);
/* WaDisableSemaphoreAndSyncFlipWait:chv */
intel_uncore_write(&dev_priv->uncore, RING_PSMI_CTL(RENDER_RING_BASE),
_MASKED_BIT_ENABLE(GEN8_RC_SEMA_IDLE_MSG_DISABLE));
/* WaDisableCSUnitClockGating:chv */
- intel_uncore_write(&dev_priv->uncore, GEN6_UCGCTL1, intel_uncore_read(&dev_priv->uncore, GEN6_UCGCTL1) |
- GEN6_CSUNIT_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN6_UCGCTL1, 0, GEN6_CSUNIT_CLOCK_GATE_DISABLE);
/* WaDisableSDEUnitClockGating:chv */
- intel_uncore_write(&dev_priv->uncore, GEN8_UCGCTL6, intel_uncore_read(&dev_priv->uncore, GEN8_UCGCTL6) |
- GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
+ intel_uncore_rmw(&dev_priv->uncore, GEN8_UCGCTL6, 0, GEN8_SDEUNIT_CLOCK_GATE_DISABLE);
/*
* WaProgramL3SqcReg1Default:chv
diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c
index 575d67b..cf89d0c 100644
--- a/drivers/gpu/drm/i915/intel_region_ttm.c
+++ b/drivers/gpu/drm/i915/intel_region_ttm.c
@@ -244,7 +244,7 @@ void intel_region_ttm_resource_free(struct intel_memory_region *mem,
struct ttm_resource_manager *man = mem->region_private;
struct ttm_buffer_object mock_bo = {};
- mock_bo.base.size = res->num_pages << PAGE_SHIFT;
+ mock_bo.base.size = res->size;
mock_bo.bdev = &mem->i915->bdev;
res->bo = &mock_bo;
diff --git a/drivers/gpu/drm/i915/intel_runtime_pm.c b/drivers/gpu/drm/i915/intel_runtime_pm.c
index 744cca5..1297467 100644
--- a/drivers/gpu/drm/i915/intel_runtime_pm.c
+++ b/drivers/gpu/drm/i915/intel_runtime_pm.c
@@ -633,6 +633,8 @@ void intel_runtime_pm_driver_release(struct intel_runtime_pm *rpm)
runtime_pm);
int count = atomic_read(&rpm->wakeref_count);
+ intel_wakeref_auto_fini(&rpm->userfault_wakeref);
+
drm_WARN(&i915->drm, count,
"i915 raw-wakerefs=%d wakelocks=%d on cleanup\n",
intel_rpm_raw_wakeref_count(count),
@@ -652,4 +654,7 @@ void intel_runtime_pm_init_early(struct intel_runtime_pm *rpm)
rpm->available = HAS_RUNTIME_PM(i915);
init_intel_runtime_pm_wakeref(rpm);
+ INIT_LIST_HEAD(&rpm->lmem_userfault_list);
+ spin_lock_init(&rpm->lmem_userfault_lock);
+ intel_wakeref_auto_init(&rpm->userfault_wakeref, rpm);
}
diff --git a/drivers/gpu/drm/i915/intel_runtime_pm.h b/drivers/gpu/drm/i915/intel_runtime_pm.h
index d9160e3..98b8b28 100644
--- a/drivers/gpu/drm/i915/intel_runtime_pm.h
+++ b/drivers/gpu/drm/i915/intel_runtime_pm.h
@@ -53,6 +53,28 @@ struct intel_runtime_pm {
bool irqs_enabled;
bool no_wakeref_tracking;
+ /*
+ * Protects access to lmem usefault list.
+ * It is required, if we are outside of the runtime suspend path,
+ * access to @lmem_userfault_list requires always first grabbing the
+ * runtime pm, to ensure we can't race against runtime suspend.
+ * Once we have that we also need to grab @lmem_userfault_lock,
+ * at which point we have exclusive access.
+ * The runtime suspend path is special since it doesn't really hold any locks,
+ * but instead has exclusive access by virtue of all other accesses requiring
+ * holding the runtime pm wakeref.
+ */
+ spinlock_t lmem_userfault_lock;
+
+ /*
+ * Keep list of userfaulted gem obj, which require to release their
+ * mmap mappings at runtime suspend path.
+ */
+ struct list_head lmem_userfault_list;
+
+ /* Manual runtime pm autosuspend delay for user GGTT/lmem mmaps */
+ struct intel_wakeref_auto userfault_wakeref;
+
#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_RUNTIME_PM)
/*
* To aide detection of wakeref leaks and general misuse, we
diff --git a/drivers/gpu/drm/i915/intel_step.c b/drivers/gpu/drm/i915/intel_step.c
index 42b3133..84a6fe7 100644
--- a/drivers/gpu/drm/i915/intel_step.c
+++ b/drivers/gpu/drm/i915/intel_step.c
@@ -131,10 +131,27 @@ static const struct intel_step_info adls_rpls_revids[] = {
[0xC] = { COMMON_GT_MEDIA_STEP(D0), .display_step = STEP_C0 },
};
+static const struct intel_step_info adlp_rplp_revids[] = {
+ [0x4] = { COMMON_GT_MEDIA_STEP(C0), .display_step = STEP_E0 },
+};
+
static const struct intel_step_info adlp_n_revids[] = {
[0x0] = { COMMON_GT_MEDIA_STEP(A0), .display_step = STEP_D0 },
};
+static u8 gmd_to_intel_step(struct drm_i915_private *i915,
+ struct intel_ip_version *gmd)
+{
+ u8 step = gmd->step + STEP_A0;
+
+ if (step >= STEP_FUTURE) {
+ drm_dbg(&i915->drm, "Using future steppings\n");
+ return STEP_FUTURE;
+ }
+
+ return step;
+}
+
static void pvc_step_init(struct drm_i915_private *i915, int pci_revid);
void intel_step_init(struct drm_i915_private *i915)
@@ -144,6 +161,18 @@ void intel_step_init(struct drm_i915_private *i915)
int revid = INTEL_REVID(i915);
struct intel_step_info step = {};
+ if (HAS_GMD_ID(i915)) {
+ step.graphics_step = gmd_to_intel_step(i915,
+ &RUNTIME_INFO(i915)->graphics.ip);
+ step.media_step = gmd_to_intel_step(i915,
+ &RUNTIME_INFO(i915)->media.ip);
+ step.display_step = gmd_to_intel_step(i915,
+ &RUNTIME_INFO(i915)->display.ip);
+ RUNTIME_INFO(i915)->step = step;
+
+ return;
+ }
+
if (IS_PONTEVECCHIO(i915)) {
pvc_step_init(i915, revid);
return;
@@ -162,6 +191,9 @@ void intel_step_init(struct drm_i915_private *i915)
} else if (IS_ADLP_N(i915)) {
revids = adlp_n_revids;
size = ARRAY_SIZE(adlp_n_revids);
+ } else if (IS_ADLP_RPLP(i915)) {
+ revids = adlp_rplp_revids;
+ size = ARRAY_SIZE(adlp_rplp_revids);
} else if (IS_ALDERLAKE_P(i915)) {
revids = adlp_revids;
size = ARRAY_SIZE(adlp_revids);
diff --git a/drivers/gpu/drm/i915/intel_step.h b/drivers/gpu/drm/i915/intel_step.h
index a6b12bf..96dfca4 100644
--- a/drivers/gpu/drm/i915/intel_step.h
+++ b/drivers/gpu/drm/i915/intel_step.h
@@ -11,6 +11,10 @@
struct drm_i915_private;
struct intel_step_info {
+ /*
+ * It is expected to have 4 number steps per letter. Deviation from
+ * the expectation breaks gmd_to_intel_step().
+ */
u8 graphics_step; /* Represents the compute tile on Xe_HPC */
u8 display_step;
u8 media_step;
@@ -23,21 +27,43 @@ struct intel_step_info {
func(A0) \
func(A1) \
func(A2) \
+ func(A3) \
func(B0) \
func(B1) \
func(B2) \
func(B3) \
func(C0) \
func(C1) \
+ func(C2) \
+ func(C3) \
func(D0) \
func(D1) \
+ func(D2) \
+ func(D3) \
func(E0) \
+ func(E1) \
+ func(E2) \
+ func(E3) \
func(F0) \
+ func(F1) \
+ func(F2) \
+ func(F3) \
func(G0) \
+ func(G1) \
+ func(G2) \
+ func(G3) \
func(H0) \
+ func(H1) \
+ func(H2) \
+ func(H3) \
func(I0) \
func(I1) \
- func(J0)
+ func(I2) \
+ func(I3) \
+ func(J0) \
+ func(J1) \
+ func(J2) \
+ func(J3)
/*
* Symbolic steppings that do not match the hardware. These are valid both as gt
diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c
index 5cd423c..8006a6c 100644
--- a/drivers/gpu/drm/i915/intel_uncore.c
+++ b/drivers/gpu/drm/i915/intel_uncore.c
@@ -29,6 +29,7 @@
#include "i915_drv.h"
#include "i915_iosf_mbi.h"
+#include "i915_reg.h"
#include "i915_trace.h"
#include "i915_vgpu.h"
#include "intel_pm.h"
@@ -104,6 +105,7 @@ static const char * const forcewake_domain_names[] = {
"vebox1",
"vebox2",
"vebox3",
+ "gsc",
};
const char *
@@ -177,8 +179,9 @@ static inline void
fw_domain_wait_ack_clear(const struct intel_uncore_forcewake_domain *d)
{
if (wait_ack_clear(d, FORCEWAKE_KERNEL)) {
- DRM_ERROR("%s: timed out waiting for forcewake ack to clear.\n",
- intel_uncore_forcewake_domain_to_str(d->id));
+ drm_err(&d->uncore->i915->drm,
+ "%s: timed out waiting for forcewake ack to clear.\n",
+ intel_uncore_forcewake_domain_to_str(d->id));
add_taint_for_CI(d->uncore->i915, TAINT_WARN); /* CI now unreliable */
}
}
@@ -225,11 +228,12 @@ fw_domain_wait_ack_with_fallback(const struct intel_uncore_forcewake_domain *d,
fw_clear(d, FORCEWAKE_KERNEL_FALLBACK);
} while (!ack_detected && pass++ < 10);
- DRM_DEBUG_DRIVER("%s had to use fallback to %s ack, 0x%x (passes %u)\n",
- intel_uncore_forcewake_domain_to_str(d->id),
- type == ACK_SET ? "set" : "clear",
- fw_ack(d),
- pass);
+ drm_dbg(&d->uncore->i915->drm,
+ "%s had to use fallback to %s ack, 0x%x (passes %u)\n",
+ intel_uncore_forcewake_domain_to_str(d->id),
+ type == ACK_SET ? "set" : "clear",
+ fw_ack(d),
+ pass);
return ack_detected ? 0 : -ETIMEDOUT;
}
@@ -254,8 +258,9 @@ static inline void
fw_domain_wait_ack_set(const struct intel_uncore_forcewake_domain *d)
{
if (wait_ack_set(d, FORCEWAKE_KERNEL)) {
- DRM_ERROR("%s: timed out waiting for forcewake ack request.\n",
- intel_uncore_forcewake_domain_to_str(d->id));
+ drm_err(&d->uncore->i915->drm,
+ "%s: timed out waiting for forcewake ack request.\n",
+ intel_uncore_forcewake_domain_to_str(d->id));
add_taint_for_CI(d->uncore->i915, TAINT_WARN); /* CI now unreliable */
}
}
@@ -888,10 +893,13 @@ void assert_forcewakes_active(struct intel_uncore *uncore,
spin_unlock_irq(&uncore->lock);
}
-/* We give fast paths for the really cool registers */
+/*
+ * We give fast paths for the really cool registers. The second range includes
+ * media domains (and the GSC starting from Xe_LPM+)
+ */
#define NEEDS_FORCE_WAKE(reg) ({ \
u32 __reg = (reg); \
- __reg < 0x40000 || __reg >= GEN11_BSD_RING_BASE; \
+ __reg < 0x40000 || __reg >= 0x116000; \
})
static int fw_range_cmp(u32 offset, const struct intel_forcewake_range *entry)
@@ -1131,6 +1139,45 @@ static const struct i915_range pvc_shadowed_regs[] = {
{ .start = 0x1F8510, .end = 0x1F8550 },
};
+static const struct i915_range mtl_shadowed_regs[] = {
+ { .start = 0x2030, .end = 0x2030 },
+ { .start = 0x2510, .end = 0x2550 },
+ { .start = 0xA008, .end = 0xA00C },
+ { .start = 0xA188, .end = 0xA188 },
+ { .start = 0xA278, .end = 0xA278 },
+ { .start = 0xA540, .end = 0xA56C },
+ { .start = 0xC050, .end = 0xC050 },
+ { .start = 0xC340, .end = 0xC340 },
+ { .start = 0xC4C8, .end = 0xC4C8 },
+ { .start = 0xC4E0, .end = 0xC4E0 },
+ { .start = 0xC600, .end = 0xC600 },
+ { .start = 0xC658, .end = 0xC658 },
+ { .start = 0xCFD4, .end = 0xCFDC },
+ { .start = 0x22030, .end = 0x22030 },
+ { .start = 0x22510, .end = 0x22550 },
+};
+
+static const struct i915_range xelpmp_shadowed_regs[] = {
+ { .start = 0x1C0030, .end = 0x1C0030 },
+ { .start = 0x1C0510, .end = 0x1C0550 },
+ { .start = 0x1C8030, .end = 0x1C8030 },
+ { .start = 0x1C8510, .end = 0x1C8550 },
+ { .start = 0x1D0030, .end = 0x1D0030 },
+ { .start = 0x1D0510, .end = 0x1D0550 },
+ { .start = 0x38A008, .end = 0x38A00C },
+ { .start = 0x38A188, .end = 0x38A188 },
+ { .start = 0x38A278, .end = 0x38A278 },
+ { .start = 0x38A540, .end = 0x38A56C },
+ { .start = 0x38A618, .end = 0x38A618 },
+ { .start = 0x38C050, .end = 0x38C050 },
+ { .start = 0x38C340, .end = 0x38C340 },
+ { .start = 0x38C4C8, .end = 0x38C4C8 },
+ { .start = 0x38C4E0, .end = 0x38C4E4 },
+ { .start = 0x38C600, .end = 0x38C600 },
+ { .start = 0x38C658, .end = 0x38C658 },
+ { .start = 0x38CFD4, .end = 0x38CFDC },
+};
+
static int mmio_range_cmp(u32 key, const struct i915_range *range)
{
if (key < range->start)
@@ -1639,25 +1686,27 @@ static const struct intel_forcewake_range __pvc_fw_ranges[] = {
GEN_FW_RANGE(0x12000, 0x12fff, 0), /*
0x12000 - 0x127ff: always on
0x12800 - 0x12fff: reserved */
- GEN_FW_RANGE(0x13000, 0x23fff, FORCEWAKE_GT), /*
+ GEN_FW_RANGE(0x13000, 0x19fff, FORCEWAKE_GT), /*
0x13000 - 0x135ff: gt
0x13600 - 0x147ff: reserved
0x14800 - 0x153ff: gt
- 0x15400 - 0x19fff: reserved
- 0x1a000 - 0x1ffff: gt
- 0x20000 - 0x21fff: reserved
- 0x22000 - 0x23fff: gt */
+ 0x15400 - 0x19fff: reserved */
+ GEN_FW_RANGE(0x1a000, 0x21fff, FORCEWAKE_RENDER), /*
+ 0x1a000 - 0x1ffff: render
+ 0x20000 - 0x21fff: reserved */
+ GEN_FW_RANGE(0x22000, 0x23fff, FORCEWAKE_GT),
GEN_FW_RANGE(0x24000, 0x2417f, 0), /*
24000 - 0x2407f: always on
24080 - 0x2417f: reserved */
- GEN_FW_RANGE(0x24180, 0x3ffff, FORCEWAKE_GT), /*
+ GEN_FW_RANGE(0x24180, 0x25fff, FORCEWAKE_GT), /*
0x24180 - 0x241ff: gt
0x24200 - 0x251ff: reserved
0x25200 - 0x252ff: gt
- 0x25300 - 0x25fff: reserved
- 0x26000 - 0x27fff: gt
- 0x28000 - 0x2ffff: reserved
- 0x30000 - 0x3ffff: gt */
+ 0x25300 - 0x25fff: reserved */
+ GEN_FW_RANGE(0x26000, 0x2ffff, FORCEWAKE_RENDER), /*
+ 0x26000 - 0x27fff: render
+ 0x28000 - 0x2ffff: reserved */
+ GEN_FW_RANGE(0x30000, 0x3ffff, FORCEWAKE_GT),
GEN_FW_RANGE(0x40000, 0x1bffff, 0),
GEN_FW_RANGE(0x1c0000, 0x1c3fff, FORCEWAKE_MEDIA_VDBOX0), /*
0x1c0000 - 0x1c2bff: VD0
@@ -1679,6 +1728,162 @@ static const struct intel_forcewake_range __pvc_fw_ranges[] = {
GEN_FW_RANGE(0x3e0000, 0x3effff, FORCEWAKE_GT),
};
+static const struct intel_forcewake_range __mtl_fw_ranges[] = {
+ GEN_FW_RANGE(0x0, 0xaff, 0),
+ GEN_FW_RANGE(0xb00, 0xbff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0xc00, 0xfff, 0),
+ GEN_FW_RANGE(0x1000, 0x1fff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0x2000, 0x26ff, FORCEWAKE_RENDER),
+ GEN_FW_RANGE(0x2700, 0x2fff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0x3000, 0x3fff, FORCEWAKE_RENDER),
+ GEN_FW_RANGE(0x4000, 0x51ff, FORCEWAKE_GT), /*
+ 0x4000 - 0x48ff: render
+ 0x4900 - 0x51ff: reserved */
+ GEN_FW_RANGE(0x5200, 0x7fff, FORCEWAKE_RENDER), /*
+ 0x5200 - 0x53ff: render
+ 0x5400 - 0x54ff: reserved
+ 0x5500 - 0x7fff: render */
+ GEN_FW_RANGE(0x8000, 0x813f, FORCEWAKE_GT),
+ GEN_FW_RANGE(0x8140, 0x817f, FORCEWAKE_RENDER), /*
+ 0x8140 - 0x815f: render
+ 0x8160 - 0x817f: reserved */
+ GEN_FW_RANGE(0x8180, 0x81ff, 0),
+ GEN_FW_RANGE(0x8200, 0x94cf, FORCEWAKE_GT), /*
+ 0x8200 - 0x87ff: gt
+ 0x8800 - 0x8dff: reserved
+ 0x8e00 - 0x8f7f: gt
+ 0x8f80 - 0x8fff: reserved
+ 0x9000 - 0x947f: gt
+ 0x9480 - 0x94cf: reserved */
+ GEN_FW_RANGE(0x94d0, 0x955f, FORCEWAKE_RENDER),
+ GEN_FW_RANGE(0x9560, 0x967f, 0), /*
+ 0x9560 - 0x95ff: always on
+ 0x9600 - 0x967f: reserved */
+ GEN_FW_RANGE(0x9680, 0x97ff, FORCEWAKE_RENDER), /*
+ 0x9680 - 0x96ff: render
+ 0x9700 - 0x97ff: reserved */
+ GEN_FW_RANGE(0x9800, 0xcfff, FORCEWAKE_GT), /*
+ 0x9800 - 0xb4ff: gt
+ 0xb500 - 0xbfff: reserved
+ 0xc000 - 0xcfff: gt */
+ GEN_FW_RANGE(0xd000, 0xd7ff, 0), /*
+ 0xd000 - 0xd3ff: always on
+ 0xd400 - 0xd7ff: reserved */
+ GEN_FW_RANGE(0xd800, 0xd87f, FORCEWAKE_RENDER),
+ GEN_FW_RANGE(0xd880, 0xdbff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0xdc00, 0xdcff, FORCEWAKE_RENDER),
+ GEN_FW_RANGE(0xdd00, 0xde7f, FORCEWAKE_GT), /*
+ 0xdd00 - 0xddff: gt
+ 0xde00 - 0xde7f: reserved */
+ GEN_FW_RANGE(0xde80, 0xe8ff, FORCEWAKE_RENDER), /*
+ 0xde80 - 0xdfff: render
+ 0xe000 - 0xe0ff: reserved
+ 0xe100 - 0xe8ff: render */
+ GEN_FW_RANGE(0xe900, 0xe9ff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0xea00, 0x147ff, 0), /*
+ 0xea00 - 0x11fff: reserved
+ 0x12000 - 0x127ff: always on
+ 0x12800 - 0x147ff: reserved */
+ GEN_FW_RANGE(0x14800, 0x19fff, FORCEWAKE_GT), /*
+ 0x14800 - 0x153ff: gt
+ 0x15400 - 0x19fff: reserved */
+ GEN_FW_RANGE(0x1a000, 0x21fff, FORCEWAKE_RENDER), /*
+ 0x1a000 - 0x1bfff: render
+ 0x1c000 - 0x21fff: reserved */
+ GEN_FW_RANGE(0x22000, 0x23fff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0x24000, 0x2ffff, 0), /*
+ 0x24000 - 0x2407f: always on
+ 0x24080 - 0x2ffff: reserved */
+ GEN_FW_RANGE(0x30000, 0x3ffff, FORCEWAKE_GT)
+};
+
+/*
+ * Note that the register ranges here are the final offsets after
+ * translation of the GSI block to the 0x380000 offset.
+ *
+ * NOTE: There are a couple MCR ranges near the bottom of this table
+ * that need to power up either VD0 or VD2 depending on which replicated
+ * instance of the register we're trying to access. Our forcewake logic
+ * at the moment doesn't have a good way to take steering into consideration,
+ * and the driver doesn't even access any registers in those ranges today,
+ * so for now we just mark those ranges as FORCEWAKE_ALL. That will ensure
+ * proper operation if we do start using the ranges in the future, and we
+ * can determine at that time whether it's worth adding extra complexity to
+ * the forcewake handling to take steering into consideration.
+ */
+static const struct intel_forcewake_range __xelpmp_fw_ranges[] = {
+ GEN_FW_RANGE(0x0, 0x115fff, 0), /* render GT range */
+ GEN_FW_RANGE(0x116000, 0x11ffff, FORCEWAKE_GSC), /*
+ 0x116000 - 0x117fff: gsc
+ 0x118000 - 0x119fff: reserved
+ 0x11a000 - 0x11efff: gsc
+ 0x11f000 - 0x11ffff: reserved */
+ GEN_FW_RANGE(0x120000, 0x1bffff, 0), /* non-GT range */
+ GEN_FW_RANGE(0x1c0000, 0x1c7fff, FORCEWAKE_MEDIA_VDBOX0), /*
+ 0x1c0000 - 0x1c3dff: VD0
+ 0x1c3e00 - 0x1c3eff: reserved
+ 0x1c3f00 - 0x1c3fff: VD0
+ 0x1c4000 - 0x1c7fff: reserved */
+ GEN_FW_RANGE(0x1c8000, 0x1cbfff, FORCEWAKE_MEDIA_VEBOX0), /*
+ 0x1c8000 - 0x1ca0ff: VE0
+ 0x1ca100 - 0x1cbfff: reserved */
+ GEN_FW_RANGE(0x1cc000, 0x1cffff, FORCEWAKE_MEDIA_VDBOX0), /*
+ 0x1cc000 - 0x1cdfff: VD0
+ 0x1ce000 - 0x1cffff: reserved */
+ GEN_FW_RANGE(0x1d0000, 0x1d7fff, FORCEWAKE_MEDIA_VDBOX2), /*
+ 0x1d0000 - 0x1d3dff: VD2
+ 0x1d3e00 - 0x1d3eff: reserved
+ 0x1d4000 - 0x1d7fff: VD2 */
+ GEN_FW_RANGE(0x1d8000, 0x1da0ff, FORCEWAKE_MEDIA_VEBOX1),
+ GEN_FW_RANGE(0x1da100, 0x380aff, 0), /*
+ 0x1da100 - 0x23ffff: reserved
+ 0x240000 - 0x37ffff: non-GT range
+ 0x380000 - 0x380aff: reserved */
+ GEN_FW_RANGE(0x380b00, 0x380bff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0x380c00, 0x380fff, 0),
+ GEN_FW_RANGE(0x381000, 0x38817f, FORCEWAKE_GT), /*
+ 0x381000 - 0x381fff: gt
+ 0x382000 - 0x383fff: reserved
+ 0x384000 - 0x384aff: gt
+ 0x384b00 - 0x3851ff: reserved
+ 0x385200 - 0x3871ff: gt
+ 0x387200 - 0x387fff: reserved
+ 0x388000 - 0x38813f: gt
+ 0x388140 - 0x38817f: reserved */
+ GEN_FW_RANGE(0x388180, 0x3882ff, 0), /*
+ 0x388180 - 0x3881ff: always on
+ 0x388200 - 0x3882ff: reserved */
+ GEN_FW_RANGE(0x388300, 0x38955f, FORCEWAKE_GT), /*
+ 0x388300 - 0x38887f: gt
+ 0x388880 - 0x388fff: reserved
+ 0x389000 - 0x38947f: gt
+ 0x389480 - 0x38955f: reserved */
+ GEN_FW_RANGE(0x389560, 0x389fff, 0), /*
+ 0x389560 - 0x3895ff: always on
+ 0x389600 - 0x389fff: reserved */
+ GEN_FW_RANGE(0x38a000, 0x38cfff, FORCEWAKE_GT), /*
+ 0x38a000 - 0x38afff: gt
+ 0x38b000 - 0x38bfff: reserved
+ 0x38c000 - 0x38cfff: gt */
+ GEN_FW_RANGE(0x38d000, 0x38d11f, 0),
+ GEN_FW_RANGE(0x38d120, 0x391fff, FORCEWAKE_GT), /*
+ 0x38d120 - 0x38dfff: gt
+ 0x38e000 - 0x38efff: reserved
+ 0x38f000 - 0x38ffff: gt
+ 0x389000 - 0x391fff: reserved */
+ GEN_FW_RANGE(0x392000, 0x392fff, 0), /*
+ 0x392000 - 0x3927ff: always on
+ 0x392800 - 0x292fff: reserved */
+ GEN_FW_RANGE(0x393000, 0x3931ff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0x393200, 0x39323f, FORCEWAKE_ALL), /* instance-based, see note above */
+ GEN_FW_RANGE(0x393240, 0x3933ff, FORCEWAKE_GT),
+ GEN_FW_RANGE(0x393400, 0x3934ff, FORCEWAKE_ALL), /* instance-based, see note above */
+ GEN_FW_RANGE(0x393500, 0x393c7f, 0), /*
+ 0x393500 - 0x393bff: reserved
+ 0x393c00 - 0x393c7f: always on */
+ GEN_FW_RANGE(0x393c80, 0x393dff, FORCEWAKE_GT),
+};
+
static void
ilk_dummy_write(struct intel_uncore *uncore)
{
@@ -2021,6 +2226,7 @@ static int __fw_domain_init(struct intel_uncore *uncore,
BUILD_BUG_ON(FORCEWAKE_MEDIA_VEBOX1 != (1 << FW_DOMAIN_ID_MEDIA_VEBOX1));
BUILD_BUG_ON(FORCEWAKE_MEDIA_VEBOX2 != (1 << FW_DOMAIN_ID_MEDIA_VEBOX2));
BUILD_BUG_ON(FORCEWAKE_MEDIA_VEBOX3 != (1 << FW_DOMAIN_ID_MEDIA_VEBOX3));
+ BUILD_BUG_ON(FORCEWAKE_GSC != (1 << FW_DOMAIN_ID_GSC));
d->mask = BIT(domain_id);
@@ -2085,17 +2291,26 @@ static int intel_uncore_fw_domains_init(struct intel_uncore *uncore)
(ret ?: (ret = __fw_domain_init((uncore__), (id__), (set__), (ack__))))
if (GRAPHICS_VER(i915) >= 11) {
- /* we'll prune the domains of missing engines later */
- intel_engine_mask_t emask = RUNTIME_INFO(i915)->platform_engine_mask;
+ intel_engine_mask_t emask;
int i;
+ /* we'll prune the domains of missing engines later */
+ emask = uncore->gt->info.engine_mask;
+
uncore->fw_get_funcs = &uncore_get_fallback;
- fw_domain_init(uncore, FW_DOMAIN_ID_RENDER,
- FORCEWAKE_RENDER_GEN9,
- FORCEWAKE_ACK_RENDER_GEN9);
- fw_domain_init(uncore, FW_DOMAIN_ID_GT,
- FORCEWAKE_GT_GEN9,
- FORCEWAKE_ACK_GT_GEN9);
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70))
+ fw_domain_init(uncore, FW_DOMAIN_ID_GT,
+ FORCEWAKE_GT_GEN9,
+ FORCEWAKE_ACK_GT_MTL);
+ else
+ fw_domain_init(uncore, FW_DOMAIN_ID_GT,
+ FORCEWAKE_GT_GEN9,
+ FORCEWAKE_ACK_GT_GEN9);
+
+ if (RCS_MASK(uncore->gt) || CCS_MASK(uncore->gt))
+ fw_domain_init(uncore, FW_DOMAIN_ID_RENDER,
+ FORCEWAKE_RENDER_GEN9,
+ FORCEWAKE_ACK_RENDER_GEN9);
for (i = 0; i < I915_MAX_VCS; i++) {
if (!__HAS_ENGINE(emask, _VCS(i)))
@@ -2113,6 +2328,10 @@ static int intel_uncore_fw_domains_init(struct intel_uncore *uncore)
FORCEWAKE_MEDIA_VEBOX_GEN11(i),
FORCEWAKE_ACK_MEDIA_VEBOX_GEN11(i));
}
+
+ if (uncore->gt->type == GT_MEDIA)
+ fw_domain_init(uncore, FW_DOMAIN_ID_GSC,
+ FORCEWAKE_REQ_GSC, FORCEWAKE_ACK_GSC);
} else if (IS_GRAPHICS_VER(i915, 9, 10)) {
uncore->fw_get_funcs = &uncore_get_fallback;
fw_domain_init(uncore, FW_DOMAIN_ID_RENDER,
@@ -2300,6 +2519,22 @@ static void uncore_raw_init(struct intel_uncore *uncore)
}
}
+static int uncore_media_forcewake_init(struct intel_uncore *uncore)
+{
+ struct drm_i915_private *i915 = uncore->i915;
+
+ if (MEDIA_VER(i915) >= 13) {
+ ASSIGN_FW_DOMAINS_TABLE(uncore, __xelpmp_fw_ranges);
+ ASSIGN_SHADOW_TABLE(uncore, xelpmp_shadowed_regs);
+ ASSIGN_WRITE_MMIO_VFUNCS(uncore, fwtable);
+ } else {
+ MISSING_CASE(MEDIA_VER(i915));
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
static int uncore_forcewake_init(struct intel_uncore *uncore)
{
struct drm_i915_private *i915 = uncore->i915;
@@ -2314,7 +2549,14 @@ static int uncore_forcewake_init(struct intel_uncore *uncore)
ASSIGN_READ_MMIO_VFUNCS(uncore, fwtable);
- if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 60)) {
+ if (uncore->gt->type == GT_MEDIA)
+ return uncore_media_forcewake_init(uncore);
+
+ if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70)) {
+ ASSIGN_FW_DOMAINS_TABLE(uncore, __mtl_fw_ranges);
+ ASSIGN_SHADOW_TABLE(uncore, mtl_shadowed_regs);
+ ASSIGN_WRITE_MMIO_VFUNCS(uncore, fwtable);
+ } else if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 60)) {
ASSIGN_FW_DOMAINS_TABLE(uncore, __pvc_fw_ranges);
ASSIGN_SHADOW_TABLE(uncore, pvc_shadowed_regs);
ASSIGN_WRITE_MMIO_VFUNCS(uncore, fwtable);
diff --git a/drivers/gpu/drm/i915/intel_uncore.h b/drivers/gpu/drm/i915/intel_uncore.h
index 5022bac8..e9e3849 100644
--- a/drivers/gpu/drm/i915/intel_uncore.h
+++ b/drivers/gpu/drm/i915/intel_uncore.h
@@ -62,6 +62,7 @@ enum forcewake_domain_id {
FW_DOMAIN_ID_MEDIA_VEBOX1,
FW_DOMAIN_ID_MEDIA_VEBOX2,
FW_DOMAIN_ID_MEDIA_VEBOX3,
+ FW_DOMAIN_ID_GSC,
FW_DOMAIN_ID_COUNT
};
@@ -82,6 +83,7 @@ enum forcewake_domains {
FORCEWAKE_MEDIA_VEBOX1 = BIT(FW_DOMAIN_ID_MEDIA_VEBOX1),
FORCEWAKE_MEDIA_VEBOX2 = BIT(FW_DOMAIN_ID_MEDIA_VEBOX2),
FORCEWAKE_MEDIA_VEBOX3 = BIT(FW_DOMAIN_ID_MEDIA_VEBOX3),
+ FORCEWAKE_GSC = BIT(FW_DOMAIN_ID_GSC),
FORCEWAKE_ALL = BIT(FW_DOMAIN_ID_COUNT) - 1,
};
@@ -380,20 +382,6 @@ __uncore_write(write_notrace, 32, l, false)
*/
__uncore_read(read64, 64, q, true)
-static inline u64
-intel_uncore_read64_2x32(struct intel_uncore *uncore,
- i915_reg_t lower_reg, i915_reg_t upper_reg)
-{
- u32 upper, lower, old_upper, loop = 0;
- upper = intel_uncore_read(uncore, upper_reg);
- do {
- old_upper = upper;
- lower = intel_uncore_read(uncore, lower_reg);
- upper = intel_uncore_read(uncore, upper_reg);
- } while (upper != old_upper && loop++ < 2);
- return (u64)upper << 32 | lower;
-}
-
#define intel_uncore_posting_read(...) ((void)intel_uncore_read_notrace(__VA_ARGS__))
#define intel_uncore_posting_read16(...) ((void)intel_uncore_read16_notrace(__VA_ARGS__))
@@ -431,15 +419,15 @@ intel_uncore_read64_2x32(struct intel_uncore *uncore,
#define intel_uncore_write64_fw(...) __raw_uncore_write64(__VA_ARGS__)
#define intel_uncore_posting_read_fw(...) ((void)intel_uncore_read_fw(__VA_ARGS__))
-static inline void intel_uncore_rmw(struct intel_uncore *uncore,
- i915_reg_t reg, u32 clear, u32 set)
+static inline u32 intel_uncore_rmw(struct intel_uncore *uncore,
+ i915_reg_t reg, u32 clear, u32 set)
{
u32 old, val;
old = intel_uncore_read(uncore, reg);
val = (old & ~clear) | set;
- if (val != old)
- intel_uncore_write(uncore, reg, val);
+ intel_uncore_write(uncore, reg, val);
+ return old;
}
static inline void intel_uncore_rmw_fw(struct intel_uncore *uncore,
@@ -453,6 +441,36 @@ static inline void intel_uncore_rmw_fw(struct intel_uncore *uncore,
intel_uncore_write_fw(uncore, reg, val);
}
+static inline u64
+intel_uncore_read64_2x32(struct intel_uncore *uncore,
+ i915_reg_t lower_reg, i915_reg_t upper_reg)
+{
+ u32 upper, lower, old_upper, loop = 0;
+ enum forcewake_domains fw_domains;
+ unsigned long flags;
+
+ fw_domains = intel_uncore_forcewake_for_reg(uncore, lower_reg,
+ FW_REG_READ);
+
+ fw_domains |= intel_uncore_forcewake_for_reg(uncore, upper_reg,
+ FW_REG_READ);
+
+ spin_lock_irqsave(&uncore->lock, flags);
+ intel_uncore_forcewake_get__locked(uncore, fw_domains);
+
+ upper = intel_uncore_read_fw(uncore, upper_reg);
+ do {
+ old_upper = upper;
+ lower = intel_uncore_read_fw(uncore, lower_reg);
+ upper = intel_uncore_read_fw(uncore, upper_reg);
+ } while (upper != old_upper && loop++ < 2);
+
+ intel_uncore_forcewake_put__locked(uncore, fw_domains);
+ spin_unlock_irqrestore(&uncore->lock, flags);
+
+ return (u64)upper << 32 | lower;
+}
+
static inline int intel_uncore_write_and_verify(struct intel_uncore *uncore,
i915_reg_t reg, u32 val,
u32 mask, u32 expected_val)
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp.c b/drivers/gpu/drm/i915/pxp/intel_pxp.c
index 69cdaad..5efe61f 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp.c
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp.c
@@ -103,19 +103,15 @@ static int create_vcs_context(struct intel_pxp *pxp)
static void destroy_vcs_context(struct intel_pxp *pxp)
{
- intel_engine_destroy_pinned_context(fetch_and_zero(&pxp->ce));
+ if (pxp->ce)
+ intel_engine_destroy_pinned_context(fetch_and_zero(&pxp->ce));
}
-void intel_pxp_init(struct intel_pxp *pxp)
+static void pxp_init_full(struct intel_pxp *pxp)
{
struct intel_gt *gt = pxp_to_gt(pxp);
int ret;
- if (!HAS_PXP(gt->i915))
- return;
-
- mutex_init(&pxp->tee_mutex);
-
/*
* we'll use the completion to check if there is a termination pending,
* so we start it as completed and we reinit it when a termination
@@ -124,8 +120,7 @@ void intel_pxp_init(struct intel_pxp *pxp)
init_completion(&pxp->termination);
complete_all(&pxp->termination);
- mutex_init(&pxp->arb_mutex);
- INIT_WORK(&pxp->session_work, intel_pxp_session_work);
+ intel_pxp_session_management_init(pxp);
ret = create_vcs_context(pxp);
if (ret)
@@ -143,11 +138,26 @@ void intel_pxp_init(struct intel_pxp *pxp)
destroy_vcs_context(pxp);
}
-void intel_pxp_fini(struct intel_pxp *pxp)
+void intel_pxp_init(struct intel_pxp *pxp)
{
- if (!intel_pxp_is_enabled(pxp))
+ struct intel_gt *gt = pxp_to_gt(pxp);
+
+ /* we rely on the mei PXP module */
+ if (!IS_ENABLED(CONFIG_INTEL_MEI_PXP))
return;
+ /*
+ * If HuC is loaded by GSC but PXP is disabled, we can skip the init of
+ * the full PXP session/object management and just init the tee channel.
+ */
+ if (HAS_PXP(gt->i915))
+ pxp_init_full(pxp);
+ else if (intel_huc_is_loaded_by_gsc(>->uc.huc) && intel_uc_uses_huc(>->uc))
+ intel_pxp_tee_component_init(pxp);
+}
+
+void intel_pxp_fini(struct intel_pxp *pxp)
+{
pxp->arb_is_valid = false;
intel_pxp_tee_component_fini(pxp);
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp.h b/drivers/gpu/drm/i915/pxp/intel_pxp.h
index 73847e5..2da3090 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp.h
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp.h
@@ -12,7 +12,6 @@
struct intel_pxp;
struct drm_i915_gem_object;
-#ifdef CONFIG_DRM_I915_PXP
struct intel_gt *pxp_to_gt(const struct intel_pxp *pxp);
bool intel_pxp_is_enabled(const struct intel_pxp *pxp);
bool intel_pxp_is_active(const struct intel_pxp *pxp);
@@ -32,36 +31,5 @@ int intel_pxp_key_check(struct intel_pxp *pxp,
bool assign);
void intel_pxp_invalidate(struct intel_pxp *pxp);
-#else
-static inline void intel_pxp_init(struct intel_pxp *pxp)
-{
-}
-
-static inline void intel_pxp_fini(struct intel_pxp *pxp)
-{
-}
-
-static inline int intel_pxp_start(struct intel_pxp *pxp)
-{
- return -ENODEV;
-}
-
-static inline bool intel_pxp_is_enabled(const struct intel_pxp *pxp)
-{
- return false;
-}
-
-static inline bool intel_pxp_is_active(const struct intel_pxp *pxp)
-{
- return false;
-}
-
-static inline int intel_pxp_key_check(struct intel_pxp *pxp,
- struct drm_i915_gem_object *obj,
- bool assign)
-{
- return -ENODEV;
-}
-#endif
#endif /* __INTEL_PXP_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h
new file mode 100644
index 0000000..739f907
--- /dev/null
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright(c) 2020, Intel Corporation. All rights reserved.
+ */
+
+#ifndef __INTEL_PXP_FW_INTERFACE_42_H__
+#define __INTEL_PXP_FW_INTERFACE_42_H__
+
+#include <linux/types.h>
+#include "intel_pxp_cmd_interface_cmn.h"
+
+/* PXP-Opcode for Init Session */
+#define PXP42_CMDID_INIT_SESSION 0x1e
+
+/* PXP-Input-Packet: Init Session (Arb-Session) */
+struct pxp42_create_arb_in {
+ struct pxp_cmd_header header;
+ u32 protection_mode;
+#define PXP42_ARB_SESSION_MODE_HEAVY 0x2
+ u32 session_id;
+} __packed;
+
+/* PXP-Output-Packet: Init Session */
+struct pxp42_create_arb_out {
+ struct pxp_cmd_header header;
+} __packed;
+
+#endif /* __INTEL_PXP_FW_INTERFACE_42_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_43.h b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_43.h
new file mode 100644
index 0000000..ad67e3f
--- /dev/null
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_43.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright(c) 2022, Intel Corporation. All rights reserved.
+ */
+
+#ifndef __INTEL_PXP_FW_INTERFACE_43_H__
+#define __INTEL_PXP_FW_INTERFACE_43_H__
+
+#include <linux/types.h>
+#include "intel_pxp_cmd_interface_cmn.h"
+
+/* PXP-Cmd-Op definitions */
+#define PXP43_CMDID_START_HUC_AUTH 0x0000003A
+
+/* PXP-Input-Packet: HUC-Authentication */
+struct pxp43_start_huc_auth_in {
+ struct pxp_cmd_header header;
+ __le64 huc_base_address;
+} __packed;
+
+/* PXP-Output-Packet: HUC-Authentication */
+struct pxp43_start_huc_auth_out {
+ struct pxp_cmd_header header;
+} __packed;
+
+#endif /* __INTEL_PXP_FW_INTERFACE_43_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h
new file mode 100644
index 0000000..c2f2339
--- /dev/null
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright(c) 2022, Intel Corporation. All rights reserved.
+ */
+
+#ifndef __INTEL_PXP_FW_INTERFACE_CMN_H__
+#define __INTEL_PXP_FW_INTERFACE_CMN_H__
+
+#include <linux/types.h>
+
+#define PXP_APIVER(x, y) (((x) & 0xFFFF) << 16 | ((y) & 0xFFFF))
+
+/*
+ * there are a lot of status codes for PXP, but we only define the cross-API
+ * common ones that we actually can handle in the kernel driver. Other failure
+ * codes should be printed to error msg for debug.
+ */
+enum pxp_status {
+ PXP_STATUS_SUCCESS = 0x0,
+ PXP_STATUS_OP_NOT_PERMITTED = 0x4013
+};
+
+/* Common PXP FW message header */
+struct pxp_cmd_header {
+ u32 api_version;
+ u32 command_id;
+ union {
+ u32 status; /* out */
+ u32 stream_id; /* in */
+ };
+ /* Length of the message (excluding the header) */
+ u32 buffer_len;
+} __packed;
+
+#endif /* __INTEL_PXP_FW_INTERFACE_CMN_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_huc.c b/drivers/gpu/drm/i915/pxp/intel_pxp_huc.c
new file mode 100644
index 0000000..2e11655
--- /dev/null
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_huc.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright(c) 2021-2022, Intel Corporation. All rights reserved.
+ */
+
+#include <drm/i915_drm.h>
+
+#include "i915_drv.h"
+
+#include "gem/i915_gem_region.h"
+#include "gt/intel_gt.h"
+
+#include "intel_pxp.h"
+#include "intel_pxp_huc.h"
+#include "intel_pxp_tee.h"
+#include "intel_pxp_types.h"
+#include "intel_pxp_cmd_interface_43.h"
+
+int intel_pxp_huc_load_and_auth(struct intel_pxp *pxp)
+{
+ struct intel_gt *gt = pxp_to_gt(pxp);
+ struct intel_huc *huc = >->uc.huc;
+ struct pxp43_start_huc_auth_in huc_in = {0};
+ struct pxp43_start_huc_auth_out huc_out = {0};
+ dma_addr_t huc_phys_addr;
+ u8 client_id = 0;
+ u8 fence_id = 0;
+ int err;
+
+ if (!pxp->pxp_component)
+ return -ENODEV;
+
+ huc_phys_addr = i915_gem_object_get_dma_address(huc->fw.obj, 0);
+
+ /* write the PXP message into the lmem (the sg list) */
+ huc_in.header.api_version = PXP_APIVER(4, 3);
+ huc_in.header.command_id = PXP43_CMDID_START_HUC_AUTH;
+ huc_in.header.status = 0;
+ huc_in.header.buffer_len = sizeof(huc_in.huc_base_address);
+ huc_in.huc_base_address = huc_phys_addr;
+
+ err = intel_pxp_tee_stream_message(pxp, client_id, fence_id,
+ &huc_in, sizeof(huc_in),
+ &huc_out, sizeof(huc_out));
+ if (err < 0) {
+ drm_err(>->i915->drm,
+ "Failed to send HuC load and auth command to GSC [%d]!\n",
+ err);
+ return err;
+ }
+
+ /*
+ * HuC does sometimes survive suspend/resume (it depends on how "deep"
+ * a sleep state the device reaches) so we can end up here on resume
+ * with HuC already loaded, in which case the GSC will return
+ * PXP_STATUS_OP_NOT_PERMITTED. We can therefore consider the GuC
+ * correctly transferred in this scenario; if the same error is ever
+ * returned with HuC not loaded we'll still catch it when we check the
+ * authentication bit later.
+ */
+ if (huc_out.header.status != PXP_STATUS_SUCCESS &&
+ huc_out.header.status != PXP_STATUS_OP_NOT_PERMITTED) {
+ drm_err(>->i915->drm,
+ "HuC load failed with GSC error = 0x%x\n",
+ huc_out.header.status);
+ return -EPROTO;
+ }
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_huc.h b/drivers/gpu/drm/i915/pxp/intel_pxp_huc.h
new file mode 100644
index 0000000..e40847a
--- /dev/null
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_huc.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright(c) 2021-2022, Intel Corporation. All rights reserved.
+ */
+
+#ifndef __INTEL_PXP_HUC_H__
+#define __INTEL_PXP_HUC_H__
+
+struct intel_pxp;
+
+int intel_pxp_huc_load_and_auth(struct intel_pxp *pxp);
+
+#endif /* __INTEL_PXP_HUC_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_irq.h b/drivers/gpu/drm/i915/pxp/intel_pxp_irq.h
index 8b57936..8c292dc 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_irq.h
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_irq.h
@@ -27,6 +27,14 @@ void intel_pxp_irq_handler(struct intel_pxp *pxp, u16 iir);
static inline void intel_pxp_irq_handler(struct intel_pxp *pxp, u16 iir)
{
}
+
+static inline void intel_pxp_irq_enable(struct intel_pxp *pxp)
+{
+}
+
+static inline void intel_pxp_irq_disable(struct intel_pxp *pxp)
+{
+}
#endif
#endif /* __INTEL_PXP_IRQ_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c
index 1bb5b52..8557236 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c
@@ -77,6 +77,7 @@ static int pxp_create_arb_session(struct intel_pxp *pxp)
drm_err(>->i915->drm, "arb session failed to go in play\n");
return ret;
}
+ drm_dbg(>->i915->drm, "PXP ARB session is alive\n");
if (!++pxp->key_instance)
++pxp->key_instance;
@@ -137,7 +138,7 @@ static void pxp_terminate_complete(struct intel_pxp *pxp)
complete_all(&pxp->termination);
}
-void intel_pxp_session_work(struct work_struct *work)
+static void pxp_session_work(struct work_struct *work)
{
struct intel_pxp *pxp = container_of(work, typeof(*pxp), session_work);
struct intel_gt *gt = pxp_to_gt(pxp);
@@ -172,3 +173,9 @@ void intel_pxp_session_work(struct work_struct *work)
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
}
+
+void intel_pxp_session_management_init(struct intel_pxp *pxp)
+{
+ mutex_init(&pxp->arb_mutex);
+ INIT_WORK(&pxp->session_work, pxp_session_work);
+}
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_session.h b/drivers/gpu/drm/i915/pxp/intel_pxp_session.h
index ba4c9d2..903ac52 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_session.h
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_session.h
@@ -8,8 +8,13 @@
#include <linux/types.h>
-struct work_struct;
+struct intel_pxp;
-void intel_pxp_session_work(struct work_struct *work);
-
+#ifdef CONFIG_DRM_I915_PXP
+void intel_pxp_session_management_init(struct intel_pxp *pxp);
+#else
+static inline void intel_pxp_session_management_init(struct intel_pxp *pxp)
+{
+}
+#endif
#endif /* __INTEL_PXP_SESSION_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c
index 4b6f565..b0c9170 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c
@@ -8,11 +8,14 @@
#include <drm/i915_pxp_tee_interface.h>
#include <drm/i915_component.h>
+#include "gem/i915_gem_lmem.h"
+
#include "i915_drv.h"
#include "intel_pxp.h"
#include "intel_pxp_session.h"
#include "intel_pxp_tee.h"
-#include "intel_pxp_tee_interface.h"
+#include "intel_pxp_cmd_interface_42.h"
+#include "intel_pxp_huc.h"
static inline struct intel_pxp *i915_dev_to_pxp(struct device *i915_kdev)
{
@@ -69,6 +72,47 @@ static int intel_pxp_tee_io_message(struct intel_pxp *pxp,
return ret;
}
+int intel_pxp_tee_stream_message(struct intel_pxp *pxp,
+ u8 client_id, u32 fence_id,
+ void *msg_in, size_t msg_in_len,
+ void *msg_out, size_t msg_out_len)
+{
+ /* TODO: for bigger objects we need to use a sg of 4k pages */
+ const size_t max_msg_size = PAGE_SIZE;
+ struct drm_i915_private *i915 = pxp_to_gt(pxp)->i915;
+ struct i915_pxp_component *pxp_component = pxp->pxp_component;
+ unsigned int offset = 0;
+ struct scatterlist *sg;
+ int ret;
+
+ if (msg_in_len > max_msg_size || msg_out_len > max_msg_size)
+ return -ENOSPC;
+
+ mutex_lock(&pxp->tee_mutex);
+
+ if (unlikely(!pxp_component || !pxp_component->ops->gsc_command)) {
+ ret = -ENODEV;
+ goto unlock;
+ }
+
+ GEM_BUG_ON(!pxp->stream_cmd.obj);
+
+ sg = i915_gem_object_get_sg_dma(pxp->stream_cmd.obj, 0, &offset);
+
+ memcpy(pxp->stream_cmd.vaddr, msg_in, msg_in_len);
+
+ ret = pxp_component->ops->gsc_command(pxp_component->tee_dev, client_id,
+ fence_id, sg, msg_in_len, sg);
+ if (ret < 0)
+ drm_err(&i915->drm, "Failed to send PXP TEE gsc command\n");
+ else
+ memcpy(msg_out, pxp->stream_cmd.vaddr, msg_out_len);
+
+unlock:
+ mutex_unlock(&pxp->tee_mutex);
+ return ret;
+}
+
/**
* i915_pxp_tee_component_bind - bind function to pass the function pointers to pxp_tee
* @i915_kdev: pointer to i915 kernel device
@@ -84,24 +128,36 @@ static int i915_pxp_tee_component_bind(struct device *i915_kdev,
{
struct drm_i915_private *i915 = kdev_to_i915(i915_kdev);
struct intel_pxp *pxp = i915_dev_to_pxp(i915_kdev);
+ struct intel_uc *uc = &pxp_to_gt(pxp)->uc;
intel_wakeref_t wakeref;
+ int ret = 0;
mutex_lock(&pxp->tee_mutex);
pxp->pxp_component = data;
pxp->pxp_component->tee_dev = tee_kdev;
mutex_unlock(&pxp->tee_mutex);
+ if (intel_uc_uses_huc(uc) && intel_huc_is_loaded_by_gsc(&uc->huc)) {
+ with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
+ /* load huc via pxp */
+ ret = intel_huc_fw_load_and_auth_via_gsc(&uc->huc);
+ if (ret < 0)
+ drm_err(&i915->drm, "failed to load huc via gsc %d\n", ret);
+ }
+ }
+
/* if we are suspended, the HW will be re-initialized on resume */
wakeref = intel_runtime_pm_get_if_in_use(&i915->runtime_pm);
if (!wakeref)
return 0;
/* the component is required to fully start the PXP HW */
- intel_pxp_init_hw(pxp);
+ if (intel_pxp_is_enabled(pxp))
+ intel_pxp_init_hw(pxp);
intel_runtime_pm_put(&i915->runtime_pm, wakeref);
- return 0;
+ return ret;
}
static void i915_pxp_tee_component_unbind(struct device *i915_kdev,
@@ -111,8 +167,9 @@ static void i915_pxp_tee_component_unbind(struct device *i915_kdev,
struct intel_pxp *pxp = i915_dev_to_pxp(i915_kdev);
intel_wakeref_t wakeref;
- with_intel_runtime_pm_if_in_use(&i915->runtime_pm, wakeref)
- intel_pxp_fini_hw(pxp);
+ if (intel_pxp_is_enabled(pxp))
+ with_intel_runtime_pm_if_in_use(&i915->runtime_pm, wakeref)
+ intel_pxp_fini_hw(pxp);
mutex_lock(&pxp->tee_mutex);
pxp->pxp_component = NULL;
@@ -124,22 +181,92 @@ static const struct component_ops i915_pxp_tee_component_ops = {
.unbind = i915_pxp_tee_component_unbind,
};
+static int alloc_streaming_command(struct intel_pxp *pxp)
+{
+ struct drm_i915_private *i915 = pxp_to_gt(pxp)->i915;
+ struct drm_i915_gem_object *obj = NULL;
+ void *cmd;
+ int err;
+
+ pxp->stream_cmd.obj = NULL;
+ pxp->stream_cmd.vaddr = NULL;
+
+ if (!IS_DGFX(i915))
+ return 0;
+
+ /* allocate lmem object of one page for PXP command memory and store it */
+ obj = i915_gem_object_create_lmem(i915, PAGE_SIZE, I915_BO_ALLOC_CONTIGUOUS);
+ if (IS_ERR(obj)) {
+ drm_err(&i915->drm, "Failed to allocate pxp streaming command!\n");
+ return PTR_ERR(obj);
+ }
+
+ err = i915_gem_object_pin_pages_unlocked(obj);
+ if (err) {
+ drm_err(&i915->drm, "Failed to pin gsc message page!\n");
+ goto out_put;
+ }
+
+ /* map the lmem into the virtual memory pointer */
+ cmd = i915_gem_object_pin_map_unlocked(obj, i915_coherent_map_type(i915, obj, true));
+ if (IS_ERR(cmd)) {
+ drm_err(&i915->drm, "Failed to map gsc message page!\n");
+ err = PTR_ERR(cmd);
+ goto out_unpin;
+ }
+
+ memset(cmd, 0, obj->base.size);
+
+ pxp->stream_cmd.obj = obj;
+ pxp->stream_cmd.vaddr = cmd;
+
+ return 0;
+
+out_unpin:
+ i915_gem_object_unpin_pages(obj);
+out_put:
+ i915_gem_object_put(obj);
+ return err;
+}
+
+static void free_streaming_command(struct intel_pxp *pxp)
+{
+ struct drm_i915_gem_object *obj = fetch_and_zero(&pxp->stream_cmd.obj);
+
+ if (!obj)
+ return;
+
+ i915_gem_object_unpin_map(obj);
+ i915_gem_object_unpin_pages(obj);
+ i915_gem_object_put(obj);
+}
+
int intel_pxp_tee_component_init(struct intel_pxp *pxp)
{
int ret;
struct intel_gt *gt = pxp_to_gt(pxp);
struct drm_i915_private *i915 = gt->i915;
+ mutex_init(&pxp->tee_mutex);
+
+ ret = alloc_streaming_command(pxp);
+ if (ret)
+ return ret;
+
ret = component_add_typed(i915->drm.dev, &i915_pxp_tee_component_ops,
I915_COMPONENT_PXP);
if (ret < 0) {
drm_err(&i915->drm, "Failed to add PXP component (%d)\n", ret);
- return ret;
+ goto out_free;
}
pxp->pxp_component_added = true;
return 0;
+
+out_free:
+ free_streaming_command(pxp);
+ return ret;
}
void intel_pxp_tee_component_fini(struct intel_pxp *pxp)
@@ -151,20 +278,22 @@ void intel_pxp_tee_component_fini(struct intel_pxp *pxp)
component_del(i915->drm.dev, &i915_pxp_tee_component_ops);
pxp->pxp_component_added = false;
+
+ free_streaming_command(pxp);
}
int intel_pxp_tee_cmd_create_arb_session(struct intel_pxp *pxp,
int arb_session_id)
{
struct drm_i915_private *i915 = pxp_to_gt(pxp)->i915;
- struct pxp_tee_create_arb_in msg_in = {0};
- struct pxp_tee_create_arb_out msg_out = {0};
+ struct pxp42_create_arb_in msg_in = {0};
+ struct pxp42_create_arb_out msg_out = {0};
int ret;
- msg_in.header.api_version = PXP_TEE_APIVER;
- msg_in.header.command_id = PXP_TEE_ARB_CMDID;
+ msg_in.header.api_version = PXP_APIVER(4, 2);
+ msg_in.header.command_id = PXP42_CMDID_INIT_SESSION;
msg_in.header.buffer_len = sizeof(msg_in) - sizeof(msg_in.header);
- msg_in.protection_mode = PXP_TEE_ARB_PROTECTION_MODE;
+ msg_in.protection_mode = PXP42_ARB_SESSION_MODE_HEAVY;
msg_in.session_id = arb_session_id;
ret = intel_pxp_tee_io_message(pxp,
@@ -174,6 +303,9 @@ int intel_pxp_tee_cmd_create_arb_session(struct intel_pxp *pxp,
if (ret)
drm_err(&i915->drm, "Failed to send tee msg ret=[%d]\n", ret);
+ else if (msg_out.header.status != 0x0)
+ drm_warn(&i915->drm, "PXP firmware failed arb session init request ret=[0x%08x]\n",
+ msg_out.header.status);
return ret;
}
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.h b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.h
index c136053..aeb3dfe 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.h
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.h
@@ -14,4 +14,9 @@ void intel_pxp_tee_component_fini(struct intel_pxp *pxp);
int intel_pxp_tee_cmd_create_arb_session(struct intel_pxp *pxp,
int arb_session_id);
+int intel_pxp_tee_stream_message(struct intel_pxp *pxp,
+ u8 client_id, u32 fence_id,
+ void *msg_in, size_t msg_in_len,
+ void *msg_out, size_t msg_out_len);
+
#endif /* __INTEL_PXP_TEE_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_tee_interface.h b/drivers/gpu/drm/i915/pxp/intel_pxp_tee_interface.h
deleted file mode 100644
index 36e9b08..0000000
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_tee_interface.h
+++ /dev/null
@@ -1,36 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-/*
- * Copyright(c) 2020, Intel Corporation. All rights reserved.
- */
-
-#ifndef __INTEL_PXP_TEE_INTERFACE_H__
-#define __INTEL_PXP_TEE_INTERFACE_H__
-
-#include <linux/types.h>
-
-#define PXP_TEE_APIVER 0x40002
-#define PXP_TEE_ARB_CMDID 0x1e
-#define PXP_TEE_ARB_PROTECTION_MODE 0x2
-
-/* PXP TEE message header */
-struct pxp_tee_cmd_header {
- u32 api_version;
- u32 command_id;
- u32 status;
- /* Length of the message (excluding the header) */
- u32 buffer_len;
-} __packed;
-
-/* PXP TEE message input to create a arbitrary session */
-struct pxp_tee_create_arb_in {
- struct pxp_tee_cmd_header header;
- u32 protection_mode;
- u32 session_id;
-} __packed;
-
-/* PXP TEE message output to create a arbitrary session */
-struct pxp_tee_create_arb_out {
- struct pxp_tee_cmd_header header;
-} __packed;
-
-#endif /* __INTEL_PXP_TEE_INTERFACE_H__ */
diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_types.h b/drivers/gpu/drm/i915/pxp/intel_pxp_types.h
index 7ce5f37..f74b1e1 100644
--- a/drivers/gpu/drm/i915/pxp/intel_pxp_types.h
+++ b/drivers/gpu/drm/i915/pxp/intel_pxp_types.h
@@ -53,6 +53,12 @@ struct intel_pxp {
/** @tee_mutex: protects the tee channel binding and messaging. */
struct mutex tee_mutex;
+ /** @stream_cmd: LMEM obj used to send stream PXP commands to the GSC */
+ struct {
+ struct drm_i915_gem_object *obj; /* contains PXP command memory */
+ void *vaddr; /* virtual memory for PXP command */
+ } stream_cmd;
+
/**
* @hw_state_invalidated: if the HW perceives an attack on the integrity
* of the encryption it will invalidate the keys and expect SW to
diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
index e050a2d..eae7d94 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
@@ -27,6 +27,7 @@
#include "gem/i915_gem_context.h"
#include "gem/i915_gem_internal.h"
+#include "gem/i915_gem_lmem.h"
#include "gem/i915_gem_region.h"
#include "gem/selftests/mock_context.h"
#include "gt/intel_context.h"
@@ -60,7 +61,6 @@ static int fake_get_pages(struct drm_i915_gem_object *obj)
#define PFN_BIAS 0x1000
struct sg_table *pages;
struct scatterlist *sg;
- unsigned int sg_page_sizes;
typeof(obj->base.size) rem;
pages = kmalloc(sizeof(*pages), GFP);
@@ -73,7 +73,6 @@ static int fake_get_pages(struct drm_i915_gem_object *obj)
return -ENOMEM;
}
- sg_page_sizes = 0;
rem = obj->base.size;
for (sg = pages->sgl; sg; sg = sg_next(sg)) {
unsigned long len = min_t(typeof(rem), rem, BIT(31));
@@ -82,13 +81,12 @@ static int fake_get_pages(struct drm_i915_gem_object *obj)
sg_set_page(sg, pfn_to_page(PFN_BIAS), len, 0);
sg_dma_address(sg) = page_to_phys(sg_page(sg));
sg_dma_len(sg) = len;
- sg_page_sizes |= len;
rem -= len;
}
GEM_BUG_ON(rem);
- __i915_gem_object_set_pages(obj, pages, sg_page_sizes);
+ __i915_gem_object_set_pages(obj, pages);
return 0;
#undef GFP
@@ -1113,15 +1111,8 @@ static int misaligned_case(struct i915_address_space *vm, struct intel_memory_re
expected_node_size = expected_vma_size;
if (HAS_64K_PAGES(vm->i915) && i915_gem_object_is_lmem(obj)) {
- /*
- * The compact-pt should expand lmem node to 2MB for the ppGTT,
- * for all other cases we should only expect 64K.
- */
expected_vma_size = round_up(size, I915_GTT_PAGE_SIZE_64K);
- if (NEEDS_COMPACT_PT(vm->i915) && !i915_is_ggtt(vm))
- expected_node_size = round_up(size, I915_GTT_PAGE_SIZE_2M);
- else
- expected_node_size = round_up(size, I915_GTT_PAGE_SIZE_64K);
+ expected_node_size = round_up(size, I915_GTT_PAGE_SIZE_64K);
}
if (vma->size != expected_vma_size || vma->node.size != expected_node_size) {
diff --git a/drivers/gpu/drm/i915/selftests/i915_perf.c b/drivers/gpu/drm/i915/selftests/i915_perf.c
index 429c6d7..24dde55 100644
--- a/drivers/gpu/drm/i915/selftests/i915_perf.c
+++ b/drivers/gpu/drm/i915/selftests/i915_perf.c
@@ -102,6 +102,12 @@ test_stream(struct i915_perf *perf)
I915_OA_FORMAT_A32u40_A4u32_B8_C8 : I915_OA_FORMAT_C4_B8,
};
struct i915_perf_stream *stream;
+ struct intel_gt *gt;
+
+ if (!props.engine)
+ return NULL;
+
+ gt = props.engine->gt;
if (!oa_config)
return NULL;
@@ -116,12 +122,12 @@ test_stream(struct i915_perf *perf)
stream->perf = perf;
- mutex_lock(&perf->lock);
+ mutex_lock(>->perf.lock);
if (i915_oa_stream_init(stream, ¶m, &props)) {
kfree(stream);
stream = NULL;
}
- mutex_unlock(&perf->lock);
+ mutex_unlock(>->perf.lock);
i915_oa_config_put(oa_config);
@@ -130,11 +136,11 @@ test_stream(struct i915_perf *perf)
static void stream_destroy(struct i915_perf_stream *stream)
{
- struct i915_perf *perf = stream->perf;
+ struct intel_gt *gt = stream->engine->gt;
- mutex_lock(&perf->lock);
+ mutex_lock(>->perf.lock);
i915_perf_destroy_locked(stream);
- mutex_unlock(&perf->lock);
+ mutex_unlock(>->perf.lock);
}
static int live_sanitycheck(void *arg)
diff --git a/drivers/gpu/drm/i915/selftests/i915_request.c b/drivers/gpu/drm/i915/selftests/i915_request.c
index 818a490..0daa866 100644
--- a/drivers/gpu/drm/i915/selftests/i915_request.c
+++ b/drivers/gpu/drm/i915/selftests/i915_request.c
@@ -299,9 +299,18 @@ __live_request_alloc(struct intel_context *ce)
return intel_context_create_request(ce);
}
-static int __igt_breadcrumbs_smoketest(void *arg)
+struct smoke_thread {
+ struct kthread_worker *worker;
+ struct kthread_work work;
+ struct smoketest *t;
+ bool stop;
+ int result;
+};
+
+static void __igt_breadcrumbs_smoketest(struct kthread_work *work)
{
- struct smoketest *t = arg;
+ struct smoke_thread *thread = container_of(work, typeof(*thread), work);
+ struct smoketest *t = thread->t;
const unsigned int max_batch = min(t->ncontexts, t->max_batch) - 1;
const unsigned int total = 4 * t->ncontexts + 1;
unsigned int num_waits = 0, num_fences = 0;
@@ -320,8 +329,10 @@ static int __igt_breadcrumbs_smoketest(void *arg)
*/
requests = kcalloc(total, sizeof(*requests), GFP_KERNEL);
- if (!requests)
- return -ENOMEM;
+ if (!requests) {
+ thread->result = -ENOMEM;
+ return;
+ }
order = i915_random_order(total, &prng);
if (!order) {
@@ -329,7 +340,7 @@ static int __igt_breadcrumbs_smoketest(void *arg)
goto out_requests;
}
- while (!kthread_should_stop()) {
+ while (!READ_ONCE(thread->stop)) {
struct i915_sw_fence *submit, *wait;
unsigned int n, count;
@@ -437,7 +448,7 @@ static int __igt_breadcrumbs_smoketest(void *arg)
kfree(order);
out_requests:
kfree(requests);
- return err;
+ thread->result = err;
}
static int mock_breadcrumbs_smoketest(void *arg)
@@ -450,7 +461,7 @@ static int mock_breadcrumbs_smoketest(void *arg)
.request_alloc = __mock_request_alloc
};
unsigned int ncpus = num_online_cpus();
- struct task_struct **threads;
+ struct smoke_thread *threads;
unsigned int n;
int ret = 0;
@@ -479,28 +490,37 @@ static int mock_breadcrumbs_smoketest(void *arg)
}
for (n = 0; n < ncpus; n++) {
- threads[n] = kthread_run(__igt_breadcrumbs_smoketest,
- &t, "igt/%d", n);
- if (IS_ERR(threads[n])) {
- ret = PTR_ERR(threads[n]);
+ struct kthread_worker *worker;
+
+ worker = kthread_create_worker(0, "igt/%d", n);
+ if (IS_ERR(worker)) {
+ ret = PTR_ERR(worker);
ncpus = n;
break;
}
- get_task_struct(threads[n]);
+ threads[n].worker = worker;
+ threads[n].t = &t;
+ threads[n].stop = false;
+ threads[n].result = 0;
+
+ kthread_init_work(&threads[n].work,
+ __igt_breadcrumbs_smoketest);
+ kthread_queue_work(worker, &threads[n].work);
}
- yield(); /* start all threads before we begin */
msleep(jiffies_to_msecs(i915_selftest.timeout_jiffies));
for (n = 0; n < ncpus; n++) {
int err;
- err = kthread_stop(threads[n]);
+ WRITE_ONCE(threads[n].stop, true);
+ kthread_flush_work(&threads[n].work);
+ err = READ_ONCE(threads[n].result);
if (err < 0 && !ret)
ret = err;
- put_task_struct(threads[n]);
+ kthread_destroy_worker(threads[n].worker);
}
pr_info("Completed %lu waits for %lu fence across %d cpus\n",
atomic_long_read(&t.num_waits),
@@ -1203,9 +1223,7 @@ static int live_all_engines(void *arg)
goto out_request;
}
- err = i915_request_await_object(request[idx], batch->obj, 0);
- if (err == 0)
- err = i915_vma_move_to_active(batch, request[idx], 0);
+ err = i915_vma_move_to_active(batch, request[idx], 0);
GEM_BUG_ON(err);
err = engine->emit_bb_start(request[idx],
@@ -1332,10 +1350,7 @@ static int live_sequential_engines(void *arg)
}
}
- err = i915_request_await_object(request[idx],
- batch->obj, false);
- if (err == 0)
- err = i915_vma_move_to_active(batch, request[idx], 0);
+ err = i915_vma_move_to_active(batch, request[idx], 0);
GEM_BUG_ON(err);
err = engine->emit_bb_start(request[idx],
@@ -1419,9 +1434,18 @@ static int live_sequential_engines(void *arg)
return err;
}
-static int __live_parallel_engine1(void *arg)
+struct parallel_thread {
+ struct kthread_worker *worker;
+ struct kthread_work work;
+ struct intel_engine_cs *engine;
+ int result;
+};
+
+static void __live_parallel_engine1(struct kthread_work *work)
{
- struct intel_engine_cs *engine = arg;
+ struct parallel_thread *thread =
+ container_of(work, typeof(*thread), work);
+ struct intel_engine_cs *engine = thread->engine;
IGT_TIMEOUT(end_time);
unsigned long count;
int err = 0;
@@ -1452,12 +1476,14 @@ static int __live_parallel_engine1(void *arg)
intel_engine_pm_put(engine);
pr_info("%s: %lu request + sync\n", engine->name, count);
- return err;
+ thread->result = err;
}
-static int __live_parallel_engineN(void *arg)
+static void __live_parallel_engineN(struct kthread_work *work)
{
- struct intel_engine_cs *engine = arg;
+ struct parallel_thread *thread =
+ container_of(work, typeof(*thread), work);
+ struct intel_engine_cs *engine = thread->engine;
IGT_TIMEOUT(end_time);
unsigned long count;
int err = 0;
@@ -1479,7 +1505,7 @@ static int __live_parallel_engineN(void *arg)
intel_engine_pm_put(engine);
pr_info("%s: %lu requests\n", engine->name, count);
- return err;
+ thread->result = err;
}
static bool wake_all(struct drm_i915_private *i915)
@@ -1505,9 +1531,11 @@ static int wait_for_all(struct drm_i915_private *i915)
return -ETIME;
}
-static int __live_parallel_spin(void *arg)
+static void __live_parallel_spin(struct kthread_work *work)
{
- struct intel_engine_cs *engine = arg;
+ struct parallel_thread *thread =
+ container_of(work, typeof(*thread), work);
+ struct intel_engine_cs *engine = thread->engine;
struct igt_spinner spin;
struct i915_request *rq;
int err = 0;
@@ -1520,7 +1548,8 @@ static int __live_parallel_spin(void *arg)
if (igt_spinner_init(&spin, engine->gt)) {
wake_all(engine->i915);
- return -ENOMEM;
+ thread->result = -ENOMEM;
+ return;
}
intel_engine_pm_get(engine);
@@ -1553,22 +1582,22 @@ static int __live_parallel_spin(void *arg)
out_spin:
igt_spinner_fini(&spin);
- return err;
+ thread->result = err;
}
static int live_parallel_engines(void *arg)
{
struct drm_i915_private *i915 = arg;
- static int (* const func[])(void *arg) = {
+ static void (* const func[])(struct kthread_work *) = {
__live_parallel_engine1,
__live_parallel_engineN,
__live_parallel_spin,
NULL,
};
const unsigned int nengines = num_uabi_engines(i915);
+ struct parallel_thread *threads;
struct intel_engine_cs *engine;
- int (* const *fn)(void *arg);
- struct task_struct **tsk;
+ void (* const *fn)(struct kthread_work *);
int err = 0;
/*
@@ -1576,8 +1605,8 @@ static int live_parallel_engines(void *arg)
* tests that we load up the system maximally.
*/
- tsk = kcalloc(nengines, sizeof(*tsk), GFP_KERNEL);
- if (!tsk)
+ threads = kcalloc(nengines, sizeof(*threads), GFP_KERNEL);
+ if (!threads)
return -ENOMEM;
for (fn = func; !err && *fn; fn++) {
@@ -1594,37 +1623,44 @@ static int live_parallel_engines(void *arg)
idx = 0;
for_each_uabi_engine(engine, i915) {
- tsk[idx] = kthread_run(*fn, engine,
- "igt/parallel:%s",
- engine->name);
- if (IS_ERR(tsk[idx])) {
- err = PTR_ERR(tsk[idx]);
+ struct kthread_worker *worker;
+
+ worker = kthread_create_worker(0, "igt/parallel:%s",
+ engine->name);
+ if (IS_ERR(worker)) {
+ err = PTR_ERR(worker);
break;
}
- get_task_struct(tsk[idx++]);
- }
- yield(); /* start all threads before we kthread_stop() */
+ threads[idx].worker = worker;
+ threads[idx].result = 0;
+ threads[idx].engine = engine;
+
+ kthread_init_work(&threads[idx].work, *fn);
+ kthread_queue_work(worker, &threads[idx].work);
+ idx++;
+ }
idx = 0;
for_each_uabi_engine(engine, i915) {
int status;
- if (IS_ERR(tsk[idx]))
+ if (!threads[idx].worker)
break;
- status = kthread_stop(tsk[idx]);
+ kthread_flush_work(&threads[idx].work);
+ status = READ_ONCE(threads[idx].result);
if (status && !err)
err = status;
- put_task_struct(tsk[idx++]);
+ kthread_destroy_worker(threads[idx++].worker);
}
if (igt_live_test_end(&t))
err = -EIO;
}
- kfree(tsk);
+ kfree(threads);
return err;
}
@@ -1669,10 +1705,11 @@ static int live_breadcrumbs_smoketest(void *arg)
{
struct drm_i915_private *i915 = arg;
const unsigned int nengines = num_uabi_engines(i915);
- const unsigned int ncpus = num_online_cpus();
+ const unsigned int ncpus = /* saturate with nengines * ncpus */
+ max_t(int, 2, DIV_ROUND_UP(num_online_cpus(), nengines));
unsigned long num_waits, num_fences;
struct intel_engine_cs *engine;
- struct task_struct **threads;
+ struct smoke_thread *threads;
struct igt_live_test live;
intel_wakeref_t wakeref;
struct smoketest *smoke;
@@ -1741,28 +1778,31 @@ static int live_breadcrumbs_smoketest(void *arg)
goto out_flush;
}
/* One ring interleaved between requests from all cpus */
- smoke[idx].max_batch /= num_online_cpus() + 1;
+ smoke[idx].max_batch /= ncpus + 1;
pr_debug("Limiting batches to %d requests on %s\n",
smoke[idx].max_batch, engine->name);
for (n = 0; n < ncpus; n++) {
- struct task_struct *tsk;
+ unsigned int i = idx * ncpus + n;
+ struct kthread_worker *worker;
- tsk = kthread_run(__igt_breadcrumbs_smoketest,
- &smoke[idx], "igt/%d.%d", idx, n);
- if (IS_ERR(tsk)) {
- ret = PTR_ERR(tsk);
+ worker = kthread_create_worker(0, "igt/%d.%d", idx, n);
+ if (IS_ERR(worker)) {
+ ret = PTR_ERR(worker);
goto out_flush;
}
- get_task_struct(tsk);
- threads[idx * ncpus + n] = tsk;
+ threads[i].worker = worker;
+ threads[i].t = &smoke[idx];
+
+ kthread_init_work(&threads[i].work,
+ __igt_breadcrumbs_smoketest);
+ kthread_queue_work(worker, &threads[i].work);
}
idx++;
}
- yield(); /* start all threads before we begin */
msleep(jiffies_to_msecs(i915_selftest.timeout_jiffies));
out_flush:
@@ -1771,17 +1811,19 @@ static int live_breadcrumbs_smoketest(void *arg)
num_fences = 0;
for_each_uabi_engine(engine, i915) {
for (n = 0; n < ncpus; n++) {
- struct task_struct *tsk = threads[idx * ncpus + n];
+ unsigned int i = idx * ncpus + n;
int err;
- if (!tsk)
+ if (!threads[i].worker)
continue;
- err = kthread_stop(tsk);
+ WRITE_ONCE(threads[i].stop, true);
+ kthread_flush_work(&threads[i].work);
+ err = READ_ONCE(threads[i].result);
if (err < 0 && !ret)
ret = err;
- put_task_struct(tsk);
+ kthread_destroy_worker(threads[i].worker);
}
num_waits += atomic_long_read(&smoke[idx].num_waits);
@@ -2891,9 +2933,18 @@ static int perf_series_engines(void *arg)
return err;
}
-static int p_sync0(void *arg)
+struct p_thread {
+ struct perf_stats p;
+ struct kthread_worker *worker;
+ struct kthread_work work;
+ struct intel_engine_cs *engine;
+ int result;
+};
+
+static void p_sync0(struct kthread_work *work)
{
- struct perf_stats *p = arg;
+ struct p_thread *thread = container_of(work, typeof(*thread), work);
+ struct perf_stats *p = &thread->p;
struct intel_engine_cs *engine = p->engine;
struct intel_context *ce;
IGT_TIMEOUT(end_time);
@@ -2902,13 +2953,16 @@ static int p_sync0(void *arg)
int err = 0;
ce = intel_context_create(engine);
- if (IS_ERR(ce))
- return PTR_ERR(ce);
+ if (IS_ERR(ce)) {
+ thread->result = PTR_ERR(ce);
+ return;
+ }
err = intel_context_pin(ce);
if (err) {
intel_context_put(ce);
- return err;
+ thread->result = err;
+ return;
}
if (intel_engine_supports_stats(engine)) {
@@ -2958,12 +3012,13 @@ static int p_sync0(void *arg)
intel_context_unpin(ce);
intel_context_put(ce);
- return err;
+ thread->result = err;
}
-static int p_sync1(void *arg)
+static void p_sync1(struct kthread_work *work)
{
- struct perf_stats *p = arg;
+ struct p_thread *thread = container_of(work, typeof(*thread), work);
+ struct perf_stats *p = &thread->p;
struct intel_engine_cs *engine = p->engine;
struct i915_request *prev = NULL;
struct intel_context *ce;
@@ -2973,13 +3028,16 @@ static int p_sync1(void *arg)
int err = 0;
ce = intel_context_create(engine);
- if (IS_ERR(ce))
- return PTR_ERR(ce);
+ if (IS_ERR(ce)) {
+ thread->result = PTR_ERR(ce);
+ return;
+ }
err = intel_context_pin(ce);
if (err) {
intel_context_put(ce);
- return err;
+ thread->result = err;
+ return;
}
if (intel_engine_supports_stats(engine)) {
@@ -3031,12 +3089,13 @@ static int p_sync1(void *arg)
intel_context_unpin(ce);
intel_context_put(ce);
- return err;
+ thread->result = err;
}
-static int p_many(void *arg)
+static void p_many(struct kthread_work *work)
{
- struct perf_stats *p = arg;
+ struct p_thread *thread = container_of(work, typeof(*thread), work);
+ struct perf_stats *p = &thread->p;
struct intel_engine_cs *engine = p->engine;
struct intel_context *ce;
IGT_TIMEOUT(end_time);
@@ -3045,13 +3104,16 @@ static int p_many(void *arg)
bool busy;
ce = intel_context_create(engine);
- if (IS_ERR(ce))
- return PTR_ERR(ce);
+ if (IS_ERR(ce)) {
+ thread->result = PTR_ERR(ce);
+ return;
+ }
err = intel_context_pin(ce);
if (err) {
intel_context_put(ce);
- return err;
+ thread->result = err;
+ return;
}
if (intel_engine_supports_stats(engine)) {
@@ -3092,26 +3154,23 @@ static int p_many(void *arg)
intel_context_unpin(ce);
intel_context_put(ce);
- return err;
+ thread->result = err;
}
static int perf_parallel_engines(void *arg)
{
struct drm_i915_private *i915 = arg;
- static int (* const func[])(void *arg) = {
+ static void (* const func[])(struct kthread_work *) = {
p_sync0,
p_sync1,
p_many,
NULL,
};
const unsigned int nengines = num_uabi_engines(i915);
+ void (* const *fn)(struct kthread_work *);
struct intel_engine_cs *engine;
- int (* const *fn)(void *arg);
struct pm_qos_request qos;
- struct {
- struct perf_stats p;
- struct task_struct *tsk;
- } *engines;
+ struct p_thread *engines;
int err = 0;
engines = kcalloc(nengines, sizeof(*engines), GFP_KERNEL);
@@ -3134,36 +3193,45 @@ static int perf_parallel_engines(void *arg)
idx = 0;
for_each_uabi_engine(engine, i915) {
+ struct kthread_worker *worker;
+
intel_engine_pm_get(engine);
memset(&engines[idx].p, 0, sizeof(engines[idx].p));
- engines[idx].p.engine = engine;
- engines[idx].tsk = kthread_run(*fn, &engines[idx].p,
- "igt:%s", engine->name);
- if (IS_ERR(engines[idx].tsk)) {
- err = PTR_ERR(engines[idx].tsk);
+ worker = kthread_create_worker(0, "igt:%s",
+ engine->name);
+ if (IS_ERR(worker)) {
+ err = PTR_ERR(worker);
intel_engine_pm_put(engine);
break;
}
- get_task_struct(engines[idx++].tsk);
- }
+ engines[idx].worker = worker;
+ engines[idx].result = 0;
+ engines[idx].p.engine = engine;
+ engines[idx].engine = engine;
- yield(); /* start all threads before we kthread_stop() */
+ kthread_init_work(&engines[idx].work, *fn);
+ kthread_queue_work(worker, &engines[idx].work);
+ idx++;
+ }
idx = 0;
for_each_uabi_engine(engine, i915) {
int status;
- if (IS_ERR(engines[idx].tsk))
+ if (!engines[idx].worker)
break;
- status = kthread_stop(engines[idx].tsk);
+ kthread_flush_work(&engines[idx].work);
+ status = READ_ONCE(engines[idx].result);
if (status && !err)
err = status;
intel_engine_pm_put(engine);
- put_task_struct(engines[idx++].tsk);
+
+ kthread_destroy_worker(engines[idx].worker);
+ idx++;
}
if (igt_live_test_end(&t))
diff --git a/drivers/gpu/drm/i915/selftests/igt_spinner.c b/drivers/gpu/drm/i915/selftests/igt_spinner.c
index 0c22594..16978ac 100644
--- a/drivers/gpu/drm/i915/selftests/igt_spinner.c
+++ b/drivers/gpu/drm/i915/selftests/igt_spinner.c
@@ -119,22 +119,6 @@ static u64 hws_address(const struct i915_vma *hws,
return hws->node.start + seqno_offset(rq->fence.context);
}
-static int move_to_active(struct i915_vma *vma,
- struct i915_request *rq,
- unsigned int flags)
-{
- int err;
-
- i915_vma_lock(vma);
- err = i915_request_await_object(rq, vma->obj,
- flags & EXEC_OBJECT_WRITE);
- if (err == 0)
- err = i915_vma_move_to_active(vma, rq, flags);
- i915_vma_unlock(vma);
-
- return err;
-}
-
struct i915_request *
igt_spinner_create_request(struct igt_spinner *spin,
struct intel_context *ce,
@@ -165,11 +149,11 @@ igt_spinner_create_request(struct igt_spinner *spin,
if (IS_ERR(rq))
return ERR_CAST(rq);
- err = move_to_active(vma, rq, 0);
+ err = igt_vma_move_to_active_unlocked(vma, rq, 0);
if (err)
goto cancel_rq;
- err = move_to_active(hws, rq, 0);
+ err = igt_vma_move_to_active_unlocked(hws, rq, 0);
if (err)
goto cancel_rq;
diff --git a/drivers/gpu/drm/i915/selftests/intel_uncore.c b/drivers/gpu/drm/i915/selftests/intel_uncore.c
index fda9bb7..e428150 100644
--- a/drivers/gpu/drm/i915/selftests/intel_uncore.c
+++ b/drivers/gpu/drm/i915/selftests/intel_uncore.c
@@ -70,6 +70,8 @@ static int intel_shadow_table_check(void)
{ gen12_shadowed_regs, ARRAY_SIZE(gen12_shadowed_regs) },
{ dg2_shadowed_regs, ARRAY_SIZE(dg2_shadowed_regs) },
{ pvc_shadowed_regs, ARRAY_SIZE(pvc_shadowed_regs) },
+ { mtl_shadowed_regs, ARRAY_SIZE(mtl_shadowed_regs) },
+ { xelpmp_shadowed_regs, ARRAY_SIZE(xelpmp_shadowed_regs) },
};
const struct i915_range *range;
unsigned int i, j;
@@ -117,6 +119,8 @@ int intel_uncore_mock_selftests(void)
{ __gen12_fw_ranges, ARRAY_SIZE(__gen12_fw_ranges), true },
{ __xehp_fw_ranges, ARRAY_SIZE(__xehp_fw_ranges), true },
{ __pvc_fw_ranges, ARRAY_SIZE(__pvc_fw_ranges), true },
+ { __mtl_fw_ranges, ARRAY_SIZE(__mtl_fw_ranges), true },
+ { __xelpmp_fw_ranges, ARRAY_SIZE(__xelpmp_fw_ranges), true },
};
int err, i;
diff --git a/drivers/gpu/drm/i915/selftests/mock_gem_device.c b/drivers/gpu/drm/i915/selftests/mock_gem_device.c
index fff11c9..f6a7c0b 100644
--- a/drivers/gpu/drm/i915/selftests/mock_gem_device.c
+++ b/drivers/gpu/drm/i915/selftests/mock_gem_device.c
@@ -67,7 +67,6 @@ static void mock_device_release(struct drm_device *dev)
intel_gt_driver_remove(to_gt(i915));
i915_gem_drain_workqueue(i915);
- i915_gem_drain_freed_objects(i915);
mock_fini_ggtt(to_gt(i915)->ggtt);
destroy_workqueue(i915->wq);
diff --git a/drivers/gpu/drm/i915/selftests/mock_region.c b/drivers/gpu/drm/i915/selftests/mock_region.c
index bac21fe..6324eb3 100644
--- a/drivers/gpu/drm/i915/selftests/mock_region.c
+++ b/drivers/gpu/drm/i915/selftests/mock_region.c
@@ -41,7 +41,7 @@ static int mock_region_get_pages(struct drm_i915_gem_object *obj)
}
pages = &obj->mm.rsgt->table;
- __i915_gem_object_set_pages(obj, pages, i915_sg_dma_sizes(pages->sgl));
+ __i915_gem_object_set_pages(obj, pages);
return 0;
diff --git a/drivers/gpu/drm/i915/vlv_sideband.c b/drivers/gpu/drm/i915/vlv_sideband.c
index c260013..6eea6e1 100644
--- a/drivers/gpu/drm/i915/vlv_sideband.c
+++ b/drivers/gpu/drm/i915/vlv_sideband.c
@@ -8,6 +8,8 @@
#include "i915_reg.h"
#include "vlv_sideband.h"
+#include "display/intel_dpio_phy.h"
+
/*
* IOSF sideband, see VLV2_SidebandMsg_HAS.docx and
* VLV_VLV2_PUNIT_HAS_0.8.docx
diff --git a/drivers/gpu/drm/i915/vlv_suspend.c b/drivers/gpu/drm/i915/vlv_suspend.c
index 664fde2..02e63ed 100644
--- a/drivers/gpu/drm/i915/vlv_suspend.c
+++ b/drivers/gpu/drm/i915/vlv_suspend.c
@@ -194,7 +194,6 @@ static void vlv_restore_gunit_s0ix_state(struct drm_i915_private *i915)
{
struct vlv_s0ix_state *s = i915->vlv_s0ix_state;
struct intel_uncore *uncore = &i915->uncore;
- u32 val;
int i;
if (!s)
@@ -262,15 +261,11 @@ static void vlv_restore_gunit_s0ix_state(struct drm_i915_private *i915)
* be restored, as they are used to control the s0ix suspend/resume
* sequence by the caller.
*/
- val = intel_uncore_read(uncore, VLV_GTLC_WAKE_CTRL);
- val &= VLV_GTLC_ALLOWWAKEREQ;
- val |= s->gtlc_wake_ctrl & ~VLV_GTLC_ALLOWWAKEREQ;
- intel_uncore_write(uncore, VLV_GTLC_WAKE_CTRL, val);
+ intel_uncore_rmw(uncore, VLV_GTLC_WAKE_CTRL, ~VLV_GTLC_ALLOWWAKEREQ,
+ s->gtlc_wake_ctrl & ~VLV_GTLC_ALLOWWAKEREQ);
- val = intel_uncore_read(uncore, VLV_GTLC_SURVIVABILITY_REG);
- val &= VLV_GFX_CLK_FORCE_ON_BIT;
- val |= s->gtlc_survive & ~VLV_GFX_CLK_FORCE_ON_BIT;
- intel_uncore_write(uncore, VLV_GTLC_SURVIVABILITY_REG, val);
+ intel_uncore_rmw(uncore, VLV_GTLC_SURVIVABILITY_REG, ~VLV_GFX_CLK_FORCE_ON_BIT,
+ s->gtlc_survive & ~VLV_GFX_CLK_FORCE_ON_BIT);
intel_uncore_write(uncore, VLV_PMWGICZ, s->pmwgicz);
@@ -308,14 +303,10 @@ static int vlv_wait_for_pw_status(struct drm_i915_private *i915,
static int vlv_force_gfx_clock(struct drm_i915_private *i915, bool force_on)
{
struct intel_uncore *uncore = &i915->uncore;
- u32 val;
int err;
- val = intel_uncore_read(uncore, VLV_GTLC_SURVIVABILITY_REG);
- val &= ~VLV_GFX_CLK_FORCE_ON_BIT;
- if (force_on)
- val |= VLV_GFX_CLK_FORCE_ON_BIT;
- intel_uncore_write(uncore, VLV_GTLC_SURVIVABILITY_REG, val);
+ intel_uncore_rmw(uncore, VLV_GTLC_SURVIVABILITY_REG, VLV_GFX_CLK_FORCE_ON_BIT,
+ force_on ? VLV_GFX_CLK_FORCE_ON_BIT : 0);
if (!force_on)
return 0;
@@ -340,11 +331,8 @@ static int vlv_allow_gt_wake(struct drm_i915_private *i915, bool allow)
u32 val;
int err;
- val = intel_uncore_read(uncore, VLV_GTLC_WAKE_CTRL);
- val &= ~VLV_GTLC_ALLOWWAKEREQ;
- if (allow)
- val |= VLV_GTLC_ALLOWWAKEREQ;
- intel_uncore_write(uncore, VLV_GTLC_WAKE_CTRL, val);
+ intel_uncore_rmw(uncore, VLV_GTLC_WAKE_CTRL, VLV_GTLC_ALLOWWAKEREQ,
+ allow ? VLV_GTLC_ALLOWWAKEREQ : 0);
intel_uncore_posting_read(uncore, VLV_GTLC_WAKE_CTRL);
mask = VLV_GTLC_ALLOWWAKEACK;
diff --git a/drivers/gpu/drm/imx/dcss/dcss-kms.c b/drivers/gpu/drm/imx/dcss/dcss-kms.c
index b4f82eb..18df388 100644
--- a/drivers/gpu/drm/imx/dcss/dcss-kms.c
+++ b/drivers/gpu/drm/imx/dcss/dcss-kms.c
@@ -7,7 +7,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge_connector.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_of.h>
@@ -21,7 +21,6 @@ DEFINE_DRM_GEM_DMA_FOPS(dcss_cma_fops);
static const struct drm_mode_config_funcs dcss_drm_mode_config_funcs = {
.fb_create = drm_gem_fb_create,
- .output_poll_changed = drm_fb_helper_output_poll_changed,
.atomic_check = drm_atomic_helper_check,
.atomic_commit = drm_atomic_helper_commit,
};
diff --git a/drivers/gpu/drm/imx/imx-drm-core.c b/drivers/gpu/drm/imx/imx-drm-core.c
index 8dd8b0f..e060fa6 100644
--- a/drivers/gpu/drm/imx/imx-drm-core.c
+++ b/drivers/gpu/drm/imx/imx-drm-core.c
@@ -16,7 +16,7 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/imx/imx-ldb.c b/drivers/gpu/drm/imx/imx-ldb.c
index 4179901..c45fc8f 100644
--- a/drivers/gpu/drm/imx/imx-ldb.c
+++ b/drivers/gpu/drm/imx/imx-ldb.c
@@ -7,6 +7,7 @@
#include <linux/clk.h>
#include <linux/component.h>
+#include <linux/i2c.h>
#include <linux/media-bus-format.h>
#include <linux/mfd/syscon.h>
#include <linux/mfd/syscon/imx6q-iomuxc-gpr.h>
@@ -23,7 +24,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_managed.h>
#include <drm/drm_of.h>
#include <drm/drm_panel.h>
diff --git a/drivers/gpu/drm/imx/imx-tve.c b/drivers/gpu/drm/imx/imx-tve.c
index ab4d1c8..d6832f5 100644
--- a/drivers/gpu/drm/imx/imx-tve.c
+++ b/drivers/gpu/drm/imx/imx-tve.c
@@ -19,7 +19,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_managed.h>
#include <drm/drm_probe_helper.h>
#include <drm/drm_simple_kms_helper.h>
diff --git a/drivers/gpu/drm/imx/parallel-display.c b/drivers/gpu/drm/imx/parallel-display.c
index 06723b2..0fa0b59 100644
--- a/drivers/gpu/drm/imx/parallel-display.c
+++ b/drivers/gpu/drm/imx/parallel-display.c
@@ -8,6 +8,7 @@
#include <linux/component.h>
#include <linux/media-bus-format.h>
#include <linux/module.h>
+#include <linux/of.h>
#include <linux/platform_device.h>
#include <linux/videodev2.h>
@@ -16,7 +17,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_managed.h>
#include <drm/drm_of.h>
#include <drm/drm_panel.h>
diff --git a/drivers/gpu/drm/ingenic/ingenic-drm-drv.c b/drivers/gpu/drm/ingenic/ingenic-drm-drv.c
index ab0515d..3d5af44 100644
--- a/drivers/gpu/drm/ingenic/ingenic-drm-drv.c
+++ b/drivers/gpu/drm/ingenic/ingenic-drm-drv.c
@@ -32,7 +32,7 @@
#include <drm/drm_encoder.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_atomic_helper.h>
@@ -1018,7 +1018,6 @@ static const struct drm_bridge_funcs ingenic_drm_bridge_funcs = {
static const struct drm_mode_config_funcs ingenic_drm_mode_config_funcs = {
.fb_create = ingenic_drm_gem_fb_create,
- .output_poll_changed = drm_fb_helper_output_poll_changed,
.atomic_check = drm_atomic_helper_check,
.atomic_commit = drm_atomic_helper_commit,
};
@@ -1629,7 +1628,11 @@ static int ingenic_drm_init(void)
return err;
}
- return platform_driver_register(&ingenic_drm_driver);
+ err = platform_driver_register(&ingenic_drm_driver);
+ if (IS_ENABLED(CONFIG_DRM_INGENIC_IPU) && err)
+ platform_driver_unregister(ingenic_ipu_driver_ptr);
+
+ return err;
}
module_init(ingenic_drm_init);
diff --git a/drivers/gpu/drm/kmb/kmb_drv.c b/drivers/gpu/drm/kmb/kmb_drv.c
index 2382ccb..d29c678 100644
--- a/drivers/gpu/drm/kmb/kmb_drv.c
+++ b/drivers/gpu/drm/kmb/kmb_drv.c
@@ -15,7 +15,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/kmb/kmb_plane.c b/drivers/gpu/drm/kmb/kmb_plane.c
index a42f63f..d172a30 100644
--- a/drivers/gpu/drm/kmb/kmb_plane.c
+++ b/drivers/gpu/drm/kmb/kmb_plane.c
@@ -9,7 +9,6 @@
#include <drm/drm_crtc.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_dma_helper.h>
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index e829317..ff00340 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -371,7 +371,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
} else {
buffer_chunk->size = lima_bo_size(bo);
- ret = drm_gem_shmem_vmap(&bo->base, &map);
+ ret = drm_gem_vmap_unlocked(&bo->base.base, &map);
if (ret) {
kvfree(et);
goto out;
@@ -379,7 +379,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
memcpy(buffer_chunk + 1, map.vaddr, buffer_chunk->size);
- drm_gem_shmem_vunmap(&bo->base, &map);
+ drm_gem_vunmap_unlocked(&bo->base.base, &map);
}
buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
diff --git a/drivers/gpu/drm/logicvc/logicvc_drm.c b/drivers/gpu/drm/logicvc/logicvc_drm.c
index cc9a4e9..9de24d9 100644
--- a/drivers/gpu/drm/logicvc/logicvc_drm.c
+++ b/drivers/gpu/drm/logicvc/logicvc_drm.c
@@ -17,7 +17,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_print.h>
diff --git a/drivers/gpu/drm/logicvc/logicvc_mode.c b/drivers/gpu/drm/logicvc/logicvc_mode.c
index d8207ff..9971950 100644
--- a/drivers/gpu/drm/logicvc/logicvc_mode.c
+++ b/drivers/gpu/drm/logicvc/logicvc_mode.c
@@ -10,7 +10,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_mode_config.h>
@@ -26,7 +25,6 @@
static const struct drm_mode_config_funcs logicvc_mode_config_funcs = {
.fb_create = drm_gem_fb_create,
- .output_poll_changed = drm_fb_helper_output_poll_changed,
.atomic_check = drm_atomic_helper_check,
.atomic_commit = drm_atomic_helper_commit,
};
diff --git a/drivers/gpu/drm/mcde/mcde_drv.c b/drivers/gpu/drm/mcde/mcde_drv.c
index 1c4482a..4aedb05 100644
--- a/drivers/gpu/drm/mcde/mcde_drv.c
+++ b/drivers/gpu/drm/mcde/mcde_drv.c
@@ -69,7 +69,7 @@
#include <drm/drm_bridge.h>
#include <drm/drm_drv.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
@@ -203,7 +203,6 @@ DEFINE_DRM_GEM_DMA_FOPS(drm_fops);
static const struct drm_driver mcde_drm_driver = {
.driver_features =
DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC,
- .lastclose = drm_fb_helper_lastclose,
.ioctls = NULL,
.fops = &drm_fops,
.name = "mcde",
diff --git a/drivers/gpu/drm/mediatek/mtk_disp_ovl.c b/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
index 002b0f6..84daeaf 100644
--- a/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
+++ b/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
@@ -29,17 +29,22 @@
#define DISP_REG_OVL_DATAPATH_CON 0x0024
#define OVL_LAYER_SMI_ID_EN BIT(0)
#define OVL_BGCLR_SEL_IN BIT(2)
+#define OVL_LAYER_AFBC_EN(n) BIT(4+n)
#define DISP_REG_OVL_ROI_BGCLR 0x0028
#define DISP_REG_OVL_SRC_CON 0x002c
#define DISP_REG_OVL_CON(n) (0x0030 + 0x20 * (n))
#define DISP_REG_OVL_SRC_SIZE(n) (0x0038 + 0x20 * (n))
#define DISP_REG_OVL_OFFSET(n) (0x003c + 0x20 * (n))
+#define DISP_REG_OVL_PITCH_MSB(n) (0x0040 + 0x20 * (n))
+#define OVL_PITCH_MSB_2ND_SUBBUF BIT(16)
#define DISP_REG_OVL_PITCH(n) (0x0044 + 0x20 * (n))
#define DISP_REG_OVL_RDMA_CTRL(n) (0x00c0 + 0x20 * (n))
#define DISP_REG_OVL_RDMA_GMC(n) (0x00c8 + 0x20 * (n))
#define DISP_REG_OVL_ADDR_MT2701 0x0040
#define DISP_REG_OVL_ADDR_MT8173 0x0f40
#define DISP_REG_OVL_ADDR(ovl, n) ((ovl)->data->addr + 0x20 * (n))
+#define DISP_REG_OVL_HDR_ADDR(ovl, n) ((ovl)->data->addr + 0x20 * (n) + 0x04)
+#define DISP_REG_OVL_HDR_PITCH(ovl, n) ((ovl)->data->addr + 0x20 * (n) + 0x08)
#define GMC_THRESHOLD_BITS 16
#define GMC_THRESHOLD_HIGH ((1 << GMC_THRESHOLD_BITS) / 4)
@@ -67,6 +72,7 @@ struct mtk_disp_ovl_data {
unsigned int layer_nr;
bool fmt_rgb565_is_0;
bool smi_id_en;
+ bool supports_afbc;
};
/*
@@ -172,7 +178,14 @@ void mtk_ovl_stop(struct device *dev)
reg = reg & ~OVL_LAYER_SMI_ID_EN;
writel_relaxed(reg, ovl->regs + DISP_REG_OVL_DATAPATH_CON);
}
+}
+static void mtk_ovl_set_afbc(struct mtk_disp_ovl *ovl, struct cmdq_pkt *cmdq_pkt,
+ int idx, bool enabled)
+{
+ mtk_ddp_write_mask(cmdq_pkt, enabled ? OVL_LAYER_AFBC_EN(idx) : 0,
+ &ovl->cmdq_reg, ovl->regs,
+ DISP_REG_OVL_DATAPATH_CON, OVL_LAYER_AFBC_EN(idx));
}
void mtk_ovl_config(struct device *dev, unsigned int w,
@@ -310,11 +323,23 @@ void mtk_ovl_layer_config(struct device *dev, unsigned int idx,
struct mtk_disp_ovl *ovl = dev_get_drvdata(dev);
struct mtk_plane_pending_state *pending = &state->pending;
unsigned int addr = pending->addr;
- unsigned int pitch = pending->pitch & 0xffff;
+ unsigned int hdr_addr = pending->hdr_addr;
+ unsigned int pitch = pending->pitch;
+ unsigned int hdr_pitch = pending->hdr_pitch;
unsigned int fmt = pending->format;
unsigned int offset = (pending->y << 16) | pending->x;
unsigned int src_size = (pending->height << 16) | pending->width;
unsigned int con;
+ bool is_afbc = pending->modifier != DRM_FORMAT_MOD_LINEAR;
+ union overlay_pitch {
+ struct split_pitch {
+ u16 lsb;
+ u16 msb;
+ } split_pitch;
+ u32 pitch;
+ } overlay_pitch;
+
+ overlay_pitch.pitch = pitch;
if (!pending->enable) {
mtk_ovl_layer_off(dev, idx, cmdq_pkt);
@@ -335,9 +360,12 @@ void mtk_ovl_layer_config(struct device *dev, unsigned int idx,
addr += pending->pitch - 1;
}
+ if (ovl->data->supports_afbc)
+ mtk_ovl_set_afbc(ovl, cmdq_pkt, idx, is_afbc);
+
mtk_ddp_write_relaxed(cmdq_pkt, con, &ovl->cmdq_reg, ovl->regs,
DISP_REG_OVL_CON(idx));
- mtk_ddp_write_relaxed(cmdq_pkt, pitch, &ovl->cmdq_reg, ovl->regs,
+ mtk_ddp_write_relaxed(cmdq_pkt, overlay_pitch.split_pitch.lsb, &ovl->cmdq_reg, ovl->regs,
DISP_REG_OVL_PITCH(idx));
mtk_ddp_write_relaxed(cmdq_pkt, src_size, &ovl->cmdq_reg, ovl->regs,
DISP_REG_OVL_SRC_SIZE(idx));
@@ -346,6 +374,20 @@ void mtk_ovl_layer_config(struct device *dev, unsigned int idx,
mtk_ddp_write_relaxed(cmdq_pkt, addr, &ovl->cmdq_reg, ovl->regs,
DISP_REG_OVL_ADDR(ovl, idx));
+ if (is_afbc) {
+ mtk_ddp_write_relaxed(cmdq_pkt, hdr_addr, &ovl->cmdq_reg, ovl->regs,
+ DISP_REG_OVL_HDR_ADDR(ovl, idx));
+ mtk_ddp_write_relaxed(cmdq_pkt,
+ OVL_PITCH_MSB_2ND_SUBBUF | overlay_pitch.split_pitch.msb,
+ &ovl->cmdq_reg, ovl->regs, DISP_REG_OVL_PITCH_MSB(idx));
+ mtk_ddp_write_relaxed(cmdq_pkt, hdr_pitch, &ovl->cmdq_reg, ovl->regs,
+ DISP_REG_OVL_HDR_PITCH(ovl, idx));
+ } else {
+ mtk_ddp_write_relaxed(cmdq_pkt,
+ overlay_pitch.split_pitch.msb,
+ &ovl->cmdq_reg, ovl->regs, DISP_REG_OVL_PITCH_MSB(idx));
+ }
+
mtk_ovl_layer_on(dev, idx, cmdq_pkt);
}
@@ -492,6 +534,15 @@ static const struct mtk_disp_ovl_data mt8192_ovl_2l_driver_data = {
.smi_id_en = true,
};
+static const struct mtk_disp_ovl_data mt8195_ovl_driver_data = {
+ .addr = DISP_REG_OVL_ADDR_MT8173,
+ .gmc_bits = 10,
+ .layer_nr = 4,
+ .fmt_rgb565_is_0 = true,
+ .smi_id_en = true,
+ .supports_afbc = true,
+};
+
static const struct of_device_id mtk_disp_ovl_driver_dt_match[] = {
{ .compatible = "mediatek,mt2701-disp-ovl",
.data = &mt2701_ovl_driver_data},
@@ -505,6 +556,8 @@ static const struct of_device_id mtk_disp_ovl_driver_dt_match[] = {
.data = &mt8192_ovl_driver_data},
{ .compatible = "mediatek,mt8192-disp-ovl-2l",
.data = &mt8192_ovl_2l_driver_data},
+ { .compatible = "mediatek,mt8195-disp-ovl",
+ .data = &mt8195_ovl_driver_data},
{},
};
MODULE_DEVICE_TABLE(of, mtk_disp_ovl_driver_dt_match);
diff --git a/drivers/gpu/drm/mediatek/mtk_dpi.c b/drivers/gpu/drm/mediatek/mtk_dpi.c
index 508a6d9..4317595 100644
--- a/drivers/gpu/drm/mediatek/mtk_dpi.c
+++ b/drivers/gpu/drm/mediatek/mtk_dpi.c
@@ -461,9 +461,6 @@ static void mtk_dpi_power_off(struct mtk_dpi *dpi)
if (--dpi->refcount != 0)
return;
- if (dpi->pinctrl && dpi->pins_gpio)
- pinctrl_select_state(dpi->pinctrl, dpi->pins_gpio);
-
mtk_dpi_disable(dpi);
clk_disable_unprepare(dpi->pixel_clk);
clk_disable_unprepare(dpi->engine_clk);
@@ -488,9 +485,6 @@ static int mtk_dpi_power_on(struct mtk_dpi *dpi)
goto err_pixel;
}
- if (dpi->pinctrl && dpi->pins_dpi)
- pinctrl_select_state(dpi->pinctrl, dpi->pins_dpi);
-
return 0;
err_pixel:
@@ -721,12 +715,18 @@ static void mtk_dpi_bridge_disable(struct drm_bridge *bridge)
struct mtk_dpi *dpi = bridge_to_dpi(bridge);
mtk_dpi_power_off(dpi);
+
+ if (dpi->pinctrl && dpi->pins_gpio)
+ pinctrl_select_state(dpi->pinctrl, dpi->pins_gpio);
}
static void mtk_dpi_bridge_enable(struct drm_bridge *bridge)
{
struct mtk_dpi *dpi = bridge_to_dpi(bridge);
+ if (dpi->pinctrl && dpi->pins_dpi)
+ pinctrl_select_state(dpi->pinctrl, dpi->pins_dpi);
+
mtk_dpi_power_on(dpi);
mtk_dpi_set_display_mode(dpi, &dpi->mode);
mtk_dpi_enable(dpi);
@@ -929,6 +929,20 @@ static const struct mtk_dpi_conf mt8183_conf = {
.csc_enable_bit = CSC_ENABLE,
};
+static const struct mtk_dpi_conf mt8188_dpintf_conf = {
+ .cal_factor = mt8195_dpintf_calculate_factor,
+ .max_clock_khz = 600000,
+ .output_fmts = mt8195_output_fmts,
+ .num_output_fmts = ARRAY_SIZE(mt8195_output_fmts),
+ .pixels_per_iter = 4,
+ .input_2pixel = false,
+ .dimension_mask = DPINTF_HPW_MASK,
+ .hvsize_mask = DPINTF_HSIZE_MASK,
+ .channel_swap_shift = DPINTF_CH_SWAP,
+ .yuv422_en_bit = DPINTF_YUV422_EN,
+ .csc_enable_bit = DPINTF_CSC_ENABLE,
+};
+
static const struct mtk_dpi_conf mt8192_conf = {
.cal_factor = mt8183_calculate_factor,
.reg_h_fre_con = 0xe0,
@@ -1079,6 +1093,9 @@ static const struct of_device_id mtk_dpi_of_ids[] = {
{ .compatible = "mediatek,mt8183-dpi",
.data = &mt8183_conf,
},
+ { .compatible = "mediatek,mt8188-dp-intf",
+ .data = &mt8188_dpintf_conf,
+ },
{ .compatible = "mediatek,mt8192-dpi",
.data = &mt8192_conf,
},
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.c b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
index 91f58db..cd5b18e 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_drv.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
@@ -17,7 +17,7 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem.h>
#include <drm/drm_gem_dma_helper.h>
@@ -387,6 +387,12 @@ static int mtk_drm_kms_init(struct drm_device *drm)
goto put_mutex_dev;
/*
+ * Ensure internal panels are at the top of the connector list before
+ * crtc creation.
+ */
+ drm_helper_move_panel_connectors_to_head(drm);
+
+ /*
* We currently support two fixed data streams, each optional,
* and each statically assigned to a crtc:
* OVL0 -> COLOR0 -> AAL -> OD -> RDMA0 -> UFOE -> DSI0 ...
@@ -631,6 +637,8 @@ static const struct of_device_id mtk_ddp_comp_dt_ids[] = {
.data = (void *)MTK_DPI },
{ .compatible = "mediatek,mt8183-dpi",
.data = (void *)MTK_DPI },
+ { .compatible = "mediatek,mt8188-dp-intf",
+ .data = (void *)MTK_DP_INTF },
{ .compatible = "mediatek,mt8192-dpi",
.data = (void *)MTK_DPI },
{ .compatible = "mediatek,mt8195-dp-intf",
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_plane.c b/drivers/gpu/drm/mediatek/mtk_drm_plane.c
index 2f5e007d..d54fbf3 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_plane.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_plane.c
@@ -11,6 +11,7 @@
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_atomic_helper.h>
+#include <linux/align.h>
#include "mtk_drm_crtc.h"
#include "mtk_drm_ddp_comp.h"
@@ -32,6 +33,14 @@ static const u32 formats[] = {
DRM_FORMAT_YUYV,
};
+static const u64 modifiers[] = {
+ DRM_FORMAT_MOD_LINEAR,
+ DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_BLOCK_SIZE_32x8 |
+ AFBC_FORMAT_MOD_SPLIT |
+ AFBC_FORMAT_MOD_SPARSE),
+ DRM_FORMAT_MOD_INVALID,
+};
+
static void mtk_plane_reset(struct drm_plane *plane)
{
struct mtk_plane_state *state;
@@ -51,6 +60,7 @@ static void mtk_plane_reset(struct drm_plane *plane)
state->base.plane = plane;
state->pending.format = DRM_FORMAT_RGB565;
+ state->pending.modifier = DRM_FORMAT_MOD_LINEAR;
}
static struct drm_plane_state *mtk_plane_duplicate_state(struct drm_plane *plane)
@@ -71,6 +81,32 @@ static struct drm_plane_state *mtk_plane_duplicate_state(struct drm_plane *plane
return &state->base;
}
+static bool mtk_plane_format_mod_supported(struct drm_plane *plane,
+ uint32_t format,
+ uint64_t modifier)
+{
+ if (modifier == DRM_FORMAT_MOD_LINEAR)
+ return true;
+
+ if (modifier != DRM_FORMAT_MOD_ARM_AFBC(
+ AFBC_FORMAT_MOD_BLOCK_SIZE_32x8 |
+ AFBC_FORMAT_MOD_SPLIT |
+ AFBC_FORMAT_MOD_SPARSE))
+ return false;
+
+ if (format != DRM_FORMAT_XRGB8888 &&
+ format != DRM_FORMAT_ARGB8888 &&
+ format != DRM_FORMAT_BGRX8888 &&
+ format != DRM_FORMAT_BGRA8888 &&
+ format != DRM_FORMAT_ABGR8888 &&
+ format != DRM_FORMAT_XBGR8888 &&
+ format != DRM_FORMAT_RGB888 &&
+ format != DRM_FORMAT_BGR888)
+ return false;
+
+ return true;
+}
+
static void mtk_drm_plane_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
@@ -119,21 +155,52 @@ static void mtk_plane_update_new_state(struct drm_plane_state *new_state,
struct drm_gem_object *gem;
struct mtk_drm_gem_obj *mtk_gem;
unsigned int pitch, format;
+ u64 modifier;
dma_addr_t addr;
+ dma_addr_t hdr_addr = 0;
+ unsigned int hdr_pitch = 0;
gem = fb->obj[0];
mtk_gem = to_mtk_gem_obj(gem);
addr = mtk_gem->dma_addr;
pitch = fb->pitches[0];
format = fb->format->format;
+ modifier = fb->modifier;
- addr += (new_state->src.x1 >> 16) * fb->format->cpp[0];
- addr += (new_state->src.y1 >> 16) * pitch;
+ if (modifier == DRM_FORMAT_MOD_LINEAR) {
+ addr += (new_state->src.x1 >> 16) * fb->format->cpp[0];
+ addr += (new_state->src.y1 >> 16) * pitch;
+ } else {
+ int width_in_blocks = ALIGN(fb->width, AFBC_DATA_BLOCK_WIDTH)
+ / AFBC_DATA_BLOCK_WIDTH;
+ int height_in_blocks = ALIGN(fb->height, AFBC_DATA_BLOCK_HEIGHT)
+ / AFBC_DATA_BLOCK_HEIGHT;
+ int x_offset_in_blocks = (new_state->src.x1 >> 16) / AFBC_DATA_BLOCK_WIDTH;
+ int y_offset_in_blocks = (new_state->src.y1 >> 16) / AFBC_DATA_BLOCK_HEIGHT;
+ int hdr_size;
+
+ hdr_pitch = width_in_blocks * AFBC_HEADER_BLOCK_SIZE;
+ pitch = width_in_blocks * AFBC_DATA_BLOCK_WIDTH *
+ AFBC_DATA_BLOCK_HEIGHT * fb->format->cpp[0];
+
+ hdr_size = ALIGN(hdr_pitch * height_in_blocks, AFBC_HEADER_ALIGNMENT);
+
+ hdr_addr = addr + hdr_pitch * y_offset_in_blocks +
+ AFBC_HEADER_BLOCK_SIZE * x_offset_in_blocks;
+ /* The data plane is offset by 1 additional block. */
+ addr = addr + hdr_size +
+ pitch * y_offset_in_blocks +
+ AFBC_DATA_BLOCK_WIDTH * AFBC_DATA_BLOCK_HEIGHT *
+ fb->format->cpp[0] * (x_offset_in_blocks + 1);
+ }
mtk_plane_state->pending.enable = true;
mtk_plane_state->pending.pitch = pitch;
+ mtk_plane_state->pending.hdr_pitch = hdr_pitch;
mtk_plane_state->pending.format = format;
+ mtk_plane_state->pending.modifier = modifier;
mtk_plane_state->pending.addr = addr;
+ mtk_plane_state->pending.hdr_addr = hdr_addr;
mtk_plane_state->pending.x = new_state->dst.x1;
mtk_plane_state->pending.y = new_state->dst.y1;
mtk_plane_state->pending.width = drm_rect_width(&new_state->dst);
@@ -172,6 +239,7 @@ static const struct drm_plane_funcs mtk_plane_funcs = {
.reset = mtk_plane_reset,
.atomic_duplicate_state = mtk_plane_duplicate_state,
.atomic_destroy_state = mtk_drm_plane_destroy_state,
+ .format_mod_supported = mtk_plane_format_mod_supported,
};
static int mtk_plane_atomic_check(struct drm_plane *plane,
@@ -253,7 +321,7 @@ int mtk_plane_init(struct drm_device *dev, struct drm_plane *plane,
err = drm_universal_plane_init(dev, plane, possible_crtcs,
&mtk_plane_funcs, formats,
- ARRAY_SIZE(formats), NULL, type, NULL);
+ ARRAY_SIZE(formats), modifiers, type, NULL);
if (err) {
DRM_ERROR("failed to initialize plane\n");
return err;
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_plane.h b/drivers/gpu/drm/mediatek/mtk_drm_plane.h
index 2d5ec66e..8f39011 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_plane.h
+++ b/drivers/gpu/drm/mediatek/mtk_drm_plane.h
@@ -10,12 +10,20 @@
#include <drm/drm_crtc.h>
#include <linux/types.h>
+#define AFBC_DATA_BLOCK_WIDTH 32
+#define AFBC_DATA_BLOCK_HEIGHT 8
+#define AFBC_HEADER_BLOCK_SIZE 16
+#define AFBC_HEADER_ALIGNMENT 1024
+
struct mtk_plane_pending_state {
bool config;
bool enable;
dma_addr_t addr;
+ dma_addr_t hdr_addr;
unsigned int pitch;
+ unsigned int hdr_pitch;
unsigned int format;
+ unsigned long long modifier;
unsigned int x;
unsigned int y;
unsigned int width;
diff --git a/drivers/gpu/drm/mediatek/mtk_hdmi.c b/drivers/gpu/drm/mediatek/mtk_hdmi.c
index 4c80b68..0a8e0a1 100644
--- a/drivers/gpu/drm/mediatek/mtk_hdmi.c
+++ b/drivers/gpu/drm/mediatek/mtk_hdmi.c
@@ -1202,9 +1202,10 @@ static enum drm_connector_status mtk_hdmi_detect(struct mtk_hdmi *hdmi)
return mtk_hdmi_update_plugged_status(hdmi);
}
-static int mtk_hdmi_bridge_mode_valid(struct drm_bridge *bridge,
- const struct drm_display_info *info,
- const struct drm_display_mode *mode)
+static enum drm_mode_status
+mtk_hdmi_bridge_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *info,
+ const struct drm_display_mode *mode)
{
struct mtk_hdmi *hdmi = hdmi_ctx_from_bridge(bridge);
struct drm_bridge *next_bridge;
@@ -1217,7 +1218,7 @@ static int mtk_hdmi_bridge_mode_valid(struct drm_bridge *bridge,
if (next_bridge) {
struct drm_display_mode adjusted_mode;
- drm_mode_copy(&adjusted_mode, mode);
+ drm_mode_init(&adjusted_mode, mode);
if (!drm_bridge_chain_mode_fixup(next_bridge, mode,
&adjusted_mode))
return MODE_BAD;
diff --git a/drivers/gpu/drm/meson/meson_drv.c b/drivers/gpu/drm/meson/meson_drv.c
index 3b24a92..79bfe39 100644
--- a/drivers/gpu/drm/meson/meson_drv.c
+++ b/drivers/gpu/drm/meson/meson_drv.c
@@ -18,7 +18,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_modeset_helper_vtables.h>
diff --git a/drivers/gpu/drm/meson/meson_encoder_cvbs.c b/drivers/gpu/drm/meson/meson_encoder_cvbs.c
index 5675bc2..3f73b21 100644
--- a/drivers/gpu/drm/meson/meson_encoder_cvbs.c
+++ b/drivers/gpu/drm/meson/meson_encoder_cvbs.c
@@ -116,9 +116,10 @@ static int meson_encoder_cvbs_get_modes(struct drm_bridge *bridge,
return i;
}
-static int meson_encoder_cvbs_mode_valid(struct drm_bridge *bridge,
- const struct drm_display_info *display_info,
- const struct drm_display_mode *mode)
+static enum drm_mode_status
+meson_encoder_cvbs_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *display_info,
+ const struct drm_display_mode *mode)
{
if (meson_cvbs_get_mode(mode))
return MODE_OK;
diff --git a/drivers/gpu/drm/mgag200/mgag200_drv.c b/drivers/gpu/drm/mgag200/mgag200_drv.c
index ece6cd1..976f0ab 100644
--- a/drivers/gpu/drm/mgag200/mgag200_drv.c
+++ b/drivers/gpu/drm/mgag200/mgag200_drv.c
@@ -11,6 +11,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/mgag200/mgag200_drv.h b/drivers/gpu/drm/mgag200/mgag200_drv.h
index f0c2349..9e604dbb 100644
--- a/drivers/gpu/drm/mgag200/mgag200_drv.h
+++ b/drivers/gpu/drm/mgag200/mgag200_drv.h
@@ -18,7 +18,6 @@
#include <drm/drm_connector.h>
#include <drm/drm_crtc.h>
#include <drm/drm_encoder.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_gem.h>
#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_plane.h>
diff --git a/drivers/gpu/drm/mgag200/mgag200_g200se.c b/drivers/gpu/drm/mgag200/mgag200_g200se.c
index be389ed..bd6e573 100644
--- a/drivers/gpu/drm/mgag200/mgag200_g200se.c
+++ b/drivers/gpu/drm/mgag200/mgag200_g200se.c
@@ -284,7 +284,8 @@ static void mgag200_g200se_04_pixpllc_atomic_update(struct drm_crtc *crtc,
pixpllcp = pixpllc->p - 1;
pixpllcs = pixpllc->s;
- xpixpllcm = pixpllcm | ((pixpllcn & BIT(8)) >> 1);
+ // For G200SE A, BIT(7) should be set unconditionally.
+ xpixpllcm = BIT(7) | pixpllcm;
xpixpllcn = pixpllcn;
xpixpllcp = (pixpllcs << 3) | pixpllcp;
diff --git a/drivers/gpu/drm/mgag200/mgag200_mode.c b/drivers/gpu/drm/mgag200/mgag200_mode.c
index bbab254..0a5aaf7 100644
--- a/drivers/gpu/drm/mgag200/mgag200_mode.c
+++ b/drivers/gpu/drm/mgag200/mgag200_mode.c
@@ -579,13 +579,13 @@ int mgag200_crtc_helper_atomic_check(struct drm_crtc *crtc, struct drm_atomic_st
struct drm_property_blob *new_gamma_lut = new_crtc_state->gamma_lut;
int ret;
- ret = drm_atomic_helper_check_crtc_state(new_crtc_state, false);
- if (ret)
- return ret;
-
if (!new_crtc_state->enable)
return 0;
+ ret = drm_atomic_helper_check_crtc_primary_plane(new_crtc_state);
+ if (ret)
+ return ret;
+
if (new_crtc_state->mode_changed) {
if (funcs->pixpllc_atomic_check) {
ret = funcs->pixpllc_atomic_check(crtc, new_state);
@@ -601,7 +601,7 @@ int mgag200_crtc_helper_atomic_check(struct drm_crtc *crtc, struct drm_atomic_st
}
}
- return drm_atomic_add_affected_planes(new_state, crtc);
+ return 0;
}
void mgag200_crtc_helper_atomic_flush(struct drm_crtc *crtc, struct drm_atomic_state *old_state)
@@ -824,7 +824,6 @@ int mgag200_mode_config_init(struct mga_device *mdev, resource_size_t vram_avail
dev->mode_config.max_width = MGAG200_MAX_FB_WIDTH;
dev->mode_config.max_height = MGAG200_MAX_FB_HEIGHT;
dev->mode_config.preferred_depth = 24;
- dev->mode_config.fb_base = mdev->vram_res->start;
dev->mode_config.funcs = &mgag200_mode_config_funcs;
dev->mode_config.helper_private = &mgag200_mode_config_helper_funcs;
diff --git a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
index 2c8b989..948785e 100644
--- a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
@@ -500,7 +500,7 @@ static const struct adreno_gpu_funcs funcs = {
#endif
.gpu_state_get = a3xx_gpu_state_get,
.gpu_state_put = adreno_gpu_state_put,
- .create_address_space = adreno_iommu_create_address_space,
+ .create_address_space = adreno_create_address_space,
.get_rptr = a3xx_get_rptr,
},
};
diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
index 7cb8d98..3e09d3a 100644
--- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
@@ -606,8 +606,7 @@ static int a4xx_pm_suspend(struct msm_gpu *gpu) {
static int a4xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value)
{
- *value = gpu_read64(gpu, REG_A4XX_RBBM_PERFCTR_CP_0_LO,
- REG_A4XX_RBBM_PERFCTR_CP_0_HI);
+ *value = gpu_read64(gpu, REG_A4XX_RBBM_PERFCTR_CP_0_LO);
return 0;
}
@@ -635,7 +634,7 @@ static const struct adreno_gpu_funcs funcs = {
#endif
.gpu_state_get = a4xx_gpu_state_get,
.gpu_state_put = adreno_gpu_state_put,
- .create_address_space = adreno_iommu_create_address_space,
+ .create_address_space = adreno_create_address_space,
.get_rptr = a4xx_get_rptr,
},
.get_timestamp = a4xx_get_timestamp,
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
index 3dcec7a..660ba0d 100644
--- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
@@ -605,11 +605,9 @@ static int a5xx_ucode_init(struct msm_gpu *gpu)
a5xx_ucode_check_version(a5xx_gpu, a5xx_gpu->pfp_bo);
}
- gpu_write64(gpu, REG_A5XX_CP_ME_INSTR_BASE_LO,
- REG_A5XX_CP_ME_INSTR_BASE_HI, a5xx_gpu->pm4_iova);
+ gpu_write64(gpu, REG_A5XX_CP_ME_INSTR_BASE_LO, a5xx_gpu->pm4_iova);
- gpu_write64(gpu, REG_A5XX_CP_PFP_INSTR_BASE_LO,
- REG_A5XX_CP_PFP_INSTR_BASE_HI, a5xx_gpu->pfp_iova);
+ gpu_write64(gpu, REG_A5XX_CP_PFP_INSTR_BASE_LO, a5xx_gpu->pfp_iova);
return 0;
}
@@ -868,8 +866,7 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
* memory rendering at this point in time and we don't want to block off
* part of the virtual memory space.
*/
- gpu_write64(gpu, REG_A5XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO,
- REG_A5XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI, 0x00000000);
+ gpu_write64(gpu, REG_A5XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO, 0x00000000);
gpu_write(gpu, REG_A5XX_RBBM_SECVID_TSB_TRUSTED_SIZE, 0x00000000);
/* Put the GPU into 64 bit by default */
@@ -908,8 +905,7 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
return ret;
/* Set the ringbuffer address */
- gpu_write64(gpu, REG_A5XX_CP_RB_BASE, REG_A5XX_CP_RB_BASE_HI,
- gpu->rb[0]->iova);
+ gpu_write64(gpu, REG_A5XX_CP_RB_BASE, gpu->rb[0]->iova);
/*
* If the microcode supports the WHERE_AM_I opcode then we can use that
@@ -936,7 +932,7 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
}
gpu_write64(gpu, REG_A5XX_CP_RB_RPTR_ADDR,
- REG_A5XX_CP_RB_RPTR_ADDR_HI, shadowptr(a5xx_gpu, gpu->rb[0]));
+ shadowptr(a5xx_gpu, gpu->rb[0]));
} else if (gpu->nr_rings > 1) {
/* Disable preemption if WHERE_AM_I isn't available */
a5xx_preempt_fini(gpu);
@@ -1239,9 +1235,9 @@ static void a5xx_fault_detect_irq(struct msm_gpu *gpu)
gpu_read(gpu, REG_A5XX_RBBM_STATUS),
gpu_read(gpu, REG_A5XX_CP_RB_RPTR),
gpu_read(gpu, REG_A5XX_CP_RB_WPTR),
- gpu_read64(gpu, REG_A5XX_CP_IB1_BASE, REG_A5XX_CP_IB1_BASE_HI),
+ gpu_read64(gpu, REG_A5XX_CP_IB1_BASE),
gpu_read(gpu, REG_A5XX_CP_IB1_BUFSZ),
- gpu_read64(gpu, REG_A5XX_CP_IB2_BASE, REG_A5XX_CP_IB2_BASE_HI),
+ gpu_read64(gpu, REG_A5XX_CP_IB2_BASE),
gpu_read(gpu, REG_A5XX_CP_IB2_BUFSZ));
/* Turn off the hangcheck timer to keep it from bothering us */
@@ -1427,8 +1423,7 @@ static int a5xx_pm_suspend(struct msm_gpu *gpu)
static int a5xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value)
{
- *value = gpu_read64(gpu, REG_A5XX_RBBM_ALWAYSON_COUNTER_LO,
- REG_A5XX_RBBM_ALWAYSON_COUNTER_HI);
+ *value = gpu_read64(gpu, REG_A5XX_RBBM_ALWAYSON_COUNTER_LO);
return 0;
}
@@ -1465,8 +1460,7 @@ static int a5xx_crashdumper_run(struct msm_gpu *gpu,
if (IS_ERR_OR_NULL(dumper->ptr))
return -EINVAL;
- gpu_write64(gpu, REG_A5XX_CP_CRASH_SCRIPT_BASE_LO,
- REG_A5XX_CP_CRASH_SCRIPT_BASE_HI, dumper->iova);
+ gpu_write64(gpu, REG_A5XX_CP_CRASH_SCRIPT_BASE_LO, dumper->iova);
gpu_write(gpu, REG_A5XX_CP_CRASH_DUMP_CNTL, 1);
@@ -1666,8 +1660,7 @@ static u64 a5xx_gpu_busy(struct msm_gpu *gpu, unsigned long *out_sample_rate)
{
u64 busy_cycles;
- busy_cycles = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO,
- REG_A5XX_RBBM_PERFCTR_RBBM_0_HI);
+ busy_cycles = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO);
*out_sample_rate = clk_get_rate(gpu->core_clk);
return busy_cycles;
@@ -1705,7 +1698,7 @@ static const struct adreno_gpu_funcs funcs = {
.gpu_busy = a5xx_gpu_busy,
.gpu_state_get = a5xx_gpu_state_get,
.gpu_state_put = a5xx_gpu_state_put,
- .create_address_space = adreno_iommu_create_address_space,
+ .create_address_space = adreno_create_address_space,
.get_rptr = a5xx_get_rptr,
},
.get_timestamp = a5xx_get_timestamp,
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c
index 8abc9a2..7658e89 100644
--- a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c
+++ b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c
@@ -137,7 +137,6 @@ void a5xx_preempt_trigger(struct msm_gpu *gpu)
/* Set the address of the incoming preemption record */
gpu_write64(gpu, REG_A5XX_CP_CONTEXT_SWITCH_RESTORE_ADDR_LO,
- REG_A5XX_CP_CONTEXT_SWITCH_RESTORE_ADDR_HI,
a5xx_gpu->preempt_iova[ring->id]);
a5xx_gpu->next_ring = ring;
@@ -211,8 +210,7 @@ void a5xx_preempt_hw_init(struct msm_gpu *gpu)
}
/* Write a 0 to signal that we aren't switching pagetables */
- gpu_write64(gpu, REG_A5XX_CP_CONTEXT_SWITCH_SMMU_INFO_LO,
- REG_A5XX_CP_CONTEXT_SWITCH_SMMU_INFO_HI, 0);
+ gpu_write64(gpu, REG_A5XX_CP_CONTEXT_SWITCH_SMMU_INFO_LO, 0);
/* Reset the preemption state */
set_preempt_state(a5xx_gpu, PREEMPT_NONE);
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
index e033d6a..6484b97 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
@@ -1213,19 +1213,17 @@ static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,
static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu)
{
- struct iommu_domain *domain;
struct msm_mmu *mmu;
- domain = iommu_domain_alloc(&platform_bus_type);
- if (!domain)
+ mmu = msm_iommu_new(gmu->dev, 0);
+ if (!mmu)
return -ENODEV;
+ if (IS_ERR(mmu))
+ return PTR_ERR(mmu);
- mmu = msm_iommu_new(gmu->dev, domain);
gmu->aspace = msm_gem_address_space_create(mmu, "gmu", 0x0, 0x80000000);
- if (IS_ERR(gmu->aspace)) {
- iommu_domain_free(domain);
+ if (IS_ERR(gmu->aspace))
return PTR_ERR(gmu->aspace);
- }
return 0;
}
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
index fdc5780..36c8fb6 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
@@ -247,8 +247,7 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
OUT_RING(ring, submit->seqno);
trace_msm_gpu_submit_flush(submit,
- gpu_read64(gpu, REG_A6XX_CP_ALWAYS_ON_COUNTER_LO,
- REG_A6XX_CP_ALWAYS_ON_COUNTER_HI));
+ gpu_read64(gpu, REG_A6XX_CP_ALWAYS_ON_COUNTER_LO));
a6xx_flush(gpu, ring);
}
@@ -947,8 +946,7 @@ static int a6xx_ucode_init(struct msm_gpu *gpu)
}
}
- gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE,
- REG_A6XX_CP_SQE_INSTR_BASE+1, a6xx_gpu->sqe_iova);
+ gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE, a6xx_gpu->sqe_iova);
return 0;
}
@@ -999,8 +997,7 @@ static int hw_init(struct msm_gpu *gpu)
* memory rendering at this point in time and we don't want to block off
* part of the virtual memory space.
*/
- gpu_write64(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO,
- REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI, 0x00000000);
+ gpu_write64(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO, 0x00000000);
gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_SIZE, 0x00000000);
/* Turn on 64 bit addressing for all blocks */
@@ -1049,11 +1046,9 @@ static int hw_init(struct msm_gpu *gpu)
if (!adreno_is_a650_family(adreno_gpu)) {
/* Set the GMEM VA range [0x100000:0x100000 + gpu->gmem - 1] */
- gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MIN_LO,
- REG_A6XX_UCHE_GMEM_RANGE_MIN_HI, 0x00100000);
+ gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MIN_LO, 0x00100000);
gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MAX_LO,
- REG_A6XX_UCHE_GMEM_RANGE_MAX_HI,
0x00100000 + adreno_gpu->gmem - 1);
}
@@ -1145,8 +1140,7 @@ static int hw_init(struct msm_gpu *gpu)
goto out;
/* Set the ringbuffer address */
- gpu_write64(gpu, REG_A6XX_CP_RB_BASE, REG_A6XX_CP_RB_BASE_HI,
- gpu->rb[0]->iova);
+ gpu_write64(gpu, REG_A6XX_CP_RB_BASE, gpu->rb[0]->iova);
/* Targets that support extended APRIV can use the RPTR shadow from
* hardware but all the other ones need to disable the feature. Targets
@@ -1178,7 +1172,6 @@ static int hw_init(struct msm_gpu *gpu)
}
gpu_write64(gpu, REG_A6XX_CP_RB_RPTR_ADDR_LO,
- REG_A6XX_CP_RB_RPTR_ADDR_HI,
shadowptr(a6xx_gpu, gpu->rb[0]));
}
@@ -1499,9 +1492,9 @@ static void a6xx_fault_detect_irq(struct msm_gpu *gpu)
gpu_read(gpu, REG_A6XX_RBBM_STATUS),
gpu_read(gpu, REG_A6XX_CP_RB_RPTR),
gpu_read(gpu, REG_A6XX_CP_RB_WPTR),
- gpu_read64(gpu, REG_A6XX_CP_IB1_BASE, REG_A6XX_CP_IB1_BASE_HI),
+ gpu_read64(gpu, REG_A6XX_CP_IB1_BASE),
gpu_read(gpu, REG_A6XX_CP_IB1_REM_SIZE),
- gpu_read64(gpu, REG_A6XX_CP_IB2_BASE, REG_A6XX_CP_IB2_BASE_HI),
+ gpu_read64(gpu, REG_A6XX_CP_IB2_BASE),
gpu_read(gpu, REG_A6XX_CP_IB2_REM_SIZE));
/* Turn off the hangcheck timer to keep it from bothering us */
@@ -1712,8 +1705,7 @@ static int a6xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value)
/* Force the GPU power on so we can read this register */
a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_PERFCOUNTER_SET);
- *value = gpu_read64(gpu, REG_A6XX_CP_ALWAYS_ON_COUNTER_LO,
- REG_A6XX_CP_ALWAYS_ON_COUNTER_HI);
+ *value = gpu_read64(gpu, REG_A6XX_CP_ALWAYS_ON_COUNTER_LO);
a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_PERFCOUNTER_SET);
@@ -1786,43 +1778,16 @@ a6xx_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu);
- struct iommu_domain *iommu;
- struct msm_mmu *mmu;
- struct msm_gem_address_space *aspace;
- u64 start, size;
-
- iommu = iommu_domain_alloc(&platform_bus_type);
- if (!iommu)
- return NULL;
+ unsigned long quirks = 0;
/*
* This allows GPU to set the bus attributes required to use system
* cache on behalf of the iommu page table walker.
*/
if (!IS_ERR_OR_NULL(a6xx_gpu->htw_llc_slice))
- adreno_set_llc_attributes(iommu);
+ quirks |= IO_PGTABLE_QUIRK_ARM_OUTER_WBWA;
- mmu = msm_iommu_new(&pdev->dev, iommu);
- if (IS_ERR(mmu)) {
- iommu_domain_free(iommu);
- return ERR_CAST(mmu);
- }
-
- /*
- * Use the aperture start or SZ_16M, whichever is greater. This will
- * ensure that we align with the allocated pagetable range while still
- * allowing room in the lower 32 bits for GMEM and whatnot
- */
- start = max_t(u64, SZ_16M, iommu->geometry.aperture_start);
- size = iommu->geometry.aperture_end - start + 1;
-
- aspace = msm_gem_address_space_create(mmu, "gpu",
- start & GENMASK_ULL(48, 0), size);
-
- if (IS_ERR(aspace) && !IS_ERR(mmu))
- mmu->funcs->destroy(mmu);
-
- return aspace;
+ return adreno_iommu_create_address_space(gpu, pdev, quirks);
}
static struct msm_gem_address_space *
@@ -1851,6 +1816,39 @@ static uint32_t a6xx_get_rptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
return ring->memptrs->rptr = gpu_read(gpu, REG_A6XX_CP_RB_RPTR);
}
+static bool a6xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
+{
+ struct msm_cp_state cp_state = {
+ .ib1_base = gpu_read64(gpu, REG_A6XX_CP_IB1_BASE),
+ .ib2_base = gpu_read64(gpu, REG_A6XX_CP_IB2_BASE),
+ .ib1_rem = gpu_read(gpu, REG_A6XX_CP_IB1_REM_SIZE),
+ .ib2_rem = gpu_read(gpu, REG_A6XX_CP_IB2_REM_SIZE),
+ };
+ bool progress;
+
+ /*
+ * Adjust the remaining data to account for what has already been
+ * fetched from memory, but not yet consumed by the SQE.
+ *
+ * This is not *technically* correct, the amount buffered could
+ * exceed the IB size due to hw prefetching ahead, but:
+ *
+ * (1) We aren't trying to find the exact position, just whether
+ * progress has been made
+ * (2) The CP_REG_TO_MEM at the end of a submit should be enough
+ * to prevent prefetching into an unrelated submit. (And
+ * either way, at some point the ROQ will be full.)
+ */
+ cp_state.ib1_rem += gpu_read(gpu, REG_A6XX_CP_CSQ_IB1_STAT) >> 16;
+ cp_state.ib2_rem += gpu_read(gpu, REG_A6XX_CP_CSQ_IB2_STAT) >> 16;
+
+ progress = !!memcmp(&cp_state, &ring->last_cp_state, sizeof(cp_state));
+
+ ring->last_cp_state = cp_state;
+
+ return progress;
+}
+
static u32 a618_get_speed_bin(u32 fuse)
{
if (fuse == 0)
@@ -1906,7 +1904,7 @@ static u32 fuse_to_supp_hw(struct device *dev, struct adreno_rev rev, u32 fuse)
if (val == UINT_MAX) {
DRM_DEV_ERROR(dev,
- "missing support for speed-bin: %u. Some OPPs may not be supported by hardware",
+ "missing support for speed-bin: %u. Some OPPs may not be supported by hardware\n",
fuse);
return UINT_MAX;
}
@@ -1916,7 +1914,7 @@ static u32 fuse_to_supp_hw(struct device *dev, struct adreno_rev rev, u32 fuse)
static int a6xx_set_supported_hw(struct device *dev, struct adreno_rev rev)
{
- u32 supp_hw = UINT_MAX;
+ u32 supp_hw;
u32 speedbin;
int ret;
@@ -1928,15 +1926,13 @@ static int a6xx_set_supported_hw(struct device *dev, struct adreno_rev rev)
if (ret == -ENOENT) {
return 0;
} else if (ret) {
- DRM_DEV_ERROR(dev,
- "failed to read speed-bin (%d). Some OPPs may not be supported by hardware",
- ret);
- goto done;
+ dev_err_probe(dev, ret,
+ "failed to read speed-bin. Some OPPs may not be supported by hardware\n");
+ return ret;
}
supp_hw = fuse_to_supp_hw(dev, rev, speedbin);
-done:
ret = devm_pm_opp_set_supported_hw(dev, &supp_hw, 1);
if (ret)
return ret;
@@ -1969,6 +1965,7 @@ static const struct adreno_gpu_funcs funcs = {
.create_address_space = a6xx_create_address_space,
.create_private_address_space = a6xx_create_private_address_space,
.get_rptr = a6xx_get_rptr,
+ .progress = a6xx_progress,
},
.get_timestamp = a6xx_get_timestamp,
};
@@ -2005,13 +2002,6 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev)
adreno_cmp_rev(ADRENO_REV(6, 3, 5, ANY_ID), info->rev)))
adreno_gpu->base.hw_apriv = true;
- /*
- * For now only clamp to idle freq for devices where this is known not
- * to cause power supply issues:
- */
- if (info && (info->revn == 618))
- gpu->clamp_to_idle = true;
-
a6xx_llc_slices_init(pdev, a6xx_gpu);
ret = a6xx_set_supported_hw(&pdev->dev, config->rev);
@@ -2026,6 +2016,13 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev)
return ERR_PTR(ret);
}
+ /*
+ * For now only clamp to idle freq for devices where this is known not
+ * to cause power supply issues:
+ */
+ if (adreno_is_a618(adreno_gpu) || adreno_is_7c3(adreno_gpu))
+ gpu->clamp_to_idle = true;
+
/* Check if there is a GMU phandle and set it up */
node = of_parse_phandle(pdev->dev.of_node, "qcom,gmu", 0);
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
index a5c3d1ed..a023d5f 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
@@ -147,8 +147,7 @@ static int a6xx_crashdumper_run(struct msm_gpu *gpu,
/* Make sure all pending memory writes are posted */
wmb();
- gpu_write64(gpu, REG_A6XX_CP_CRASH_SCRIPT_BASE_LO,
- REG_A6XX_CP_CRASH_SCRIPT_BASE_HI, dumper->iova);
+ gpu_write64(gpu, REG_A6XX_CP_CRASH_SCRIPT_BASE_LO, dumper->iova);
gpu_write(gpu, REG_A6XX_CP_CRASH_DUMP_CNTL, 1);
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index 5a0e849..57586c7 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -191,37 +191,38 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)
return zap_shader_load_mdt(gpu, adreno_gpu->info->zapfw, pasid);
}
-void adreno_set_llc_attributes(struct iommu_domain *iommu)
+struct msm_gem_address_space *
+adreno_create_address_space(struct msm_gpu *gpu,
+ struct platform_device *pdev)
{
- iommu_set_pgtable_quirks(iommu, IO_PGTABLE_QUIRK_ARM_OUTER_WBWA);
+ return adreno_iommu_create_address_space(gpu, pdev, 0);
}
struct msm_gem_address_space *
adreno_iommu_create_address_space(struct msm_gpu *gpu,
- struct platform_device *pdev)
+ struct platform_device *pdev,
+ unsigned long quirks)
{
- struct iommu_domain *iommu;
+ struct iommu_domain_geometry *geometry;
struct msm_mmu *mmu;
struct msm_gem_address_space *aspace;
u64 start, size;
- iommu = iommu_domain_alloc(&platform_bus_type);
- if (!iommu)
- return NULL;
-
- mmu = msm_iommu_new(&pdev->dev, iommu);
- if (IS_ERR(mmu)) {
- iommu_domain_free(iommu);
+ mmu = msm_iommu_new(&pdev->dev, quirks);
+ if (IS_ERR_OR_NULL(mmu))
return ERR_CAST(mmu);
- }
+
+ geometry = msm_iommu_get_geometry(mmu);
+ if (IS_ERR(geometry))
+ return ERR_CAST(geometry);
/*
* Use the aperture start or SZ_16M, whichever is greater. This will
* ensure that we align with the allocated pagetable range while still
* allowing room in the lower 32 bits for GMEM and whatnot
*/
- start = max_t(u64, SZ_16M, iommu->geometry.aperture_start);
- size = iommu->geometry.aperture_end - start + 1;
+ start = max_t(u64, SZ_16M, geometry->aperture_start);
+ size = geometry->aperture_end - start + 1;
aspace = msm_gem_address_space_create(mmu, "gpu",
start & GENMASK_ULL(48, 0), size);
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
index e7adc5c..5d4b1c9 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
@@ -335,10 +335,13 @@ void adreno_show_object(struct drm_printer *p, void **ptr, int len,
* attached targets
*/
struct msm_gem_address_space *
-adreno_iommu_create_address_space(struct msm_gpu *gpu,
- struct platform_device *pdev);
+adreno_create_address_space(struct msm_gpu *gpu,
+ struct platform_device *pdev);
-void adreno_set_llc_attributes(struct iommu_domain *iommu);
+struct msm_gem_address_space *
+adreno_iommu_create_address_space(struct msm_gpu *gpu,
+ struct platform_device *pdev,
+ unsigned long quirks);
int adreno_read_speedbin(struct device *dev, u32 *speedbin);
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c
index 2c14646..0f71e8f 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c
@@ -237,12 +237,13 @@ static void dpu_encoder_phys_vid_setup_timing_engine(
unsigned long lock_flags;
struct dpu_hw_intf_cfg intf_cfg = { 0 };
+ drm_mode_init(&mode, &phys_enc->cached_mode);
+
if (!phys_enc->hw_ctl->ops.setup_intf_cfg) {
DPU_ERROR("invalid encoder %d\n", phys_enc != NULL);
return;
}
- mode = phys_enc->cached_mode;
if (!phys_enc->hw_intf->ops.setup_timing_gen) {
DPU_ERROR("timing engine setup is not supported\n");
return;
@@ -634,7 +635,9 @@ static int dpu_encoder_phys_vid_get_frame_count(
{
struct intf_status s = {0};
u32 fetch_start = 0;
- struct drm_display_mode mode = phys_enc->cached_mode;
+ struct drm_display_mode mode;
+
+ drm_mode_init(&mode, &phys_enc->cached_mode);
if (!dpu_encoder_phys_vid_is_master(phys_enc))
return -EINVAL;
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c
index f436a1f..d955403 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c
@@ -434,6 +434,12 @@ static const struct dpu_format dpu_format_map[] = {
DPU_CHROMA_H2V1, DPU_FORMAT_FLAG_YUV,
DPU_FETCH_LINEAR, 2),
+ PSEUDO_YUV_FMT_LOOSE(P010,
+ 0, COLOR_8BIT, COLOR_8BIT, COLOR_8BIT,
+ C1_B_Cb, C2_R_Cr,
+ DPU_CHROMA_420, DPU_FORMAT_FLAG_DX | DPU_FORMAT_FLAG_YUV,
+ DPU_FETCH_LINEAR, 2),
+
INTERLEAVED_YUV_FMT(VYUY,
0, COLOR_8BIT, COLOR_8BIT, COLOR_8BIT,
C2_R_Cr, C0_G_Y, C1_B_Cb, C0_G_Y,
@@ -524,12 +530,26 @@ static const struct dpu_format dpu_format_map_ubwc[] = {
true, 4, DPU_FORMAT_FLAG_DX | DPU_FORMAT_FLAG_COMPRESSED,
DPU_FETCH_UBWC, 2, DPU_TILE_HEIGHT_UBWC),
+ INTERLEAVED_RGB_FMT_TILED(XRGB2101010,
+ COLOR_8BIT, COLOR_8BIT, COLOR_8BIT, COLOR_8BIT,
+ C2_R_Cr, C0_G_Y, C1_B_Cb, C3_ALPHA, 4,
+ true, 4, DPU_FORMAT_FLAG_DX | DPU_FORMAT_FLAG_COMPRESSED,
+ DPU_FETCH_UBWC, 2, DPU_TILE_HEIGHT_UBWC),
+
PSEUDO_YUV_FMT_TILED(NV12,
0, COLOR_8BIT, COLOR_8BIT, COLOR_8BIT,
C1_B_Cb, C2_R_Cr,
DPU_CHROMA_420, DPU_FORMAT_FLAG_YUV |
DPU_FORMAT_FLAG_COMPRESSED,
DPU_FETCH_UBWC, 4, DPU_TILE_HEIGHT_NV12),
+
+ PSEUDO_YUV_FMT_TILED(P010,
+ 0, COLOR_8BIT, COLOR_8BIT, COLOR_8BIT,
+ C1_B_Cb, C2_R_Cr,
+ DPU_CHROMA_420, DPU_FORMAT_FLAG_DX |
+ DPU_FORMAT_FLAG_YUV |
+ DPU_FORMAT_FLAG_COMPRESSED,
+ DPU_FETCH_UBWC, 4, DPU_TILE_HEIGHT_UBWC),
};
/* _dpu_get_v_h_subsample_rate - Get subsample rates for all formats we support
@@ -571,13 +591,15 @@ static int _dpu_format_get_media_color_ubwc(const struct dpu_format *fmt)
{DRM_FORMAT_XBGR8888, COLOR_FMT_RGBA8888_UBWC},
{DRM_FORMAT_XRGB8888, COLOR_FMT_RGBA8888_UBWC},
{DRM_FORMAT_ABGR2101010, COLOR_FMT_RGBA1010102_UBWC},
+ {DRM_FORMAT_XRGB2101010, COLOR_FMT_RGBA1010102_UBWC},
{DRM_FORMAT_XBGR2101010, COLOR_FMT_RGBA1010102_UBWC},
{DRM_FORMAT_BGR565, COLOR_FMT_RGB565_UBWC},
};
int color_fmt = -1;
int i;
- if (fmt->base.pixel_format == DRM_FORMAT_NV12) {
+ if (fmt->base.pixel_format == DRM_FORMAT_NV12 ||
+ fmt->base.pixel_format == DRM_FORMAT_P010) {
if (DPU_FORMAT_IS_DX(fmt)) {
if (fmt->unpack_tight)
color_fmt = COLOR_FMT_NV12_BPP10_UBWC;
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c
index 27f029f..2196e20 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c
@@ -156,6 +156,7 @@ static const uint32_t plane_formats[] = {
DRM_FORMAT_RGBX8888,
DRM_FORMAT_BGRX8888,
DRM_FORMAT_XBGR8888,
+ DRM_FORMAT_XRGB2101010,
DRM_FORMAT_RGB888,
DRM_FORMAT_BGR888,
DRM_FORMAT_RGB565,
@@ -184,6 +185,7 @@ static const uint32_t plane_formats_yuv[] = {
DRM_FORMAT_RGBA8888,
DRM_FORMAT_BGRX8888,
DRM_FORMAT_BGRA8888,
+ DRM_FORMAT_XRGB2101010,
DRM_FORMAT_XRGB8888,
DRM_FORMAT_XBGR8888,
DRM_FORMAT_RGBX8888,
@@ -208,6 +210,7 @@ static const uint32_t plane_formats_yuv[] = {
DRM_FORMAT_RGBX4444,
DRM_FORMAT_BGRX4444,
+ DRM_FORMAT_P010,
DRM_FORMAT_NV12,
DRM_FORMAT_NV21,
DRM_FORMAT_NV16,
@@ -318,6 +321,18 @@ static const struct dpu_caps sc7180_dpu_caps = {
.pixel_ram_size = DEFAULT_PIXEL_RAM_SIZE,
};
+static const struct dpu_caps sm6115_dpu_caps = {
+ .max_mixer_width = DEFAULT_DPU_OUTPUT_LINE_WIDTH,
+ .max_mixer_blendstages = 0x4,
+ .qseed_type = DPU_SSPP_SCALER_QSEED3LITE,
+ .smart_dma_rev = DPU_SSPP_SMART_DMA_V2, /* TODO: v2.5 */
+ .ubwc_version = DPU_HW_UBWC_VER_20,
+ .has_dim_layer = true,
+ .has_idle_pc = true,
+ .max_linewidth = 2160,
+ .pixel_ram_size = DEFAULT_PIXEL_RAM_SIZE,
+};
+
static const struct dpu_caps sm8150_dpu_caps = {
.max_mixer_width = DEFAULT_DPU_OUTPUT_LINE_WIDTH,
.max_mixer_blendstages = 0xb,
@@ -472,6 +487,19 @@ static const struct dpu_mdp_cfg sc8180x_mdp[] = {
},
};
+static const struct dpu_mdp_cfg sm6115_mdp[] = {
+ {
+ .name = "top_0", .id = MDP_TOP,
+ .base = 0x0, .len = 0x494,
+ .features = 0,
+ .highest_bank_bit = 0x1,
+ .clk_ctrls[DPU_CLK_CTRL_VIG0] = {
+ .reg_off = 0x2ac, .bit_off = 0},
+ .clk_ctrls[DPU_CLK_CTRL_DMA0] = {
+ .reg_off = 0x2ac, .bit_off = 8},
+ },
+};
+
static const struct dpu_mdp_cfg sm8250_mdp[] = {
{
.name = "top_0", .id = MDP_TOP,
@@ -849,6 +877,16 @@ static const struct dpu_sspp_cfg sc7180_sspp[] = {
sdm845_dma_sblk_2, 9, SSPP_TYPE_DMA, DPU_CLK_CTRL_CURSOR1),
};
+static const struct dpu_sspp_sub_blks sm6115_vig_sblk_0 =
+ _VIG_SBLK("0", 2, DPU_SSPP_SCALER_QSEED3LITE);
+
+static const struct dpu_sspp_cfg sm6115_sspp[] = {
+ SSPP_BLK("sspp_0", SSPP_VIG0, 0x4000, VIG_SM8250_MASK,
+ sm6115_vig_sblk_0, 0, SSPP_TYPE_VIG, DPU_CLK_CTRL_VIG0),
+ SSPP_BLK("sspp_8", SSPP_DMA0, 0x24000, DMA_SDM845_MASK,
+ sdm845_dma_sblk_0, 1, SSPP_TYPE_DMA, DPU_CLK_CTRL_DMA0),
+};
+
static const struct dpu_sspp_sub_blks sm8250_vig_sblk_0 =
_VIG_SBLK("0", 5, DPU_SSPP_SCALER_QSEED3LITE);
static const struct dpu_sspp_sub_blks sm8250_vig_sblk_1 =
@@ -1175,6 +1213,13 @@ static const struct dpu_pingpong_cfg sm8150_pp[] = {
-1),
};
+static const struct dpu_pingpong_cfg sc7280_pp[] = {
+ PP_BLK("pingpong_0", PINGPONG_0, 0x59000, 0, sc7280_pp_sblk, -1, -1),
+ PP_BLK("pingpong_1", PINGPONG_1, 0x6a000, 0, sc7280_pp_sblk, -1, -1),
+ PP_BLK("pingpong_2", PINGPONG_2, 0x6b000, 0, sc7280_pp_sblk, -1, -1),
+ PP_BLK("pingpong_3", PINGPONG_3, 0x6c000, 0, sc7280_pp_sblk, -1, -1),
+};
+
static struct dpu_pingpong_cfg qcm2290_pp[] = {
PP_BLK("pingpong_0", PINGPONG_0, 0x70000, 0, sdm845_pp_sblk,
DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 8),
@@ -1198,13 +1243,6 @@ static const struct dpu_merge_3d_cfg sm8150_merge_3d[] = {
MERGE_3D_BLK("merge_3d_2", MERGE_3D_2, 0x83200),
};
-static const struct dpu_pingpong_cfg sc7280_pp[] = {
- PP_BLK("pingpong_0", PINGPONG_0, 0x59000, 0, sc7280_pp_sblk, -1, -1),
- PP_BLK("pingpong_1", PINGPONG_1, 0x6a000, 0, sc7280_pp_sblk, -1, -1),
- PP_BLK("pingpong_2", PINGPONG_2, 0x6b000, 0, sc7280_pp_sblk, -1, -1),
- PP_BLK("pingpong_3", PINGPONG_3, 0x6c000, 0, sc7280_pp_sblk, -1, -1),
-};
-
/*************************************************************
* DSC sub blocks config
*************************************************************/
@@ -1587,6 +1625,35 @@ static const struct dpu_perf_cfg sc7180_perf_data = {
.bw_inefficiency_factor = 120,
};
+static const struct dpu_perf_cfg sm6115_perf_data = {
+ .max_bw_low = 3100000,
+ .max_bw_high = 4000000,
+ .min_core_ib = 2400000,
+ .min_llcc_ib = 800000,
+ .min_dram_ib = 800000,
+ .min_prefill_lines = 24,
+ .danger_lut_tbl = {0xff, 0xffff, 0x0},
+ .safe_lut_tbl = {0xfff0, 0xff00, 0xffff},
+ .qos_lut_tbl = {
+ {.nentry = ARRAY_SIZE(sc7180_qos_linear),
+ .entries = sc7180_qos_linear
+ },
+ {.nentry = ARRAY_SIZE(sc7180_qos_macrotile),
+ .entries = sc7180_qos_macrotile
+ },
+ {.nentry = ARRAY_SIZE(sc7180_qos_nrt),
+ .entries = sc7180_qos_nrt
+ },
+ /* TODO: macrotile-qseed is different from macrotile */
+ },
+ .cdp_cfg = {
+ {.rd_enable = 1, .wr_enable = 1},
+ {.rd_enable = 1, .wr_enable = 0}
+ },
+ .clk_inefficiency_factor = 105,
+ .bw_inefficiency_factor = 120,
+};
+
static const struct dpu_perf_cfg sm8150_perf_data = {
.max_bw_low = 12800000,
.max_bw_high = 12800000,
@@ -1798,6 +1865,28 @@ static const struct dpu_mdss_cfg sc7180_dpu_cfg = {
.mdss_irqs = IRQ_SC7180_MASK,
};
+static const struct dpu_mdss_cfg sm6115_dpu_cfg = {
+ .caps = &sm6115_dpu_caps,
+ .mdp_count = ARRAY_SIZE(sm6115_mdp),
+ .mdp = sm6115_mdp,
+ .ctl_count = ARRAY_SIZE(qcm2290_ctl),
+ .ctl = qcm2290_ctl,
+ .sspp_count = ARRAY_SIZE(sm6115_sspp),
+ .sspp = sm6115_sspp,
+ .mixer_count = ARRAY_SIZE(qcm2290_lm),
+ .mixer = qcm2290_lm,
+ .dspp_count = ARRAY_SIZE(qcm2290_dspp),
+ .dspp = qcm2290_dspp,
+ .pingpong_count = ARRAY_SIZE(qcm2290_pp),
+ .pingpong = qcm2290_pp,
+ .intf_count = ARRAY_SIZE(qcm2290_intf),
+ .intf = qcm2290_intf,
+ .vbif_count = ARRAY_SIZE(sdm845_vbif),
+ .vbif = sdm845_vbif,
+ .perf = &sm6115_perf_data,
+ .mdss_irqs = IRQ_SC7180_MASK,
+};
+
static const struct dpu_mdss_cfg sm8150_dpu_cfg = {
.caps = &sm8150_dpu_caps,
.mdp_count = ARRAY_SIZE(sdm845_mdp),
@@ -1932,6 +2021,7 @@ static const struct dpu_mdss_hw_cfg_handler cfg_handler[] = {
{ .hw_rev = DPU_HW_VER_510, .dpu_cfg = &sc8180x_dpu_cfg},
{ .hw_rev = DPU_HW_VER_600, .dpu_cfg = &sm8250_dpu_cfg},
{ .hw_rev = DPU_HW_VER_620, .dpu_cfg = &sc7180_dpu_cfg},
+ { .hw_rev = DPU_HW_VER_630, .dpu_cfg = &sm6115_dpu_cfg},
{ .hw_rev = DPU_HW_VER_650, .dpu_cfg = &qcm2290_dpu_cfg},
{ .hw_rev = DPU_HW_VER_720, .dpu_cfg = &sc7280_dpu_cfg},
};
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h
index 38aa38a..3b645d5 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h
@@ -44,6 +44,7 @@
#define DPU_HW_VER_510 DPU_HW_VER(5, 1, 1) /* sc8180 */
#define DPU_HW_VER_600 DPU_HW_VER(6, 0, 0) /* sm8250 */
#define DPU_HW_VER_620 DPU_HW_VER(6, 2, 0) /* sc7180 v1.0 */
+#define DPU_HW_VER_630 DPU_HW_VER(6, 3, 0) /* sm6115|sm4250 */
#define DPU_HW_VER_650 DPU_HW_VER(6, 5, 0) /* qcm2290|sm4125 */
#define DPU_HW_VER_720 DPU_HW_VER(7, 2, 0) /* sc7280 */
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_dsc.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_dsc.c
index f2ddcfb..3662df6 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_dsc.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_dsc.c
@@ -42,7 +42,7 @@ static void dpu_hw_dsc_config(struct dpu_hw_dsc *hw_dsc,
u32 initial_lines)
{
struct dpu_hw_blk_reg_map *c = &hw_dsc->hw;
- u32 data, lsb, bpp;
+ u32 data;
u32 slice_last_group_size;
u32 det_thresh_flatness;
bool is_cmd_mode = !(mode & DSC_MODE_VIDEO);
@@ -56,14 +56,7 @@ static void dpu_hw_dsc_config(struct dpu_hw_dsc *hw_dsc,
data = (initial_lines << 20);
data |= ((slice_last_group_size - 1) << 18);
/* bpp is 6.4 format, 4 LSBs bits are for fractional part */
- data |= dsc->bits_per_pixel << 12;
- lsb = dsc->bits_per_pixel % 4;
- bpp = dsc->bits_per_pixel / 4;
- bpp *= 4;
- bpp <<= 4;
- bpp |= lsb;
-
- data |= bpp << 8;
+ data |= (dsc->bits_per_pixel << 8);
data |= (dsc->block_pred_enable << 7);
data |= (dsc->line_buf_depth << 3);
data |= (dsc->simple_422 << 2);
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
index 5e6e262..b7119951 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
@@ -194,7 +194,7 @@ struct dpu_debugfs_regset32 {
struct dpu_kms *dpu_kms;
};
-static int _dpu_debugfs_show_regset32(struct seq_file *s, void *data)
+static int dpu_regset32_show(struct seq_file *s, void *data)
{
struct dpu_debugfs_regset32 *regset = s->private;
struct dpu_kms *dpu_kms = regset->dpu_kms;
@@ -227,19 +227,7 @@ static int _dpu_debugfs_show_regset32(struct seq_file *s, void *data)
return 0;
}
-
-static int dpu_debugfs_open_regset32(struct inode *inode,
- struct file *file)
-{
- return single_open(file, _dpu_debugfs_show_regset32, inode->i_private);
-}
-
-static const struct file_operations dpu_fops_regset32 = {
- .open = dpu_debugfs_open_regset32,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
+DEFINE_SHOW_ATTRIBUTE(dpu_regset32);
void dpu_debugfs_create_regset32(const char *name, umode_t mode,
void *parent,
@@ -259,7 +247,7 @@ void dpu_debugfs_create_regset32(const char *name, umode_t mode,
regset->blk_len = length;
regset->dpu_kms = dpu_kms;
- debugfs_create_file(name, mode, parent, regset, &dpu_fops_regset32);
+ debugfs_create_file(name, mode, parent, regset, &dpu_regset32_fops);
}
static int dpu_kms_debugfs_init(struct msm_kms *kms, struct drm_minor *minor)
@@ -1304,6 +1292,7 @@ static const struct of_device_id dpu_dt_match[] = {
{ .compatible = "qcom,sc7180-dpu", },
{ .compatible = "qcom,sc7280-dpu", },
{ .compatible = "qcom,sc8180x-dpu", },
+ { .compatible = "qcom,sm6115-dpu", },
{ .compatible = "qcom,sm8150-dpu", },
{ .compatible = "qcom,sm8250-dpu", },
{}
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c
index 658005f..8671902 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c
@@ -69,9 +69,11 @@ static const uint32_t qcom_compressed_supported_formats[] = {
DRM_FORMAT_ARGB8888,
DRM_FORMAT_XBGR8888,
DRM_FORMAT_XRGB8888,
+ DRM_FORMAT_XRGB2101010,
DRM_FORMAT_BGR565,
DRM_FORMAT_NV12,
+ DRM_FORMAT_P010,
};
/**
diff --git a/drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c b/drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c
index 964573d..9a1a076 100644
--- a/drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c
+++ b/drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c
@@ -387,7 +387,7 @@ static int mdp4_kms_init(struct drm_device *dev)
struct msm_drm_private *priv = dev->dev_private;
struct mdp4_kms *mdp4_kms;
struct msm_kms *kms = NULL;
- struct iommu_domain *iommu;
+ struct msm_mmu *mmu;
struct msm_gem_address_space *aspace;
int irq, ret;
u32 major, minor;
@@ -499,10 +499,15 @@ static int mdp4_kms_init(struct drm_device *dev)
mdp4_disable(mdp4_kms);
mdelay(16);
- iommu = iommu_domain_alloc(pdev->dev.bus);
- if (iommu) {
- struct msm_mmu *mmu = msm_iommu_new(&pdev->dev, iommu);
-
+ mmu = msm_iommu_new(&pdev->dev, 0);
+ if (IS_ERR(mmu)) {
+ ret = PTR_ERR(mmu);
+ goto fail;
+ } else if (!mmu) {
+ DRM_DEV_INFO(dev->dev, "no iommu, fallback to phys "
+ "contig buffers for scanout\n");
+ aspace = NULL;
+ } else {
aspace = msm_gem_address_space_create(mmu,
"mdp4", 0x1000, 0x100000000 - 0x1000);
@@ -514,10 +519,6 @@ static int mdp4_kms_init(struct drm_device *dev)
}
kms->aspace = aspace;
- } else {
- DRM_DEV_INFO(dev->dev, "no iommu, fallback to phys "
- "contig buffers for scanout\n");
- aspace = NULL;
}
ret = modeset_init(mdp4_kms);
diff --git a/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c b/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c
index b0d2183..29ae5c9 100644
--- a/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c
+++ b/drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c
@@ -203,7 +203,7 @@ static int mdp5_set_split_display(struct msm_kms *kms,
slave_encoder);
}
-static void mdp5_destroy(struct platform_device *pdev);
+static void mdp5_destroy(struct mdp5_kms *mdp5_kms);
static void mdp5_kms_destroy(struct msm_kms *kms)
{
@@ -223,7 +223,7 @@ static void mdp5_kms_destroy(struct msm_kms *kms)
}
mdp_kms_destroy(&mdp5_kms->base);
- mdp5_destroy(mdp5_kms->pdev);
+ mdp5_destroy(mdp5_kms);
}
#ifdef CONFIG_DEBUG_FS
@@ -559,6 +559,8 @@ static int mdp5_kms_init(struct drm_device *dev)
int irq, i, ret;
ret = mdp5_init(to_platform_device(dev->dev), dev);
+ if (ret)
+ return ret;
/* priv->kms would have been populated by the MDP5 driver */
kms = priv->kms;
@@ -632,9 +634,8 @@ static int mdp5_kms_init(struct drm_device *dev)
return ret;
}
-static void mdp5_destroy(struct platform_device *pdev)
+static void mdp5_destroy(struct mdp5_kms *mdp5_kms)
{
- struct mdp5_kms *mdp5_kms = platform_get_drvdata(pdev);
int i;
if (mdp5_kms->ctlm)
@@ -648,7 +649,7 @@ static void mdp5_destroy(struct platform_device *pdev)
kfree(mdp5_kms->intfs[i]);
if (mdp5_kms->rpm_enabled)
- pm_runtime_disable(&pdev->dev);
+ pm_runtime_disable(&mdp5_kms->pdev->dev);
drm_atomic_private_obj_fini(&mdp5_kms->glob_state);
drm_modeset_lock_fini(&mdp5_kms->glob_state_lock);
@@ -797,8 +798,6 @@ static int mdp5_init(struct platform_device *pdev, struct drm_device *dev)
goto fail;
}
- platform_set_drvdata(pdev, mdp5_kms);
-
spin_lock_init(&mdp5_kms->resource_lock);
mdp5_kms->dev = dev;
@@ -839,6 +838,9 @@ static int mdp5_init(struct platform_device *pdev, struct drm_device *dev)
*/
clk_set_rate(mdp5_kms->core_clk, 200000000);
+ /* set uninit-ed kms */
+ priv->kms = &mdp5_kms->base.base;
+
pm_runtime_enable(&pdev->dev);
mdp5_kms->rpm_enabled = true;
@@ -890,13 +892,10 @@ static int mdp5_init(struct platform_device *pdev, struct drm_device *dev)
if (ret)
goto fail;
- /* set uninit-ed kms */
- priv->kms = &mdp5_kms->base.base;
-
return 0;
fail:
if (mdp5_kms)
- mdp5_destroy(pdev);
+ mdp5_destroy(mdp5_kms);
return ret;
}
@@ -953,7 +952,8 @@ static int mdp5_dev_remove(struct platform_device *pdev)
static __maybe_unused int mdp5_runtime_suspend(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
- struct mdp5_kms *mdp5_kms = platform_get_drvdata(pdev);
+ struct msm_drm_private *priv = platform_get_drvdata(pdev);
+ struct mdp5_kms *mdp5_kms = to_mdp5_kms(to_mdp_kms(priv->kms));
DBG("");
@@ -963,7 +963,8 @@ static __maybe_unused int mdp5_runtime_suspend(struct device *dev)
static __maybe_unused int mdp5_runtime_resume(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
- struct mdp5_kms *mdp5_kms = platform_get_drvdata(pdev);
+ struct msm_drm_private *priv = platform_get_drvdata(pdev);
+ struct mdp5_kms *mdp5_kms = to_mdp5_kms(to_mdp_kms(priv->kms));
DBG("");
diff --git a/drivers/gpu/drm/msm/dp/dp_display.c b/drivers/gpu/drm/msm/dp/dp_display.c
index a49f6db..7ff60e5 100644
--- a/drivers/gpu/drm/msm/dp/dp_display.c
+++ b/drivers/gpu/drm/msm/dp/dp_display.c
@@ -857,7 +857,7 @@ static int dp_display_set_mode(struct msm_dp *dp_display,
dp = container_of(dp_display, struct dp_display_private, dp_display);
- dp->panel->dp_mode.drm_mode = mode->drm_mode;
+ drm_mode_copy(&dp->panel->dp_mode.drm_mode, &mode->drm_mode);
dp->panel->dp_mode.bpp = mode->bpp;
dp->panel->dp_mode.capabilities = mode->capabilities;
dp_panel_init_panel_info(dp->panel);
@@ -1600,20 +1600,10 @@ static int dp_display_get_next_bridge(struct msm_dp *dp)
int msm_dp_modeset_init(struct msm_dp *dp_display, struct drm_device *dev,
struct drm_encoder *encoder)
{
- struct msm_drm_private *priv;
+ struct msm_drm_private *priv = dev->dev_private;
struct dp_display_private *dp_priv;
int ret;
- if (WARN_ON(!encoder) || WARN_ON(!dp_display) || WARN_ON(!dev))
- return -EINVAL;
-
- priv = dev->dev_private;
-
- if (priv->num_bridges == ARRAY_SIZE(priv->bridges)) {
- DRM_DEV_ERROR(dev->dev, "too many bridges\n");
- return -ENOSPC;
- }
-
dp_display->drm_dev = dev;
dp_priv = container_of(dp_display, struct dp_display_private, dp_display);
diff --git a/drivers/gpu/drm/msm/dp/dp_link.c b/drivers/gpu/drm/msm/dp/dp_link.c
index 36bb619..f1f1d64 100644
--- a/drivers/gpu/drm/msm/dp/dp_link.c
+++ b/drivers/gpu/drm/msm/dp/dp_link.c
@@ -49,23 +49,26 @@ static int dp_aux_link_power_up(struct drm_dp_aux *aux,
struct dp_link_info *link)
{
u8 value;
- int err;
+ ssize_t len;
+ int i;
if (link->revision < 0x11)
return 0;
- err = drm_dp_dpcd_readb(aux, DP_SET_POWER, &value);
- if (err < 0)
- return err;
+ len = drm_dp_dpcd_readb(aux, DP_SET_POWER, &value);
+ if (len < 0)
+ return len;
value &= ~DP_SET_POWER_MASK;
value |= DP_SET_POWER_D0;
- err = drm_dp_dpcd_writeb(aux, DP_SET_POWER, value);
- if (err < 0)
- return err;
-
- usleep_range(1000, 2000);
+ /* retry for 1ms to give the sink time to wake up */
+ for (i = 0; i < 3; i++) {
+ len = drm_dp_dpcd_writeb(aux, DP_SET_POWER, value);
+ usleep_range(1000, 2000);
+ if (len == 1)
+ break;
+ }
return 0;
}
diff --git a/drivers/gpu/drm/msm/dsi/dsi.c b/drivers/gpu/drm/msm/dsi/dsi.c
index 8a95c74..31fdee2 100644
--- a/drivers/gpu/drm/msm/dsi/dsi.c
+++ b/drivers/gpu/drm/msm/dsi/dsi.c
@@ -211,14 +211,9 @@ void __exit msm_dsi_unregister(void)
int msm_dsi_modeset_init(struct msm_dsi *msm_dsi, struct drm_device *dev,
struct drm_encoder *encoder)
{
- struct msm_drm_private *priv;
+ struct msm_drm_private *priv = dev->dev_private;
int ret;
- if (WARN_ON(!encoder) || WARN_ON(!msm_dsi) || WARN_ON(!dev))
- return -EINVAL;
-
- priv = dev->dev_private;
-
if (priv->num_bridges == ARRAY_SIZE(priv->bridges)) {
DRM_DEV_ERROR(dev->dev, "too many bridges\n");
return -ENOSPC;
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index 7fbf391..89aadd3 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -21,6 +21,7 @@
#include <video/mipi_display.h>
+#include <drm/display/drm_dsc_helper.h>
#include <drm/drm_of.h>
#include "dsi.h"
@@ -33,7 +34,7 @@
#define DSI_RESET_TOGGLE_DELAY_MS 20
-static int dsi_populate_dsc_params(struct drm_dsc_config *dsc);
+static int dsi_populate_dsc_params(struct msm_dsi_host *msm_host, struct drm_dsc_config *dsc);
static int dsi_get_version(const void __iomem *base, u32 *major, u32 *minor)
{
@@ -842,17 +843,15 @@ static void dsi_ctrl_config(struct msm_dsi_host *msm_host, bool enable,
static void dsi_update_dsc_timing(struct msm_dsi_host *msm_host, bool is_cmd_mode, u32 hdisplay)
{
struct drm_dsc_config *dsc = msm_host->dsc;
- u32 reg, intf_width, reg_ctrl, reg_ctrl2;
+ u32 reg, reg_ctrl, reg_ctrl2;
u32 slice_per_intf, total_bytes_per_intf;
u32 pkt_per_line;
- u32 bytes_in_slice;
u32 eol_byte_num;
/* first calculate dsc parameters and then program
* compress mode registers
*/
- intf_width = hdisplay;
- slice_per_intf = DIV_ROUND_UP(intf_width, dsc->slice_width);
+ slice_per_intf = DIV_ROUND_UP(hdisplay, dsc->slice_width);
/* If slice_per_pkt is greater than slice_per_intf
* then default to 1. This can happen during partial
@@ -861,12 +860,7 @@ static void dsi_update_dsc_timing(struct msm_dsi_host *msm_host, bool is_cmd_mod
if (slice_per_intf > dsc->slice_count)
dsc->slice_count = 1;
- slice_per_intf = DIV_ROUND_UP(hdisplay, dsc->slice_width);
- bytes_in_slice = DIV_ROUND_UP(dsc->slice_width * dsc->bits_per_pixel, 8);
-
- dsc->slice_chunk_size = bytes_in_slice;
-
- total_bytes_per_intf = bytes_in_slice * slice_per_intf;
+ total_bytes_per_intf = dsc->slice_chunk_size * slice_per_intf;
eol_byte_num = total_bytes_per_intf % 3;
pkt_per_line = slice_per_intf / dsc->slice_count;
@@ -892,7 +886,7 @@ static void dsi_update_dsc_timing(struct msm_dsi_host *msm_host, bool is_cmd_mod
reg_ctrl |= reg;
reg_ctrl2 &= ~DSI_COMMAND_COMPRESSION_MODE_CTRL2_STREAM0_SLICE_WIDTH__MASK;
- reg_ctrl2 |= DSI_COMMAND_COMPRESSION_MODE_CTRL2_STREAM0_SLICE_WIDTH(bytes_in_slice);
+ reg_ctrl2 |= DSI_COMMAND_COMPRESSION_MODE_CTRL2_STREAM0_SLICE_WIDTH(dsc->slice_chunk_size);
dsi_write(msm_host, REG_DSI_COMMAND_COMPRESSION_MODE_CTRL, reg_ctrl);
dsi_write(msm_host, REG_DSI_COMMAND_COMPRESSION_MODE_CTRL2, reg_ctrl2);
@@ -915,6 +909,7 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi)
u32 va_end = va_start + mode->vdisplay;
u32 hdisplay = mode->hdisplay;
u32 wc;
+ int ret;
DBG("");
@@ -950,7 +945,9 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi)
/* we do the calculations for dsc parameters here so that
* panel can use these parameters
*/
- dsi_populate_dsc_params(dsc);
+ ret = dsi_populate_dsc_params(msm_host, dsc);
+ if (ret)
+ return;
/* Divide the display by 3 but keep back/font porch and
* pulse width same
@@ -1754,18 +1751,20 @@ static char bpg_offset[DSC_NUM_BUF_RANGES] = {
2, 0, 0, -2, -4, -6, -8, -8, -8, -10, -10, -12, -12, -12, -12
};
-static int dsi_populate_dsc_params(struct drm_dsc_config *dsc)
+static int dsi_populate_dsc_params(struct msm_dsi_host *msm_host, struct drm_dsc_config *dsc)
{
- int mux_words_size;
- int groups_per_line, groups_total;
- int min_rate_buffer_size;
- int hrd_delay;
- int pre_num_extra_mux_bits, num_extra_mux_bits;
- int slice_bits;
- int target_bpp_x16;
- int data;
- int final_value, final_scale;
int i;
+ u16 bpp = dsc->bits_per_pixel >> 4;
+
+ if (dsc->bits_per_pixel & 0xf) {
+ DRM_DEV_ERROR(&msm_host->pdev->dev, "DSI does not support fractional bits_per_pixel\n");
+ return -EINVAL;
+ }
+
+ if (dsc->bits_per_component != 8) {
+ DRM_DEV_ERROR(&msm_host->pdev->dev, "DSI does not support bits_per_component != 8 yet\n");
+ return -EOPNOTSUPP;
+ }
dsc->rc_model_size = 8192;
dsc->first_line_bpg_offset = 12;
@@ -1783,16 +1782,21 @@ static int dsi_populate_dsc_params(struct drm_dsc_config *dsc)
for (i = 0; i < DSC_NUM_BUF_RANGES; i++) {
dsc->rc_range_params[i].range_min_qp = min_qp[i];
dsc->rc_range_params[i].range_max_qp = max_qp[i];
- dsc->rc_range_params[i].range_bpg_offset = bpg_offset[i];
+ /*
+ * Range BPG Offset contains two's-complement signed values that fill
+ * 8 bits, yet the registers and DCS PPS field are only 6 bits wide.
+ */
+ dsc->rc_range_params[i].range_bpg_offset = bpg_offset[i] & DSC_RANGE_BPG_OFFSET_MASK;
}
- dsc->initial_offset = 6144; /* Not bpp 12 */
- if (dsc->bits_per_pixel != 8)
+ dsc->initial_offset = 6144; /* Not bpp 12 */
+ if (bpp != 8)
dsc->initial_offset = 2048; /* bpp = 12 */
- mux_words_size = 48; /* bpc == 8/10 */
- if (dsc->bits_per_component == 12)
- mux_words_size = 64;
+ if (dsc->bits_per_component <= 10)
+ dsc->mux_word_size = DSC_MUX_WORD_SIZE_8_10_BPC;
+ else
+ dsc->mux_word_size = DSC_MUX_WORD_SIZE_12_BPC;
dsc->initial_xmit_delay = 512;
dsc->initial_scale_value = 32;
@@ -1804,63 +1808,8 @@ static int dsi_populate_dsc_params(struct drm_dsc_config *dsc)
dsc->flatness_max_qp = 12;
dsc->rc_quant_incr_limit0 = 11;
dsc->rc_quant_incr_limit1 = 11;
- dsc->mux_word_size = DSC_MUX_WORD_SIZE_8_10_BPC;
- /* FIXME: need to call drm_dsc_compute_rc_parameters() so that rest of
- * params are calculated
- */
- groups_per_line = DIV_ROUND_UP(dsc->slice_width, 3);
- dsc->slice_chunk_size = dsc->slice_width * dsc->bits_per_pixel / 8;
- if ((dsc->slice_width * dsc->bits_per_pixel) % 8)
- dsc->slice_chunk_size++;
-
- /* rbs-min */
- min_rate_buffer_size = dsc->rc_model_size - dsc->initial_offset +
- dsc->initial_xmit_delay * dsc->bits_per_pixel +
- groups_per_line * dsc->first_line_bpg_offset;
-
- hrd_delay = DIV_ROUND_UP(min_rate_buffer_size, dsc->bits_per_pixel);
-
- dsc->initial_dec_delay = hrd_delay - dsc->initial_xmit_delay;
-
- dsc->initial_scale_value = 8 * dsc->rc_model_size /
- (dsc->rc_model_size - dsc->initial_offset);
-
- slice_bits = 8 * dsc->slice_chunk_size * dsc->slice_height;
-
- groups_total = groups_per_line * dsc->slice_height;
-
- data = dsc->first_line_bpg_offset * 2048;
-
- dsc->nfl_bpg_offset = DIV_ROUND_UP(data, (dsc->slice_height - 1));
-
- pre_num_extra_mux_bits = 3 * (mux_words_size + (4 * dsc->bits_per_component + 4) - 2);
-
- num_extra_mux_bits = pre_num_extra_mux_bits - (mux_words_size -
- ((slice_bits - pre_num_extra_mux_bits) % mux_words_size));
-
- data = 2048 * (dsc->rc_model_size - dsc->initial_offset + num_extra_mux_bits);
- dsc->slice_bpg_offset = DIV_ROUND_UP(data, groups_total);
-
- /* bpp * 16 + 0.5 */
- data = dsc->bits_per_pixel * 16;
- data *= 2;
- data++;
- data /= 2;
- target_bpp_x16 = data;
-
- data = (dsc->initial_xmit_delay * target_bpp_x16) / 16;
- final_value = dsc->rc_model_size - data + num_extra_mux_bits;
- dsc->final_offset = final_value;
-
- final_scale = 8 * dsc->rc_model_size / (dsc->rc_model_size - final_value);
-
- data = (final_scale - 9) * (dsc->nfl_bpg_offset + dsc->slice_bpg_offset);
- dsc->scale_increment_interval = (2048 * dsc->final_offset) / data;
-
- dsc->scale_decrement_interval = groups_per_line / (dsc->initial_scale_value - 8);
-
- return 0;
+ return drm_dsc_compute_rc_parameters(dsc);
}
static int dsi_host_parse_dt(struct msm_dsi_host *msm_host)
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
index 7fc0975..ee60513 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
@@ -549,6 +549,8 @@ static const struct of_device_id dsi_phy_dt_match[] = {
#ifdef CONFIG_DRM_MSM_DSI_14NM_PHY
{ .compatible = "qcom,dsi-phy-14nm",
.data = &dsi_phy_14nm_cfgs },
+ { .compatible = "qcom,dsi-phy-14nm-2290",
+ .data = &dsi_phy_14nm_2290_cfgs },
{ .compatible = "qcom,dsi-phy-14nm-660",
.data = &dsi_phy_14nm_660_cfgs },
{ .compatible = "qcom,dsi-phy-14nm-8953",
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.h b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.h
index 60a99c6..1096afe 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.h
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.h
@@ -50,6 +50,7 @@ extern const struct msm_dsi_phy_cfg dsi_phy_20nm_cfgs;
extern const struct msm_dsi_phy_cfg dsi_phy_28nm_8960_cfgs;
extern const struct msm_dsi_phy_cfg dsi_phy_14nm_cfgs;
extern const struct msm_dsi_phy_cfg dsi_phy_14nm_660_cfgs;
+extern const struct msm_dsi_phy_cfg dsi_phy_14nm_2290_cfgs;
extern const struct msm_dsi_phy_cfg dsi_phy_14nm_8953_cfgs;
extern const struct msm_dsi_phy_cfg dsi_phy_10nm_cfgs;
extern const struct msm_dsi_phy_cfg dsi_phy_10nm_8998_cfgs;
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_14nm.c b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_14nm.c
index 0f8f4ca..9f488ad 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_14nm.c
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_14nm.c
@@ -1081,3 +1081,20 @@ const struct msm_dsi_phy_cfg dsi_phy_14nm_8953_cfgs = {
.io_start = { 0x1a94400, 0x1a96400 },
.num_dsi_phy = 2,
};
+
+const struct msm_dsi_phy_cfg dsi_phy_14nm_2290_cfgs = {
+ .has_phy_lane = true,
+ .regulator_data = dsi_phy_14nm_17mA_regulators,
+ .num_regulators = ARRAY_SIZE(dsi_phy_14nm_17mA_regulators),
+ .ops = {
+ .enable = dsi_14nm_phy_enable,
+ .disable = dsi_14nm_phy_disable,
+ .pll_init = dsi_pll_14nm_init,
+ .save_pll_state = dsi_14nm_pll_save_state,
+ .restore_pll_state = dsi_14nm_pll_restore_state,
+ },
+ .min_pll_rate = VCO_MIN_RATE,
+ .max_pll_rate = VCO_MAX_RATE,
+ .io_start = { 0x5e94400 },
+ .num_dsi_phy = 1,
+};
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi.c b/drivers/gpu/drm/msm/hdmi/hdmi.c
index f28fb21..4d3fdc8 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi.c
+++ b/drivers/gpu/drm/msm/hdmi/hdmi.c
@@ -68,16 +68,17 @@ static void msm_hdmi_destroy(struct hdmi *hdmi)
destroy_workqueue(hdmi->workq);
msm_hdmi_hdcp_destroy(hdmi);
+ if (hdmi->i2c)
+ msm_hdmi_i2c_destroy(hdmi->i2c);
+}
+
+static void msm_hdmi_put_phy(struct hdmi *hdmi)
+{
if (hdmi->phy_dev) {
put_device(hdmi->phy_dev);
hdmi->phy = NULL;
hdmi->phy_dev = NULL;
}
-
- if (hdmi->i2c)
- msm_hdmi_i2c_destroy(hdmi->i2c);
-
- platform_set_drvdata(hdmi->pdev, NULL);
}
static int msm_hdmi_get_phy(struct hdmi *hdmi)
@@ -93,22 +94,18 @@ static int msm_hdmi_get_phy(struct hdmi *hdmi)
}
phy_pdev = of_find_device_by_node(phy_node);
- if (phy_pdev)
- hdmi->phy = platform_get_drvdata(phy_pdev);
-
of_node_put(phy_node);
- if (!phy_pdev) {
- DRM_DEV_ERROR(&pdev->dev, "phy driver is not ready\n");
- return -EPROBE_DEFER;
- }
+ if (!phy_pdev)
+ return dev_err_probe(&pdev->dev, -EPROBE_DEFER, "phy driver is not ready\n");
+
+ hdmi->phy = platform_get_drvdata(phy_pdev);
if (!hdmi->phy) {
- DRM_DEV_ERROR(&pdev->dev, "phy driver is not ready\n");
put_device(&phy_pdev->dev);
- return -EPROBE_DEFER;
+ return dev_err_probe(&pdev->dev, -EPROBE_DEFER, "phy driver is not ready\n");
}
- hdmi->phy_dev = get_device(&phy_pdev->dev);
+ hdmi->phy_dev = &phy_pdev->dev;
return 0;
}
@@ -117,142 +114,10 @@ static int msm_hdmi_get_phy(struct hdmi *hdmi)
* we are to EPROBE_DEFER we want to do it here, rather than later
* at modeset_init() time
*/
-static struct hdmi *msm_hdmi_init(struct platform_device *pdev)
+static int msm_hdmi_init(struct hdmi *hdmi)
{
- struct hdmi_platform_config *config = pdev->dev.platform_data;
- struct hdmi *hdmi = NULL;
- struct resource *res;
- int i, ret;
-
- hdmi = devm_kzalloc(&pdev->dev, sizeof(*hdmi), GFP_KERNEL);
- if (!hdmi) {
- ret = -ENOMEM;
- goto fail;
- }
-
- hdmi->pdev = pdev;
- hdmi->config = config;
- spin_lock_init(&hdmi->reg_lock);
-
- ret = drm_of_find_panel_or_bridge(pdev->dev.of_node, 1, 0, NULL, &hdmi->next_bridge);
- if (ret && ret != -ENODEV)
- goto fail;
-
- hdmi->mmio = msm_ioremap(pdev, config->mmio_name);
- if (IS_ERR(hdmi->mmio)) {
- ret = PTR_ERR(hdmi->mmio);
- goto fail;
- }
-
- /* HDCP needs physical address of hdmi register */
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
- config->mmio_name);
- if (!res) {
- ret = -EINVAL;
- goto fail;
- }
- hdmi->mmio_phy_addr = res->start;
-
- hdmi->qfprom_mmio = msm_ioremap(pdev, config->qfprom_mmio_name);
- if (IS_ERR(hdmi->qfprom_mmio)) {
- DRM_DEV_INFO(&pdev->dev, "can't find qfprom resource\n");
- hdmi->qfprom_mmio = NULL;
- }
-
- hdmi->hpd_regs = devm_kcalloc(&pdev->dev,
- config->hpd_reg_cnt,
- sizeof(hdmi->hpd_regs[0]),
- GFP_KERNEL);
- if (!hdmi->hpd_regs) {
- ret = -ENOMEM;
- goto fail;
- }
- for (i = 0; i < config->hpd_reg_cnt; i++)
- hdmi->hpd_regs[i].supply = config->hpd_reg_names[i];
-
- ret = devm_regulator_bulk_get(&pdev->dev, config->hpd_reg_cnt, hdmi->hpd_regs);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to get hpd regulator: %d\n", ret);
- goto fail;
- }
-
- hdmi->pwr_regs = devm_kcalloc(&pdev->dev,
- config->pwr_reg_cnt,
- sizeof(hdmi->pwr_regs[0]),
- GFP_KERNEL);
- if (!hdmi->pwr_regs) {
- ret = -ENOMEM;
- goto fail;
- }
-
- for (i = 0; i < config->pwr_reg_cnt; i++)
- hdmi->pwr_regs[i].supply = config->pwr_reg_names[i];
-
- ret = devm_regulator_bulk_get(&pdev->dev, config->pwr_reg_cnt, hdmi->pwr_regs);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to get pwr regulator: %d\n", ret);
- goto fail;
- }
-
- hdmi->hpd_clks = devm_kcalloc(&pdev->dev,
- config->hpd_clk_cnt,
- sizeof(hdmi->hpd_clks[0]),
- GFP_KERNEL);
- if (!hdmi->hpd_clks) {
- ret = -ENOMEM;
- goto fail;
- }
- for (i = 0; i < config->hpd_clk_cnt; i++) {
- struct clk *clk;
-
- clk = msm_clk_get(pdev, config->hpd_clk_names[i]);
- if (IS_ERR(clk)) {
- ret = PTR_ERR(clk);
- DRM_DEV_ERROR(&pdev->dev, "failed to get hpd clk: %s (%d)\n",
- config->hpd_clk_names[i], ret);
- goto fail;
- }
-
- hdmi->hpd_clks[i] = clk;
- }
-
- hdmi->pwr_clks = devm_kcalloc(&pdev->dev,
- config->pwr_clk_cnt,
- sizeof(hdmi->pwr_clks[0]),
- GFP_KERNEL);
- if (!hdmi->pwr_clks) {
- ret = -ENOMEM;
- goto fail;
- }
- for (i = 0; i < config->pwr_clk_cnt; i++) {
- struct clk *clk;
-
- clk = msm_clk_get(pdev, config->pwr_clk_names[i]);
- if (IS_ERR(clk)) {
- ret = PTR_ERR(clk);
- DRM_DEV_ERROR(&pdev->dev, "failed to get pwr clk: %s (%d)\n",
- config->pwr_clk_names[i], ret);
- goto fail;
- }
-
- hdmi->pwr_clks[i] = clk;
- }
-
- hdmi->hpd_gpiod = devm_gpiod_get_optional(&pdev->dev, "hpd", GPIOD_IN);
- /* This will catch e.g. -EPROBE_DEFER */
- if (IS_ERR(hdmi->hpd_gpiod)) {
- ret = PTR_ERR(hdmi->hpd_gpiod);
- DRM_DEV_ERROR(&pdev->dev, "failed to get hpd gpio: (%d)\n", ret);
- goto fail;
- }
-
- if (!hdmi->hpd_gpiod)
- DBG("failed to get HPD gpio");
-
- if (hdmi->hpd_gpiod)
- gpiod_set_consumer_name(hdmi->hpd_gpiod, "HDMI_HPD");
-
- pm_runtime_enable(&pdev->dev);
+ struct platform_device *pdev = hdmi->pdev;
+ int ret;
hdmi->workq = alloc_ordered_workqueue("msm_hdmi", 0);
@@ -264,25 +129,18 @@ static struct hdmi *msm_hdmi_init(struct platform_device *pdev)
goto fail;
}
- ret = msm_hdmi_get_phy(hdmi);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to get phy\n");
- goto fail;
- }
-
hdmi->hdcp_ctrl = msm_hdmi_hdcp_init(hdmi);
if (IS_ERR(hdmi->hdcp_ctrl)) {
dev_warn(&pdev->dev, "failed to init hdcp: disabled\n");
hdmi->hdcp_ctrl = NULL;
}
- return hdmi;
+ return 0;
fail:
- if (hdmi)
- msm_hdmi_destroy(hdmi);
+ msm_hdmi_destroy(hdmi);
- return ERR_PTR(ret);
+ return ret;
}
/* Second part of initialization, the drm/kms level modeset_init,
@@ -297,7 +155,6 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi,
struct drm_device *dev, struct drm_encoder *encoder)
{
struct msm_drm_private *priv = dev->dev_private;
- struct platform_device *pdev = hdmi->pdev;
int ret;
if (priv->num_bridges == ARRAY_SIZE(priv->bridges)) {
@@ -337,13 +194,6 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi,
drm_connector_attach_encoder(hdmi->connector, hdmi->encoder);
- hdmi->irq = irq_of_parse_and_map(pdev->dev.of_node, 0);
- if (!hdmi->irq) {
- ret = -EINVAL;
- DRM_DEV_ERROR(dev->dev, "failed to get irq\n");
- goto fail;
- }
-
ret = devm_request_irq(dev->dev, hdmi->irq,
msm_hdmi_irq, IRQF_TRIGGER_HIGH,
"hdmi_isr", hdmi);
@@ -363,8 +213,6 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi,
priv->bridges[priv->num_bridges++] = hdmi->bridge;
- platform_set_drvdata(pdev, hdmi);
-
return 0;
fail:
@@ -392,7 +240,7 @@ int msm_hdmi_modeset_init(struct hdmi *hdmi,
static const char *hpd_reg_names_8960[] = {"core-vdda"};
static const char *hpd_clk_names_8960[] = {"core", "master_iface", "slave_iface"};
-static struct hdmi_platform_config hdmi_tx_8960_config = {
+static const struct hdmi_platform_config hdmi_tx_8960_config = {
HDMI_CFG(hpd_reg, 8960),
HDMI_CFG(hpd_clk, 8960),
};
@@ -402,7 +250,7 @@ static const char *pwr_clk_names_8x74[] = {"extp", "alt_iface"};
static const char *hpd_clk_names_8x74[] = {"iface", "core", "mdp_core"};
static unsigned long hpd_clk_freq_8x74[] = {0, 19200000, 0};
-static struct hdmi_platform_config hdmi_tx_8974_config = {
+static const struct hdmi_platform_config hdmi_tx_8974_config = {
HDMI_CFG(pwr_reg, 8x74),
HDMI_CFG(pwr_clk, 8x74),
HDMI_CFG(hpd_clk, 8x74),
@@ -517,26 +365,12 @@ static int msm_hdmi_register_audio_driver(struct hdmi *hdmi, struct device *dev)
static int msm_hdmi_bind(struct device *dev, struct device *master, void *data)
{
struct msm_drm_private *priv = dev_get_drvdata(master);
- struct hdmi_platform_config *hdmi_cfg;
- struct hdmi *hdmi;
- struct device_node *of_node = dev->of_node;
+ struct hdmi *hdmi = dev_get_drvdata(dev);
int err;
- hdmi_cfg = (struct hdmi_platform_config *)
- of_device_get_match_data(dev);
- if (!hdmi_cfg) {
- DRM_DEV_ERROR(dev, "unknown hdmi_cfg: %pOFn\n", of_node);
- return -ENXIO;
- }
-
- hdmi_cfg->mmio_name = "core_physical";
- hdmi_cfg->qfprom_mmio_name = "qfprom_physical";
-
- dev->platform_data = hdmi_cfg;
-
- hdmi = msm_hdmi_init(to_platform_device(dev));
- if (IS_ERR(hdmi))
- return PTR_ERR(hdmi);
+ err = msm_hdmi_init(hdmi);
+ if (err)
+ return err;
priv->hdmi = hdmi;
err = msm_hdmi_register_audio_driver(hdmi, dev);
@@ -569,12 +403,150 @@ static const struct component_ops msm_hdmi_ops = {
static int msm_hdmi_dev_probe(struct platform_device *pdev)
{
+ const struct hdmi_platform_config *config;
+ struct device *dev = &pdev->dev;
+ struct hdmi *hdmi;
+ struct resource *res;
+ int i, ret;
+
+ config = of_device_get_match_data(dev);
+ if (!config)
+ return -EINVAL;
+
+ hdmi = devm_kzalloc(&pdev->dev, sizeof(*hdmi), GFP_KERNEL);
+ if (!hdmi)
+ return -ENOMEM;
+
+ hdmi->pdev = pdev;
+ hdmi->config = config;
+ spin_lock_init(&hdmi->reg_lock);
+
+ ret = drm_of_find_panel_or_bridge(pdev->dev.of_node, 1, 0, NULL, &hdmi->next_bridge);
+ if (ret && ret != -ENODEV)
+ return ret;
+
+ hdmi->mmio = msm_ioremap(pdev, "core_physical");
+ if (IS_ERR(hdmi->mmio))
+ return PTR_ERR(hdmi->mmio);
+
+ /* HDCP needs physical address of hdmi register */
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ "core_physical");
+ if (!res)
+ return -EINVAL;
+ hdmi->mmio_phy_addr = res->start;
+
+ hdmi->qfprom_mmio = msm_ioremap(pdev, "qfprom_physical");
+ if (IS_ERR(hdmi->qfprom_mmio)) {
+ DRM_DEV_INFO(&pdev->dev, "can't find qfprom resource\n");
+ hdmi->qfprom_mmio = NULL;
+ }
+
+ hdmi->irq = platform_get_irq(pdev, 0);
+ if (hdmi->irq < 0)
+ return hdmi->irq;
+
+ hdmi->hpd_regs = devm_kcalloc(&pdev->dev,
+ config->hpd_reg_cnt,
+ sizeof(hdmi->hpd_regs[0]),
+ GFP_KERNEL);
+ if (!hdmi->hpd_regs)
+ return -ENOMEM;
+
+ for (i = 0; i < config->hpd_reg_cnt; i++)
+ hdmi->hpd_regs[i].supply = config->hpd_reg_names[i];
+
+ ret = devm_regulator_bulk_get(&pdev->dev, config->hpd_reg_cnt, hdmi->hpd_regs);
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to get hpd regulators\n");
+
+ hdmi->pwr_regs = devm_kcalloc(&pdev->dev,
+ config->pwr_reg_cnt,
+ sizeof(hdmi->pwr_regs[0]),
+ GFP_KERNEL);
+ if (!hdmi->pwr_regs)
+ return -ENOMEM;
+
+ for (i = 0; i < config->pwr_reg_cnt; i++)
+ hdmi->pwr_regs[i].supply = config->pwr_reg_names[i];
+
+ ret = devm_regulator_bulk_get(&pdev->dev, config->pwr_reg_cnt, hdmi->pwr_regs);
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to get pwr regulators\n");
+
+ hdmi->hpd_clks = devm_kcalloc(&pdev->dev,
+ config->hpd_clk_cnt,
+ sizeof(hdmi->hpd_clks[0]),
+ GFP_KERNEL);
+ if (!hdmi->hpd_clks)
+ return -ENOMEM;
+
+ for (i = 0; i < config->hpd_clk_cnt; i++) {
+ struct clk *clk;
+
+ clk = msm_clk_get(pdev, config->hpd_clk_names[i]);
+ if (IS_ERR(clk))
+ return dev_err_probe(dev, PTR_ERR(clk),
+ "failed to get hpd clk: %s\n",
+ config->hpd_clk_names[i]);
+
+ hdmi->hpd_clks[i] = clk;
+ }
+
+ hdmi->pwr_clks = devm_kcalloc(&pdev->dev,
+ config->pwr_clk_cnt,
+ sizeof(hdmi->pwr_clks[0]),
+ GFP_KERNEL);
+ if (!hdmi->pwr_clks)
+ return -ENOMEM;
+
+ for (i = 0; i < config->pwr_clk_cnt; i++) {
+ struct clk *clk;
+
+ clk = msm_clk_get(pdev, config->pwr_clk_names[i]);
+ if (IS_ERR(clk))
+ return dev_err_probe(dev, PTR_ERR(clk),
+ "failed to get pwr clk: %s\n",
+ config->pwr_clk_names[i]);
+
+ hdmi->pwr_clks[i] = clk;
+ }
+
+ hdmi->hpd_gpiod = devm_gpiod_get_optional(&pdev->dev, "hpd", GPIOD_IN);
+ /* This will catch e.g. -EPROBE_DEFER */
+ if (IS_ERR(hdmi->hpd_gpiod))
+ return dev_err_probe(dev, PTR_ERR(hdmi->hpd_gpiod),
+ "failed to get hpd gpio\n");
+
+ if (!hdmi->hpd_gpiod)
+ DBG("failed to get HPD gpio");
+
+ if (hdmi->hpd_gpiod)
+ gpiod_set_consumer_name(hdmi->hpd_gpiod, "HDMI_HPD");
+
+ ret = msm_hdmi_get_phy(hdmi);
+ if (ret) {
+ DRM_DEV_ERROR(&pdev->dev, "failed to get phy\n");
+ return ret;
+ }
+
+ ret = devm_pm_runtime_enable(&pdev->dev);
+ if (ret)
+ return ret;
+
+ platform_set_drvdata(pdev, hdmi);
+
return component_add(&pdev->dev, &msm_hdmi_ops);
}
static int msm_hdmi_dev_remove(struct platform_device *pdev)
{
+ struct hdmi *hdmi = dev_get_drvdata(&pdev->dev);
+
component_del(&pdev->dev, &msm_hdmi_ops);
+
+ msm_hdmi_put_phy(hdmi);
+
return 0;
}
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi.h b/drivers/gpu/drm/msm/hdmi/hdmi.h
index 04a7438..e8dbee5 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi.h
+++ b/drivers/gpu/drm/msm/hdmi/hdmi.h
@@ -86,9 +86,6 @@ struct hdmi {
/* platform config data (ie. from DT, or pdata) */
struct hdmi_platform_config {
- const char *mmio_name;
- const char *qfprom_mmio_name;
-
/* regulators that need to be on for hpd: */
const char **hpd_reg_names;
int hpd_reg_cnt;
diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
index 105b5b4..8b0b0ac 100644
--- a/drivers/gpu/drm/msm/msm_drv.c
+++ b/drivers/gpu/drm/msm/msm_drv.c
@@ -277,7 +277,6 @@ static int msm_drm_uninit(struct device *dev)
struct msm_gem_address_space *msm_kms_init_aspace(struct drm_device *dev)
{
- struct iommu_domain *domain;
struct msm_gem_address_space *aspace;
struct msm_mmu *mmu;
struct device *mdp_dev = dev->dev;
@@ -293,22 +292,21 @@ struct msm_gem_address_space *msm_kms_init_aspace(struct drm_device *dev)
else
iommu_dev = mdss_dev;
- domain = iommu_domain_alloc(iommu_dev->bus);
- if (!domain) {
+ mmu = msm_iommu_new(iommu_dev, 0);
+ if (IS_ERR(mmu))
+ return ERR_CAST(mmu);
+
+ if (!mmu) {
drm_info(dev, "no IOMMU, fallback to phys contig buffers for scanout\n");
return NULL;
}
- mmu = msm_iommu_new(iommu_dev, domain);
- if (IS_ERR(mmu)) {
- iommu_domain_free(domain);
- return ERR_CAST(mmu);
- }
-
aspace = msm_gem_address_space_create(mmu, "mdp_kms",
0x1000, 0x100000000 - 0x1000);
- if (IS_ERR(aspace))
+ if (IS_ERR(aspace)) {
+ dev_err(mdp_dev, "aspace create, error %pe\n", aspace);
mmu->funcs->destroy(mmu);
+ }
return aspace;
}
@@ -420,7 +418,6 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv)
priv->dev = ddev;
priv->wq = alloc_ordered_workqueue("msm", 0);
- priv->hangcheck_period = DRM_MSM_HANGCHECK_DEFAULT_PERIOD;
INIT_LIST_HEAD(&priv->objects);
mutex_init(&priv->obj_lock);
@@ -819,6 +816,7 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void *data,
case MSM_INFO_GET_OFFSET:
case MSM_INFO_GET_IOVA:
case MSM_INFO_SET_IOVA:
+ case MSM_INFO_GET_FLAGS:
/* value returned as immediate, not pointer, so len==0: */
if (args->len)
return -EINVAL;
@@ -846,6 +844,15 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void *data,
case MSM_INFO_SET_IOVA:
ret = msm_ioctl_gem_info_set_iova(dev, file, obj, args->value);
break;
+ case MSM_INFO_GET_FLAGS:
+ if (obj->import_attach) {
+ ret = -EINVAL;
+ break;
+ }
+ /* Hide internal kernel-only flags: */
+ args->value = to_msm_bo(obj)->flags & MSM_BO_FLAGS;
+ ret = 0;
+ break;
case MSM_INFO_SET_NAME:
/* length check should leave room for terminating null: */
if (args->len >= sizeof(msm_obj->name)) {
diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h
index b2ea262..d4e0ef6 100644
--- a/drivers/gpu/drm/msm/msm_drv.h
+++ b/drivers/gpu/drm/msm/msm_drv.h
@@ -224,7 +224,13 @@ struct msm_drm_private {
struct drm_atomic_state *pm_state;
- /* For hang detection, in ms */
+ /**
+ * hangcheck_period: For hang detection, in ms
+ *
+ * Note that in practice, a submit/job will get at least two hangcheck
+ * periods, due to checking for progress being implemented as simply
+ * "have the CP position registers changed since last time?"
+ */
unsigned int hangcheck_period;
/**
diff --git a/drivers/gpu/drm/msm/msm_fbdev.c b/drivers/gpu/drm/msm/msm_fbdev.c
index 46168ec..31e1e30 100644
--- a/drivers/gpu/drm/msm/msm_fbdev.c
+++ b/drivers/gpu/drm/msm/msm_fbdev.c
@@ -93,7 +93,7 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
goto fail;
}
- fbi = drm_fb_helper_alloc_fbi(helper);
+ fbi = drm_fb_helper_alloc_info(helper);
if (IS_ERR(fbi)) {
DRM_DEV_ERROR(dev->dev, "failed to allocate fb info\n");
ret = PTR_ERR(fbi);
@@ -109,8 +109,6 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
drm_fb_helper_fill_info(fbi, helper, sizes);
- dev->mode_config.fb_base = paddr;
-
fbi->screen_base = msm_gem_get_vaddr(bo);
if (IS_ERR(fbi->screen_base)) {
ret = PTR_ERR(fbi->screen_base);
@@ -184,7 +182,7 @@ void msm_fbdev_free(struct drm_device *dev)
DBG();
- drm_fb_helper_unregister_fbi(helper);
+ drm_fb_helper_unregister_info(helper);
drm_fb_helper_fini(helper);
diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c
index 1de14e6..051bdbc 100644
--- a/drivers/gpu/drm/msm/msm_gem_shrinker.c
+++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c
@@ -15,7 +15,7 @@
/* Default disabled for now until it has some more testing on the different
* iommu combinations that can be paired with the driver:
*/
-static bool enable_eviction = false;
+static bool enable_eviction = true;
MODULE_PARM_DESC(enable_eviction, "Enable swappable GEM buffers");
module_param(enable_eviction, bool, 0600);
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 45a3e5c..73a2ca1 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -334,8 +334,7 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
if (ret)
return ret;
- /* exclusive fences must be ordered */
- if (no_implicit && !write)
+ if (no_implicit)
continue;
ret = drm_sched_job_add_implicit_dependencies(&submit->base,
diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index 021f4e2..30ed45a 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -492,6 +492,21 @@ static void hangcheck_timer_reset(struct msm_gpu *gpu)
round_jiffies_up(jiffies + msecs_to_jiffies(priv->hangcheck_period)));
}
+static bool made_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
+{
+ if (ring->hangcheck_progress_retries >= DRM_MSM_HANGCHECK_PROGRESS_RETRIES)
+ return false;
+
+ if (!gpu->funcs->progress)
+ return false;
+
+ if (!gpu->funcs->progress(gpu, ring))
+ return false;
+
+ ring->hangcheck_progress_retries++;
+ return true;
+}
+
static void hangcheck_handler(struct timer_list *t)
{
struct msm_gpu *gpu = from_timer(gpu, t, hangcheck_timer);
@@ -502,9 +517,12 @@ static void hangcheck_handler(struct timer_list *t)
if (fence != ring->hangcheck_fence) {
/* some progress has been made.. ya! */
ring->hangcheck_fence = fence;
- } else if (fence_before(fence, ring->fctx->last_fence)) {
+ ring->hangcheck_progress_retries = 0;
+ } else if (fence_before(fence, ring->fctx->last_fence) &&
+ !made_progress(gpu, ring)) {
/* no progress and not done.. hung! */
ring->hangcheck_fence = fence;
+ ring->hangcheck_progress_retries = 0;
DRM_DEV_ERROR(dev->dev, "%s: hangcheck detected gpu lockup rb %d!\n",
gpu->name, ring->id);
DRM_DEV_ERROR(dev->dev, "%s: completed fence: %u\n",
@@ -830,6 +848,7 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
struct msm_gpu *gpu, const struct msm_gpu_funcs *funcs,
const char *name, struct msm_gpu_config *config)
{
+ struct msm_drm_private *priv = drm->dev_private;
int i, ret, nr_rings = config->nr_rings;
void *memptrs;
uint64_t memptrs_iova;
@@ -857,6 +876,16 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
kthread_init_work(&gpu->recover_work, recover_worker);
kthread_init_work(&gpu->fault_work, fault_worker);
+ priv->hangcheck_period = DRM_MSM_HANGCHECK_DEFAULT_PERIOD;
+
+ /*
+ * If progress detection is supported, halve the hangcheck timer
+ * duration, as it takes two iterations of the hangcheck handler
+ * to detect a hang.
+ */
+ if (funcs->progress)
+ priv->hangcheck_period /= 2;
+
timer_setup(&gpu->hangcheck_timer, hangcheck_handler, 0);
spin_lock_init(&gpu->perf_lock);
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index 58a72e6..651786b 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -78,6 +78,15 @@ struct msm_gpu_funcs {
struct msm_gem_address_space *(*create_private_address_space)
(struct msm_gpu *gpu);
uint32_t (*get_rptr)(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
+
+ /**
+ * progress: Has the GPU made progress?
+ *
+ * Return true if GPU position in cmdstream has advanced (or changed)
+ * since the last call. To avoid false negatives, this should account
+ * for cmdstream that is buffered in this FIFO upstream of the CP fw.
+ */
+ bool (*progress)(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
};
/* Additional state for iommu faults: */
@@ -237,6 +246,7 @@ struct msm_gpu {
#define DRM_MSM_INACTIVE_PERIOD 66 /* in ms (roughly four frames) */
#define DRM_MSM_HANGCHECK_DEFAULT_PERIOD 500 /* in ms */
+#define DRM_MSM_HANGCHECK_PROGRESS_RETRIES 3
struct timer_list hangcheck_timer;
/* Fault info for most recent iova fault: */
@@ -540,7 +550,7 @@ static inline void gpu_rmw(struct msm_gpu *gpu, u32 reg, u32 mask, u32 or)
msm_rmw(gpu->mmio + (reg << 2), mask, or);
}
-static inline u64 gpu_read64(struct msm_gpu *gpu, u32 lo, u32 hi)
+static inline u64 gpu_read64(struct msm_gpu *gpu, u32 reg)
{
u64 val;
@@ -558,17 +568,17 @@ static inline u64 gpu_read64(struct msm_gpu *gpu, u32 lo, u32 hi)
* when the lo is read, so make sure to read the lo first to trigger
* that
*/
- val = (u64) msm_readl(gpu->mmio + (lo << 2));
- val |= ((u64) msm_readl(gpu->mmio + (hi << 2)) << 32);
+ val = (u64) msm_readl(gpu->mmio + (reg << 2));
+ val |= ((u64) msm_readl(gpu->mmio + ((reg + 1) << 2)) << 32);
return val;
}
-static inline void gpu_write64(struct msm_gpu *gpu, u32 lo, u32 hi, u64 val)
+static inline void gpu_write64(struct msm_gpu *gpu, u32 reg, u64 val)
{
/* Why not a writeq here? Read the screed above */
- msm_writel(lower_32_bits(val), gpu->mmio + (lo << 2));
- msm_writel(upper_32_bits(val), gpu->mmio + (hi << 2));
+ msm_writel(lower_32_bits(val), gpu->mmio + (reg << 2));
+ msm_writel(upper_32_bits(val), gpu->mmio + ((reg + 1) << 2));
}
int msm_gpu_pm_suspend(struct msm_gpu *gpu);
diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c
index 5577cea..c250758 100644
--- a/drivers/gpu/drm/msm/msm_iommu.c
+++ b/drivers/gpu/drm/msm/msm_iommu.c
@@ -186,6 +186,13 @@ int msm_iommu_pagetable_params(struct msm_mmu *mmu,
return 0;
}
+struct iommu_domain_geometry *msm_iommu_get_geometry(struct msm_mmu *mmu)
+{
+ struct msm_iommu *iommu = to_msm_iommu(mmu);
+
+ return &iommu->domain->geometry;
+}
+
static const struct msm_mmu_funcs pagetable_funcs = {
.map = msm_iommu_pagetable_map,
.unmap = msm_iommu_pagetable_unmap,
@@ -367,17 +374,23 @@ static const struct msm_mmu_funcs funcs = {
.resume_translation = msm_iommu_resume_translation,
};
-struct msm_mmu *msm_iommu_new(struct device *dev, struct iommu_domain *domain)
+struct msm_mmu *msm_iommu_new(struct device *dev, unsigned long quirks)
{
+ struct iommu_domain *domain;
struct msm_iommu *iommu;
int ret;
+ domain = iommu_domain_alloc(dev->bus);
if (!domain)
- return ERR_PTR(-ENODEV);
+ return NULL;
+
+ iommu_set_pgtable_quirks(domain, quirks);
iommu = kzalloc(sizeof(*iommu), GFP_KERNEL);
- if (!iommu)
+ if (!iommu) {
+ iommu_domain_free(domain);
return ERR_PTR(-ENOMEM);
+ }
iommu->domain = domain;
msm_mmu_init(&iommu->base, dev, &funcs, MSM_MMU_IOMMU);
@@ -386,6 +399,7 @@ struct msm_mmu *msm_iommu_new(struct device *dev, struct iommu_domain *domain)
ret = iommu_attach_device(iommu->domain, dev);
if (ret) {
+ iommu_domain_free(domain);
kfree(iommu);
return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/msm/msm_mdss.c b/drivers/gpu/drm/msm/msm_mdss.c
index e13c5c1..86b28ad 100644
--- a/drivers/gpu/drm/msm/msm_mdss.c
+++ b/drivers/gpu/drm/msm/msm_mdss.c
@@ -22,6 +22,7 @@
#define HW_REV 0x0
#define HW_INTR_STATUS 0x0010
+#define UBWC_DEC_HW_VERSION 0x58
#define UBWC_STATIC 0x144
#define UBWC_CTRL_2 0x150
#define UBWC_PREDICTION_MODE 0x154
@@ -174,9 +175,63 @@ static int _msm_mdss_irq_domain_add(struct msm_mdss *msm_mdss)
return 0;
}
+#define UBWC_1_0 0x10000000
+#define UBWC_2_0 0x20000000
+#define UBWC_3_0 0x30000000
+#define UBWC_4_0 0x40000000
+
+static void msm_mdss_setup_ubwc_dec_20(struct msm_mdss *msm_mdss,
+ u32 ubwc_static)
+{
+ writel_relaxed(ubwc_static, msm_mdss->mmio + UBWC_STATIC);
+}
+
+static void msm_mdss_setup_ubwc_dec_30(struct msm_mdss *msm_mdss,
+ unsigned int ubwc_version,
+ u32 ubwc_swizzle,
+ u32 highest_bank_bit,
+ u32 macrotile_mode)
+{
+ u32 value = (ubwc_swizzle & 0x1) |
+ (highest_bank_bit & 0x3) << 4 |
+ (macrotile_mode & 0x1) << 12;
+
+ if (ubwc_version == UBWC_3_0)
+ value |= BIT(10);
+
+ if (ubwc_version == UBWC_1_0)
+ value |= BIT(8);
+
+ writel_relaxed(value, msm_mdss->mmio + UBWC_STATIC);
+}
+
+static void msm_mdss_setup_ubwc_dec_40(struct msm_mdss *msm_mdss,
+ unsigned int ubwc_version,
+ u32 ubwc_swizzle,
+ u32 ubwc_static,
+ u32 highest_bank_bit,
+ u32 macrotile_mode)
+{
+ u32 value = (ubwc_swizzle & 0x7) |
+ (ubwc_static & 0x1) << 3 |
+ (highest_bank_bit & 0x7) << 4 |
+ (macrotile_mode & 0x1) << 12;
+
+ writel_relaxed(value, msm_mdss->mmio + UBWC_STATIC);
+
+ if (ubwc_version == UBWC_3_0) {
+ writel_relaxed(1, msm_mdss->mmio + UBWC_CTRL_2);
+ writel_relaxed(0, msm_mdss->mmio + UBWC_PREDICTION_MODE);
+ } else {
+ writel_relaxed(2, msm_mdss->mmio + UBWC_CTRL_2);
+ writel_relaxed(1, msm_mdss->mmio + UBWC_PREDICTION_MODE);
+ }
+}
+
static int msm_mdss_enable(struct msm_mdss *msm_mdss)
{
int ret;
+ u32 hw_rev;
/*
* Several components have AXI clocks that can only be turned on if
@@ -198,26 +253,39 @@ static int msm_mdss_enable(struct msm_mdss *msm_mdss)
if (msm_mdss->is_mdp5)
return 0;
+ hw_rev = readl_relaxed(msm_mdss->mmio + HW_REV);
+ dev_dbg(msm_mdss->dev, "HW_REV: 0x%x\n", hw_rev);
+ dev_dbg(msm_mdss->dev, "UBWC_DEC_HW_VERSION: 0x%x\n",
+ readl_relaxed(msm_mdss->mmio + UBWC_DEC_HW_VERSION));
+
/*
* ubwc config is part of the "mdss" region which is not accessible
* from the rest of the driver. hardcode known configurations here
+ *
+ * Decoder version can be read from the UBWC_DEC_HW_VERSION reg,
+ * UBWC_n and the rest of params comes from hw_catalog.
+ * Unforunately this driver can not access hw catalog, so we have to
+ * hardcode them here.
*/
- switch (readl_relaxed(msm_mdss->mmio + HW_REV)) {
+ switch (hw_rev) {
case DPU_HW_VER_500:
case DPU_HW_VER_501:
- writel_relaxed(0x420, msm_mdss->mmio + UBWC_STATIC);
+ msm_mdss_setup_ubwc_dec_30(msm_mdss, UBWC_3_0, 0, 2, 0);
break;
case DPU_HW_VER_600:
- /* TODO: 0x102e for LP_DDR4 */
- writel_relaxed(0x103e, msm_mdss->mmio + UBWC_STATIC);
- writel_relaxed(2, msm_mdss->mmio + UBWC_CTRL_2);
- writel_relaxed(1, msm_mdss->mmio + UBWC_PREDICTION_MODE);
+ /* TODO: highest_bank_bit = 2 for LP_DDR4 */
+ msm_mdss_setup_ubwc_dec_40(msm_mdss, UBWC_4_0, 6, 1, 3, 1);
break;
case DPU_HW_VER_620:
- writel_relaxed(0x1e, msm_mdss->mmio + UBWC_STATIC);
+ /* UBWC_2_0 */
+ msm_mdss_setup_ubwc_dec_20(msm_mdss, 0x1e);
+ break;
+ case DPU_HW_VER_630:
+ /* UBWC_2_0 */
+ msm_mdss_setup_ubwc_dec_20(msm_mdss, 0x11f);
break;
case DPU_HW_VER_720:
- writel_relaxed(0x101e, msm_mdss->mmio + UBWC_STATIC);
+ msm_mdss_setup_ubwc_dec_40(msm_mdss, UBWC_3_0, 6, 1, 1, 1);
break;
}
@@ -445,6 +513,7 @@ static const struct of_device_id mdss_dt_match[] = {
{ .compatible = "qcom,sc7180-mdss" },
{ .compatible = "qcom,sc7280-mdss" },
{ .compatible = "qcom,sc8180x-mdss" },
+ { .compatible = "qcom,sm6115-mdss" },
{ .compatible = "qcom,sm8150-mdss" },
{ .compatible = "qcom,sm8250-mdss" },
{}
diff --git a/drivers/gpu/drm/msm/msm_mmu.h b/drivers/gpu/drm/msm/msm_mmu.h
index de158e1..74cd81e 100644
--- a/drivers/gpu/drm/msm/msm_mmu.h
+++ b/drivers/gpu/drm/msm/msm_mmu.h
@@ -40,7 +40,7 @@ static inline void msm_mmu_init(struct msm_mmu *mmu, struct device *dev,
mmu->type = type;
}
-struct msm_mmu *msm_iommu_new(struct device *dev, struct iommu_domain *domain);
+struct msm_mmu *msm_iommu_new(struct device *dev, unsigned long quirks);
struct msm_mmu *msm_gpummu_new(struct device *dev, struct msm_gpu *gpu);
static inline void msm_mmu_set_fault_handler(struct msm_mmu *mmu, void *arg,
@@ -58,5 +58,6 @@ void msm_gpummu_params(struct msm_mmu *mmu, dma_addr_t *pt_base,
int msm_iommu_pagetable_params(struct msm_mmu *mmu, phys_addr_t *ttbr,
int *asid);
+struct iommu_domain_geometry *msm_iommu_get_geometry(struct msm_mmu *mmu);
#endif /* __MSM_MMU_H__ */
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h b/drivers/gpu/drm/msm/msm_ringbuffer.h
index 2a5045a..698b333 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.h
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.h
@@ -35,6 +35,11 @@ struct msm_rbmemptrs {
volatile u64 ttbr0;
};
+struct msm_cp_state {
+ uint64_t ib1_base, ib2_base;
+ uint32_t ib1_rem, ib2_rem;
+};
+
struct msm_ringbuffer {
struct msm_gpu *gpu;
int id;
@@ -64,6 +69,29 @@ struct msm_ringbuffer {
uint64_t memptrs_iova;
struct msm_fence_context *fctx;
+ /**
+ * hangcheck_progress_retries:
+ *
+ * The number of extra hangcheck duration cycles that we have given
+ * due to it appearing that the GPU is making forward progress.
+ *
+ * For GPU generations which support progress detection (see.
+ * msm_gpu_funcs::progress()), if the GPU appears to be making progress
+ * (ie. the CP has advanced in the command stream, we'll allow up to
+ * DRM_MSM_HANGCHECK_PROGRESS_RETRIES expirations of the hangcheck timer
+ * before killing the job. But to detect progress we need two sample
+ * points, so the duration of the hangcheck timer is halved. In other
+ * words we'll let the submit run for up to:
+ *
+ * (DRM_MSM_HANGCHECK_DEFAULT_PERIOD / 2) * (DRM_MSM_HANGCHECK_PROGRESS_RETRIES + 1)
+ */
+ int hangcheck_progress_retries;
+
+ /**
+ * last_cp_state: The state of the CP at the last call to gpu->progress()
+ */
+ struct msm_cp_state last_cp_state;
+
/*
* preempt_lock protects preemption and serializes wptr updates against
* preemption. Can be aquired from irq context.
diff --git a/drivers/gpu/drm/mxsfb/lcdif_drv.c b/drivers/gpu/drm/mxsfb/lcdif_drv.c
index 075002e..cc2ceb3 100644
--- a/drivers/gpu/drm/mxsfb/lcdif_drv.c
+++ b/drivers/gpu/drm/mxsfb/lcdif_drv.c
@@ -16,7 +16,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_mode_config.h>
diff --git a/drivers/gpu/drm/mxsfb/lcdif_kms.c b/drivers/gpu/drm/mxsfb/lcdif_kms.c
index b1092aa..262bc43 100644
--- a/drivers/gpu/drm/mxsfb/lcdif_kms.c
+++ b/drivers/gpu/drm/mxsfb/lcdif_kms.c
@@ -5,6 +5,7 @@
* This code is based on drivers/gpu/drm/mxsfb/mxsfb*
*/
+#include <linux/bitfield.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/iopoll.h>
@@ -15,6 +16,7 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
+#include <drm/drm_color_mgmt.h>
#include <drm/drm_crtc.h>
#include <drm/drm_encoder.h>
#include <drm/drm_fb_dma_helper.h>
@@ -31,13 +33,126 @@
/* -----------------------------------------------------------------------------
* CRTC
*/
+
+/*
+ * For conversion from YCbCr to RGB, the CSC operates as follows:
+ *
+ * |R| |A1 A2 A3| |Y + D1|
+ * |G| = |B1 B2 B3| * |Cb + D2|
+ * |B| |C1 C2 C3| |Cr + D3|
+ *
+ * The A, B and C coefficients are expressed as Q2.8 fixed point values, and
+ * the D coefficients as Q0.8. Despite the reference manual stating the
+ * opposite, the D1, D2 and D3 offset values are added to Y, Cb and Cr, not
+ * subtracted. They must thus be programmed with negative values.
+ */
+static const u32 lcdif_yuv2rgb_coeffs[3][2][6] = {
+ [DRM_COLOR_YCBCR_BT601] = {
+ [DRM_COLOR_YCBCR_LIMITED_RANGE] = {
+ /*
+ * BT.601 limited range:
+ *
+ * |R| |1.1644 0.0000 1.5960| |Y - 16 |
+ * |G| = |1.1644 -0.3917 -0.8129| * |Cb - 128|
+ * |B| |1.1644 2.0172 0.0000| |Cr - 128|
+ */
+ CSC0_COEF0_A1(0x12a) | CSC0_COEF0_A2(0x000),
+ CSC0_COEF1_A3(0x199) | CSC0_COEF1_B1(0x12a),
+ CSC0_COEF2_B2(0x79c) | CSC0_COEF2_B3(0x730),
+ CSC0_COEF3_C1(0x12a) | CSC0_COEF3_C2(0x204),
+ CSC0_COEF4_C3(0x000) | CSC0_COEF4_D1(0x1f0),
+ CSC0_COEF5_D2(0x180) | CSC0_COEF5_D3(0x180),
+ },
+ [DRM_COLOR_YCBCR_FULL_RANGE] = {
+ /*
+ * BT.601 full range:
+ *
+ * |R| |1.0000 0.0000 1.4020| |Y - 0 |
+ * |G| = |1.0000 -0.3441 -0.7141| * |Cb - 128|
+ * |B| |1.0000 1.7720 0.0000| |Cr - 128|
+ */
+ CSC0_COEF0_A1(0x100) | CSC0_COEF0_A2(0x000),
+ CSC0_COEF1_A3(0x167) | CSC0_COEF1_B1(0x100),
+ CSC0_COEF2_B2(0x7a8) | CSC0_COEF2_B3(0x749),
+ CSC0_COEF3_C1(0x100) | CSC0_COEF3_C2(0x1c6),
+ CSC0_COEF4_C3(0x000) | CSC0_COEF4_D1(0x000),
+ CSC0_COEF5_D2(0x180) | CSC0_COEF5_D3(0x180),
+ },
+ },
+ [DRM_COLOR_YCBCR_BT709] = {
+ [DRM_COLOR_YCBCR_LIMITED_RANGE] = {
+ /*
+ * Rec.709 limited range:
+ *
+ * |R| |1.1644 0.0000 1.7927| |Y - 16 |
+ * |G| = |1.1644 -0.2132 -0.5329| * |Cb - 128|
+ * |B| |1.1644 2.1124 0.0000| |Cr - 128|
+ */
+ CSC0_COEF0_A1(0x12a) | CSC0_COEF0_A2(0x000),
+ CSC0_COEF1_A3(0x1cb) | CSC0_COEF1_B1(0x12a),
+ CSC0_COEF2_B2(0x7c9) | CSC0_COEF2_B3(0x778),
+ CSC0_COEF3_C1(0x12a) | CSC0_COEF3_C2(0x21d),
+ CSC0_COEF4_C3(0x000) | CSC0_COEF4_D1(0x1f0),
+ CSC0_COEF5_D2(0x180) | CSC0_COEF5_D3(0x180),
+ },
+ [DRM_COLOR_YCBCR_FULL_RANGE] = {
+ /*
+ * Rec.709 full range:
+ *
+ * |R| |1.0000 0.0000 1.5748| |Y - 0 |
+ * |G| = |1.0000 -0.1873 -0.4681| * |Cb - 128|
+ * |B| |1.0000 1.8556 0.0000| |Cr - 128|
+ */
+ CSC0_COEF0_A1(0x100) | CSC0_COEF0_A2(0x000),
+ CSC0_COEF1_A3(0x193) | CSC0_COEF1_B1(0x100),
+ CSC0_COEF2_B2(0x7d0) | CSC0_COEF2_B3(0x788),
+ CSC0_COEF3_C1(0x100) | CSC0_COEF3_C2(0x1db),
+ CSC0_COEF4_C3(0x000) | CSC0_COEF4_D1(0x000),
+ CSC0_COEF5_D2(0x180) | CSC0_COEF5_D3(0x180),
+ },
+ },
+ [DRM_COLOR_YCBCR_BT2020] = {
+ [DRM_COLOR_YCBCR_LIMITED_RANGE] = {
+ /*
+ * BT.2020 limited range:
+ *
+ * |R| |1.1644 0.0000 1.6787| |Y - 16 |
+ * |G| = |1.1644 -0.1874 -0.6505| * |Cb - 128|
+ * |B| |1.1644 2.1418 0.0000| |Cr - 128|
+ */
+ CSC0_COEF0_A1(0x12a) | CSC0_COEF0_A2(0x000),
+ CSC0_COEF1_A3(0x1ae) | CSC0_COEF1_B1(0x12a),
+ CSC0_COEF2_B2(0x7d0) | CSC0_COEF2_B3(0x759),
+ CSC0_COEF3_C1(0x12a) | CSC0_COEF3_C2(0x224),
+ CSC0_COEF4_C3(0x000) | CSC0_COEF4_D1(0x1f0),
+ CSC0_COEF5_D2(0x180) | CSC0_COEF5_D3(0x180),
+ },
+ [DRM_COLOR_YCBCR_FULL_RANGE] = {
+ /*
+ * BT.2020 full range:
+ *
+ * |R| |1.0000 0.0000 1.4746| |Y - 0 |
+ * |G| = |1.0000 -0.1646 -0.5714| * |Cb - 128|
+ * |B| |1.0000 1.8814 0.0000| |Cr - 128|
+ */
+ CSC0_COEF0_A1(0x100) | CSC0_COEF0_A2(0x000),
+ CSC0_COEF1_A3(0x179) | CSC0_COEF1_B1(0x100),
+ CSC0_COEF2_B2(0x7d6) | CSC0_COEF2_B3(0x76e),
+ CSC0_COEF3_C1(0x100) | CSC0_COEF3_C2(0x1e2),
+ CSC0_COEF4_C3(0x000) | CSC0_COEF4_D1(0x000),
+ CSC0_COEF5_D2(0x180) | CSC0_COEF5_D3(0x180),
+ },
+ },
+};
+
static void lcdif_set_formats(struct lcdif_drm_private *lcdif,
+ struct drm_plane_state *plane_state,
const u32 bus_format)
{
struct drm_device *drm = lcdif->drm;
- const u32 format = lcdif->crtc.primary->state->fb->format->format;
-
- writel(CSC0_CTRL_BYPASS, lcdif->base + LCDC_V8_CSC0_CTRL);
+ const u32 format = plane_state->fb->format->format;
+ bool in_yuv = false;
+ bool out_yuv = false;
switch (bus_format) {
case MEDIA_BUS_FMT_RGB565_1X16:
@@ -51,24 +166,7 @@ static void lcdif_set_formats(struct lcdif_drm_private *lcdif,
case MEDIA_BUS_FMT_UYVY8_1X16:
writel(DISP_PARA_LINE_PATTERN_UYVY_H,
lcdif->base + LCDC_V8_DISP_PARA);
-
- /* CSC: BT.601 Full Range RGB to YCbCr coefficients. */
- writel(CSC0_COEF0_A2(0x096) | CSC0_COEF0_A1(0x04c),
- lcdif->base + LCDC_V8_CSC0_COEF0);
- writel(CSC0_COEF1_B1(0x7d5) | CSC0_COEF1_A3(0x01d),
- lcdif->base + LCDC_V8_CSC0_COEF1);
- writel(CSC0_COEF2_B3(0x080) | CSC0_COEF2_B2(0x7ac),
- lcdif->base + LCDC_V8_CSC0_COEF2);
- writel(CSC0_COEF3_C2(0x795) | CSC0_COEF3_C1(0x080),
- lcdif->base + LCDC_V8_CSC0_COEF3);
- writel(CSC0_COEF4_D1(0x000) | CSC0_COEF4_C3(0x7ec),
- lcdif->base + LCDC_V8_CSC0_COEF4);
- writel(CSC0_COEF5_D3(0x080) | CSC0_COEF5_D2(0x080),
- lcdif->base + LCDC_V8_CSC0_COEF5);
-
- writel(CSC0_CTRL_CSC_MODE_RGB2YCbCr,
- lcdif->base + LCDC_V8_CSC0_CTRL);
-
+ out_yuv = true;
break;
default:
dev_err(drm->dev, "Unknown media bus format 0x%x\n", bus_format);
@@ -76,6 +174,7 @@ static void lcdif_set_formats(struct lcdif_drm_private *lcdif,
}
switch (format) {
+ /* RGB Formats */
case DRM_FORMAT_RGB565:
writel(CTRLDESCL0_5_BPP_16_RGB565,
lcdif->base + LCDC_V8_CTRLDESCL0_5);
@@ -100,10 +199,84 @@ static void lcdif_set_formats(struct lcdif_drm_private *lcdif,
writel(CTRLDESCL0_5_BPP_32_ARGB8888,
lcdif->base + LCDC_V8_CTRLDESCL0_5);
break;
+
+ /* YUV Formats */
+ case DRM_FORMAT_YUYV:
+ writel(CTRLDESCL0_5_BPP_YCbCr422 | CTRLDESCL0_5_YUV_FORMAT_VY2UY1,
+ lcdif->base + LCDC_V8_CTRLDESCL0_5);
+ in_yuv = true;
+ break;
+ case DRM_FORMAT_YVYU:
+ writel(CTRLDESCL0_5_BPP_YCbCr422 | CTRLDESCL0_5_YUV_FORMAT_UY2VY1,
+ lcdif->base + LCDC_V8_CTRLDESCL0_5);
+ in_yuv = true;
+ break;
+ case DRM_FORMAT_UYVY:
+ writel(CTRLDESCL0_5_BPP_YCbCr422 | CTRLDESCL0_5_YUV_FORMAT_Y2VY1U,
+ lcdif->base + LCDC_V8_CTRLDESCL0_5);
+ in_yuv = true;
+ break;
+ case DRM_FORMAT_VYUY:
+ writel(CTRLDESCL0_5_BPP_YCbCr422 | CTRLDESCL0_5_YUV_FORMAT_Y2UY1V,
+ lcdif->base + LCDC_V8_CTRLDESCL0_5);
+ in_yuv = true;
+ break;
+
default:
dev_err(drm->dev, "Unknown pixel format 0x%x\n", format);
break;
}
+
+ /*
+ * The CSC differentiates between "YCbCr" and "YUV", but the reference
+ * manual doesn't detail how they differ. Experiments showed that the
+ * luminance value is unaffected, only the calculations involving chroma
+ * values differ. The YCbCr mode behaves as expected, with chroma values
+ * being offset by 128. The YUV mode isn't fully understood.
+ */
+ if (!in_yuv && out_yuv) {
+ /* RGB -> YCbCr */
+ writel(CSC0_CTRL_CSC_MODE_RGB2YCbCr,
+ lcdif->base + LCDC_V8_CSC0_CTRL);
+
+ /*
+ * CSC: BT.601 Limited Range RGB to YCbCr coefficients.
+ *
+ * |Y | | 0.2568 0.5041 0.0979| |R| |16 |
+ * |Cb| = |-0.1482 -0.2910 0.4392| * |G| + |128|
+ * |Cr| | 0.4392 0.4392 -0.3678| |B| |128|
+ */
+ writel(CSC0_COEF0_A2(0x081) | CSC0_COEF0_A1(0x041),
+ lcdif->base + LCDC_V8_CSC0_COEF0);
+ writel(CSC0_COEF1_B1(0x7db) | CSC0_COEF1_A3(0x019),
+ lcdif->base + LCDC_V8_CSC0_COEF1);
+ writel(CSC0_COEF2_B3(0x070) | CSC0_COEF2_B2(0x7b6),
+ lcdif->base + LCDC_V8_CSC0_COEF2);
+ writel(CSC0_COEF3_C2(0x7a2) | CSC0_COEF3_C1(0x070),
+ lcdif->base + LCDC_V8_CSC0_COEF3);
+ writel(CSC0_COEF4_D1(0x010) | CSC0_COEF4_C3(0x7ee),
+ lcdif->base + LCDC_V8_CSC0_COEF4);
+ writel(CSC0_COEF5_D3(0x080) | CSC0_COEF5_D2(0x080),
+ lcdif->base + LCDC_V8_CSC0_COEF5);
+ } else if (in_yuv && !out_yuv) {
+ /* YCbCr -> RGB */
+ const u32 *coeffs =
+ lcdif_yuv2rgb_coeffs[plane_state->color_encoding]
+ [plane_state->color_range];
+
+ writel(CSC0_CTRL_CSC_MODE_YCbCr2RGB,
+ lcdif->base + LCDC_V8_CSC0_CTRL);
+
+ writel(coeffs[0], lcdif->base + LCDC_V8_CSC0_COEF0);
+ writel(coeffs[1], lcdif->base + LCDC_V8_CSC0_COEF1);
+ writel(coeffs[2], lcdif->base + LCDC_V8_CSC0_COEF2);
+ writel(coeffs[3], lcdif->base + LCDC_V8_CSC0_COEF3);
+ writel(coeffs[4], lcdif->base + LCDC_V8_CSC0_COEF4);
+ writel(coeffs[5], lcdif->base + LCDC_V8_CSC0_COEF5);
+ } else {
+ /* RGB -> RGB, YCbCr -> YCbCr: bypass colorspace converter. */
+ writel(CSC0_CTRL_BYPASS, lcdif->base + LCDC_V8_CSC0_CTRL);
+ }
}
static void lcdif_set_mode(struct lcdif_drm_private *lcdif, u32 bus_flags)
@@ -142,14 +315,36 @@ static void lcdif_set_mode(struct lcdif_drm_private *lcdif, u32 bus_flags)
CTRLDESCL0_1_WIDTH(m->hdisplay),
lcdif->base + LCDC_V8_CTRLDESCL0_1);
- writel(CTRLDESCL0_3_PITCH(lcdif->crtc.primary->state->fb->pitches[0]),
- lcdif->base + LCDC_V8_CTRLDESCL0_3);
+ /*
+ * Undocumented P_SIZE and T_SIZE register but those written in the
+ * downstream kernel those registers control the AXI burst size. As of
+ * now there are two known values:
+ * 1 - 128Byte
+ * 2 - 256Byte
+ * Downstream set it to 256B burst size to improve the memory
+ * efficiency so set it here too.
+ */
+ ctrl = CTRLDESCL0_3_P_SIZE(2) | CTRLDESCL0_3_T_SIZE(2) |
+ CTRLDESCL0_3_PITCH(lcdif->crtc.primary->state->fb->pitches[0]);
+ writel(ctrl, lcdif->base + LCDC_V8_CTRLDESCL0_3);
}
static void lcdif_enable_controller(struct lcdif_drm_private *lcdif)
{
u32 reg;
+ /* Set FIFO Panic watermarks, low 1/3, high 2/3 . */
+ writel(FIELD_PREP(PANIC0_THRES_LOW_MASK, 1 * PANIC0_THRES_MAX / 3) |
+ FIELD_PREP(PANIC0_THRES_HIGH_MASK, 2 * PANIC0_THRES_MAX / 3),
+ lcdif->base + LCDC_V8_PANIC0_THRES);
+
+ /*
+ * Enable FIFO Panic, this does not generate interrupt, but
+ * boosts NoC priority based on FIFO Panic watermarks.
+ */
+ writel(INT_ENABLE_D1_PLANE_PANIC_EN,
+ lcdif->base + LCDC_V8_INT_ENABLE_D1);
+
reg = readl(lcdif->base + LCDC_V8_DISP_PARA);
reg |= DISP_PARA_DISP_ON;
writel(reg, lcdif->base + LCDC_V8_DISP_PARA);
@@ -177,6 +372,9 @@ static void lcdif_disable_controller(struct lcdif_drm_private *lcdif)
reg = readl(lcdif->base + LCDC_V8_DISP_PARA);
reg &= ~DISP_PARA_DISP_ON;
writel(reg, lcdif->base + LCDC_V8_DISP_PARA);
+
+ /* Disable FIFO Panic NoC priority booster. */
+ writel(0, lcdif->base + LCDC_V8_INT_ENABLE_D1);
}
static void lcdif_reset_block(struct lcdif_drm_private *lcdif)
@@ -188,6 +386,7 @@ static void lcdif_reset_block(struct lcdif_drm_private *lcdif)
}
static void lcdif_crtc_mode_set_nofb(struct lcdif_drm_private *lcdif,
+ struct drm_plane_state *plane_state,
struct drm_bridge_state *bridge_state,
const u32 bus_format)
{
@@ -210,7 +409,7 @@ static void lcdif_crtc_mode_set_nofb(struct lcdif_drm_private *lcdif,
/* Mandatory eLCDIF reset as per the Reference Manual */
lcdif_reset_block(lcdif);
- lcdif_set_formats(lcdif, bus_format);
+ lcdif_set_formats(lcdif, plane_state, bus_format);
lcdif_set_mode(lcdif, bus_flags);
}
@@ -293,7 +492,7 @@ static void lcdif_crtc_atomic_enable(struct drm_crtc *crtc,
pm_runtime_get_sync(drm->dev);
- lcdif_crtc_mode_set_nofb(lcdif, bridge_state, bus_format);
+ lcdif_crtc_mode_set_nofb(lcdif, new_pstate, bridge_state, bus_format);
/* Write cur_buf as well to avoid an initial corrupt frame */
paddr = drm_fb_dma_get_gem_addr(new_pstate->fb, new_pstate, 0);
@@ -437,12 +636,19 @@ static const struct drm_plane_funcs lcdif_plane_funcs = {
};
static const u32 lcdif_primary_plane_formats[] = {
+ /* RGB */
DRM_FORMAT_RGB565,
DRM_FORMAT_RGB888,
DRM_FORMAT_XBGR8888,
DRM_FORMAT_XRGB1555,
DRM_FORMAT_XRGB4444,
DRM_FORMAT_XRGB8888,
+
+ /* Packed YCbCr */
+ DRM_FORMAT_YUYV,
+ DRM_FORMAT_YVYU,
+ DRM_FORMAT_UYVY,
+ DRM_FORMAT_VYUY,
};
static const u64 lcdif_modifiers[] = {
@@ -456,6 +662,11 @@ static const u64 lcdif_modifiers[] = {
int lcdif_kms_init(struct lcdif_drm_private *lcdif)
{
+ const u32 supported_encodings = BIT(DRM_COLOR_YCBCR_BT601) |
+ BIT(DRM_COLOR_YCBCR_BT709) |
+ BIT(DRM_COLOR_YCBCR_BT2020);
+ const u32 supported_ranges = BIT(DRM_COLOR_YCBCR_LIMITED_RANGE) |
+ BIT(DRM_COLOR_YCBCR_FULL_RANGE);
struct drm_encoder *encoder = &lcdif->encoder;
struct drm_crtc *crtc = &lcdif->crtc;
int ret;
@@ -471,6 +682,14 @@ int lcdif_kms_init(struct lcdif_drm_private *lcdif)
if (ret)
return ret;
+ ret = drm_plane_create_color_properties(&lcdif->planes.primary,
+ supported_encodings,
+ supported_ranges,
+ DRM_COLOR_YCBCR_BT601,
+ DRM_COLOR_YCBCR_LIMITED_RANGE);
+ if (ret)
+ return ret;
+
drm_crtc_helper_add(crtc, &lcdif_crtc_helper_funcs);
ret = drm_crtc_init_with_planes(lcdif->drm, crtc,
&lcdif->planes.primary, NULL,
diff --git a/drivers/gpu/drm/mxsfb/lcdif_regs.h b/drivers/gpu/drm/mxsfb/lcdif_regs.h
index c702206..c55dfb2 100644
--- a/drivers/gpu/drm/mxsfb/lcdif_regs.h
+++ b/drivers/gpu/drm/mxsfb/lcdif_regs.h
@@ -130,7 +130,7 @@
#define CTRL_FETCH_START_OPTION_BPV BIT(9)
#define CTRL_FETCH_START_OPTION_RESV GENMASK(9, 8)
#define CTRL_FETCH_START_OPTION_MASK GENMASK(9, 8)
-#define CTRL_NEG BIT(4)
+#define CTRL_NEG BIT(4)
#define CTRL_INV_PXCK BIT(3)
#define CTRL_INV_DE BIT(2)
#define CTRL_INV_VS BIT(1)
@@ -138,9 +138,9 @@
#define DISP_PARA_DISP_ON BIT(31)
#define DISP_PARA_SWAP_EN BIT(30)
-#define DISP_PARA_LINE_PATTERN_UYVY_H (GENMASK(29, 28) | BIT(26))
-#define DISP_PARA_LINE_PATTERN_RGB565 GENMASK(28, 26)
-#define DISP_PARA_LINE_PATTERN_RGB888 0
+#define DISP_PARA_LINE_PATTERN_UYVY_H (0xd << 26)
+#define DISP_PARA_LINE_PATTERN_RGB565 (0x7 << 26)
+#define DISP_PARA_LINE_PATTERN_RGB888 (0x0 << 26)
#define DISP_PARA_LINE_PATTERN_MASK GENMASK(29, 26)
#define DISP_PARA_DISP_MODE_MASK GENMASK(25, 24)
#define DISP_PARA_BGND_R_MASK GENMASK(23, 16)
@@ -186,10 +186,14 @@
#define INT_ENABLE_D1_PLANE_PANIC_EN BIT(0)
#define CTRLDESCL0_1_HEIGHT(n) (((n) & 0xffff) << 16)
-#define CTRLDESCL0_1_HEIGHT_MASK GENMASK(31, 16)
+#define CTRLDESCL0_1_HEIGHT_MASK GENMASK(31, 16)
#define CTRLDESCL0_1_WIDTH(n) ((n) & 0xffff)
#define CTRLDESCL0_1_WIDTH_MASK GENMASK(15, 0)
+#define CTRLDESCL0_3_P_SIZE(n) (((n) << 20) & CTRLDESCL0_3_P_SIZE_MASK)
+#define CTRLDESCL0_3_P_SIZE_MASK GENMASK(22, 20)
+#define CTRLDESCL0_3_T_SIZE(n) (((n) << 16) & CTRLDESCL0_3_T_SIZE_MASK)
+#define CTRLDESCL0_3_T_SIZE_MASK GENMASK(17, 16)
#define CTRLDESCL0_3_PITCH(n) ((n) & 0xffff)
#define CTRLDESCL0_3_PITCH_MASK GENMASK(15, 0)
@@ -198,21 +202,24 @@
#define CTRLDESCL0_5_EN BIT(31)
#define CTRLDESCL0_5_SHADOW_LOAD_EN BIT(30)
-#define CTRLDESCL0_5_BPP_16_RGB565 BIT(26)
-#define CTRLDESCL0_5_BPP_16_ARGB1555 (BIT(26) | BIT(24))
-#define CTRLDESCL0_5_BPP_16_ARGB4444 (BIT(26) | BIT(25))
-#define CTRLDESCL0_5_BPP_YCbCr422 (BIT(26) | BIT(25) | BIT(24))
-#define CTRLDESCL0_5_BPP_24_RGB888 BIT(27)
-#define CTRLDESCL0_5_BPP_32_ARGB8888 (BIT(27) | BIT(24))
-#define CTRLDESCL0_5_BPP_32_ABGR8888 (BIT(27) | BIT(25))
+#define CTRLDESCL0_5_BPP_16_RGB565 (0x4 << 24)
+#define CTRLDESCL0_5_BPP_16_ARGB1555 (0x5 << 24)
+#define CTRLDESCL0_5_BPP_16_ARGB4444 (0x6 << 24)
+#define CTRLDESCL0_5_BPP_YCbCr422 (0x7 << 24)
+#define CTRLDESCL0_5_BPP_24_RGB888 (0x8 << 24)
+#define CTRLDESCL0_5_BPP_32_ARGB8888 (0x9 << 24)
+#define CTRLDESCL0_5_BPP_32_ABGR8888 (0xa << 24)
#define CTRLDESCL0_5_BPP_MASK GENMASK(27, 24)
-#define CTRLDESCL0_5_YUV_FORMAT_Y2VY1U 0
-#define CTRLDESCL0_5_YUV_FORMAT_Y2UY1V BIT(14)
-#define CTRLDESCL0_5_YUV_FORMAT_VY2UY1 BIT(15)
-#define CTRLDESCL0_5_YUV_FORMAT_UY2VY1 (BIT(15) | BIT(14))
+#define CTRLDESCL0_5_YUV_FORMAT_Y2VY1U (0x0 << 14)
+#define CTRLDESCL0_5_YUV_FORMAT_Y2UY1V (0x1 << 14)
+#define CTRLDESCL0_5_YUV_FORMAT_VY2UY1 (0x2 << 14)
+#define CTRLDESCL0_5_YUV_FORMAT_UY2VY1 (0x3 << 14)
#define CTRLDESCL0_5_YUV_FORMAT_MASK GENMASK(15, 14)
-#define CSC0_CTRL_CSC_MODE_RGB2YCbCr GENMASK(2, 1)
+#define CSC0_CTRL_CSC_MODE_YUV2RGB (0x0 << 1)
+#define CSC0_CTRL_CSC_MODE_YCbCr2RGB (0x1 << 1)
+#define CSC0_CTRL_CSC_MODE_RGB2YUV (0x2 << 1)
+#define CSC0_CTRL_CSC_MODE_RGB2YCbCr (0x3 << 1)
#define CSC0_CTRL_CSC_MODE_MASK GENMASK(2, 1)
#define CSC0_CTRL_BYPASS BIT(0)
@@ -248,6 +255,7 @@
#define PANIC0_THRES_LOW_MASK GENMASK(24, 16)
#define PANIC0_THRES_HIGH_MASK GENMASK(8, 0)
+#define PANIC0_THRES_MAX 511
#define LCDIF_MIN_XRES 120
#define LCDIF_MIN_YRES 120
diff --git a/drivers/gpu/drm/mxsfb/mxsfb_drv.c b/drivers/gpu/drm/mxsfb/mxsfb_drv.c
index b29b332..810edea 100644
--- a/drivers/gpu/drm/mxsfb/mxsfb_drv.c
+++ b/drivers/gpu/drm/mxsfb/mxsfb_drv.c
@@ -20,7 +20,7 @@
#include <drm/drm_bridge.h>
#include <drm/drm_connector.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
diff --git a/drivers/gpu/drm/nouveau/Kbuild b/drivers/gpu/drm/nouveau/Kbuild
index 60586fb..5e56170 100644
--- a/drivers/gpu/drm/nouveau/Kbuild
+++ b/drivers/gpu/drm/nouveau/Kbuild
@@ -54,10 +54,6 @@
nouveau-y += nouveau_connector.o
nouveau-y += nouveau_display.o
nouveau-y += nouveau_dp.o
-nouveau-y += nouveau_fbcon.o
-nouveau-y += nv04_fbcon.o
-nouveau-y += nv50_fbcon.o
-nouveau-y += nvc0_fbcon.o
include $(src)/dispnv04/Kbuild
include $(src)/dispnv50/Kbuild
diff --git a/drivers/gpu/drm/nouveau/dispnv04/crtc.c b/drivers/gpu/drm/nouveau/dispnv04/crtc.c
index ee92d57..0e0f117 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/crtc.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/crtc.c
@@ -23,6 +23,7 @@
* DEALINGS IN THE SOFTWARE.
*/
#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_plane_helper.h>
#include <drm/drm_vblank.h>
@@ -37,7 +38,6 @@
#include "nouveau_crtc.h"
#include "hw.h"
#include "nvreg.h"
-#include "nouveau_fbcon.h"
#include "disp.h"
#include "nouveau_dma.h"
@@ -761,7 +761,8 @@ static void nv_crtc_destroy(struct drm_crtc *crtc)
nouveau_bo_unmap(nv_crtc->cursor.nvbo);
nouveau_bo_unpin(nv_crtc->cursor.nvbo);
nouveau_bo_ref(NULL, &nv_crtc->cursor.nvbo);
- nvif_notify_dtor(&nv_crtc->vblank);
+ nvif_event_dtor(&nv_crtc->vblank);
+ nvif_head_dtor(&nv_crtc->head);
kfree(nv_crtc);
}
@@ -914,14 +915,6 @@ nv04_crtc_mode_set_base_atomic(struct drm_crtc *crtc,
struct drm_framebuffer *fb,
int x, int y, enum mode_set_atomic state)
{
- struct nouveau_drm *drm = nouveau_drm(crtc->dev);
- struct drm_device *dev = drm->dev;
-
- if (state == ENTER_ATOMIC_MODE_SET)
- nouveau_fbcon_accel_save_disable(dev);
- else
- nouveau_fbcon_accel_restore(dev);
-
return nv04_crtc_do_mode_set_base(crtc, fb, x, y, true);
}
@@ -1080,10 +1073,10 @@ nv04_finish_page_flip(struct nouveau_channel *chan,
}
int
-nv04_flip_complete(struct nvif_notify *notify)
+nv04_flip_complete(struct nvif_event *event, void *argv, u32 argc)
{
- struct nouveau_cli *cli = (void *)notify->object->client;
- struct nouveau_drm *drm = cli->drm;
+ struct nv04_display *disp = container_of(event, typeof(*disp), flip);
+ struct nouveau_drm *drm = disp->drm;
struct nouveau_channel *chan = drm->channel;
struct nv04_page_flip_state state;
@@ -1094,7 +1087,7 @@ nv04_flip_complete(struct nvif_notify *notify)
state.bpp / 8);
}
- return NVIF_NOTIFY_KEEP;
+ return NVIF_EVENT_KEEP;
}
static int
@@ -1279,13 +1272,13 @@ static const struct drm_plane_funcs nv04_primary_plane_funcs = {
DRM_PLANE_NON_ATOMIC_FUNCS,
};
-static int nv04_crtc_vblank_handler(struct nvif_notify *notify)
+static int
+nv04_crtc_vblank_handler(struct nvif_event *event, void *repv, u32 repc)
{
- struct nouveau_crtc *nv_crtc =
- container_of(notify, struct nouveau_crtc, vblank);
+ struct nouveau_crtc *nv_crtc = container_of(event, struct nouveau_crtc, vblank);
drm_crtc_handle_vblank(&nv_crtc->base);
- return NVIF_NOTIFY_KEEP;
+ return NVIF_EVENT_KEEP;
}
int
@@ -1341,14 +1334,10 @@ nv04_crtc_create(struct drm_device *dev, int crtc_num)
nv04_cursor_init(nv_crtc);
- ret = nvif_notify_ctor(&disp->disp.object, "kmsVbl", nv04_crtc_vblank_handler,
- false, NV04_DISP_NTFY_VBLANK,
- &(struct nvif_notify_head_req_v0) {
- .head = nv_crtc->index,
- },
- sizeof(struct nvif_notify_head_req_v0),
- sizeof(struct nvif_notify_head_rep_v0),
- &nv_crtc->vblank);
+ ret = nvif_head_ctor(&disp->disp, nv_crtc->base.name, nv_crtc->index, &nv_crtc->head);
+ if (ret)
+ return ret;
- return ret;
+ return nvif_head_vblank_event_ctor(&nv_crtc->head, "kmsVbl", nv04_crtc_vblank_handler,
+ false, &nv_crtc->vblank);
}
diff --git a/drivers/gpu/drm/nouveau/dispnv04/disp.c b/drivers/gpu/drm/nouveau/dispnv04/disp.c
index 99fee4d8..e9ac3fb 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/disp.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/disp.c
@@ -61,7 +61,7 @@ nv04_display_fini(struct drm_device *dev, bool runtime, bool suspend)
struct drm_crtc *crtc;
/* Disable flip completion events. */
- nvif_notify_put(&disp->flip);
+ nvif_event_block(&disp->flip);
/* Disable vblank interrupts. */
NVWriteCRTC(dev, 0, NV_PCRTC_INTR_EN_0, 0);
@@ -121,7 +121,7 @@ nv04_display_init(struct drm_device *dev, bool resume, bool runtime)
encoder->enc_save(&encoder->base.base);
/* Enable flip completion events. */
- nvif_notify_get(&disp->flip);
+ nvif_event_allow(&disp->flip);
if (!resume)
return 0;
@@ -202,7 +202,7 @@ nv04_display_destroy(struct drm_device *dev)
nouveau_hw_save_vga_fonts(dev, 0);
- nvif_notify_dtor(&disp->flip);
+ nvif_event_dtor(&disp->flip);
nouveau_display(dev)->priv = NULL;
vfree(disp);
@@ -227,6 +227,8 @@ nv04_display_create(struct drm_device *dev)
if (!disp)
return -ENOMEM;
+ disp->drm = drm;
+
nvif_object_map(&drm->client.device.object, NULL, 0);
nouveau_display(dev)->priv = disp;
@@ -239,9 +241,10 @@ nv04_display_create(struct drm_device *dev)
/* Request page flip completion event. */
if (drm->channel) {
- nvif_notify_ctor(&drm->channel->nvsw, "kmsFlip", nv04_flip_complete,
- false, NV04_NVSW_NTFY_UEVENT,
- NULL, 0, 0, &disp->flip);
+ ret = nvif_event_ctor(&drm->channel->nvsw, "kmsFlip", 0, nv04_flip_complete,
+ true, NULL, 0, &disp->flip);
+ if (ret)
+ return ret;
}
nouveau_hw_save_vga_fonts(dev, 1);
diff --git a/drivers/gpu/drm/nouveau/dispnv04/disp.h b/drivers/gpu/drm/nouveau/dispnv04/disp.h
index f0a2412..11a6663 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/disp.h
+++ b/drivers/gpu/drm/nouveau/dispnv04/disp.h
@@ -6,6 +6,8 @@
#include "nouveau_display.h"
+#include <nvif/event.h>
+
struct nouveau_encoder;
enum nv04_fp_display_regs {
@@ -84,7 +86,8 @@ struct nv04_display {
uint32_t saved_vga_font[4][16384];
uint32_t dac_users[4];
struct nouveau_bo *image[2];
- struct nvif_notify flip;
+ struct nvif_event flip;
+ struct nouveau_drm *drm;
};
static inline struct nv04_display *
@@ -179,5 +182,5 @@ nouveau_bios_run_init_table(struct drm_device *dev, u16 table,
);
}
-int nv04_flip_complete(struct nvif_notify *);
+int nv04_flip_complete(struct nvif_event *, void *, u32);
#endif
diff --git a/drivers/gpu/drm/nouveau/dispnv50/crc.c b/drivers/gpu/drm/nouveau/dispnv50/crc.c
index b834e8a..9c942fb 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/crc.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/crc.c
@@ -463,7 +463,7 @@ void nv50_crc_atomic_set(struct nv50_head *head,
if (!outp)
return;
- func->set_src(head, outp->or, nv50_crc_source_type(outp, asyh->crc.src),
+ func->set_src(head, outp->outp.or.id, nv50_crc_source_type(outp, asyh->crc.src),
&crc->ctx[crc->ctx_idx]);
}
diff --git a/drivers/gpu/drm/nouveau/dispnv50/disp.c b/drivers/gpu/drm/nouveau/dispnv50/disp.c
index 33c97d5..edcb252 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/disp.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/disp.c
@@ -46,8 +46,8 @@
#include <nvif/class.h>
#include <nvif/cl0002.h>
-#include <nvif/cl5070.h>
#include <nvif/event.h>
+#include <nvif/if0012.h>
#include <nvif/if0014.h>
#include <nvif/timer.h>
@@ -64,7 +64,6 @@
#include "nouveau_connector.h"
#include "nouveau_encoder.h"
#include "nouveau_fence.h"
-#include "nouveau_fbcon.h"
#include <subdev/bios/dp.h>
@@ -131,7 +130,7 @@ nv50_dmac_kick(struct nvif_push *push)
{
struct nv50_dmac *dmac = container_of(push, typeof(*dmac), _push);
- dmac->cur = push->cur - (u32 *)dmac->_push.mem.object.map.ptr;
+ dmac->cur = push->cur - (u32 __iomem *)dmac->_push.mem.object.map.ptr;
if (dmac->put != dmac->cur) {
/* Push buffer fetches are not coherent with BAR1, we need to ensure
* writes have been flushed right through to VRAM before writing PUT.
@@ -194,7 +193,7 @@ nv50_dmac_wait(struct nvif_push *push, u32 size)
if (WARN_ON(size > dmac->max))
return -EINVAL;
- dmac->cur = push->cur - (u32 *)dmac->_push.mem.object.map.ptr;
+ dmac->cur = push->cur - (u32 __iomem *)dmac->_push.mem.object.map.ptr;
if (dmac->cur + size >= dmac->max) {
int ret = nv50_dmac_wind(dmac);
if (ret)
@@ -317,52 +316,6 @@ nv50_outp_dump_caps(struct nouveau_drm *drm,
outp->base.base.name, outp->caps.dp_interlace);
}
-static void
-nv50_outp_release(struct nouveau_encoder *nv_encoder)
-{
- struct nv50_disp *disp = nv50_disp(nv_encoder->base.base.dev);
- struct {
- struct nv50_disp_mthd_v1 base;
- } args = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_RELEASE,
- .base.hasht = nv_encoder->dcb->hasht,
- .base.hashm = nv_encoder->dcb->hashm,
- };
-
- nvif_mthd(&disp->disp->object, 0, &args, sizeof(args));
- nv_encoder->or = -1;
- nv_encoder->link = 0;
-}
-
-static int
-nv50_outp_acquire(struct nouveau_encoder *nv_encoder, bool hda)
-{
- struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev);
- struct nv50_disp *disp = nv50_disp(drm->dev);
- struct {
- struct nv50_disp_mthd_v1 base;
- struct nv50_disp_acquire_v0 info;
- } args = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_ACQUIRE,
- .base.hasht = nv_encoder->dcb->hasht,
- .base.hashm = nv_encoder->dcb->hashm,
- .info.hda = hda,
- };
- int ret;
-
- ret = nvif_mthd(&disp->disp->object, 0, &args, sizeof(args));
- if (ret) {
- NV_ERROR(drm, "error acquiring output path: %d\n", ret);
- return ret;
- }
-
- nv_encoder->or = args.info.or;
- nv_encoder->link = args.info.link;
- return 0;
-}
-
static int
nv50_outp_atomic_check_view(struct drm_encoder *encoder,
struct drm_crtc_state *crtc_state,
@@ -489,9 +442,9 @@ nv50_dac_atomic_disable(struct drm_encoder *encoder, struct drm_atomic_state *st
struct nv50_core *core = nv50_disp(encoder->dev)->core;
const u32 ctrl = NVDEF(NV507D, DAC_SET_CONTROL, OWNER, NONE);
- core->func->dac->ctrl(core, nv_encoder->or, ctrl, NULL);
+ core->func->dac->ctrl(core, nv_encoder->outp.or.id, ctrl, NULL);
nv_encoder->crtc = NULL;
- nv50_outp_release(nv_encoder);
+ nvif_outp_release(&nv_encoder->outp);
}
static void
@@ -516,9 +469,9 @@ nv50_dac_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *sta
ctrl |= NVDEF(NV507D, DAC_SET_CONTROL, PROTOCOL, RGB_CRT);
- nv50_outp_acquire(nv_encoder, false);
+ nvif_outp_acquire_rgb_crt(&nv_encoder->outp);
- core->func->dac->ctrl(core, nv_encoder->or, ctrl, asyh);
+ core->func->dac->ctrl(core, nv_encoder->outp.or.id, ctrl, asyh);
asyh->or.depth = 0;
nv_encoder->crtc = &nv_crtc->base;
@@ -634,7 +587,7 @@ nv50_audio_component_get_eld(struct device *kdev, int port, int dev_id,
nv_connector = nouveau_connector(nv_encoder->audio.connector);
nv_crtc = nouveau_crtc(nv_encoder->crtc);
- if (!nv_crtc || nv_encoder->or != port || nv_crtc->index != dev_id)
+ if (!nv_crtc || nv_encoder->outp.or.id != port || nv_crtc->index != dev_id)
continue;
*enabled = nv_encoder->audio.enabled;
@@ -718,33 +671,37 @@ nv50_audio_component_fini(struct nouveau_drm *drm)
/******************************************************************************
* Audio
*****************************************************************************/
+static bool
+nv50_audio_supported(struct drm_encoder *encoder)
+{
+ struct nv50_disp *disp = nv50_disp(encoder->dev);
+
+ if (disp->disp->object.oclass <= GT200_DISP ||
+ disp->disp->object.oclass == GT206_DISP)
+ return false;
+
+ return true;
+}
+
static void
nv50_audio_disable(struct drm_encoder *encoder, struct nouveau_crtc *nv_crtc)
{
struct nouveau_drm *drm = nouveau_drm(encoder->dev);
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
- struct nv50_disp *disp = nv50_disp(encoder->dev);
- struct {
- struct nv50_disp_mthd_v1 base;
- struct nv50_disp_sor_hda_eld_v0 eld;
- } args = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_SOR_HDA_ELD,
- .base.hasht = nv_encoder->dcb->hasht,
- .base.hashm = (0xf0ff & nv_encoder->dcb->hashm) |
- (0x0100 << nv_crtc->index),
- };
+ struct nvif_outp *outp = &nv_encoder->outp;
+
+ if (!nv50_audio_supported(encoder))
+ return;
mutex_lock(&drm->audio.lock);
if (nv_encoder->audio.enabled) {
nv_encoder->audio.enabled = false;
nv_encoder->audio.connector = NULL;
- nvif_mthd(&disp->disp->object, 0, &args, sizeof(args));
+ nvif_outp_hda_eld(&nv_encoder->outp, nv_crtc->index, NULL, 0);
}
mutex_unlock(&drm->audio.lock);
- nv50_audio_component_eld_notify(drm->audio.component, nv_encoder->or,
- nv_crtc->index);
+ nv50_audio_component_eld_notify(drm->audio.component, outp->or.id, nv_crtc->index);
}
static void
@@ -754,159 +711,101 @@ nv50_audio_enable(struct drm_encoder *encoder, struct nouveau_crtc *nv_crtc,
{
struct nouveau_drm *drm = nouveau_drm(encoder->dev);
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
- struct nv50_disp *disp = nv50_disp(encoder->dev);
- struct __packed {
- struct {
- struct nv50_disp_mthd_v1 mthd;
- struct nv50_disp_sor_hda_eld_v0 eld;
- } base;
- u8 data[sizeof(nv_connector->base.eld)];
- } args = {
- .base.mthd.version = 1,
- .base.mthd.method = NV50_DISP_MTHD_V1_SOR_HDA_ELD,
- .base.mthd.hasht = nv_encoder->dcb->hasht,
- .base.mthd.hashm = (0xf0ff & nv_encoder->dcb->hashm) |
- (0x0100 << nv_crtc->index),
- };
+ struct nvif_outp *outp = &nv_encoder->outp;
- if (!drm_detect_monitor_audio(nv_connector->edid))
+ if (!nv50_audio_supported(encoder) || !drm_detect_monitor_audio(nv_connector->edid))
return;
mutex_lock(&drm->audio.lock);
- memcpy(args.data, nv_connector->base.eld, sizeof(args.data));
-
- nvif_mthd(&disp->disp->object, 0, &args,
- sizeof(args.base) + drm_eld_size(args.data));
+ nvif_outp_hda_eld(&nv_encoder->outp, nv_crtc->index, nv_connector->base.eld,
+ drm_eld_size(nv_connector->base.eld));
nv_encoder->audio.enabled = true;
nv_encoder->audio.connector = &nv_connector->base;
mutex_unlock(&drm->audio.lock);
- nv50_audio_component_eld_notify(drm->audio.component, nv_encoder->or,
- nv_crtc->index);
+ nv50_audio_component_eld_notify(drm->audio.component, outp->or.id, nv_crtc->index);
}
/******************************************************************************
* HDMI
*****************************************************************************/
static void
-nv50_hdmi_disable(struct drm_encoder *encoder, struct nouveau_crtc *nv_crtc)
-{
- struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
- struct nv50_disp *disp = nv50_disp(encoder->dev);
- struct {
- struct nv50_disp_mthd_v1 base;
- struct nv50_disp_sor_hdmi_pwr_v0 pwr;
- } args = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_SOR_HDMI_PWR,
- .base.hasht = nv_encoder->dcb->hasht,
- .base.hashm = (0xf0ff & nv_encoder->dcb->hashm) |
- (0x0100 << nv_crtc->index),
- };
-
- nvif_mthd(&disp->disp->object, 0, &args, sizeof(args));
-}
-
-static void
nv50_hdmi_enable(struct drm_encoder *encoder, struct nouveau_crtc *nv_crtc,
struct nouveau_connector *nv_connector, struct drm_atomic_state *state,
- struct drm_display_mode *mode)
+ struct drm_display_mode *mode, bool hda)
{
struct nouveau_drm *drm = nouveau_drm(encoder->dev);
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
- struct nv50_disp *disp = nv50_disp(encoder->dev);
- struct {
- struct nv50_disp_mthd_v1 base;
- struct nv50_disp_sor_hdmi_pwr_v0 pwr;
- u8 infoframes[2 * 17]; /* two frames, up to 17 bytes each */
- } args = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_SOR_HDMI_PWR,
- .base.hasht = nv_encoder->dcb->hasht,
- .base.hashm = (0xf0ff & nv_encoder->dcb->hashm) |
- (0x0100 << nv_crtc->index),
- .pwr.state = 1,
- .pwr.rekey = 56, /* binary driver, and tegra, constant */
- };
- struct drm_hdmi_info *hdmi;
+ struct drm_hdmi_info *hdmi = &nv_connector->base.display_info.hdmi;
+ union hdmi_infoframe infoframe = { 0 };
+ const u8 rekey = 56; /* binary driver, and tegra, constant */
+ u8 scdc = 0;
u32 max_ac_packet;
- union hdmi_infoframe avi_frame;
- union hdmi_infoframe vendor_frame;
- bool high_tmds_clock_ratio = false, scrambling = false;
- u8 config;
- int ret;
- int size;
-
- if (!drm_detect_hdmi_monitor(nv_connector->edid))
- return;
-
- hdmi = &nv_connector->base.display_info.hdmi;
-
- ret = drm_hdmi_avi_infoframe_from_display_mode(&avi_frame.avi,
- &nv_connector->base, mode);
- if (!ret) {
- drm_hdmi_avi_infoframe_quant_range(&avi_frame.avi,
- &nv_connector->base, mode,
- HDMI_QUANTIZATION_RANGE_FULL);
- /* We have an AVI InfoFrame, populate it to the display */
- args.pwr.avi_infoframe_length
- = hdmi_infoframe_pack(&avi_frame, args.infoframes, 17);
- }
-
- ret = drm_hdmi_vendor_infoframe_from_display_mode(&vendor_frame.vendor.hdmi,
- &nv_connector->base, mode);
- if (!ret) {
- /* We have a Vendor InfoFrame, populate it to the display */
- args.pwr.vendor_infoframe_length
- = hdmi_infoframe_pack(&vendor_frame,
- args.infoframes
- + args.pwr.avi_infoframe_length,
- 17);
- }
+ struct {
+ struct nvif_outp_infoframe_v0 infoframe;
+ u8 data[17];
+ } args = { 0 };
+ int ret, size;
max_ac_packet = mode->htotal - mode->hdisplay;
- max_ac_packet -= args.pwr.rekey;
+ max_ac_packet -= rekey;
max_ac_packet -= 18; /* constant from tegra */
- args.pwr.max_ac_packet = max_ac_packet / 32;
+ max_ac_packet /= 32;
if (hdmi->scdc.scrambling.supported) {
- high_tmds_clock_ratio = mode->clock > 340000;
- scrambling = high_tmds_clock_ratio ||
- hdmi->scdc.scrambling.low_rates;
+ const bool high_tmds_clock_ratio = mode->clock > 340000;
+
+ ret = drm_scdc_readb(nv_encoder->i2c, SCDC_TMDS_CONFIG, &scdc);
+ if (ret < 0) {
+ NV_ERROR(drm, "Failure to read SCDC_TMDS_CONFIG: %d\n", ret);
+ return;
+ }
+
+ scdc &= ~(SCDC_TMDS_BIT_CLOCK_RATIO_BY_40 | SCDC_SCRAMBLING_ENABLE);
+ if (high_tmds_clock_ratio || hdmi->scdc.scrambling.low_rates)
+ scdc |= SCDC_SCRAMBLING_ENABLE;
+ if (high_tmds_clock_ratio)
+ scdc |= SCDC_TMDS_BIT_CLOCK_RATIO_BY_40;
+
+ ret = drm_scdc_writeb(nv_encoder->i2c, SCDC_TMDS_CONFIG, scdc);
+ if (ret < 0)
+ NV_ERROR(drm, "Failure to write SCDC_TMDS_CONFIG = 0x%02x: %d\n",
+ scdc, ret);
}
- args.pwr.scdc =
- NV50_DISP_SOR_HDMI_PWR_V0_SCDC_SCRAMBLE * scrambling |
- NV50_DISP_SOR_HDMI_PWR_V0_SCDC_DIV_BY_4 * high_tmds_clock_ratio;
+ ret = nvif_outp_acquire_tmds(&nv_encoder->outp, nv_crtc->index, true,
+ max_ac_packet, rekey, scdc, hda);
+ if (ret)
+ return;
- size = sizeof(args.base)
- + sizeof(args.pwr)
- + args.pwr.avi_infoframe_length
- + args.pwr.vendor_infoframe_length;
- nvif_mthd(&disp->disp->object, 0, &args, size);
+ /* AVI InfoFrame. */
+ args.infoframe.version = 0;
+ args.infoframe.head = nv_crtc->index;
+
+ if (!drm_hdmi_avi_infoframe_from_display_mode(&infoframe.avi, &nv_connector->base, mode)) {
+ drm_hdmi_avi_infoframe_quant_range(&infoframe.avi, &nv_connector->base, mode,
+ HDMI_QUANTIZATION_RANGE_FULL);
+
+ size = hdmi_infoframe_pack(&infoframe, args.data, ARRAY_SIZE(args.data));
+ } else {
+ size = 0;
+ }
+
+ nvif_outp_infoframe(&nv_encoder->outp, NVIF_OUTP_INFOFRAME_V0_AVI, &args.infoframe, size);
+
+ /* Vendor InfoFrame. */
+ memset(&args.data, 0, sizeof(args.data));
+ if (!drm_hdmi_vendor_infoframe_from_display_mode(&infoframe.vendor.hdmi,
+ &nv_connector->base, mode))
+ size = hdmi_infoframe_pack(&infoframe, args.data, ARRAY_SIZE(args.data));
+ else
+ size = 0;
+
+ nvif_outp_infoframe(&nv_encoder->outp, NVIF_OUTP_INFOFRAME_V0_VSI, &args.infoframe, size);
nv50_audio_enable(encoder, nv_crtc, nv_connector, state, mode);
-
- /* If SCDC is supported by the downstream monitor, update
- * divider / scrambling settings to what we programmed above.
- */
- if (!hdmi->scdc.scrambling.supported)
- return;
-
- ret = drm_scdc_readb(nv_encoder->i2c, SCDC_TMDS_CONFIG, &config);
- if (ret < 0) {
- NV_ERROR(drm, "Failure to read SCDC_TMDS_CONFIG: %d\n", ret);
- return;
- }
- config &= ~(SCDC_TMDS_BIT_CLOCK_RATIO_BY_40 | SCDC_SCRAMBLING_ENABLE);
- config |= SCDC_TMDS_BIT_CLOCK_RATIO_BY_40 * high_tmds_clock_ratio;
- config |= SCDC_SCRAMBLING_ENABLE * scrambling;
- ret = drm_scdc_writeb(nv_encoder->i2c, SCDC_TMDS_CONFIG, config);
- if (ret < 0)
- NV_ERROR(drm, "Failure to write SCDC_TMDS_CONFIG = 0x%02x: %d\n",
- config, ret);
}
/******************************************************************************
@@ -979,16 +878,6 @@ nv50_msto_prepare(struct drm_atomic_state *state,
struct nv50_mstc *mstc = msto->mstc;
struct nv50_mstm *mstm = mstc->mstm;
struct drm_dp_mst_atomic_payload *payload;
- struct {
- struct nv50_disp_mthd_v1 base;
- struct nv50_disp_sor_dp_mst_vcpi_v0 vcpi;
- } args = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_SOR_DP_MST_VCPI,
- .base.hasht = mstm->outp->dcb->hasht,
- .base.hashm = (0xf0ff & mstm->outp->dcb->hashm) |
- (0x0100 << msto->head->base.index),
- };
NV_ATOMIC(drm, "%s: msto prepare\n", msto->encoder.name);
@@ -997,22 +886,16 @@ nv50_msto_prepare(struct drm_atomic_state *state,
// TODO: Figure out if we want to do a better job of handling VCPI allocation failures here?
if (msto->disabled) {
drm_dp_remove_payload(mgr, mst_state, payload);
+
+ nvif_outp_dp_mst_vcpi(&mstm->outp->outp, msto->head->base.index, 0, 0, 0, 0);
} else {
if (msto->enabled)
drm_dp_add_payload_part1(mgr, mst_state, payload);
- args.vcpi.start_slot = payload->vc_start_slot;
- args.vcpi.num_slots = payload->time_slots;
- args.vcpi.pbn = payload->pbn;
- args.vcpi.aligned_pbn = payload->time_slots * mst_state->pbn_div;
+ nvif_outp_dp_mst_vcpi(&mstm->outp->outp, msto->head->base.index,
+ payload->vc_start_slot, payload->time_slots,
+ payload->pbn, payload->time_slots * mst_state->pbn_div);
}
-
- NV_ATOMIC(drm, "%s: %s: %02x %02x %04x %04x\n",
- msto->encoder.name, msto->head->base.base.name,
- args.vcpi.start_slot, args.vcpi.num_slots,
- args.vcpi.pbn, args.vcpi.aligned_pbn);
-
- nvif_mthd(&drm->display->disp.object, 0, &args, sizeof(args));
}
static int
@@ -1107,10 +990,12 @@ nv50_msto_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *st
if (WARN_ON(!mstc))
return;
- if (!mstm->links++)
- nv50_outp_acquire(mstm->outp, false /*XXX: MST audio.*/);
+ if (!mstm->links++) {
+ /*XXX: MST audio. */
+ nvif_outp_acquire_dp(&mstm->outp->outp, mstm->outp->dp.dpcd, 0, 0, false, true);
+ }
- if (mstm->outp->link & 1)
+ if (mstm->outp->outp.or.link & 1)
proto = NV917D_SOR_SET_CONTROL_PROTOCOL_DP_A;
else
proto = NV917D_SOR_SET_CONTROL_PROTOCOL_DP_B;
@@ -1405,7 +1290,7 @@ nv50_mstm_prepare(struct drm_atomic_state *state,
if (mstm->disabled) {
if (!mstm->links)
- nv50_outp_release(mstm->outp);
+ nvif_outp_release(&mstm->outp->outp);
mstm->disabled = false;
}
}
@@ -1473,26 +1358,6 @@ nv50_mstm_remove(struct nv50_mstm *mstm)
drm_dp_mst_topology_mgr_set_mst(&mstm->mgr, false);
}
-static int
-nv50_mstm_enable(struct nv50_mstm *mstm, int state)
-{
- struct nouveau_encoder *outp = mstm->outp;
- struct {
- struct nv50_disp_mthd_v1 base;
- struct nv50_disp_sor_dp_mst_link_v0 mst;
- } args = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_SOR_DP_MST_LINK,
- .base.hasht = outp->dcb->hasht,
- .base.hashm = outp->dcb->hashm,
- .mst.state = state,
- };
- struct nouveau_drm *drm = nouveau_drm(outp->base.base.dev);
- struct nvif_object *disp = &drm->display->disp.object;
-
- return nvif_mthd(disp, 0, &args, sizeof(args));
-}
-
int
nv50_mstm_detect(struct nouveau_encoder *outp)
{
@@ -1513,16 +1378,10 @@ nv50_mstm_detect(struct nouveau_encoder *outp)
return ret;
/* And start enabling */
- ret = nv50_mstm_enable(mstm, true);
+ ret = drm_dp_mst_topology_mgr_set_mst(&mstm->mgr, true);
if (ret)
return ret;
- ret = drm_dp_mst_topology_mgr_set_mst(&mstm->mgr, true);
- if (ret) {
- nv50_mstm_enable(mstm, false);
- return ret;
- }
-
mstm->is_mst = true;
return 1;
}
@@ -1623,7 +1482,7 @@ nv50_sor_update(struct nouveau_encoder *nv_encoder, u8 head,
asyh->or.depth = depth;
}
- core->func->sor->ctrl(core, nv_encoder->or, nv_encoder->ctrl, asyh);
+ core->func->sor->ctrl(core, nv_encoder->outp.or.id, nv_encoder->ctrl, asyh);
}
/* TODO: Should we extend this to PWM-only backlights?
@@ -1666,8 +1525,7 @@ nv50_sor_atomic_disable(struct drm_encoder *encoder, struct drm_atomic_state *st
nv_encoder->update(nv_encoder, nv_crtc->index, NULL, 0, 0);
nv50_audio_disable(encoder, nv_crtc);
- nv50_hdmi_disable(&nv_encoder->base.base, nv_crtc);
- nv50_outp_release(nv_encoder);
+ nvif_outp_release(&nv_encoder->outp);
nv_encoder->crtc = NULL;
}
@@ -1679,16 +1537,8 @@ nv50_sor_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *sta
struct nv50_head_atom *asyh =
nv50_head_atom(drm_atomic_get_new_crtc_state(state, &nv_crtc->base));
struct drm_display_mode *mode = &asyh->state.adjusted_mode;
- struct {
- struct nv50_disp_mthd_v1 base;
- struct nv50_disp_sor_lvds_script_v0 lvds;
- } lvds = {
- .base.version = 1,
- .base.method = NV50_DISP_MTHD_V1_SOR_LVDS_SCRIPT,
- .base.hasht = nv_encoder->dcb->hasht,
- .base.hashm = nv_encoder->dcb->hashm,
- };
struct nv50_disp *disp = nv50_disp(encoder->dev);
+ struct nvif_outp *outp = &nv_encoder->outp;
struct drm_device *dev = encoder->dev;
struct nouveau_drm *drm = nouveau_drm(dev);
struct nouveau_connector *nv_connector;
@@ -1696,7 +1546,7 @@ nv50_sor_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *sta
struct nouveau_backlight *backlight;
#endif
struct nvbios *bios = &drm->vbios;
- bool hda = false;
+ bool lvds_dual = false, lvds_8bpc = false, hda = false;
u8 proto = NV507D_SOR_SET_CONTROL_PROTOCOL_CUSTOM;
u8 depth = NV837D_SOR_SET_CONTROL_PIXEL_DEPTH_DEFAULT;
@@ -1707,11 +1557,16 @@ nv50_sor_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *sta
disp->disp->object.oclass >= GF110_DISP) &&
drm_detect_monitor_audio(nv_connector->edid))
hda = true;
- nv50_outp_acquire(nv_encoder, hda);
switch (nv_encoder->dcb->type) {
case DCB_OUTPUT_TMDS:
- if (nv_encoder->link & 1) {
+ if (disp->disp->object.oclass == NV50_DISP ||
+ !drm_detect_hdmi_monitor(nv_connector->edid))
+ nvif_outp_acquire_tmds(outp, nv_crtc->index, false, 0, 0, 0, false);
+ else
+ nv50_hdmi_enable(encoder, nv_crtc, nv_connector, state, mode, hda);
+
+ if (nv_encoder->outp.or.link & 1) {
proto = NV507D_SOR_SET_CONTROL_PROTOCOL_SINGLE_TMDS_A;
/* Only enable dual-link if:
* - Need to (i.e. rate > 165MHz)
@@ -1726,44 +1581,41 @@ nv50_sor_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *sta
} else {
proto = NV507D_SOR_SET_CONTROL_PROTOCOL_SINGLE_TMDS_B;
}
-
- nv50_hdmi_enable(&nv_encoder->base.base, nv_crtc, nv_connector, state, mode);
break;
case DCB_OUTPUT_LVDS:
proto = NV507D_SOR_SET_CONTROL_PROTOCOL_LVDS_CUSTOM;
if (bios->fp_no_ddc) {
- if (bios->fp.dual_link)
- lvds.lvds.script |= 0x0100;
- if (bios->fp.if_is_24bit)
- lvds.lvds.script |= 0x0200;
+ lvds_dual = bios->fp.dual_link;
+ lvds_8bpc = bios->fp.if_is_24bit;
} else {
if (nv_connector->type == DCB_CONNECTOR_LVDS_SPWG) {
if (((u8 *)nv_connector->edid)[121] == 2)
- lvds.lvds.script |= 0x0100;
+ lvds_dual = true;
} else
if (mode->clock >= bios->fp.duallink_transition_clk) {
- lvds.lvds.script |= 0x0100;
+ lvds_dual = true;
}
- if (lvds.lvds.script & 0x0100) {
+ if (lvds_dual) {
if (bios->fp.strapless_is_24bit & 2)
- lvds.lvds.script |= 0x0200;
+ lvds_8bpc = true;
} else {
if (bios->fp.strapless_is_24bit & 1)
- lvds.lvds.script |= 0x0200;
+ lvds_8bpc = true;
}
if (asyh->or.bpc == 8)
- lvds.lvds.script |= 0x0200;
+ lvds_8bpc = true;
}
- nvif_mthd(&disp->disp->object, 0, &lvds, sizeof(lvds));
+ nvif_outp_acquire_lvds(&nv_encoder->outp, lvds_dual, lvds_8bpc);
break;
case DCB_OUTPUT_DP:
+ nvif_outp_acquire_dp(&nv_encoder->outp, nv_encoder->dp.dpcd, 0, 0, hda, false);
depth = nv50_dp_bpc_to_depth(asyh->or.bpc);
- if (nv_encoder->link & 1)
+ if (nv_encoder->outp.or.link & 1)
proto = NV887D_SOR_SET_CONTROL_PROTOCOL_DP_A;
else
proto = NV887D_SOR_SET_CONTROL_PROTOCOL_DP_B;
@@ -1921,9 +1773,9 @@ nv50_pior_atomic_disable(struct drm_encoder *encoder, struct drm_atomic_state *s
struct nv50_core *core = nv50_disp(encoder->dev)->core;
const u32 ctrl = NVDEF(NV507D, PIOR_SET_CONTROL, OWNER, NONE);
- core->func->pior->ctrl(core, nv_encoder->or, ctrl, NULL);
+ core->func->pior->ctrl(core, nv_encoder->outp.or.id, ctrl, NULL);
nv_encoder->crtc = NULL;
- nv50_outp_release(nv_encoder);
+ nvif_outp_release(&nv_encoder->outp);
}
static void
@@ -1944,8 +1796,6 @@ nv50_pior_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *st
break;
}
- nv50_outp_acquire(nv_encoder, false);
-
switch (asyh->or.bpc) {
case 10: asyh->or.depth = NV837D_PIOR_SET_CONTROL_PIXEL_DEPTH_BPP_30_444; break;
case 8: asyh->or.depth = NV837D_PIOR_SET_CONTROL_PIXEL_DEPTH_BPP_24_444; break;
@@ -1955,15 +1805,19 @@ nv50_pior_atomic_enable(struct drm_encoder *encoder, struct drm_atomic_state *st
switch (nv_encoder->dcb->type) {
case DCB_OUTPUT_TMDS:
+ ctrl |= NVDEF(NV507D, PIOR_SET_CONTROL, PROTOCOL, EXT_TMDS_ENC);
+ nvif_outp_acquire_tmds(&nv_encoder->outp, false, false, 0, 0, 0, false);
+ break;
case DCB_OUTPUT_DP:
ctrl |= NVDEF(NV507D, PIOR_SET_CONTROL, PROTOCOL, EXT_TMDS_ENC);
+ nvif_outp_acquire_dp(&nv_encoder->outp, nv_encoder->dp.dpcd, 0, 0, false, false);
break;
default:
BUG();
break;
}
- core->func->pior->ctrl(core, nv_encoder->or, ctrl, asyh);
+ core->func->pior->ctrl(core, nv_encoder->outp.or.id, ctrl, asyh);
nv_encoder->crtc = &nv_crtc->base;
}
@@ -2587,7 +2441,7 @@ nv50_disp_atomic_state_alloc(struct drm_device *dev)
static const struct drm_mode_config_funcs
nv50_disp_func = {
.fb_create = nouveau_user_framebuffer_create,
- .output_poll_changed = nouveau_fbcon_output_poll_changed,
+ .output_poll_changed = drm_fb_helper_output_poll_changed,
.atomic_check = nv50_disp_atomic_check,
.atomic_commit = nv50_disp_atomic_commit,
.atomic_state_alloc = nv50_disp_atomic_state_alloc,
diff --git a/drivers/gpu/drm/nouveau/dispnv50/head.c b/drivers/gpu/drm/nouveau/dispnv50/head.c
index c3c57be..f006e56 100644
--- a/drivers/gpu/drm/nouveau/dispnv50/head.c
+++ b/drivers/gpu/drm/nouveau/dispnv50/head.c
@@ -517,7 +517,8 @@ nv50_head_destroy(struct drm_crtc *crtc)
{
struct nv50_head *head = nv50_head(crtc);
- nvif_notify_dtor(&head->base.vblank);
+ nvif_event_dtor(&head->base.vblank);
+ nvif_head_dtor(&head->base.head);
nv50_lut_fini(&head->olut);
drm_crtc_cleanup(crtc);
kfree(head);
@@ -554,15 +555,15 @@ nvd9_head_func = {
.late_register = nv50_head_late_register,
};
-static int nv50_head_vblank_handler(struct nvif_notify *notify)
+static int
+nv50_head_vblank_handler(struct nvif_event *event, void *repv, u32 repc)
{
- struct nouveau_crtc *nv_crtc =
- container_of(notify, struct nouveau_crtc, vblank);
+ struct nouveau_crtc *nv_crtc = container_of(event, struct nouveau_crtc, vblank);
if (drm_crtc_handle_vblank(&nv_crtc->base))
nv50_crc_handle_vblank(nv50_head(&nv_crtc->base));
- return NVIF_NOTIFY_KEEP;
+ return NVIF_EVENT_KEEP;
}
struct nv50_head *
@@ -624,14 +625,12 @@ nv50_head_create(struct drm_device *dev, int index)
}
}
- ret = nvif_notify_ctor(&disp->disp->object, "kmsVbl", nv50_head_vblank_handler,
- false, NV04_DISP_NTFY_VBLANK,
- &(struct nvif_notify_head_req_v0) {
- .head = nv_crtc->index,
- },
- sizeof(struct nvif_notify_head_req_v0),
- sizeof(struct nvif_notify_head_rep_v0),
- &nv_crtc->vblank);
+ ret = nvif_head_ctor(disp->disp, head->base.base.name, head->base.index, &head->base.head);
+ if (ret)
+ return ERR_PTR(ret);
+
+ ret = nvif_head_vblank_event_ctor(&head->base.head, "kmsVbl", nv50_head_vblank_handler,
+ false, &nv_crtc->vblank);
if (ret)
return ERR_PTR(ret);
diff --git a/drivers/gpu/drm/nouveau/include/nvfw/acr.h b/drivers/gpu/drm/nouveau/include/nvfw/acr.h
index e65d6a8..6f19560 100644
--- a/drivers/gpu/drm/nouveau/include/nvfw/acr.h
+++ b/drivers/gpu/drm/nouveau/include/nvfw/acr.h
@@ -39,6 +39,23 @@ struct wpr_header_v1 {
void wpr_header_v1_dump(struct nvkm_subdev *, const struct wpr_header_v1 *);
+struct wpr_generic_header {
+#define WPR_GENERIC_HEADER_ID_LSF_UCODE_DESC 1
+#define WPR_GENERIC_HEADER_ID_LSF_WPR_HEADER 2
+#define WPR_GENERIC_HEADER_ID_LSF_SHARED_SUB_WPR 3
+#define WPR_GENERIC_HEADER_ID_LSF_LSB_HEADER 4
+ u16 identifier;
+ u16 version;
+ u32 size;
+};
+
+struct wpr_header_v2 {
+ struct wpr_generic_header hdr;
+ struct wpr_header_v1 wpr;
+};
+
+void wpr_header_v2_dump(struct nvkm_subdev *, const struct wpr_header_v2 *);
+
struct lsf_signature {
u8 prd_keys[2][16];
u8 dbg_keys[2][16];
@@ -89,6 +106,74 @@ struct lsb_header_v1 {
void lsb_header_v1_dump(struct nvkm_subdev *, struct lsb_header_v1 *);
+struct lsb_header_v2 {
+ struct wpr_generic_header hdr;
+ struct lsf_signature_v2 {
+ struct wpr_generic_header hdr;
+ u32 falcon_id;
+ u8 prd_present;
+ u8 dbg_present;
+ u16 reserved;
+ u32 sig_size;
+ u8 prod_sig[2][384 + 128];
+ u8 debug_sig[2][384 + 128];
+ u16 sig_algo_ver;
+ u16 sig_algo;
+ u16 hash_algo_ver;
+ u16 hash_algo;
+ u32 sig_algo_padding_type;
+ u8 depmap[11 * 2 * 4];
+ u32 depmap_count;
+ u8 supports_versioning;
+ u8 pad[3];
+ u32 ls_ucode_version;
+ u32 ls_ucode_id;
+ u32 ucode_ls_encrypted;
+ u32 ls_eng_algo_type;
+ u32 ls_eng_algo_ver;
+ u8 ls_enc_iv[16];
+ u8 rsvd[36];
+ } signature;
+ u32 ucode_off;
+ u32 ucode_size;
+ u32 data_size;
+ u32 bl_code_size;
+ u32 bl_imem_off;
+ u32 bl_data_off;
+ u32 bl_data_size;
+ u32 rsvd0;
+ u32 app_code_off;
+ u32 app_code_size;
+ u32 app_data_off;
+ u32 app_data_size;
+ u32 app_imem_offset;
+ u32 app_dmem_offset;
+ u32 flags;
+ u32 monitor_code_offset;
+ u32 monitor_data_offset;
+ u32 manifest_offset;
+ struct hs_fmc_params {
+ u8 hs_fmc;
+ u8 padding[3];
+ u16 pkc_algo;
+ u16 pkc_algo_version;
+ u32 engid_mask;
+ u32 ucode_id;
+ u32 fuse_ver;
+ u8 pkc_signature[384 + 128];
+ u8 pkc_key[2048];
+ u8 rsvd[4];
+ } hs_fmc_params;
+ struct hs_ovl_sig_blob_params {
+ u8 hs_ovl_sig_blob_present;
+ u32 hs_ovl_sig_blob_offset;
+ u32 hs_ovl_sig_blob_size;
+ } hs_ovl_sig_blob_params;
+ u8 rsvd[20];
+};
+
+void lsb_header_v2_dump(struct nvkm_subdev *, struct lsb_header_v2 *);
+
struct flcn_acr_desc {
union {
u8 reserved_dmem[0x200];
diff --git a/drivers/gpu/drm/nouveau/include/nvfw/hs.h b/drivers/gpu/drm/nouveau/include/nvfw/hs.h
index b53bbc4..8c4cd08 100644
--- a/drivers/gpu/drm/nouveau/include/nvfw/hs.h
+++ b/drivers/gpu/drm/nouveau/include/nvfw/hs.h
@@ -17,6 +17,20 @@ struct nvfw_hs_header {
const struct nvfw_hs_header *nvfw_hs_header(struct nvkm_subdev *, const void *);
+struct nvfw_hs_header_v2 {
+ u32 sig_prod_offset;
+ u32 sig_prod_size;
+ u32 patch_loc;
+ u32 patch_sig;
+ u32 meta_data_offset;
+ u32 meta_data_size;
+ u32 num_sig;
+ u32 header_offset;
+ u32 header_size;
+};
+
+const struct nvfw_hs_header_v2 *nvfw_hs_header_v2(struct nvkm_subdev *, const void *);
+
struct nvfw_hs_load_header {
u32 non_sec_code_off;
u32 non_sec_code_size;
@@ -28,4 +42,18 @@ struct nvfw_hs_load_header {
const struct nvfw_hs_load_header *
nvfw_hs_load_header(struct nvkm_subdev *, const void *);
+
+struct nvfw_hs_load_header_v2 {
+ u32 os_code_offset;
+ u32 os_code_size;
+ u32 os_data_offset;
+ u32 os_data_size;
+ u32 num_apps;
+ struct {
+ u32 offset;
+ u32 size;
+ } app[0];
+};
+
+const struct nvfw_hs_load_header_v2 *nvfw_hs_load_header_v2(struct nvkm_subdev *, const void *);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvfw/ls.h b/drivers/gpu/drm/nouveau/include/nvfw/ls.h
index f63692a..d531121 100644
--- a/drivers/gpu/drm/nouveau/include/nvfw/ls.h
+++ b/drivers/gpu/drm/nouveau/include/nvfw/ls.h
@@ -50,4 +50,55 @@ struct nvfw_ls_desc_v1 {
const struct nvfw_ls_desc_v1 *
nvfw_ls_desc_v1(struct nvkm_subdev *, const void *);
+
+struct nvfw_ls_desc_v2 {
+ u32 descriptor_size;
+ u32 image_size;
+ u32 tools_version;
+ u32 app_version;
+ char date[64];
+ u32 secure_bootloader;
+ u32 bootloader_start_offset;
+ u32 bootloader_size;
+ u32 bootloader_imem_offset;
+ u32 bootloader_entry_point;
+ u32 app_start_offset;
+ u32 app_size;
+ u32 app_imem_offset;
+ u32 app_imem_entry;
+ u32 app_dmem_offset;
+ u32 app_resident_code_offset;
+ u32 app_resident_code_size;
+ u32 app_resident_data_offset;
+ u32 app_resident_data_size;
+ u32 nb_imem_overlays;
+ u32 nb_dmem_overlays;
+ struct {
+ u32 start;
+ u32 size;
+ } load_ovl[64];
+};
+
+const struct nvfw_ls_desc_v2 *nvfw_ls_desc_v2(struct nvkm_subdev *, const void *);
+
+struct nvfw_ls_hsbl_bin_hdr {
+ u32 bin_magic;
+ u32 bin_ver;
+ u32 bin_size;
+ u32 header_offset;
+};
+
+const struct nvfw_ls_hsbl_bin_hdr *nvfw_ls_hsbl_bin_hdr(struct nvkm_subdev *, const void *);
+
+struct nvfw_ls_hsbl_hdr {
+ u32 sig_prod_offset;
+ u32 sig_prod_size;
+ u32 patch_loc;
+ u32 patch_sig;
+ u32 meta_data_offset;
+ u32 meta_data_size;
+ u32 num_sig;
+};
+
+const struct nvfw_ls_hsbl_hdr *nvfw_ls_hsbl_hdr(struct nvkm_subdev *, const void *);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvfw/sec2.h b/drivers/gpu/drm/nouveau/include/nvfw/sec2.h
index 9a37ad4..b3331d6 100644
--- a/drivers/gpu/drm/nouveau/include/nvfw/sec2.h
+++ b/drivers/gpu/drm/nouveau/include/nvfw/sec2.h
@@ -10,6 +10,7 @@ struct nv_sec2_args {
};
#define NV_SEC2_UNIT_INIT 0x01
+#define NV_SEC2_UNIT_UNLOAD 0x06
#define NV_SEC2_UNIT_ACR 0x08
struct nv_sec2_init_msg {
@@ -33,6 +34,29 @@ struct nv_sec2_init_msg {
u16 sw_managed_area_size;
};
+struct nv_sec2_init_msg_v1 {
+ struct nvfw_falcon_msg hdr;
+#define NV_SEC2_INIT_MSG_INIT 0x00
+ u8 msg_type;
+
+ u8 num_queues;
+ u16 os_debug_entry_point;
+
+ struct {
+ u32 offset;
+ u16 size;
+ u8 index;
+#define NV_SEC2_INIT_MSG_QUEUE_ID_CMDQ 0x00
+#define NV_SEC2_INIT_MSG_QUEUE_ID_MSGQ 0x01
+ u8 id;
+ } queue_info[2];
+
+ u32 sw_managed_area_offset;
+ u16 sw_managed_area_size;
+
+ u32 unkn[8];
+};
+
struct nv_sec2_acr_cmd {
struct nvfw_falcon_cmd hdr;
#define NV_SEC2_ACR_CMD_BOOTSTRAP_FALCON 0x00
@@ -57,4 +81,25 @@ struct nv_sec2_acr_bootstrap_falcon_msg {
u32 error_code;
u32 falcon_id;
};
+
+#define NV_SEC2_UNIT_V2_INIT 0x01
+#define NV_SEC2_UNIT_V2_UNLOAD 0x05
+#define NV_SEC2_UNIT_V2_ACR 0x07
+
+struct nv_sec2_acr_bootstrap_falcon_cmd_v1 {
+ struct nv_sec2_acr_cmd cmd;
+#define NV_SEC2_ACR_BOOTSTRAP_FALCON_FLAGS_RESET_YES 0x00000000
+#define NV_SEC2_ACR_BOOTSTRAP_FALCON_FLAGS_RESET_NO 0x00000001
+ u32 flags;
+ u32 falcon_id;
+ u32 unkn08;
+ u32 unkn0c;
+};
+
+struct nv_sec2_acr_bootstrap_falcon_msg_v1 {
+ struct nv_sec2_acr_msg msg;
+ u32 error_code;
+ u32 falcon_id;
+ u32 unkn08;
+};
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl0046.h b/drivers/gpu/drm/nouveau/include/nvif/cl0046.h
index d490d40..eca7c39 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/cl0046.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/cl0046.h
@@ -2,28 +2,5 @@
#ifndef __NVIF_CL0046_H__
#define __NVIF_CL0046_H__
-#define NV04_DISP_NTFY_VBLANK 0x00
#define NV04_DISP_NTFY_CONN 0x01
-
-struct nv04_disp_mthd_v0 {
- __u8 version;
-#define NV04_DISP_SCANOUTPOS 0x00
- __u8 method;
- __u8 head;
- __u8 pad03[5];
-};
-
-struct nv04_disp_scanoutpos_v0 {
- __u8 version;
- __u8 pad01[7];
- __s64 time[2];
- __u16 vblanks;
- __u16 vblanke;
- __u16 vtotal;
- __u16 vline;
- __u16 hblanks;
- __u16 hblanke;
- __u16 htotal;
- __u16 hline;
-};
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl006b.h b/drivers/gpu/drm/nouveau/include/nvif/cl006b.h
deleted file mode 100644
index c960c44..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cl006b.h
+++ /dev/null
@@ -1,12 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CL006B_H__
-#define __NVIF_CL006B_H__
-
-struct nv03_channel_dma_v0 {
- __u8 version;
- __u8 chid;
- __u8 pad02[2];
- __u32 offset;
- __u64 pushbuf;
-};
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl0080.h b/drivers/gpu/drm/nouveau/include/nvif/cl0080.h
index 59759c4..8b5a240 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/cl0080.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/cl0080.h
@@ -68,7 +68,7 @@ struct nv_device_time_v0 {
/* Returns the number of available runlists. */
#define NV_DEVICE_HOST_RUNLISTS NV_DEVICE_HOST(0x00000000)
-/* Returns the number of available channels. */
+/* Returns the number of available channels (0 if per-runlist). */
#define NV_DEVICE_HOST_CHANNELS NV_DEVICE_HOST(0x00000001)
/* Returns a mask of available engine types on runlist(data). */
@@ -90,4 +90,6 @@ struct nv_device_time_v0 {
#define NV_DEVICE_HOST_RUNLIST_ENGINES_SEC2 0x00004000
#define NV_DEVICE_HOST_RUNLIST_ENGINES_NVDEC 0x00008000
#define NV_DEVICE_HOST_RUNLIST_ENGINES_NVENC 0x00010000
+/* Returns the number of available channels on runlist(data). */
+#define NV_DEVICE_HOST_RUNLIST_CHANNELS NV_DEVICE_HOST(0x00000101)
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl506e.h b/drivers/gpu/drm/nouveau/include/nvif/cl506e.h
deleted file mode 100644
index 9df289c..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cl506e.h
+++ /dev/null
@@ -1,13 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CL506E_H__
-#define __NVIF_CL506E_H__
-
-struct nv50_channel_dma_v0 {
- __u8 version;
- __u8 chid;
- __u8 pad02[6];
- __u64 vmm;
- __u64 pushbuf;
- __u64 offset;
-};
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl506f.h b/drivers/gpu/drm/nouveau/include/nvif/cl506f.h
deleted file mode 100644
index 327c96a..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cl506f.h
+++ /dev/null
@@ -1,14 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CL506F_H__
-#define __NVIF_CL506F_H__
-
-struct nv50_channel_gpfifo_v0 {
- __u8 version;
- __u8 chid;
- __u8 pad02[2];
- __u32 ilength;
- __u64 ioffset;
- __u64 pushbuf;
- __u64 vmm;
-};
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl5070.h b/drivers/gpu/drm/nouveau/include/nvif/cl5070.h
deleted file mode 100644
index 56affb6..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cl5070.h
+++ /dev/null
@@ -1,92 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CL5070_H__
-#define __NVIF_CL5070_H__
-
-#define NV50_DISP_MTHD 0x00
-
-struct nv50_disp_mthd_v0 {
- __u8 version;
-#define NV50_DISP_SCANOUTPOS 0x00
- __u8 method;
- __u8 head;
- __u8 pad03[5];
-};
-
-struct nv50_disp_scanoutpos_v0 {
- __u8 version;
- __u8 pad01[7];
- __s64 time[2];
- __u16 vblanks;
- __u16 vblanke;
- __u16 vtotal;
- __u16 vline;
- __u16 hblanks;
- __u16 hblanke;
- __u16 htotal;
- __u16 hline;
-};
-
-struct nv50_disp_mthd_v1 {
- __u8 version;
-#define NV50_DISP_MTHD_V1_ACQUIRE 0x01
-#define NV50_DISP_MTHD_V1_RELEASE 0x02
-#define NV50_DISP_MTHD_V1_SOR_HDA_ELD 0x21
-#define NV50_DISP_MTHD_V1_SOR_HDMI_PWR 0x22
-#define NV50_DISP_MTHD_V1_SOR_LVDS_SCRIPT 0x23
-#define NV50_DISP_MTHD_V1_SOR_DP_MST_LINK 0x25
-#define NV50_DISP_MTHD_V1_SOR_DP_MST_VCPI 0x26
- __u8 method;
- __u16 hasht;
- __u16 hashm;
- __u8 pad06[2];
-};
-
-struct nv50_disp_acquire_v0 {
- __u8 version;
- __u8 or;
- __u8 link;
- __u8 hda;
- __u8 pad04[4];
-};
-
-struct nv50_disp_sor_hda_eld_v0 {
- __u8 version;
- __u8 pad01[7];
- __u8 data[];
-};
-
-struct nv50_disp_sor_hdmi_pwr_v0 {
- __u8 version;
- __u8 state;
- __u8 max_ac_packet;
- __u8 rekey;
- __u8 avi_infoframe_length;
- __u8 vendor_infoframe_length;
-#define NV50_DISP_SOR_HDMI_PWR_V0_SCDC_SCRAMBLE (1 << 0)
-#define NV50_DISP_SOR_HDMI_PWR_V0_SCDC_DIV_BY_4 (1 << 1)
- __u8 scdc;
- __u8 pad07[1];
-};
-
-struct nv50_disp_sor_lvds_script_v0 {
- __u8 version;
- __u8 pad01[1];
- __u16 script;
- __u8 pad04[4];
-};
-
-struct nv50_disp_sor_dp_mst_link_v0 {
- __u8 version;
- __u8 state;
- __u8 pad02[6];
-};
-
-struct nv50_disp_sor_dp_mst_vcpi_v0 {
- __u8 version;
- __u8 pad01[1];
- __u8 start_slot;
- __u8 num_slots;
- __u16 pbn;
- __u16 aligned_pbn;
-};
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl826e.h b/drivers/gpu/drm/nouveau/include/nvif/cl826e.h
deleted file mode 100644
index 1b6496d3..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cl826e.h
+++ /dev/null
@@ -1,15 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CL826E_H__
-#define __NVIF_CL826E_H__
-
-struct g82_channel_dma_v0 {
- __u8 version;
- __u8 chid;
- __u8 pad02[6];
- __u64 vmm;
- __u64 pushbuf;
- __u64 offset;
-};
-
-#define NV826E_V0_NTFY_NON_STALL_INTERRUPT 0x00
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl826f.h b/drivers/gpu/drm/nouveau/include/nvif/cl826f.h
deleted file mode 100644
index 14860226..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cl826f.h
+++ /dev/null
@@ -1,16 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CL826F_H__
-#define __NVIF_CL826F_H__
-
-struct g82_channel_gpfifo_v0 {
- __u8 version;
- __u8 chid;
- __u8 pad02[2];
- __u32 ilength;
- __u64 ioffset;
- __u64 pushbuf;
- __u64 vmm;
-};
-
-#define NV826F_V0_NTFY_NON_STALL_INTERRUPT 0x00
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl906f.h b/drivers/gpu/drm/nouveau/include/nvif/cl906f.h
deleted file mode 100644
index 3823d68..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cl906f.h
+++ /dev/null
@@ -1,16 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CL906F_H__
-#define __NVIF_CL906F_H__
-
-struct fermi_channel_gpfifo_v0 {
- __u8 version;
- __u8 chid;
- __u8 pad02[2];
- __u32 ilength;
- __u64 ioffset;
- __u64 vmm;
-};
-
-#define NV906F_V0_NTFY_NON_STALL_INTERRUPT 0x00
-#define NV906F_V0_NTFY_KILLED 0x01
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/cla06f.h b/drivers/gpu/drm/nouveau/include/nvif/cla06f.h
deleted file mode 100644
index cfa18f1..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/cla06f.h
+++ /dev/null
@@ -1,18 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CLA06F_H__
-#define __NVIF_CLA06F_H__
-
-struct kepler_channel_gpfifo_a_v0 {
- __u8 version;
- __u8 priv;
- __u16 chid;
- __u32 ilength;
- __u64 ioffset;
- __u64 runlist;
- __u64 vmm;
- __u64 inst;
-};
-
-#define NVA06F_V0_NTFY_NON_STALL_INTERRUPT 0x00
-#define NVA06F_V0_NTFY_KILLED 0x01
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/class.h b/drivers/gpu/drm/nouveau/include/nvif/class.h
index 8641db6..ad1e5de 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/class.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/class.h
@@ -32,11 +32,17 @@
#define NVIF_CLASS_VMM_GM200 /* ifb00d.h */ 0x8000b00d
#define NVIF_CLASS_VMM_GP100 /* ifc00d.h */ 0x8000c00d
+#define NVIF_CLASS_EVENT /* if000e.h */ 0x8000000e
+
#define NVIF_CLASS_DISP /* if0010.h */ 0x80000010
#define NVIF_CLASS_CONN /* if0011.h */ 0x80000011
#define NVIF_CLASS_OUTP /* if0012.h */ 0x80000012
+#define NVIF_CLASS_HEAD /* if0013.h */ 0x80000013
#define NVIF_CLASS_DISP_CHAN /* if0014.h */ 0x80000014
+#define NVIF_CLASS_CHAN /* if0020.h */ 0x80000020
+#define NVIF_CLASS_CGRP /* if0021.h */ 0x80000021
+
/* the below match nvidia-assigned (either in hw, or sw) class numbers */
#define NV_NULL_CLASS 0x00000030
@@ -58,25 +64,30 @@
#define NV04_DISP /* cl0046.h */ 0x00000046
#define VOLTA_USERMODE_A 0x0000c361
+#define TURING_USERMODE_A 0x0000c461
+#define AMPERE_USERMODE_A 0x0000c561
#define MAXWELL_FAULT_BUFFER_A /* clb069.h */ 0x0000b069
#define VOLTA_FAULT_BUFFER_A /* clb069.h */ 0x0000c369
-#define NV03_CHANNEL_DMA /* cl506b.h */ 0x0000006b
-#define NV10_CHANNEL_DMA /* cl506b.h */ 0x0000006e
-#define NV17_CHANNEL_DMA /* cl506b.h */ 0x0000176e
-#define NV40_CHANNEL_DMA /* cl506b.h */ 0x0000406e
+#define NV03_CHANNEL_DMA /* if0020.h */ 0x0000006b
+#define NV10_CHANNEL_DMA /* if0020.h */ 0x0000006e
+#define NV17_CHANNEL_DMA /* if0020.h */ 0x0000176e
+#define NV40_CHANNEL_DMA /* if0020.h */ 0x0000406e
-#define NV50_CHANNEL_GPFIFO /* cl506f.h */ 0x0000506f
-#define G82_CHANNEL_GPFIFO /* cl826f.h */ 0x0000826f
-#define FERMI_CHANNEL_GPFIFO /* cl906f.h */ 0x0000906f
-#define KEPLER_CHANNEL_GPFIFO_A /* cla06f.h */ 0x0000a06f
-#define KEPLER_CHANNEL_GPFIFO_B /* cla06f.h */ 0x0000a16f
-#define MAXWELL_CHANNEL_GPFIFO_A /* cla06f.h */ 0x0000b06f
-#define PASCAL_CHANNEL_GPFIFO_A /* cla06f.h */ 0x0000c06f
-#define VOLTA_CHANNEL_GPFIFO_A /* clc36f.h */ 0x0000c36f
-#define TURING_CHANNEL_GPFIFO_A /* clc36f.h */ 0x0000c46f
-#define AMPERE_CHANNEL_GPFIFO_B /* clc36f.h */ 0x0000c76f
+#define KEPLER_CHANNEL_GROUP_A /* if0021.h */ 0x0000a06c
+
+#define NV50_CHANNEL_GPFIFO /* if0020.h */ 0x0000506f
+#define G82_CHANNEL_GPFIFO /* if0020.h */ 0x0000826f
+#define FERMI_CHANNEL_GPFIFO /* if0020.h */ 0x0000906f
+#define KEPLER_CHANNEL_GPFIFO_A /* if0020.h */ 0x0000a06f
+#define KEPLER_CHANNEL_GPFIFO_B /* if0020.h */ 0x0000a16f
+#define MAXWELL_CHANNEL_GPFIFO_A /* if0020.h */ 0x0000b06f
+#define PASCAL_CHANNEL_GPFIFO_A /* if0020.h */ 0x0000c06f
+#define VOLTA_CHANNEL_GPFIFO_A /* if0020.h */ 0x0000c36f
+#define TURING_CHANNEL_GPFIFO_A /* if0020.h */ 0x0000c46f
+#define AMPERE_CHANNEL_GPFIFO_A /* if0020.h */ 0x0000c56f
+#define AMPERE_CHANNEL_GPFIFO_B /* if0020.h */ 0x0000c76f
#define NV50_DISP /* if0010.h */ 0x00005070
#define G82_DISP /* if0010.h */ 0x00008270
@@ -179,6 +190,8 @@
#define TURING_A /* cl9097.h */ 0x0000c597
+#define AMPERE_B /* cl9097.h */ 0x0000c797
+
#define NV74_BSP 0x000074b0
#define GT212_MSVLD 0x000085b1
@@ -206,6 +219,7 @@
#define PASCAL_DMA_COPY_B 0x0000c1b5
#define VOLTA_DMA_COPY_A 0x0000c3b5
#define TURING_DMA_COPY_A 0x0000c5b5
+#define AMPERE_DMA_COPY_A 0x0000c6b5
#define AMPERE_DMA_COPY_B 0x0000c7b5
#define FERMI_DECOMPRESS 0x000090b8
@@ -222,6 +236,7 @@
#define PASCAL_COMPUTE_B 0x0000c1c0
#define VOLTA_COMPUTE_A 0x0000c3c0
#define TURING_COMPUTE_A 0x0000c5c0
+#define AMPERE_COMPUTE_B 0x0000c7c0
#define NV74_CIPHER 0x000074c1
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/clb069.h b/drivers/gpu/drm/nouveau/include/nvif/clb069.h
index eef5d02..d7689de3 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/clb069.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/clb069.h
@@ -8,5 +8,8 @@ struct nvif_clb069_v0 {
__u32 put;
};
-#define NVB069_V0_NTFY_FAULT 0x00
+union nvif_clb069_event_args {
+ struct nvif_clb069_event_vn {
+ } vn;
+};
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/clc36f.h b/drivers/gpu/drm/nouveau/include/nvif/clc36f.h
deleted file mode 100644
index f668858..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/clc36f.h
+++ /dev/null
@@ -1,19 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_CLC36F_H__
-#define __NVIF_CLC36F_H__
-
-struct volta_channel_gpfifo_a_v0 {
- __u8 version;
- __u8 priv;
- __u16 chid;
- __u32 ilength;
- __u64 ioffset;
- __u64 runlist;
- __u64 vmm;
- __u64 inst;
- __u32 token;
-};
-
-#define NVC36F_V0_NTFY_NON_STALL_INTERRUPT 0x00
-#define NVC36F_V0_NTFY_KILLED 0x01
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/conn.h b/drivers/gpu/drm/nouveau/include/nvif/conn.h
index f72a8f1..dc355e1 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/conn.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/conn.h
@@ -2,6 +2,7 @@
#ifndef __NVIF_CONN_H__
#define __NVIF_CONN_H__
#include <nvif/object.h>
+#include <nvif/event.h>
struct nvif_disp;
struct nvif_conn {
@@ -11,8 +12,17 @@ struct nvif_conn {
int nvif_conn_ctor(struct nvif_disp *, const char *name, int id, struct nvif_conn *);
void nvif_conn_dtor(struct nvif_conn *);
+static inline int
+nvif_conn_id(struct nvif_conn *conn)
+{
+ return conn->object.handle;
+}
+
#define NVIF_CONN_HPD_STATUS_UNSUPPORTED 0 /* negative if query fails */
#define NVIF_CONN_HPD_STATUS_NOT_PRESENT 1
#define NVIF_CONN_HPD_STATUS_PRESENT 2
int nvif_conn_hpd_status(struct nvif_conn *);
+
+int nvif_conn_event_ctor(struct nvif_conn *, const char *name, nvif_event_func, u8 types,
+ struct nvif_event *);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/disp.h b/drivers/gpu/drm/nouveau/include/nvif/disp.h
index 742632a..56eb729 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/disp.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/disp.h
@@ -7,6 +7,7 @@ struct nvif_disp {
struct nvif_object object;
unsigned long conn_mask;
unsigned long outp_mask;
+ unsigned long head_mask;
};
int nvif_disp_ctor(struct nvif_device *, const char *name, s32 oclass,
diff --git a/drivers/gpu/drm/nouveau/include/nvif/event.h b/drivers/gpu/drm/nouveau/include/nvif/event.h
index a6b1ee4..68bf663 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/event.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/event.h
@@ -1,63 +1,36 @@
/* SPDX-License-Identifier: MIT */
#ifndef __NVIF_EVENT_H__
#define __NVIF_EVENT_H__
+#include <nvif/object.h>
+#include <nvif/if000e.h>
+struct nvif_event;
-struct nvif_notify_req_v0 {
- __u8 version;
- __u8 reply;
- __u8 pad02[5];
-#define NVIF_NOTIFY_V0_ROUTE_NVIF 0x00
- __u8 route;
- __u64 token; /* must be unique */
- __u8 data[]; /* request data (below) */
+#define NVIF_EVENT_KEEP 0
+#define NVIF_EVENT_DROP 1
+typedef int (*nvif_event_func)(struct nvif_event *, void *repv, u32 repc);
+
+struct nvif_event {
+ struct nvif_object object;
+ nvif_event_func func;
};
-struct nvif_notify_rep_v0 {
- __u8 version;
- __u8 pad01[6];
- __u8 route;
- __u64 token;
- __u8 data[]; /* reply data (below) */
-};
+static inline bool
+nvif_event_constructed(struct nvif_event *event)
+{
+ return nvif_object_constructed(&event->object);
+}
-struct nvif_notify_head_req_v0 {
- /* nvif_notify_req ... */
- __u8 version;
- __u8 head;
- __u8 pad02[6];
-};
+int nvif_event_ctor_(struct nvif_object *, const char *, u32, nvif_event_func, bool,
+ struct nvif_event_v0 *, u32, bool, struct nvif_event *);
-struct nvif_notify_head_rep_v0 {
- /* nvif_notify_rep ... */
- __u8 version;
- __u8 pad01[7];
-};
+static inline int
+nvif_event_ctor(struct nvif_object *parent, const char *name, u32 handle, nvif_event_func func,
+ bool wait, struct nvif_event_v0 *args, u32 argc, struct nvif_event *event)
+{
+ return nvif_event_ctor_(parent, name, handle, func, wait, args, argc, true, event);
+}
-struct nvif_notify_conn_req_v0 {
- /* nvif_notify_req ... */
- __u8 version;
-#define NVIF_NOTIFY_CONN_V0_PLUG 0x01
-#define NVIF_NOTIFY_CONN_V0_UNPLUG 0x02
-#define NVIF_NOTIFY_CONN_V0_IRQ 0x04
-#define NVIF_NOTIFY_CONN_V0_ANY 0x07
- __u8 mask;
- __u8 conn;
- __u8 pad03[5];
-};
-
-struct nvif_notify_conn_rep_v0 {
- /* nvif_notify_rep ... */
- __u8 version;
- __u8 mask;
- __u8 pad02[6];
-};
-
-struct nvif_notify_uevent_req {
- /* nvif_notify_req ... */
-};
-
-struct nvif_notify_uevent_rep {
- /* nvif_notify_rep ... */
-};
-
+void nvif_event_dtor(struct nvif_event *);
+int nvif_event_allow(struct nvif_event *);
+int nvif_event_block(struct nvif_event *);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/head.h b/drivers/gpu/drm/nouveau/include/nvif/head.h
new file mode 100644
index 0000000..3ec3699
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvif/head.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVIF_HEAD_H__
+#define __NVIF_HEAD_H__
+#include <nvif/object.h>
+#include <nvif/event.h>
+struct nvif_disp;
+
+struct nvif_head {
+ struct nvif_object object;
+};
+
+int nvif_head_ctor(struct nvif_disp *, const char *name, int id, struct nvif_head *);
+void nvif_head_dtor(struct nvif_head *);
+
+static inline int
+nvif_head_id(struct nvif_head *head)
+{
+ return head->object.handle;
+}
+
+int nvif_head_vblank_event_ctor(struct nvif_head *, const char *name, nvif_event_func, bool wait,
+ struct nvif_event *);
+#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if0004.h b/drivers/gpu/drm/nouveau/include/nvif/if0004.h
index d324c73..1d916a1 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/if0004.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/if0004.h
@@ -2,7 +2,10 @@
#ifndef __NVIF_IF0004_H__
#define __NVIF_IF0004_H__
-#define NV04_NVSW_NTFY_UEVENT 0x00
+union nv04_nvsw_event_args {
+ struct nv04_nvsw_event_vn {
+ } vn;
+};
#define NV04_NVSW_GET_REF 0x00
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if000e.h b/drivers/gpu/drm/nouveau/include/nvif/if000e.h
new file mode 100644
index 0000000..90a936c
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvif/if000e.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVIF_IF000E_H__
+#define __NVIF_IF000E_H__
+
+union nvif_event_args {
+ struct nvif_event_v0 {
+ __u8 version;
+ __u8 wait;
+ __u8 pad02[6];
+ __u8 data[];
+ } v0;
+};
+
+#define NVIF_EVENT_V0_ALLOW 0x00
+#define NVIF_EVENT_V0_BLOCK 0x01
+
+union nvif_event_allow_args {
+ struct nvif_event_allow_vn {
+ } vn;
+};
+
+union nvif_event_block_args {
+ struct nvif_event_block_vn {
+ } vn;
+};
+#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if0010.h b/drivers/gpu/drm/nouveau/include/nvif/if0010.h
index fc236ef..4c835bb 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/if0010.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/if0010.h
@@ -8,6 +8,7 @@ union nvif_disp_args {
__u8 pad01[3];
__u32 conn_mask;
__u32 outp_mask;
+ __u32 head_mask;
} v0;
};
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if0011.h b/drivers/gpu/drm/nouveau/include/nvif/if0011.h
index 04ba658..69b0b77 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/if0011.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/if0011.h
@@ -10,6 +10,17 @@ union nvif_conn_args {
} v0;
};
+union nvif_conn_event_args {
+ struct nvif_conn_event_v0 {
+ __u8 version;
+#define NVIF_CONN_EVENT_V0_PLUG 0x01
+#define NVIF_CONN_EVENT_V0_UNPLUG 0x02
+#define NVIF_CONN_EVENT_V0_IRQ 0x04
+ __u8 types;
+ __u8 pad02[6];
+ } v0;
+};
+
#define NVIF_CONN_V0_HPD_STATUS 0x00000000
union nvif_conn_hpd_status_args {
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if0012.h b/drivers/gpu/drm/nouveau/include/nvif/if0012.h
index 243bd35..eb99d84 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/if0012.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/if0012.h
@@ -11,6 +11,13 @@ union nvif_outp_args {
};
#define NVIF_OUTP_V0_LOAD_DETECT 0x00
+#define NVIF_OUTP_V0_ACQUIRE 0x01
+#define NVIF_OUTP_V0_RELEASE 0x02
+#define NVIF_OUTP_V0_INFOFRAME 0x03
+#define NVIF_OUTP_V0_HDA_ELD 0x04
+#define NVIF_OUTP_V0_DP_AUX_PWR 0x05
+#define NVIF_OUTP_V0_DP_RETRAIN 0x06
+#define NVIF_OUTP_V0_DP_MST_VCPI 0x07
union nvif_outp_load_detect_args {
struct nvif_outp_load_detect_v0 {
@@ -20,4 +27,95 @@ union nvif_outp_load_detect_args {
__u32 data; /*TODO: move vbios loadval parsing into nvkm */
} v0;
};
+
+union nvif_outp_acquire_args {
+ struct nvif_outp_acquire_v0 {
+ __u8 version;
+#define NVIF_OUTP_ACQUIRE_V0_RGB_CRT 0x00
+#define NVIF_OUTP_ACQUIRE_V0_TV 0x01
+#define NVIF_OUTP_ACQUIRE_V0_TMDS 0x02
+#define NVIF_OUTP_ACQUIRE_V0_LVDS 0x03
+#define NVIF_OUTP_ACQUIRE_V0_DP 0x04
+ __u8 proto;
+ __u8 or;
+ __u8 link;
+ __u8 pad04[4];
+ union {
+ struct {
+ __u8 head;
+ __u8 hdmi;
+ __u8 hdmi_max_ac_packet;
+ __u8 hdmi_rekey;
+#define NVIF_OUTP_ACQUIRE_V0_TMDS_HDMI_SCDC_SCRAMBLE (1 << 0)
+#define NVIF_OUTP_ACQUIRE_V0_TMDS_HDMI_SCDC_DIV_BY_4 (1 << 1)
+ __u8 hdmi_scdc;
+ __u8 hdmi_hda;
+ __u8 pad06[2];
+ } tmds;
+ struct {
+ __u8 dual;
+ __u8 bpc8;
+ __u8 pad02[6];
+ } lvds;
+ struct {
+ __u8 link_nr; /* 0 = highest possible. */
+ __u8 link_bw; /* 0 = highest possible, DP BW code otherwise. */
+ __u8 hda;
+ __u8 mst;
+ __u8 pad04[4];
+ __u8 dpcd[16];
+ } dp;
+ };
+ } v0;
+};
+
+union nvif_outp_release_args {
+ struct nvif_outp_release_vn {
+ } vn;
+};
+
+union nvif_outp_infoframe_args {
+ struct nvif_outp_infoframe_v0 {
+ __u8 version;
+#define NVIF_OUTP_INFOFRAME_V0_AVI 0
+#define NVIF_OUTP_INFOFRAME_V0_VSI 1
+ __u8 type;
+ __u8 head;
+ __u8 pad03[5];
+ __u8 data[];
+ } v0;
+};
+
+union nvif_outp_hda_eld_args {
+ struct nvif_outp_hda_eld_v0 {
+ __u8 version;
+ __u8 head;
+ __u8 pad02[6];
+ __u8 data[];
+ } v0;
+};
+
+union nvif_outp_dp_aux_pwr_args {
+ struct nvif_outp_dp_aux_pwr_v0 {
+ __u8 version;
+ __u8 state;
+ __u8 pad02[6];
+ } v0;
+};
+
+union nvif_outp_dp_retrain_args {
+ struct nvif_outp_dp_retrain_vn {
+ } vn;
+};
+
+union nvif_outp_dp_mst_vcpi_args {
+ struct nvif_outp_dp_mst_vcpi_v0 {
+ __u8 version;
+ __u8 head;
+ __u8 start_slot;
+ __u8 num_slots;
+ __u16 pbn;
+ __u16 aligned_pbn;
+ } v0;
+};
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if0013.h b/drivers/gpu/drm/nouveau/include/nvif/if0013.h
new file mode 100644
index 0000000..6756c74
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvif/if0013.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVIF_IF0013_H__
+#define __NVIF_IF0013_H__
+
+union nvif_head_args {
+ struct nvif_head_v0 {
+ __u8 version;
+ __u8 id;
+ __u8 pad02[6];
+ } v0;
+};
+
+union nvif_head_event_args {
+ struct nvif_head_event_vn {
+ } vn;
+};
+
+#define NVIF_HEAD_V0_SCANOUTPOS 0x00
+
+union nvif_head_scanoutpos_args {
+ struct nvif_head_scanoutpos_v0 {
+ __u8 version;
+ __u8 pad01[7];
+ __s64 time[2];
+ __u16 vblanks;
+ __u16 vblanke;
+ __u16 vtotal;
+ __u16 vline;
+ __u16 hblanks;
+ __u16 hblanke;
+ __u16 htotal;
+ __u16 hline;
+ } v0;
+};
+#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if0020.h b/drivers/gpu/drm/nouveau/include/nvif/if0020.h
new file mode 100644
index 0000000..085e0ae
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvif/if0020.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVIF_IF0020_H__
+#define __NVIF_IF0020_H__
+
+union nvif_chan_args {
+ struct nvif_chan_v0 {
+ __u8 version;
+ __u8 namelen;
+ __u8 runlist;
+ __u8 runq;
+ __u8 priv;
+ __u8 pad05;
+ __u16 devm;
+ __u64 vmm;
+
+ __u64 ctxdma;
+ __u64 offset;
+ __u64 length;
+
+ __u64 huserd;
+ __u64 ouserd;
+
+ __u32 token;
+ __u16 chid;
+ __u8 pad3e;
+#define NVIF_CHAN_V0_INST_APER_VRAM 0
+#define NVIF_CHAN_V0_INST_APER_HOST 1
+#define NVIF_CHAN_V0_INST_APER_NCOH 2
+#define NVIF_CHAN_V0_INST_APER_INST 0xff
+ __u8 aper;
+ __u64 inst;
+
+ __u8 name[];
+ } v0;
+};
+
+union nvif_chan_event_args {
+ struct nvif_chan_event_v0 {
+ __u8 version;
+#define NVIF_CHAN_EVENT_V0_NON_STALL_INTR 0x00
+#define NVIF_CHAN_EVENT_V0_KILLED 0x01
+ __u8 type;
+ } v0;
+};
+#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/if0021.h b/drivers/gpu/drm/nouveau/include/nvif/if0021.h
new file mode 100644
index 0000000..5013def90
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvif/if0021.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVIF_IF0021_H__
+#define __NVIF_IF0021_H__
+
+union nvif_cgrp_args {
+ struct nvif_cgrp_v0 {
+ __u8 version;
+ __u8 namelen;
+ __u8 runlist;
+ __u8 pad03[3];
+ __u16 cgid;
+ __u64 vmm;
+ __u8 name[];
+ } v0;
+};
+#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/ioctl.h b/drivers/gpu/drm/nouveau/include/nvif/ioctl.h
index 886c63f..4e047bb 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/ioctl.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/ioctl.h
@@ -15,10 +15,6 @@ struct nvif_ioctl_v0 {
#define NVIF_IOCTL_V0_WR 0x06
#define NVIF_IOCTL_V0_MAP 0x07
#define NVIF_IOCTL_V0_UNMAP 0x08
-#define NVIF_IOCTL_V0_NTFY_NEW 0x09
-#define NVIF_IOCTL_V0_NTFY_DEL 0x0a
-#define NVIF_IOCTL_V0_NTFY_GET 0x0b
-#define NVIF_IOCTL_V0_NTFY_PUT 0x0c
__u8 type;
__u8 pad02[4];
#define NVIF_IOCTL_V0_OWNER_NVIF 0x00
@@ -63,6 +59,14 @@ struct nvif_ioctl_new_v0 {
struct nvif_ioctl_del {
};
+struct nvif_ioctl_mthd_v0 {
+ /* nvif_ioctl ... */
+ __u8 version;
+ __u8 method;
+ __u8 pad02[6];
+ __u8 data[]; /* method data (class.h) */
+};
+
struct nvif_ioctl_rd_v0 {
/* nvif_ioctl ... */
__u8 version;
@@ -95,43 +99,4 @@ struct nvif_ioctl_map_v0 {
struct nvif_ioctl_unmap {
};
-
-struct nvif_ioctl_ntfy_new_v0 {
- /* nvif_ioctl ... */
- __u8 version;
- __u8 event;
- __u8 index;
- __u8 pad03[5];
- __u8 data[]; /* event request data (event.h) */
-};
-
-struct nvif_ioctl_ntfy_del_v0 {
- /* nvif_ioctl ... */
- __u8 version;
- __u8 index;
- __u8 pad02[6];
-};
-
-struct nvif_ioctl_ntfy_get_v0 {
- /* nvif_ioctl ... */
- __u8 version;
- __u8 index;
- __u8 pad02[6];
-};
-
-struct nvif_ioctl_ntfy_put_v0 {
- /* nvif_ioctl ... */
- __u8 version;
- __u8 index;
- __u8 pad02[6];
-};
-
-struct nvif_ioctl_mthd_v0 {
- /* nvif_ioctl ... */
- __u8 version;
- __u8 method;
- __u8 pad02[6];
- __u8 data[]; /* method data (class.h) */
-};
-
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/notify.h b/drivers/gpu/drm/nouveau/include/nvif/notify.h
deleted file mode 100644
index 39f6b7e..0000000
--- a/drivers/gpu/drm/nouveau/include/nvif/notify.h
+++ /dev/null
@@ -1,35 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVIF_NOTIFY_H__
-#define __NVIF_NOTIFY_H__
-
-struct nvif_notify {
- struct nvif_object *object;
- const char *name;
- int index;
-
-#define NVIF_NOTIFY_USER 0
-#define NVIF_NOTIFY_WORK 1
- unsigned long flags;
- atomic_t putcnt;
- void (*dtor)(struct nvif_notify *);
-#define NVIF_NOTIFY_DROP 0
-#define NVIF_NOTIFY_KEEP 1
- int (*func)(struct nvif_notify *);
-
- /* this is const for a *very* good reason - the data might be on the
- * stack from an irq handler. if you're not nvif/notify.c then you
- * should probably think twice before casting it away...
- */
- const void *data;
- u32 size;
- struct work_struct work;
-};
-
-int nvif_notify_ctor(struct nvif_object *, const char *name,
- int (*func)(struct nvif_notify *), bool work, u8 type,
- void *data, u32 size, u32 reply, struct nvif_notify *);
-int nvif_notify_dtor(struct nvif_notify *);
-int nvif_notify_get(struct nvif_notify *);
-int nvif_notify_put(struct nvif_notify *);
-int nvif_notify(const void *, u32, const void *, u32);
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvif/outp.h b/drivers/gpu/drm/nouveau/include/nvif/outp.h
index 0d6aa07..45daadec3 100644
--- a/drivers/gpu/drm/nouveau/include/nvif/outp.h
+++ b/drivers/gpu/drm/nouveau/include/nvif/outp.h
@@ -2,13 +2,32 @@
#ifndef __NVIF_OUTP_H__
#define __NVIF_OUTP_H__
#include <nvif/object.h>
+#include <nvif/if0012.h>
struct nvif_disp;
struct nvif_outp {
struct nvif_object object;
+
+ struct {
+ int id;
+ int link;
+ } or;
};
int nvif_outp_ctor(struct nvif_disp *, const char *name, int id, struct nvif_outp *);
void nvif_outp_dtor(struct nvif_outp *);
int nvif_outp_load_detect(struct nvif_outp *, u32 loadval);
+int nvif_outp_acquire_rgb_crt(struct nvif_outp *);
+int nvif_outp_acquire_tmds(struct nvif_outp *, int head,
+ bool hdmi, u8 max_ac_packet, u8 rekey, u8 scdc, bool hda);
+int nvif_outp_acquire_lvds(struct nvif_outp *, bool dual, bool bpc8);
+int nvif_outp_acquire_dp(struct nvif_outp *, u8 dpcd[16],
+ int link_nr, int link_bw, bool hda, bool mst);
+void nvif_outp_release(struct nvif_outp *);
+int nvif_outp_infoframe(struct nvif_outp *, u8 type, struct nvif_outp_infoframe_v0 *, u32 size);
+int nvif_outp_hda_eld(struct nvif_outp *, int head, void *data, u32 size);
+int nvif_outp_dp_aux_pwr(struct nvif_outp *, bool enable);
+int nvif_outp_dp_retrain(struct nvif_outp *);
+int nvif_outp_dp_mst_vcpi(struct nvif_outp *, int head,
+ u8 start_slot, u8 num_slots, u16 pbn, u16 aligned_pbn);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/client.h b/drivers/gpu/drm/nouveau/include/nvkm/core/client.h
index 2f86606..0d9fc74 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/client.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/client.h
@@ -10,28 +10,19 @@ struct nvkm_client {
u64 device;
u32 debug;
- struct nvkm_client_notify *notify[32];
struct rb_root objroot;
void *data;
- int (*ntfy)(const void *, u32, const void *, u32);
+ int (*event)(u64 token, void *argv, u32 argc);
struct list_head umem;
spinlock_t lock;
};
-int nvkm_client_new(const char *name, u64 device, const char *cfg,
- const char *dbg,
- int (*)(const void *, u32, const void *, u32),
- struct nvkm_client **);
+int nvkm_client_new(const char *name, u64 device, const char *cfg, const char *dbg,
+ int (*)(u64, void *, u32), struct nvkm_client **);
struct nvkm_client *nvkm_client_search(struct nvkm_client *, u64 handle);
-int nvkm_client_notify_new(struct nvkm_object *, struct nvkm_event *,
- void *data, u32 size);
-int nvkm_client_notify_del(struct nvkm_client *, int index);
-int nvkm_client_notify_get(struct nvkm_client *, int index);
-int nvkm_client_notify_put(struct nvkm_client *, int index);
-
/* logging for client-facing objects */
#define nvif_printk(o,l,p,f,a...) do { \
const struct nvkm_object *_object = (o); \
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/device.h b/drivers/gpu/drm/nouveau/include/nvkm/core/device.h
index efede1f..f65b500 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/device.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/device.h
@@ -2,6 +2,7 @@
#ifndef __NVKM_DEVICE_H__
#define __NVKM_DEVICE_H__
#include <core/oclass.h>
+#include <core/intr.h>
enum nvkm_subdev_type;
enum nvkm_device_type {
@@ -60,6 +61,16 @@ struct nvkm_device {
#undef NVKM_LAYOUT_INST
#undef NVKM_LAYOUT_ONCE
struct list_head subdev;
+
+ struct {
+ struct list_head intr;
+ struct list_head prio[NVKM_INTR_PRIO_NR];
+ spinlock_t lock;
+ int irq;
+ bool alloc;
+ bool armed;
+ bool legacy_done;
+ } intr;
};
struct nvkm_subdev *nvkm_device_subdev(struct nvkm_device *, int type, int inst);
@@ -72,6 +83,7 @@ struct nvkm_device_func {
int (*preinit)(struct nvkm_device *);
int (*init)(struct nvkm_device *);
void (*fini)(struct nvkm_device *, bool suspend);
+ int (*irq)(struct nvkm_device *);
resource_size_t (*resource_addr)(struct nvkm_device *, unsigned bar);
resource_size_t (*resource_size)(struct nvkm_device *, unsigned bar);
bool cpu_coherent;
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h b/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h
index e58923b..b67b9c1 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h
@@ -12,12 +12,6 @@ struct nvkm_engine {
const struct nvkm_engine_func *func;
struct nvkm_subdev subdev;
spinlock_t lock;
-
- struct {
- refcount_t refcount;
- struct mutex mutex;
- bool enabled;
- } use;
};
struct nvkm_engine_func {
@@ -27,6 +21,7 @@ struct nvkm_engine_func {
int (*info)(struct nvkm_engine *, u64 mthd, u64 *data);
int (*init)(struct nvkm_engine *);
int (*fini)(struct nvkm_engine *, bool suspend);
+ int (*reset)(struct nvkm_engine *);
void (*intr)(struct nvkm_engine *);
void (*tile)(struct nvkm_engine *, int region, struct nvkm_fb_tile *);
bool (*chsw_load)(struct nvkm_engine *);
@@ -54,6 +49,7 @@ int nvkm_engine_new_(const struct nvkm_engine_func *, struct nvkm_device *,
struct nvkm_engine *nvkm_engine_ref(struct nvkm_engine *);
void nvkm_engine_unref(struct nvkm_engine **);
+int nvkm_engine_reset(struct nvkm_engine *);
void nvkm_engine_tile(struct nvkm_engine *, int region);
bool nvkm_engine_chsw_load(struct nvkm_engine *);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/event.h b/drivers/gpu/drm/nouveau/include/nvkm/core/event.h
index a7a413f..82b267c 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/event.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/event.h
@@ -2,34 +2,76 @@
#ifndef __NVKM_EVENT_H__
#define __NVKM_EVENT_H__
#include <core/os.h>
-struct nvkm_notify;
struct nvkm_object;
+struct nvkm_oclass;
+struct nvkm_uevent;
struct nvkm_event {
const struct nvkm_event_func *func;
+ struct nvkm_subdev *subdev;
int types_nr;
int index_nr;
spinlock_t refs_lock;
spinlock_t list_lock;
- struct list_head list;
int *refs;
+
+ struct list_head ntfy;
};
struct nvkm_event_func {
- int (*ctor)(struct nvkm_object *, void *data, u32 size,
- struct nvkm_notify *);
- void (*send)(void *data, u32 size, struct nvkm_notify *);
void (*init)(struct nvkm_event *, int type, int index);
void (*fini)(struct nvkm_event *, int type, int index);
};
-int nvkm_event_init(const struct nvkm_event_func *func, int types_nr,
- int index_nr, struct nvkm_event *);
+int __nvkm_event_init(const struct nvkm_event_func *func, struct nvkm_subdev *, int types_nr,
+ int index_nr, struct nvkm_event *);
+
+/* Each nvkm_event needs its own lockdep class due to inter-dependencies, to
+ * prevent lockdep false-positives.
+ *
+ * Inlining the spinlock initialisation ensures each is unique.
+ */
+static __always_inline int
+nvkm_event_init(const struct nvkm_event_func *func, struct nvkm_subdev *subdev,
+ int types_nr, int index_nr, struct nvkm_event *event)
+{
+ spin_lock_init(&event->refs_lock);
+ spin_lock_init(&event->list_lock);
+ return __nvkm_event_init(func, subdev, types_nr, index_nr, event);
+}
+
void nvkm_event_fini(struct nvkm_event *);
-void nvkm_event_get(struct nvkm_event *, u32 types, int index);
-void nvkm_event_put(struct nvkm_event *, u32 types, int index);
-void nvkm_event_send(struct nvkm_event *, u32 types, int index,
- void *data, u32 size);
+
+#define NVKM_EVENT_KEEP 0
+#define NVKM_EVENT_DROP 1
+struct nvkm_event_ntfy;
+typedef int (*nvkm_event_func)(struct nvkm_event_ntfy *, u32 bits);
+
+struct nvkm_event_ntfy {
+ struct nvkm_event *event;
+ int id;
+ u32 bits;
+ bool wait;
+ nvkm_event_func func;
+
+ atomic_t allowed;
+ bool running;
+
+ struct list_head head;
+};
+
+void nvkm_event_ntfy(struct nvkm_event *, int id, u32 bits);
+bool nvkm_event_ntfy_valid(struct nvkm_event *, int id, u32 bits);
+void nvkm_event_ntfy_add(struct nvkm_event *, int id, u32 bits, bool wait, nvkm_event_func,
+ struct nvkm_event_ntfy *);
+void nvkm_event_ntfy_del(struct nvkm_event_ntfy *);
+void nvkm_event_ntfy_allow(struct nvkm_event_ntfy *);
+void nvkm_event_ntfy_block(struct nvkm_event_ntfy *);
+
+typedef int (*nvkm_uevent_func)(struct nvkm_object *, u64 token, u32 bits);
+
+int nvkm_uevent_new(const struct nvkm_oclass *, void *argv, u32 argc, struct nvkm_object **);
+int nvkm_uevent_add(struct nvkm_uevent *, struct nvkm_event *, int id, u32 bits, nvkm_uevent_func);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/falcon.h b/drivers/gpu/drm/nouveau/include/nvkm/core/falcon.h
index fd9a3f9..b857cf1 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/falcon.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/falcon.h
@@ -1,34 +1,166 @@
#ifndef __NVKM_FALCON_H__
#define __NVKM_FALCON_H__
+#include <core/firmware.h>
#include <engine/falcon.h>
+enum nvkm_falcon_mem {
+ IMEM,
+ DMEM,
+ EMEM,
+};
+
+static inline const char *
+nvkm_falcon_mem(enum nvkm_falcon_mem mem)
+{
+ switch (mem) {
+ case IMEM: return "imem";
+ case DMEM: return "dmem";
+ case EMEM: return "emem";
+ default:
+ WARN_ON(1);
+ return "?mem";
+ }
+}
+
+struct nvkm_falcon_func_pio {
+ int min;
+ int max;
+ void (*wr_init)(struct nvkm_falcon *, u8 port, bool sec, u32 mem_base);
+ void (*wr)(struct nvkm_falcon *, u8 port, const u8 *img, int len, u16 tag);
+ void (*rd_init)(struct nvkm_falcon *, u8 port, u32 mem_base);
+ void (*rd)(struct nvkm_falcon *, u8 port, const u8 *img, int len);
+};
+
+struct nvkm_falcon_func_dma {
+ int (*init)(struct nvkm_falcon *, u64 dma_addr, int xfer_len,
+ enum nvkm_falcon_mem, bool sec, u32 *cmd);
+ void (*xfer)(struct nvkm_falcon *, u32 mem_base, u32 dma_base, u32 cmd);
+ bool (*done)(struct nvkm_falcon *);
+};
+
int nvkm_falcon_ctor(const struct nvkm_falcon_func *, struct nvkm_subdev *owner,
const char *name, u32 addr, struct nvkm_falcon *);
void nvkm_falcon_dtor(struct nvkm_falcon *);
+int nvkm_falcon_reset(struct nvkm_falcon *);
+int nvkm_falcon_pio_wr(struct nvkm_falcon *, const u8 *img, u32 img_base, u8 port,
+ enum nvkm_falcon_mem mem_type, u32 mem_base, int len, u16 tag, bool sec);
+int nvkm_falcon_pio_rd(struct nvkm_falcon *, u8 port, enum nvkm_falcon_mem type, u32 mem_base,
+ const u8 *img, u32 img_base, int len);
+int nvkm_falcon_dma_wr(struct nvkm_falcon *, const u8 *img, u64 dma_addr, u32 dma_base,
+ enum nvkm_falcon_mem mem_type, u32 mem_base, int len, bool sec);
+
+int gm200_flcn_reset_wait_mem_scrubbing(struct nvkm_falcon *);
+int gm200_flcn_disable(struct nvkm_falcon *);
+int gm200_flcn_enable(struct nvkm_falcon *);
+void gm200_flcn_bind_inst(struct nvkm_falcon *, int, u64);
+int gm200_flcn_bind_stat(struct nvkm_falcon *, bool);
+extern const struct nvkm_falcon_func_pio gm200_flcn_imem_pio;
+extern const struct nvkm_falcon_func_pio gm200_flcn_dmem_pio;
+void gm200_flcn_tracepc(struct nvkm_falcon *);
+
+int gp102_flcn_reset_eng(struct nvkm_falcon *);
+extern const struct nvkm_falcon_func_pio gp102_flcn_emem_pio;
+
+int ga102_flcn_select(struct nvkm_falcon *);
+int ga102_flcn_reset_prep(struct nvkm_falcon *);
+int ga102_flcn_reset_wait_mem_scrubbing(struct nvkm_falcon *);
+extern const struct nvkm_falcon_func_dma ga102_flcn_dma;
void nvkm_falcon_v1_load_imem(struct nvkm_falcon *,
void *, u32, u32, u16, u8, bool);
void nvkm_falcon_v1_load_dmem(struct nvkm_falcon *, void *, u32, u32, u8);
-void nvkm_falcon_v1_read_dmem(struct nvkm_falcon *, u32, u32, u8, void *);
-void nvkm_falcon_v1_bind_context(struct nvkm_falcon *, struct nvkm_memory *);
-int nvkm_falcon_v1_wait_for_halt(struct nvkm_falcon *, u32);
-int nvkm_falcon_v1_clear_interrupt(struct nvkm_falcon *, u32);
-void nvkm_falcon_v1_set_start_addr(struct nvkm_falcon *, u32 start_addr);
void nvkm_falcon_v1_start(struct nvkm_falcon *);
-int nvkm_falcon_v1_enable(struct nvkm_falcon *);
-void nvkm_falcon_v1_disable(struct nvkm_falcon *);
-void gp102_sec2_flcn_bind_context(struct nvkm_falcon *, struct nvkm_memory *);
-int gp102_sec2_flcn_enable(struct nvkm_falcon *);
+#define FLCN_PRINTK(f,l,p,fmt,a...) ({ \
+ if ((f)->owner->name != (f)->name) \
+ nvkm_printk___((f)->owner, (f)->user, NV_DBG_##l, p, "%s:"fmt, (f)->name, ##a); \
+ else \
+ nvkm_printk___((f)->owner, (f)->user, NV_DBG_##l, p, fmt, ##a); \
+})
+#define FLCN_DBG(f,fmt,a...) FLCN_PRINTK((f), DEBUG, info, " "fmt"\n", ##a)
+#define FLCN_ERR(f,fmt,a...) FLCN_PRINTK((f), ERROR, err, " "fmt"\n", ##a)
+#define FLCN_ERRON(f,c,fmt,a...) \
+ ({ bool _cond = (c); _cond ? FLCN_ERR(f, fmt, ##a) : FLCN_DBG(f, fmt, ##a); _cond; })
-#define FLCN_PRINTK(t,f,fmt,a...) do { \
- if ((f)->owner->name != (f)->name) \
- nvkm_##t((f)->owner, "%s: "fmt"\n", (f)->name, ##a); \
- else \
- nvkm_##t((f)->owner, fmt"\n", ##a); \
-} while(0)
-#define FLCN_DBG(f,fmt,a...) FLCN_PRINTK(debug, (f), fmt, ##a)
-#define FLCN_ERR(f,fmt,a...) FLCN_PRINTK(error, (f), fmt, ##a)
+
+struct nvkm_falcon_fw {
+ const struct nvkm_falcon_fw_func {
+ int (*signature)(struct nvkm_falcon_fw *, u32 *sig_base_src);
+ int (*reset)(struct nvkm_falcon_fw *);
+ int (*setup)(struct nvkm_falcon_fw *);
+ int (*load)(struct nvkm_falcon_fw *);
+ int (*load_bld)(struct nvkm_falcon_fw *);
+ int (*boot)(struct nvkm_falcon_fw *,
+ u32 *mbox0, u32 *mbox1, u32 mbox0_ok, u32 irqsclr);
+ } *func;
+ struct nvkm_firmware fw;
+
+ u32 sig_base_prd;
+ u32 sig_base_dbg;
+ u32 sig_base_img;
+ u32 sig_size;
+ int sig_nr;
+ u8 *sigs;
+ u32 fuse_ver;
+ u32 engine_id;
+ u32 ucode_id;
+
+ u32 nmem_base_img;
+ u32 nmem_base;
+ u32 nmem_size;
+
+ u32 imem_base_img;
+ u32 imem_base;
+ u32 imem_size;
+
+ u32 dmem_base_img;
+ u32 dmem_base;
+ u32 dmem_size;
+ u32 dmem_sign;
+
+ u8 *boot;
+ u32 boot_size;
+ u32 boot_addr;
+
+ struct nvkm_falcon *falcon;
+ struct nvkm_memory *inst;
+ struct nvkm_vmm *vmm;
+ struct nvkm_vma *vma;
+};
+
+int nvkm_falcon_fw_ctor(const struct nvkm_falcon_fw_func *, const char *name, struct nvkm_device *,
+ bool bl, const void *src, u32 len, struct nvkm_falcon *,
+ struct nvkm_falcon_fw *);
+int nvkm_falcon_fw_ctor_hs(const struct nvkm_falcon_fw_func *, const char *name,
+ struct nvkm_subdev *, const char *bl, const char *img, int ver,
+ struct nvkm_falcon *falcon, struct nvkm_falcon_fw *fw);
+int nvkm_falcon_fw_ctor_hs_v2(const struct nvkm_falcon_fw_func *, const char *name,
+ struct nvkm_subdev *, const char *img, int ver, struct nvkm_falcon *,
+ struct nvkm_falcon_fw *);
+int nvkm_falcon_fw_sign(struct nvkm_falcon_fw *, u32 sig_base_img, u32 sig_size, const u8 *sigs,
+ int sig_nr_prd, u32 sig_base_prd, int sig_nr_dbg, u32 sig_base_dbg);
+int nvkm_falcon_fw_patch(struct nvkm_falcon_fw *);
+void nvkm_falcon_fw_dtor(struct nvkm_falcon_fw *);
+int nvkm_falcon_fw_oneinit(struct nvkm_falcon_fw *, struct nvkm_falcon *, struct nvkm_vmm *,
+ struct nvkm_memory *inst);
+int nvkm_falcon_fw_boot(struct nvkm_falcon_fw *, struct nvkm_subdev *user,
+ bool release, u32 *pmbox0, u32 *pmbox1, u32 mbox0_ok, u32 irqsclr);
+
+extern const struct nvkm_falcon_fw_func gm200_flcn_fw;
+int gm200_flcn_fw_signature(struct nvkm_falcon_fw *, u32 *);
+int gm200_flcn_fw_reset(struct nvkm_falcon_fw *);
+int gm200_flcn_fw_load(struct nvkm_falcon_fw *);
+int gm200_flcn_fw_boot(struct nvkm_falcon_fw *, u32 *, u32 *, u32, u32);
+
+int ga100_flcn_fw_signature(struct nvkm_falcon_fw *, u32 *);
+
+extern const struct nvkm_falcon_fw_func ga102_flcn_fw;
+int ga102_flcn_fw_load(struct nvkm_falcon_fw *);
+int ga102_flcn_fw_boot(struct nvkm_falcon_fw *, u32 *, u32 *, u32, u32);
+
+#define FLCNFW_PRINTK(f,l,p,fmt,a...) FLCN_PRINTK((f)->falcon, l, p, "%s: "fmt, (f)->fw.name, ##a)
+#define FLCNFW_DBG(f,fmt,a...) FLCNFW_PRINTK((f), DEBUG, info, fmt"\n", ##a)
+#define FLCNFW_ERR(f,fmt,a...) FLCNFW_PRINTK((f), ERROR, err, fmt"\n", ##a)
/**
* struct nvfw_falcon_msg - header for all messages
@@ -72,6 +204,7 @@ int nvkm_falcon_msgq_new(struct nvkm_falcon_qmgr *, const char *name,
void nvkm_falcon_msgq_del(struct nvkm_falcon_msgq **);
void nvkm_falcon_msgq_init(struct nvkm_falcon_msgq *,
u32 index, u32 offset, u32 size);
+bool nvkm_falcon_msgq_empty(struct nvkm_falcon_msgq *);
int nvkm_falcon_msgq_recv_initmsg(struct nvkm_falcon_msgq *, void *, u32 size);
void nvkm_falcon_msgq_recv(struct nvkm_falcon_msgq *);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/firmware.h b/drivers/gpu/drm/nouveau/include/nvkm/core/firmware.h
index 85bcb80..d4e507e 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/firmware.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/firmware.h
@@ -1,9 +1,34 @@
/* SPDX-License-Identifier: MIT */
#ifndef __NVKM_FIRMWARE_H__
#define __NVKM_FIRMWARE_H__
+#include <core/memory.h>
#include <core/option.h>
#include <core/subdev.h>
+struct nvkm_firmware {
+ const struct nvkm_firmware_func {
+ enum nvkm_firmware_type {
+ NVKM_FIRMWARE_IMG_RAM,
+ NVKM_FIRMWARE_IMG_DMA,
+ } type;
+ } *func;
+ const char *name;
+ struct nvkm_device *device;
+
+ int len;
+ u8 *img;
+ u64 phys;
+
+ struct nvkm_firmware_mem {
+ struct nvkm_memory memory;
+ struct scatterlist sgl;
+ } mem;
+};
+
+int nvkm_firmware_ctor(const struct nvkm_firmware_func *, const char *name, struct nvkm_device *,
+ const void *ptr, int len, struct nvkm_firmware *);
+void nvkm_firmware_dtor(struct nvkm_firmware *);
+
int nvkm_firmware_get(const struct nvkm_subdev *, const char *fwname, int ver,
const struct firmware **);
void nvkm_firmware_put(const struct firmware *);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/intr.h b/drivers/gpu/drm/nouveau/include/nvkm/core/intr.h
new file mode 100644
index 0000000..a003d6a
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/intr.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVKM_INTR_H__
+#define __NVKM_INTR_H__
+#include <core/os.h>
+struct nvkm_device;
+struct nvkm_subdev;
+
+enum nvkm_intr_prio {
+ NVKM_INTR_PRIO_VBLANK = 0,
+ NVKM_INTR_PRIO_NORMAL,
+ NVKM_INTR_PRIO_NR
+};
+
+enum nvkm_intr_type {
+ NVKM_INTR_SUBDEV = -1, /* lookup vector by requesting subdev, in mapping table. */
+ NVKM_INTR_VECTOR_0 = 0,
+};
+
+struct nvkm_intr {
+ const struct nvkm_intr_func {
+ bool (*pending)(struct nvkm_intr *);
+ void (*unarm)(struct nvkm_intr *);
+ void (*rearm)(struct nvkm_intr *);
+ void (*block)(struct nvkm_intr *, int leaf, u32 mask);
+ void (*allow)(struct nvkm_intr *, int leaf, u32 mask);
+ void (*reset)(struct nvkm_intr *, int leaf, u32 mask);
+ } *func;
+ const struct nvkm_intr_data {
+ int type; /* enum nvkm_subdev_type (+ve), enum nvkm_intr_type (-ve) */
+ int inst;
+ int leaf;
+ u32 mask; /* 0-terminated. */
+ bool legacy; /* auto-create "legacy" nvkm_subdev_intr() handler */
+ } *data;
+
+ struct nvkm_subdev *subdev;
+ int leaves;
+ u32 *stat;
+ u32 *mask;
+
+ struct list_head head;
+};
+
+void nvkm_intr_ctor(struct nvkm_device *);
+void nvkm_intr_dtor(struct nvkm_device *);
+int nvkm_intr_install(struct nvkm_device *);
+void nvkm_intr_unarm(struct nvkm_device *);
+void nvkm_intr_rearm(struct nvkm_device *);
+
+int nvkm_intr_add(const struct nvkm_intr_func *, const struct nvkm_intr_data *,
+ struct nvkm_subdev *, int leaves, struct nvkm_intr *);
+void nvkm_intr_block(struct nvkm_subdev *, enum nvkm_intr_type);
+void nvkm_intr_allow(struct nvkm_subdev *, enum nvkm_intr_type);
+
+struct nvkm_inth;
+typedef irqreturn_t (*nvkm_inth_func)(struct nvkm_inth *);
+
+struct nvkm_inth {
+ struct nvkm_intr *intr;
+ int leaf;
+ u32 mask;
+ nvkm_inth_func func;
+
+ atomic_t allowed;
+
+ struct list_head head;
+};
+
+int nvkm_inth_add(struct nvkm_intr *, enum nvkm_intr_type, enum nvkm_intr_prio,
+ struct nvkm_subdev *, nvkm_inth_func, struct nvkm_inth *);
+void nvkm_inth_allow(struct nvkm_inth *);
+void nvkm_inth_block(struct nvkm_inth *);
+#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/layout.h b/drivers/gpu/drm/nouveau/include/nvkm/core/layout.h
index 7afe157..58108de 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/layout.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/layout.h
@@ -1,8 +1,10 @@
/* SPDX-License-Identifier: MIT */
+NVKM_LAYOUT_ONCE(NVKM_SUBDEV_TOP , struct nvkm_top , top)
+NVKM_LAYOUT_ONCE(NVKM_SUBDEV_GSP , struct nvkm_gsp , gsp)
+NVKM_LAYOUT_ONCE(NVKM_SUBDEV_VFN , struct nvkm_vfn , vfn)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_PCI , struct nvkm_pci , pci)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_VBIOS , struct nvkm_bios , bios)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_DEVINIT , struct nvkm_devinit , devinit)
-NVKM_LAYOUT_ONCE(NVKM_SUBDEV_TOP , struct nvkm_top , top)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_PRIVRING, struct nvkm_subdev , privring)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_GPIO , struct nvkm_gpio , gpio)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_I2C , struct nvkm_i2c , i2c)
@@ -23,7 +25,6 @@ NVKM_LAYOUT_ONCE(NVKM_SUBDEV_VOLT , struct nvkm_volt , volt)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_ICCSENSE, struct nvkm_iccsense, iccsense)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_THERM , struct nvkm_therm , therm)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_CLK , struct nvkm_clk , clk)
-NVKM_LAYOUT_ONCE(NVKM_SUBDEV_GSP , struct nvkm_gsp , gsp)
NVKM_LAYOUT_INST(NVKM_SUBDEV_IOCTRL , struct nvkm_subdev , ioctrl, 3)
NVKM_LAYOUT_ONCE(NVKM_SUBDEV_FLA , struct nvkm_subdev , fla)
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/memory.h b/drivers/gpu/drm/nouveau/include/nvkm/core/memory.h
index 74d3f1a..d3b6a68d 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/memory.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/memory.h
@@ -37,6 +37,7 @@ struct nvkm_memory_func {
void (*release)(struct nvkm_memory *);
int (*map)(struct nvkm_memory *, u64 offset, struct nvkm_vmm *,
struct nvkm_vma *, void *argv, u32 argc);
+ int (*kmap)(struct nvkm_memory *, struct nvkm_memory **);
};
struct nvkm_memory_ptrs {
@@ -63,6 +64,7 @@ void nvkm_memory_tags_put(struct nvkm_memory *, struct nvkm_device *,
#define nvkm_memory_boot(p,v) (p)->func->boot((p),(v))
#define nvkm_memory_map(p,o,vm,va,av,ac) \
(p)->func->map((p),(o),(vm),(va),(av),(ac))
+#define nvkm_memory_kmap(p,i) ((p)->func->kmap ? (p)->func->kmap((p), (i)) : -ENOSYS)
/* accessor macros - kmap()/done() must bracket use of the other accessor
* macros to guarantee correct behaviour across all chipsets
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/notify.h b/drivers/gpu/drm/nouveau/include/nvkm/core/notify.h
deleted file mode 100644
index 3d358a6..0000000
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/notify.h
+++ /dev/null
@@ -1,39 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NVKM_NOTIFY_H__
-#define __NVKM_NOTIFY_H__
-#include <core/os.h>
-struct nvkm_object;
-
-struct nvkm_notify {
- struct nvkm_event *event;
- struct list_head head;
-#define NVKM_NOTIFY_USER 0
-#define NVKM_NOTIFY_WORK 1
- unsigned long flags;
- int block;
-#define NVKM_NOTIFY_DROP 0
-#define NVKM_NOTIFY_KEEP 1
- int (*func)(struct nvkm_notify *);
-
- /* set by nvkm_event ctor */
- u32 types;
- int index;
- u32 size;
-
- struct work_struct work;
- /* this is const for a *very* good reason - the data might be on the
- * stack from an irq handler. if you're not core/notify.c then you
- * should probably think twice before casting it away...
- */
- const void *data;
-};
-
-int nvkm_notify_init(struct nvkm_object *, struct nvkm_event *,
- int (*func)(struct nvkm_notify *), bool work,
- void *data, u32 size, u32 reply,
- struct nvkm_notify *);
-void nvkm_notify_fini(struct nvkm_notify *);
-void nvkm_notify_get(struct nvkm_notify *);
-void nvkm_notify_put(struct nvkm_notify *);
-void nvkm_notify_send(struct nvkm_notify *, void *data, u32 size);
-#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/object.h b/drivers/gpu/drm/nouveau/include/nvkm/core/object.h
index 7efcd5d..ed1f663 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/object.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/object.h
@@ -4,6 +4,7 @@
#include <core/oclass.h>
struct nvkm_event;
struct nvkm_gpuobj;
+struct nvkm_uevent;
struct nvkm_object {
const struct nvkm_object_func *func;
@@ -43,6 +44,7 @@ struct nvkm_object_func {
int (*bind)(struct nvkm_object *, struct nvkm_gpuobj *, int align,
struct nvkm_gpuobj **);
int (*sclass)(struct nvkm_object *, int index, struct nvkm_oclass *);
+ int (*uevent)(struct nvkm_object *, void *argv, u32 argc, struct nvkm_uevent *);
};
void nvkm_object_ctor(const struct nvkm_object_func *,
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/os.h b/drivers/gpu/drm/nouveau/include/nvkm/core/os.h
index d7ba320..4486d98 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/os.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/os.h
@@ -34,4 +34,24 @@ nvkm_blob_dtor(struct nvkm_blob *blob)
blob->data = NULL;
blob->size = 0;
}
+
+#define nvkm_list_find_next(p,h,m,c) ({ \
+ typeof(p) _p = NULL; \
+ list_for_each_entry_continue(p, (h), m) { \
+ if (c) { \
+ _p = p; \
+ break; \
+ } \
+ } \
+ _p; \
+})
+#define nvkm_list_find(p,h,m,c) \
+ (p = container_of((h), typeof(*p), m), nvkm_list_find_next(p, (h), m, (c)))
+#define nvkm_list_foreach(p,h,m,c) \
+ for (p = nvkm_list_find(p, (h), m, (c)); p; p = nvkm_list_find_next(p, (h), m, (c)))
+
+/*FIXME: remove after */
+#define nvkm_fifo_chan nvkm_chan
+#define nvkm_fifo_chan_func nvkm_chan_func
+#define nvkm_fifo_cgrp nvkm_cgrp
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h b/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h
index 96113c8..bce6e1b 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h
@@ -17,10 +17,19 @@ struct nvkm_subdev {
struct nvkm_device *device;
enum nvkm_subdev_type type;
int inst;
+
char name[16];
u32 debug;
- struct list_head head;
+ struct {
+ refcount_t refcount;
+ struct mutex mutex;
+ bool enabled;
+ } use;
+
+ struct nvkm_inth inth;
+
+ struct list_head head;
void **pself;
bool oneinit;
};
@@ -38,22 +47,41 @@ struct nvkm_subdev_func {
extern const char *nvkm_subdev_type[NVKM_SUBDEV_NR];
int nvkm_subdev_new_(const struct nvkm_subdev_func *, struct nvkm_device *, enum nvkm_subdev_type,
int inst, struct nvkm_subdev **);
-void nvkm_subdev_ctor(const struct nvkm_subdev_func *, struct nvkm_device *,
- enum nvkm_subdev_type, int inst, struct nvkm_subdev *);
+void __nvkm_subdev_ctor(const struct nvkm_subdev_func *, struct nvkm_device *,
+ enum nvkm_subdev_type, int inst, struct nvkm_subdev *);
+
+static inline void
+nvkm_subdev_ctor(const struct nvkm_subdev_func *func, struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_subdev *subdev)
+{
+ __nvkm_subdev_ctor(func, device, type, inst, subdev);
+ mutex_init(&subdev->use.mutex);
+}
+
void nvkm_subdev_disable(struct nvkm_device *, enum nvkm_subdev_type, int inst);
void nvkm_subdev_del(struct nvkm_subdev **);
+int nvkm_subdev_ref(struct nvkm_subdev *);
+void nvkm_subdev_unref(struct nvkm_subdev *);
int nvkm_subdev_preinit(struct nvkm_subdev *);
+int nvkm_subdev_oneinit(struct nvkm_subdev *);
int nvkm_subdev_init(struct nvkm_subdev *);
int nvkm_subdev_fini(struct nvkm_subdev *, bool suspend);
int nvkm_subdev_info(struct nvkm_subdev *, u64, u64 *);
void nvkm_subdev_intr(struct nvkm_subdev *);
/* subdev logging */
-#define nvkm_printk_(s,l,p,f,a...) do { \
- const struct nvkm_subdev *_subdev = (s); \
- if (CONFIG_NOUVEAU_DEBUG >= (l) && _subdev->debug >= (l)) \
- dev_##p(_subdev->device->dev, "%s: "f, _subdev->name, ##a); \
+#define nvkm_printk_ok(s,u,l) \
+ ((CONFIG_NOUVEAU_DEBUG >= (l)) && ((s)->debug >= (l) || ((u) && (u)->debug >= (l))))
+#define nvkm_printk___(s,u,l,p,f,a...) do { \
+ if (nvkm_printk_ok((s), (u), (l))) { \
+ if ((u) && (u) != (s)) \
+ dev_##p((s)->device->dev, "%s(%s):"f, (s)->name, (u)->name, ##a); \
+ else \
+ dev_##p((s)->device->dev, "%s:"f, (s)->name, ##a); \
+ } \
} while(0)
+#define nvkm_printk__(s,l,p,f,a...) nvkm_printk___((s), (s), (l), p, f, ##a)
+#define nvkm_printk_(s,l,p,f,a...) nvkm_printk__((s), (l), p, " "f, ##a)
#define nvkm_printk(s,l,p,f,a...) nvkm_printk_((s), NV_DBG_##l, p, f, ##a)
#define nvkm_fatal(s,f,a...) nvkm_printk((s), FATAL, crit, f, ##a)
#define nvkm_error(s,f,a...) nvkm_printk((s), ERROR, err, f, ##a)
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/tegra.h b/drivers/gpu/drm/nouveau/include/nvkm/core/tegra.h
index 924009d..ccee53d 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/tegra.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/tegra.h
@@ -8,7 +8,6 @@ struct nvkm_device_tegra {
const struct nvkm_device_tegra_func *func;
struct nvkm_device device;
struct platform_device *pdev;
- int irq;
struct reset_control *rst;
struct clk *clk;
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/ce.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/ce.h
index cfd2da8..b616a1e 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/ce.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/ce.h
@@ -12,4 +12,6 @@ int gp100_ce_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct n
int gp102_ce_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_engine **);
int gv100_ce_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_engine **);
int tu102_ce_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_engine **);
+int ga100_ce_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_engine **);
+int ga102_ce_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_engine **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/disp.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/disp.h
index 8b5d8a4..ad9aef2 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/disp.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/disp.h
@@ -16,6 +16,7 @@ struct nvkm_disp {
struct list_head conns;
struct nvkm_event hpd;
+#define NVKM_DISP_HEAD_EVENT_VBLANK BIT(0)
struct nvkm_event vblank;
struct {
@@ -31,13 +32,7 @@ struct nvkm_disp {
struct {
unsigned long mask;
int nr;
- } wndw, head, dac;
-
- struct {
- unsigned long mask;
- int nr;
- u32 lvdsconf;
- } sor;
+ } wndw, head, dac, sor;
struct {
unsigned long mask;
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h
index b593407..cd86d91 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h
@@ -16,15 +16,16 @@ enum nvkm_falcon_dmaidx {
struct nvkm_falcon {
const struct nvkm_falcon_func *func;
- const struct nvkm_subdev *owner;
+ struct nvkm_subdev *owner;
const char *name;
u32 addr;
+ u32 addr2;
struct mutex mutex;
struct mutex dmem_mutex;
bool oneinit;
- const struct nvkm_subdev *user;
+ struct nvkm_subdev *user;
u8 version;
u8 secret;
@@ -50,13 +51,42 @@ struct nvkm_falcon {
struct nvkm_engine engine;
};
-int nvkm_falcon_get(struct nvkm_falcon *, const struct nvkm_subdev *);
-void nvkm_falcon_put(struct nvkm_falcon *, const struct nvkm_subdev *);
+int nvkm_falcon_get(struct nvkm_falcon *, struct nvkm_subdev *);
+void nvkm_falcon_put(struct nvkm_falcon *, struct nvkm_subdev *);
int nvkm_falcon_new_(const struct nvkm_falcon_func *, struct nvkm_device *,
enum nvkm_subdev_type, int inst, bool enable, u32 addr, struct nvkm_engine **);
struct nvkm_falcon_func {
+ int (*disable)(struct nvkm_falcon *);
+ int (*enable)(struct nvkm_falcon *);
+ int (*select)(struct nvkm_falcon *);
+ u32 addr2;
+ bool reset_pmc;
+ int (*reset_eng)(struct nvkm_falcon *);
+ int (*reset_prep)(struct nvkm_falcon *);
+ int (*reset_wait_mem_scrubbing)(struct nvkm_falcon *);
+
+ u32 debug;
+ void (*bind_inst)(struct nvkm_falcon *, int target, u64 addr);
+ int (*bind_stat)(struct nvkm_falcon *, bool intr);
+ bool bind_intr;
+
+ const struct nvkm_falcon_func_pio *imem_pio;
+ const struct nvkm_falcon_func_dma *imem_dma;
+
+ const struct nvkm_falcon_func_pio *dmem_pio;
+ const struct nvkm_falcon_func_dma *dmem_dma;
+
+ u32 emem_addr;
+ const struct nvkm_falcon_func_pio *emem_pio;
+
+ struct {
+ u32 head;
+ u32 tail;
+ u32 stride;
+ } cmdq, msgq;
+
struct {
u32 *data;
u32 size;
@@ -66,29 +96,11 @@ struct nvkm_falcon_func {
u32 size;
} data;
void (*init)(struct nvkm_falcon *);
- void (*intr)(struct nvkm_falcon *, struct nvkm_fifo_chan *);
-
- u32 debug;
- u32 fbif;
+ void (*intr)(struct nvkm_falcon *, struct nvkm_chan *);
void (*load_imem)(struct nvkm_falcon *, void *, u32, u32, u16, u8, bool);
void (*load_dmem)(struct nvkm_falcon *, void *, u32, u32, u8);
- void (*read_dmem)(struct nvkm_falcon *, u32, u32, u8, void *);
- u32 emem_addr;
- void (*bind_context)(struct nvkm_falcon *, struct nvkm_memory *);
- int (*wait_for_halt)(struct nvkm_falcon *, u32);
- int (*clear_interrupt)(struct nvkm_falcon *, u32);
- void (*set_start_addr)(struct nvkm_falcon *, u32 start_addr);
void (*start)(struct nvkm_falcon *);
- int (*enable)(struct nvkm_falcon *falcon);
- void (*disable)(struct nvkm_falcon *falcon);
- int (*reset)(struct nvkm_falcon *);
-
- struct {
- u32 head;
- u32 tail;
- u32 stride;
- } cmdq, msgq;
struct nvkm_sclass sclass[];
};
@@ -116,13 +128,5 @@ nvkm_falcon_mask(struct nvkm_falcon *falcon, u32 addr, u32 mask, u32 val)
void nvkm_falcon_load_imem(struct nvkm_falcon *, void *, u32, u32, u16, u8,
bool);
void nvkm_falcon_load_dmem(struct nvkm_falcon *, void *, u32, u32, u8);
-void nvkm_falcon_read_dmem(struct nvkm_falcon *, u32, u32, u8, void *);
-void nvkm_falcon_bind_context(struct nvkm_falcon *, struct nvkm_memory *);
-void nvkm_falcon_set_start_addr(struct nvkm_falcon *, u32);
void nvkm_falcon_start(struct nvkm_falcon *);
-int nvkm_falcon_wait_for_halt(struct nvkm_falcon *, u32);
-int nvkm_falcon_clear_interrupt(struct nvkm_falcon *, u32);
-int nvkm_falcon_enable(struct nvkm_falcon *);
-void nvkm_falcon_disable(struct nvkm_falcon *);
-int nvkm_falcon_reset(struct nvkm_falcon *);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/fifo.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/fifo.h
index 1509991..221abd6 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/fifo.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/fifo.h
@@ -6,56 +6,76 @@
#include <core/event.h>
struct nvkm_fault_data;
-#define NVKM_FIFO_CHID_NR 4096
#define NVKM_FIFO_ENGN_NR 16
-struct nvkm_fifo_engn {
- struct nvkm_object *object;
- int refcount;
- int usecount;
-};
+struct nvkm_chan {
+ const struct nvkm_chan_func *func;
+ char name[64];
+ struct nvkm_cgrp *cgrp;
+ int runq;
-struct nvkm_fifo_chan {
- const struct nvkm_fifo_chan_func *func;
- struct nvkm_fifo *fifo;
- u32 engm;
- struct nvkm_object object;
-
- struct list_head head;
- u16 chid;
struct nvkm_gpuobj *inst;
- struct nvkm_gpuobj *push;
struct nvkm_vmm *vmm;
- u64 addr;
- u32 size;
+ struct nvkm_gpuobj *push;
+ int id;
- struct nvkm_fifo_engn engn[NVKM_FIFO_ENGN_NR];
+ struct {
+ struct nvkm_memory *mem;
+ u32 base;
+ } userd;
+
+ u32 ramfc_offset;
+ struct nvkm_gpuobj *ramfc;
+ struct nvkm_gpuobj *cache;
+ struct nvkm_gpuobj *eng;
+ struct nvkm_gpuobj *pgd;
+ struct nvkm_ramht *ramht;
+
+ spinlock_t lock;
+ atomic_t blocked;
+ atomic_t errored;
+
+ struct list_head cctxs;
+ struct list_head head;
};
+struct nvkm_chan *nvkm_chan_get_chid(struct nvkm_engine *, int id, unsigned long *irqflags);
+struct nvkm_chan *nvkm_chan_get_inst(struct nvkm_engine *, u64 inst, unsigned long *irqflags);
+void nvkm_chan_put(struct nvkm_chan **, unsigned long irqflags);
+
struct nvkm_fifo {
const struct nvkm_fifo_func *func;
struct nvkm_engine engine;
- DECLARE_BITMAP(mask, NVKM_FIFO_CHID_NR);
- int nr;
- struct list_head chan;
+ struct nvkm_chid *chid;
+ struct nvkm_chid *cgid;
+
+ struct list_head runqs;
+ struct list_head runls;
+
+ struct {
+#define NVKM_FIFO_NONSTALL_EVENT BIT(0)
+ struct nvkm_event event;
+ struct nvkm_inth intr;
+ } nonstall;
+
+ struct {
+ u32 chan_msec;
+ } timeout;
+
+ struct {
+ struct nvkm_memory *mem;
+ struct nvkm_vma *bar1;
+ } userd;
+
spinlock_t lock;
struct mutex mutex;
-
- struct nvkm_event uevent; /* async user trigger */
- struct nvkm_event kevent; /* channel killed */
};
void nvkm_fifo_fault(struct nvkm_fifo *, struct nvkm_fault_data *);
void nvkm_fifo_pause(struct nvkm_fifo *, unsigned long *);
void nvkm_fifo_start(struct nvkm_fifo *, unsigned long *);
-
-void nvkm_fifo_chan_put(struct nvkm_fifo *, unsigned long flags,
- struct nvkm_fifo_chan **);
-struct nvkm_fifo_chan *
-nvkm_fifo_chan_inst(struct nvkm_fifo *, u64 inst, unsigned long *flags);
-struct nvkm_fifo_chan *
-nvkm_fifo_chan_chid(struct nvkm_fifo *, int chid, unsigned long *flags);
+bool nvkm_fifo_ctxsw_in_progress(struct nvkm_engine *);
int nv04_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int nv10_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
@@ -63,6 +83,7 @@ int nv17_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct
int nv40_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int nv50_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int g84_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
+int g98_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int gf100_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int gk104_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int gk110_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
@@ -70,10 +91,9 @@ int gk208_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct
int gk20a_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int gm107_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int gm200_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
-int gm20b_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int gp100_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
-int gp10b_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int gv100_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int tu102_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
+int ga100_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
int ga102_fifo_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_fifo **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/gr.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/gr.h
index b28b752..a2333cf 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/gr.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/gr.h
@@ -54,4 +54,5 @@ int gp108_gr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct n
int gp10b_gr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_gr **);
int gv100_gr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_gr **);
int tu102_gr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_gr **);
+int ga102_gr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_gr **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/nvdec.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/nvdec.h
index 97bd309..9baf197 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/nvdec.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/nvdec.h
@@ -12,4 +12,5 @@ struct nvkm_nvdec {
};
int gm107_nvdec_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_nvdec **);
+int ga102_nvdec_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_nvdec **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/sec2.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/sec2.h
index 06264c8..8d48fb2 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/sec2.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/sec2.h
@@ -10,15 +10,18 @@ struct nvkm_sec2 {
struct nvkm_engine engine;
struct nvkm_falcon falcon;
+ atomic_t running;
+ atomic_t initmsg;
+
struct nvkm_falcon_qmgr *qmgr;
struct nvkm_falcon_cmdq *cmdq;
struct nvkm_falcon_msgq *msgq;
struct work_struct work;
- bool initmsg_received;
};
int gp102_sec2_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_sec2 **);
int gp108_sec2_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_sec2 **);
int tu102_sec2_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_sec2 **);
+int ga102_sec2_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_sec2 **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/acr.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/acr.h
index c0b254f..73d2a6a 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/acr.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/acr.h
@@ -36,7 +36,7 @@ struct nvkm_acr {
const struct nvkm_acr_func *func;
struct nvkm_subdev subdev;
- struct list_head hsfw, hsf;
+ struct list_head hsfw;
struct list_head lsfw, lsf;
u64 managed_falcons;
@@ -50,6 +50,7 @@ struct nvkm_acr {
struct nvkm_vmm *vmm;
bool done;
+ struct nvkm_acr_lsf *rtos;
const struct firmware *wpr_fw;
bool wpr_comp;
@@ -64,7 +65,9 @@ int gm20b_acr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct
int gp102_acr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_acr **);
int gp108_acr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_acr **);
int gp10b_acr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_acr **);
+int gv100_acr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_acr **);
int tu102_acr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_acr **);
+int ga102_acr_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_acr **);
struct nvkm_acr_lsfw {
const struct nvkm_acr_lsf_func *func;
@@ -77,6 +80,7 @@ struct nvkm_acr_lsfw {
const struct firmware *sig;
+ bool secure_bootloader;
u32 bootloader_size;
u32 bootloader_imem_offset;
@@ -87,10 +91,19 @@ struct nvkm_acr_lsfw {
u32 app_resident_code_size;
u32 app_resident_data_offset;
u32 app_resident_data_size;
+ u32 app_imem_offset;
+ u32 app_dmem_offset;
u32 ucode_size;
u32 data_size;
+ u32 fuse_ver;
+ u32 engine_id;
+ u32 ucode_id;
+ u32 sig_size;
+ u32 sig_nr;
+ u8 *sigs;
+
struct {
u32 lsb;
u32 img;
@@ -105,10 +118,10 @@ struct nvkm_acr_lsf_func {
#define NVKM_ACR_LSF_DMACTL_REQ_CTX 0x00000004
#define NVKM_ACR_LSF_FORCE_PRIV_LOAD 0x00000008
u32 flags;
+ u32 bl_entry;
u32 bld_size;
void (*bld_write)(struct nvkm_acr *, u32 bld, struct nvkm_acr_lsfw *);
void (*bld_patch)(struct nvkm_acr *, u32 bld, s64 adjust);
- int (*boot)(struct nvkm_falcon *);
u64 bootstrap_falcons;
int (*bootstrap_falcon)(struct nvkm_falcon *, enum nvkm_acr_lsf_id);
int (*bootstrap_multiple_falcons)(struct nvkm_falcon *, u32 mask);
@@ -122,8 +135,20 @@ int
nvkm_acr_lsfw_load_sig_image_desc_v1(struct nvkm_subdev *, struct nvkm_falcon *,
enum nvkm_acr_lsf_id, const char *path,
int ver, const struct nvkm_acr_lsf_func *);
+
+int
+nvkm_acr_lsfw_load_sig_image_desc_v2(struct nvkm_subdev *, struct nvkm_falcon *,
+ enum nvkm_acr_lsf_id, const char *path,
+ int ver, const struct nvkm_acr_lsf_func *);
+
int
nvkm_acr_lsfw_load_bl_inst_data_sig(struct nvkm_subdev *, struct nvkm_falcon *,
enum nvkm_acr_lsf_id, const char *path,
int ver, const struct nvkm_acr_lsf_func *);
+
+int
+nvkm_acr_lsfw_load_bl_sig_net(struct nvkm_subdev *, struct nvkm_falcon *,
+ enum nvkm_acr_lsf_id, const char *path,
+ int ver, const struct nvkm_acr_lsf_func *,
+ const void *, u32, const void *, u32);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h
index 9c78f07..e40bbf37 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h
@@ -2,18 +2,21 @@
#define __NVKM_FAULT_H__
#include <core/subdev.h>
#include <core/event.h>
-#include <core/notify.h>
struct nvkm_fault {
const struct nvkm_fault_func *func;
struct nvkm_subdev subdev;
+ struct nvkm_inth info_fault;
+
struct nvkm_fault_buffer *buffer[2];
int buffer_nr;
+#define NVKM_FAULT_BUFFER_EVENT_PENDING BIT(0)
struct nvkm_event event;
- struct nvkm_notify nrpfb;
+ struct nvkm_event_ntfy nrpfb;
+ struct work_struct nrpfb_work;
struct nvkm_device_oclass user;
};
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h
index ef6a629..4076837 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h
@@ -35,6 +35,11 @@ struct nvkm_fb {
struct nvkm_blob vpr_scrubber;
+ struct {
+ struct page *flush_page;
+ dma_addr_t flush_page_addr;
+ } sysmem;
+
struct nvkm_ram *ram;
struct {
@@ -53,6 +58,8 @@ struct nvkm_fb {
struct nvkm_memory *mmu_wr;
};
+int nvkm_fb_mem_unlock(struct nvkm_fb *);
+
void nvkm_fb_tile_init(struct nvkm_fb *, int region, u32 addr, u32 size,
u32 pitch, u32 flags, struct nvkm_fb_tile *);
void nvkm_fb_tile_fini(struct nvkm_fb *, int region, struct nvkm_fb_tile *);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gpio.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/gpio.h
index 0e46ea1..537c4fc 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gpio.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/gpio.h
@@ -8,9 +8,6 @@
#include <subdev/bios/gpio.h>
struct nvkm_gpio_ntfy_req {
-#define NVKM_GPIO_HI 0x01
-#define NVKM_GPIO_LO 0x02
-#define NVKM_GPIO_TOGGLED 0x03
u8 mask;
u8 line;
};
@@ -23,6 +20,9 @@ struct nvkm_gpio {
const struct nvkm_gpio_func *func;
struct nvkm_subdev subdev;
+#define NVKM_GPIO_HI BIT(0)
+#define NVKM_GPIO_LO BIT(1)
+#define NVKM_GPIO_TOGGLED (NVKM_GPIO_HI | NVKM_GPIO_LO)
struct nvkm_event event;
};
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h
index cf42a59..72619d7 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h
@@ -5,9 +5,12 @@
#include <core/falcon.h>
struct nvkm_gsp {
+ const struct nvkm_gsp_func *func;
struct nvkm_subdev subdev;
+
struct nvkm_falcon falcon;
};
int gv100_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_gsp **);
+int ga102_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_gsp **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/i2c.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/i2c.h
index 146e132..40a1065 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/i2c.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/i2c.h
@@ -7,20 +7,6 @@
#include <subdev/bios.h>
#include <subdev/bios/i2c.h>
-struct nvkm_i2c_ntfy_req {
-#define NVKM_I2C_PLUG 0x01
-#define NVKM_I2C_UNPLUG 0x02
-#define NVKM_I2C_IRQ 0x04
-#define NVKM_I2C_DONE 0x08
-#define NVKM_I2C_ANY 0x0f
- u8 mask;
- u8 port;
-};
-
-struct nvkm_i2c_ntfy_rep {
- u8 mask;
-};
-
struct nvkm_i2c_bus_probe {
struct i2c_board_info dev;
u8 udelay; /* set to 0 to use the standard delay */
@@ -79,6 +65,11 @@ struct nvkm_i2c {
struct list_head bus;
struct list_head aux;
+#define NVKM_I2C_PLUG BIT(0)
+#define NVKM_I2C_UNPLUG BIT(1)
+#define NVKM_I2C_IRQ BIT(2)
+#define NVKM_I2C_DONE BIT(3)
+#define NVKM_I2C_ANY (NVKM_I2C_PLUG | NVKM_I2C_UNPLUG | NVKM_I2C_IRQ | NVKM_I2C_DONE)
struct nvkm_event event;
};
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/instmem.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/instmem.h
index f967b97..fcdaefc 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/instmem.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/instmem.h
@@ -28,7 +28,7 @@ u32 nvkm_instmem_rd32(struct nvkm_instmem *, u32 addr);
void nvkm_instmem_wr32(struct nvkm_instmem *, u32 addr, u32 data);
int nvkm_instobj_new(struct nvkm_instmem *, u32 size, u32 align, bool zero,
struct nvkm_memory **);
-
+int nvkm_instobj_wrap(struct nvkm_device *, struct nvkm_memory *, struct nvkm_memory **);
int nv04_instmem_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_instmem **);
int nv40_instmem_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_instmem **);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/ltc.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/ltc.h
index d32a326..6429404 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/ltc.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/ltc.h
@@ -4,7 +4,8 @@
#include <core/subdev.h>
#include <core/mm.h>
-#define NVKM_LTC_MAX_ZBC_CNT 16
+#define NVKM_LTC_MAX_ZBC_COLOR_CNT 32
+#define NVKM_LTC_MAX_ZBC_DEPTH_CNT 16
struct nvkm_ltc {
const struct nvkm_ltc_func *func;
@@ -18,11 +19,13 @@ struct nvkm_ltc {
u32 tag_base;
struct nvkm_memory *tag_ram;
- int zbc_min;
- int zbc_max;
- u32 zbc_color[NVKM_LTC_MAX_ZBC_CNT][4];
- u32 zbc_depth[NVKM_LTC_MAX_ZBC_CNT];
- u32 zbc_stencil[NVKM_LTC_MAX_ZBC_CNT];
+ int zbc_color_min;
+ int zbc_color_max;
+ u32 zbc_color[NVKM_LTC_MAX_ZBC_COLOR_CNT][4];
+ int zbc_depth_min;
+ int zbc_depth_max;
+ u32 zbc_depth[NVKM_LTC_MAX_ZBC_DEPTH_CNT];
+ u32 zbc_stencil[NVKM_LTC_MAX_ZBC_DEPTH_CNT];
};
void nvkm_ltc_tags_clear(struct nvkm_device *, u32 first, u32 count);
@@ -41,4 +44,5 @@ int gm200_ltc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct
int gp100_ltc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_ltc **);
int gp102_ltc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_ltc **);
int gp10b_ltc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_ltc **);
+int ga102_ltc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_ltc **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h
index cb86a56..127ac54 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h
@@ -6,15 +6,14 @@
struct nvkm_mc {
const struct nvkm_mc_func *func;
struct nvkm_subdev subdev;
+
+ struct nvkm_intr intr;
};
void nvkm_mc_enable(struct nvkm_device *, enum nvkm_subdev_type, int);
void nvkm_mc_disable(struct nvkm_device *, enum nvkm_subdev_type, int);
bool nvkm_mc_enabled(struct nvkm_device *, enum nvkm_subdev_type, int);
void nvkm_mc_reset(struct nvkm_device *, enum nvkm_subdev_type, int);
-void nvkm_mc_intr(struct nvkm_device *, bool *handled);
-void nvkm_mc_intr_unarm(struct nvkm_device *);
-void nvkm_mc_intr_rearm(struct nvkm_device *);
void nvkm_mc_intr_mask(struct nvkm_device *, enum nvkm_subdev_type, int, bool enable);
void nvkm_mc_unk260(struct nvkm_device *, u32 data);
@@ -31,6 +30,5 @@ int gk104_mc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct n
int gk20a_mc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_mc **);
int gp100_mc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_mc **);
int gp10b_mc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_mc **);
-int tu102_mc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_mc **);
int ga100_mc_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_mc **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
index 74c19bd..3c10310 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
@@ -13,7 +13,6 @@ struct nvkm_pci {
const struct nvkm_pci_func *func;
struct nvkm_subdev subdev;
struct pci_dev *pdev;
- int irq;
struct {
struct agp_bridge_data *bridge;
@@ -38,6 +37,7 @@ void nvkm_pci_wr08(struct nvkm_pci *, u16 addr, u8 data);
void nvkm_pci_wr32(struct nvkm_pci *, u16 addr, u32 data);
u32 nvkm_pci_mask(struct nvkm_pci *, u16 addr, u32 mask, u32 value);
void nvkm_pci_rom_shadow(struct nvkm_pci *, bool shadow);
+void nvkm_pci_msi_rearm(struct nvkm_device *);
int nv04_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int nv40_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/top.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/top.h
index ee75c55..73e717b 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/top.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/top.h
@@ -21,6 +21,7 @@ struct nvkm_top_device {
struct list_head head;
};
+int nvkm_top_parse(struct nvkm_device *);
u32 nvkm_top_addr(struct nvkm_device *, enum nvkm_subdev_type, int);
u32 nvkm_top_reset(struct nvkm_device *, enum nvkm_subdev_type, int);
u32 nvkm_top_intr_mask(struct nvkm_device *, enum nvkm_subdev_type, int);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/vfn.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/vfn.h
new file mode 100644
index 0000000..cc6d079
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/vfn.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVKM_VFN_H__
+#define __NVKM_VFN_H__
+#include <core/subdev.h>
+
+struct nvkm_vfn {
+ const struct nvkm_vfn_func *func;
+ struct nvkm_subdev subdev;
+
+ struct {
+ u32 priv;
+ u32 user;
+ } addr;
+
+ struct nvkm_intr intr;
+
+ struct nvkm_device_oclass user;
+};
+
+int gv100_vfn_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_vfn **);
+int tu102_vfn_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_vfn **);
+int ga100_vfn_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_vfn **);
+#endif
diff --git a/drivers/gpu/drm/nouveau/nouveau_abi16.c b/drivers/gpu/drm/nouveau/nouveau_abi16.c
index 5bee655..82dab51 100644
--- a/drivers/gpu/drm/nouveau/nouveau_abi16.c
+++ b/drivers/gpu/drm/nouveau/nouveau_abi16.c
@@ -27,7 +27,6 @@
#include <nvif/ioctl.h>
#include <nvif/class.h>
#include <nvif/cl0002.h>
-#include <nvif/cla06f.h>
#include <nvif/unpack.h>
#include "nouveau_drv.h"
@@ -253,7 +252,7 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS)
struct nouveau_abi16 *abi16 = nouveau_abi16_get(file_priv);
struct nouveau_abi16_chan *chan;
struct nvif_device *device;
- u64 engine;
+ u64 engine, runm;
int ret;
if (unlikely(!abi16))
@@ -263,6 +262,7 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS)
return nouveau_abi16_put(abi16, -ENODEV);
device = &abi16->device;
+ engine = NV_DEVICE_HOST_RUNLIST_ENGINES_GR;
/* hack to allow channel engine type specification on kepler */
if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) {
@@ -276,19 +276,18 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS)
default:
return nouveau_abi16_put(abi16, -ENOSYS);
}
- } else {
- engine = NV_DEVICE_HOST_RUNLIST_ENGINES_GR;
- }
- if (engine != NV_DEVICE_HOST_RUNLIST_ENGINES_CE)
- engine = nvif_fifo_runlist(device, engine);
- else
- engine = nvif_fifo_runlist_ce(device);
- init->fb_ctxdma_handle = engine;
- init->tt_ctxdma_handle = 0;
+ init->fb_ctxdma_handle = 0;
+ init->tt_ctxdma_handle = 0;
+ }
}
- if (init->fb_ctxdma_handle == ~0 || init->tt_ctxdma_handle == ~0)
+ if (engine != NV_DEVICE_HOST_RUNLIST_ENGINES_CE)
+ runm = nvif_fifo_runlist(device, engine);
+ else
+ runm = nvif_fifo_runlist_ce(device);
+
+ if (!runm || init->fb_ctxdma_handle == ~0 || init->tt_ctxdma_handle == ~0)
return nouveau_abi16_put(abi16, -EINVAL);
/* allocate "abi16 channel" data and make up a handle for it */
@@ -300,8 +299,8 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS)
list_add(&chan->head, &abi16->channels);
/* create channel object and initialise dma and fence management */
- ret = nouveau_channel_new(drm, device, init->fb_ctxdma_handle,
- init->tt_ctxdma_handle, false, &chan->chan);
+ ret = nouveau_channel_new(drm, device, false, runm, init->fb_ctxdma_handle,
+ init->tt_ctxdma_handle, &chan->chan);
if (ret)
goto done;
diff --git a/drivers/gpu/drm/nouveau/nouveau_backlight.c b/drivers/gpu/drm/nouveau/nouveau_backlight.c
index a614582..40409a2 100644
--- a/drivers/gpu/drm/nouveau/nouveau_backlight.c
+++ b/drivers/gpu/drm/nouveau/nouveau_backlight.c
@@ -264,7 +264,11 @@ nva3_set_intensity(struct backlight_device *bd)
u32 div, val;
div = nvif_rd32(device, NV50_PDISP_SOR_PWM_DIV(or));
- val = (bd->props.brightness * div) / 100;
+
+ val = backlight_get_brightness(bd);
+ if (val)
+ val = (val * div) / 100;
+
if (div) {
nvif_wr32(device, NV50_PDISP_SOR_PWM_CTL(or),
val |
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c
index 126b3c6..a11871e 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo.c
@@ -532,7 +532,7 @@ nouveau_bo_map(struct nouveau_bo *nvbo)
if (ret)
return ret;
- ret = ttm_bo_kmap(&nvbo->bo, 0, nvbo->bo.resource->num_pages, &nvbo->kmap);
+ ret = ttm_bo_kmap(&nvbo->bo, 0, PFN_UP(nvbo->bo.base.size), &nvbo->kmap);
ttm_bo_unreserve(&nvbo->bo);
return ret;
@@ -856,6 +856,9 @@ nouveau_bo_move_init(struct nouveau_drm *drm)
int (*init)(struct nouveau_channel *, u32 handle);
} _methods[] = {
{ "COPY", 4, 0xc7b5, nve0_bo_move_copy, nve0_bo_move_init },
+ { "GRCE", 0, 0xc7b5, nve0_bo_move_copy, nvc0_bo_move_init },
+ { "COPY", 4, 0xc6b5, nve0_bo_move_copy, nve0_bo_move_init },
+ { "GRCE", 0, 0xc6b5, nve0_bo_move_copy, nvc0_bo_move_init },
{ "COPY", 4, 0xc5b5, nve0_bo_move_copy, nve0_bo_move_init },
{ "GRCE", 0, 0xc5b5, nve0_bo_move_copy, nvc0_bo_move_init },
{ "COPY", 4, 0xc3b5, nve0_bo_move_copy, nve0_bo_move_init },
@@ -1236,7 +1239,7 @@ vm_fault_t nouveau_ttm_fault_reserve_notify(struct ttm_buffer_object *bo)
} else {
/* make sure bo is in mappable vram */
if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_TESLA ||
- bo->resource->start + bo->resource->num_pages < mappable)
+ bo->resource->start + PFN_UP(bo->resource->size) < mappable)
return 0;
for (i = 0; i < nvbo->placement.num_placement; ++i) {
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo0039.c b/drivers/gpu/drm/nouveau/nouveau_bo0039.c
index 7390132..e2ce44a 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo0039.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo0039.c
@@ -52,7 +52,7 @@ nv04_bo_move_m2mf(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
u32 src_offset = old_reg->start << PAGE_SHIFT;
u32 dst_ctxdma = nouveau_bo_mem_ctxdma(bo, chan, new_reg);
u32 dst_offset = new_reg->start << PAGE_SHIFT;
- u32 page_count = new_reg->num_pages;
+ u32 page_count = PFN_UP(new_reg->size);
int ret;
ret = PUSH_WAIT(push, 3);
@@ -62,7 +62,7 @@ nv04_bo_move_m2mf(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
PUSH_MTHD(push, NV039, SET_CONTEXT_DMA_BUFFER_IN, src_ctxdma,
SET_CONTEXT_DMA_BUFFER_OUT, dst_ctxdma);
- page_count = new_reg->num_pages;
+ page_count = PFN_UP(new_reg->size);
while (page_count) {
int line_count = (page_count > 2047) ? 2047 : page_count;
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo5039.c b/drivers/gpu/drm/nouveau/nouveau_bo5039.c
index 4c75c7b..c6cf362 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo5039.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo5039.c
@@ -41,7 +41,7 @@ nv50_bo_move_m2mf(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
{
struct nouveau_mem *mem = nouveau_mem(old_reg);
struct nvif_push *push = chan->chan.push;
- u64 length = (new_reg->num_pages << PAGE_SHIFT);
+ u64 length = new_reg->size;
u64 src_offset = mem->vma[0].addr;
u64 dst_offset = mem->vma[1].addr;
int src_tiled = !!mem->kind;
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo74c1.c b/drivers/gpu/drm/nouveau/nouveau_bo74c1.c
index ed6c09d..9b7ba31 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo74c1.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo74c1.c
@@ -44,7 +44,7 @@ nv84_bo_move_exec(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
if (ret)
return ret;
- PUSH_NVSQ(push, NV74C1, 0x0304, new_reg->num_pages << PAGE_SHIFT,
+ PUSH_NVSQ(push, NV74C1, 0x0304, new_reg->size,
0x0308, upper_32_bits(mem->vma[0].addr),
0x030c, lower_32_bits(mem->vma[0].addr),
0x0310, upper_32_bits(mem->vma[1].addr),
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo85b5.c b/drivers/gpu/drm/nouveau/nouveau_bo85b5.c
index dec29b2..a15a38a 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo85b5.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo85b5.c
@@ -44,10 +44,10 @@ nva3_bo_move_copy(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
struct nvif_push *push = chan->chan.push;
u64 src_offset = mem->vma[0].addr;
u64 dst_offset = mem->vma[1].addr;
- u32 page_count = new_reg->num_pages;
+ u32 page_count = PFN_UP(new_reg->size);
int ret;
- page_count = new_reg->num_pages;
+ page_count = PFN_UP(new_reg->size);
while (page_count) {
int line_count = (page_count > 8191) ? 8191 : page_count;
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo9039.c b/drivers/gpu/drm/nouveau/nouveau_bo9039.c
index 776b049..d2bb2687 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo9039.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo9039.c
@@ -42,10 +42,10 @@ nvc0_bo_move_m2mf(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
struct nouveau_mem *mem = nouveau_mem(old_reg);
u64 src_offset = mem->vma[0].addr;
u64 dst_offset = mem->vma[1].addr;
- u32 page_count = new_reg->num_pages;
+ u32 page_count = PFN_UP(new_reg->size);
int ret;
- page_count = new_reg->num_pages;
+ page_count = PFN_UP(new_reg->size);
while (page_count) {
int line_count = (page_count > 2047) ? 2047 : page_count;
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo90b5.c b/drivers/gpu/drm/nouveau/nouveau_bo90b5.c
index 8499f58..4618f4f5 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo90b5.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo90b5.c
@@ -37,10 +37,10 @@ nvc0_bo_move_copy(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
struct nvif_push *push = chan->chan.push;
u64 src_offset = mem->vma[0].addr;
u64 dst_offset = mem->vma[1].addr;
- u32 page_count = new_reg->num_pages;
+ u32 page_count = PFN_UP(new_reg->size);
int ret;
- page_count = new_reg->num_pages;
+ page_count = PFN_UP(new_reg->size);
while (page_count) {
int line_count = (page_count > 8191) ? 8191 : page_count;
diff --git a/drivers/gpu/drm/nouveau/nouveau_boa0b5.c b/drivers/gpu/drm/nouveau/nouveau_boa0b5.c
index 5752124..07a5c63 100644
--- a/drivers/gpu/drm/nouveau/nouveau_boa0b5.c
+++ b/drivers/gpu/drm/nouveau/nouveau_boa0b5.c
@@ -58,7 +58,7 @@ nve0_bo_move_copy(struct nouveau_channel *chan, struct ttm_buffer_object *bo,
PITCH_IN, PAGE_SIZE,
PITCH_OUT, PAGE_SIZE,
LINE_LENGTH_IN, PAGE_SIZE,
- LINE_COUNT, new_reg->num_pages);
+ LINE_COUNT, PFN_UP(new_reg->size));
PUSH_IMMD(push, NVA0B5, LAUNCH_DMA,
NVDEF(NVA0B5, LAUNCH_DMA, DATA_TRANSFER_TYPE, NON_PIPELINED) |
diff --git a/drivers/gpu/drm/nouveau/nouveau_chan.c b/drivers/gpu/drm/nouveau/nouveau_chan.c
index 48dea5d..e648ecd 100644
--- a/drivers/gpu/drm/nouveau/nouveau_chan.c
+++ b/drivers/gpu/drm/nouveau/nouveau_chan.c
@@ -25,12 +25,7 @@
#include <nvif/class.h>
#include <nvif/cl0002.h>
-#include <nvif/cl006b.h>
-#include <nvif/cl506f.h>
-#include <nvif/cl906f.h>
-#include <nvif/cla06f.h>
-#include <nvif/clc36f.h>
-#include <nvif/ioctl.h>
+#include <nvif/if0020.h>
#include "nouveau_drv.h"
#include "nouveau_dma.h"
@@ -46,15 +41,17 @@ int nouveau_vram_pushbuf;
module_param_named(vram_pushbuf, nouveau_vram_pushbuf, int, 0400);
static int
-nouveau_channel_killed(struct nvif_notify *ntfy)
+nouveau_channel_killed(struct nvif_event *event, void *repv, u32 repc)
{
- struct nouveau_channel *chan = container_of(ntfy, typeof(*chan), kill);
+ struct nouveau_channel *chan = container_of(event, typeof(*chan), kill);
struct nouveau_cli *cli = (void *)chan->user.client;
+
NV_PRINTK(warn, cli, "channel %d killed!\n", chan->chid);
atomic_set(&chan->killed, 1);
if (chan->fence)
nouveau_fence_context_kill(chan->fence, -ENODEV);
- return NVIF_NOTIFY_DROP;
+
+ return NVIF_EVENT_DROP;
}
int
@@ -96,8 +93,9 @@ nouveau_channel_del(struct nouveau_channel **pchan)
nvif_object_dtor(&chan->nvsw);
nvif_object_dtor(&chan->gart);
nvif_object_dtor(&chan->vram);
- nvif_notify_dtor(&chan->kill);
+ nvif_event_dtor(&chan->kill);
nvif_object_dtor(&chan->user);
+ nvif_mem_dtor(&chan->mem_userd);
nvif_object_dtor(&chan->push.ctxdma);
nouveau_vma_del(&chan->push.vma);
nouveau_bo_unmap(chan->push.buffer);
@@ -247,134 +245,113 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device,
}
static int
-nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device,
- u64 runlist, bool priv, struct nouveau_channel **pchan)
+nouveau_channel_ctor(struct nouveau_drm *drm, struct nvif_device *device, bool priv, u64 runm,
+ struct nouveau_channel **pchan)
{
- static const u16 oclasses[] = { AMPERE_CHANNEL_GPFIFO_B,
- TURING_CHANNEL_GPFIFO_A,
- VOLTA_CHANNEL_GPFIFO_A,
- PASCAL_CHANNEL_GPFIFO_A,
- MAXWELL_CHANNEL_GPFIFO_A,
- KEPLER_CHANNEL_GPFIFO_B,
- KEPLER_CHANNEL_GPFIFO_A,
- FERMI_CHANNEL_GPFIFO,
- G82_CHANNEL_GPFIFO,
- NV50_CHANNEL_GPFIFO,
- 0 };
- const u16 *oclass = oclasses;
- union {
- struct nv50_channel_gpfifo_v0 nv50;
- struct fermi_channel_gpfifo_v0 fermi;
- struct kepler_channel_gpfifo_a_v0 kepler;
- struct volta_channel_gpfifo_a_v0 volta;
+ static const struct {
+ s32 oclass;
+ int version;
+ } hosts[] = {
+ { AMPERE_CHANNEL_GPFIFO_B, 0 },
+ { AMPERE_CHANNEL_GPFIFO_A, 0 },
+ { TURING_CHANNEL_GPFIFO_A, 0 },
+ { VOLTA_CHANNEL_GPFIFO_A, 0 },
+ { PASCAL_CHANNEL_GPFIFO_A, 0 },
+ { MAXWELL_CHANNEL_GPFIFO_A, 0 },
+ { KEPLER_CHANNEL_GPFIFO_B, 0 },
+ { KEPLER_CHANNEL_GPFIFO_A, 0 },
+ { FERMI_CHANNEL_GPFIFO , 0 },
+ { G82_CHANNEL_GPFIFO , 0 },
+ { NV50_CHANNEL_GPFIFO , 0 },
+ { NV40_CHANNEL_DMA , 0 },
+ { NV17_CHANNEL_DMA , 0 },
+ { NV10_CHANNEL_DMA , 0 },
+ { NV03_CHANNEL_DMA , 0 },
+ {}
+ };
+ struct {
+ struct nvif_chan_v0 chan;
+ char name[TASK_COMM_LEN+16];
} args;
+ struct nouveau_cli *cli = (void *)device->object.client;
struct nouveau_channel *chan;
- u32 size;
- int ret;
+ const u64 plength = 0x10000;
+ const u64 ioffset = plength;
+ const u64 ilength = 0x02000;
+ char name[TASK_COMM_LEN];
+ int cid, ret;
+ u64 size;
+
+ cid = nvif_mclass(&device->object, hosts);
+ if (cid < 0)
+ return cid;
+
+ if (hosts[cid].oclass < NV50_CHANNEL_GPFIFO)
+ size = plength;
+ else
+ size = ioffset + ilength;
/* allocate dma push buffer */
- ret = nouveau_channel_prep(drm, device, 0x12000, &chan);
+ ret = nouveau_channel_prep(drm, device, size, &chan);
*pchan = chan;
if (ret)
return ret;
/* create channel object */
- do {
- if (oclass[0] >= VOLTA_CHANNEL_GPFIFO_A) {
- args.volta.version = 0;
- args.volta.ilength = 0x02000;
- args.volta.ioffset = 0x10000 + chan->push.addr;
- args.volta.runlist = runlist;
- args.volta.vmm = nvif_handle(&chan->vmm->vmm.object);
- args.volta.priv = priv;
- size = sizeof(args.volta);
- } else
- if (oclass[0] >= KEPLER_CHANNEL_GPFIFO_A) {
- args.kepler.version = 0;
- args.kepler.ilength = 0x02000;
- args.kepler.ioffset = 0x10000 + chan->push.addr;
- args.kepler.runlist = runlist;
- args.kepler.vmm = nvif_handle(&chan->vmm->vmm.object);
- args.kepler.priv = priv;
- size = sizeof(args.kepler);
- } else
- if (oclass[0] >= FERMI_CHANNEL_GPFIFO) {
- args.fermi.version = 0;
- args.fermi.ilength = 0x02000;
- args.fermi.ioffset = 0x10000 + chan->push.addr;
- args.fermi.vmm = nvif_handle(&chan->vmm->vmm.object);
- size = sizeof(args.fermi);
- } else {
- args.nv50.version = 0;
- args.nv50.ilength = 0x02000;
- args.nv50.ioffset = 0x10000 + chan->push.addr;
- args.nv50.pushbuf = nvif_handle(&chan->push.ctxdma);
- args.nv50.vmm = nvif_handle(&chan->vmm->vmm.object);
- size = sizeof(args.nv50);
- }
+ args.chan.version = 0;
+ args.chan.namelen = sizeof(args.name);
+ args.chan.runlist = __ffs64(runm);
+ args.chan.runq = 0;
+ args.chan.priv = priv;
+ args.chan.devm = BIT(0);
+ if (hosts[cid].oclass < NV50_CHANNEL_GPFIFO) {
+ args.chan.vmm = 0;
+ args.chan.ctxdma = nvif_handle(&chan->push.ctxdma);
+ args.chan.offset = chan->push.addr;
+ args.chan.length = 0;
+ } else {
+ args.chan.vmm = nvif_handle(&chan->vmm->vmm.object);
+ if (hosts[cid].oclass < FERMI_CHANNEL_GPFIFO)
+ args.chan.ctxdma = nvif_handle(&chan->push.ctxdma);
+ else
+ args.chan.ctxdma = 0;
+ args.chan.offset = ioffset + chan->push.addr;
+ args.chan.length = ilength;
+ }
+ args.chan.huserd = 0;
+ args.chan.ouserd = 0;
- ret = nvif_object_ctor(&device->object, "abi16ChanUser", 0,
- *oclass++, &args, size, &chan->user);
- if (ret == 0) {
- if (chan->user.oclass >= VOLTA_CHANNEL_GPFIFO_A) {
- chan->chid = args.volta.chid;
- chan->inst = args.volta.inst;
- chan->token = args.volta.token;
- } else
- if (chan->user.oclass >= KEPLER_CHANNEL_GPFIFO_A) {
- chan->chid = args.kepler.chid;
- chan->inst = args.kepler.inst;
- } else
- if (chan->user.oclass >= FERMI_CHANNEL_GPFIFO) {
- chan->chid = args.fermi.chid;
- } else {
- chan->chid = args.nv50.chid;
- }
+ /* allocate userd */
+ if (hosts[cid].oclass >= VOLTA_CHANNEL_GPFIFO_A) {
+ ret = nvif_mem_ctor(&cli->mmu, "abi16ChanUSERD", NVIF_CLASS_MEM_GF100,
+ NVIF_MEM_VRAM | NVIF_MEM_COHERENT | NVIF_MEM_MAPPABLE,
+ 0, PAGE_SIZE, NULL, 0, &chan->mem_userd);
+ if (ret)
return ret;
- }
- } while (*oclass);
- nouveau_channel_del(pchan);
- return ret;
-}
+ args.chan.huserd = nvif_handle(&chan->mem_userd.object);
+ args.chan.ouserd = 0;
-static int
-nouveau_channel_dma(struct nouveau_drm *drm, struct nvif_device *device,
- struct nouveau_channel **pchan)
-{
- static const u16 oclasses[] = { NV40_CHANNEL_DMA,
- NV17_CHANNEL_DMA,
- NV10_CHANNEL_DMA,
- NV03_CHANNEL_DMA,
- 0 };
- const u16 *oclass = oclasses;
- struct nv03_channel_dma_v0 args;
- struct nouveau_channel *chan;
- int ret;
+ chan->userd = &chan->mem_userd.object;
+ } else {
+ chan->userd = &chan->user;
+ }
- /* allocate dma push buffer */
- ret = nouveau_channel_prep(drm, device, 0x10000, &chan);
- *pchan = chan;
- if (ret)
+ get_task_comm(name, current);
+ snprintf(args.name, sizeof(args.name), "%s[%d]", name, task_pid_nr(current));
+
+ ret = nvif_object_ctor(&device->object, "abi16ChanUser", 0, hosts[cid].oclass,
+ &args, sizeof(args), &chan->user);
+ if (ret) {
+ nouveau_channel_del(pchan);
return ret;
+ }
- /* create channel object */
- args.version = 0;
- args.pushbuf = nvif_handle(&chan->push.ctxdma);
- args.offset = chan->push.addr;
-
- do {
- ret = nvif_object_ctor(&device->object, "abi16ChanUser", 0,
- *oclass++, &args, sizeof(args),
- &chan->user);
- if (ret == 0) {
- chan->chid = args.chid;
- return ret;
- }
- } while (ret && *oclass);
-
- nouveau_channel_del(pchan);
- return ret;
+ chan->runlist = args.chan.runlist;
+ chan->chid = args.chan.chid;
+ chan->inst = args.chan.inst;
+ chan->token = args.chan.token;
+ return 0;
}
static int
@@ -385,18 +362,24 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart)
struct nv_dma_v0 args = {};
int ret, i;
- ret = nvif_object_map(&chan->user, NULL, 0);
+ ret = nvif_object_map(chan->userd, NULL, 0);
if (ret)
return ret;
- if (chan->user.oclass >= FERMI_CHANNEL_GPFIFO &&
- chan->user.oclass < AMPERE_CHANNEL_GPFIFO_B) {
- ret = nvif_notify_ctor(&chan->user, "abi16ChanKilled",
- nouveau_channel_killed,
- true, NV906F_V0_NTFY_KILLED,
- NULL, 0, 0, &chan->kill);
+ if (chan->user.oclass >= FERMI_CHANNEL_GPFIFO) {
+ struct {
+ struct nvif_event_v0 base;
+ struct nvif_chan_event_v0 host;
+ } args;
+
+ args.host.version = 0;
+ args.host.type = NVIF_CHAN_EVENT_V0_KILLED;
+
+ ret = nvif_event_ctor(&chan->user, "abi16ChanKilled", chan->chid,
+ nouveau_channel_killed, false,
+ &args.base, sizeof(args), &chan->kill);
if (ret == 0)
- ret = nvif_notify_get(&chan->kill);
+ ret = nvif_event_allow(&chan->kill);
if (ret) {
NV_ERROR(drm, "Failed to request channel kill "
"notification: %d\n", ret);
@@ -503,24 +486,18 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart)
int
nouveau_channel_new(struct nouveau_drm *drm, struct nvif_device *device,
- u32 arg0, u32 arg1, bool priv,
- struct nouveau_channel **pchan)
+ bool priv, u64 runm, u32 vram, u32 gart, struct nouveau_channel **pchan)
{
struct nouveau_cli *cli = (void *)device->object.client;
int ret;
- /* hack until fencenv50 is fixed, and agp access relaxed */
- ret = nouveau_channel_ind(drm, device, arg0, priv, pchan);
+ ret = nouveau_channel_ctor(drm, device, priv, runm, pchan);
if (ret) {
- NV_PRINTK(dbg, cli, "ib channel create, %d\n", ret);
- ret = nouveau_channel_dma(drm, device, pchan);
- if (ret) {
- NV_PRINTK(dbg, cli, "dma channel create, %d\n", ret);
- return ret;
- }
+ NV_PRINTK(dbg, cli, "channel create, %d\n", ret);
+ return ret;
}
- ret = nouveau_channel_init(*pchan, arg0, arg1);
+ ret = nouveau_channel_init(*pchan, vram, gart);
if (ret) {
NV_PRINTK(err, cli, "channel failed to initialise, %d\n", ret);
nouveau_channel_del(pchan);
@@ -534,6 +511,12 @@ nouveau_channel_new(struct nouveau_drm *drm, struct nvif_device *device,
return ret;
}
+void
+nouveau_channels_fini(struct nouveau_drm *drm)
+{
+ kfree(drm->runl);
+}
+
int
nouveau_channels_init(struct nouveau_drm *drm)
{
@@ -541,20 +524,53 @@ nouveau_channels_init(struct nouveau_drm *drm)
struct nv_device_info_v1 m;
struct {
struct nv_device_info_v1_data channels;
+ struct nv_device_info_v1_data runlists;
} v;
} args = {
.m.version = 1,
.m.count = sizeof(args.v) / sizeof(args.v.channels),
.v.channels.mthd = NV_DEVICE_HOST_CHANNELS,
+ .v.runlists.mthd = NV_DEVICE_HOST_RUNLISTS,
};
struct nvif_object *device = &drm->client.device.object;
- int ret;
+ int ret, i;
ret = nvif_object_mthd(device, NV_DEVICE_V0_INFO, &args, sizeof(args));
- if (ret || args.v.channels.mthd == NV_DEVICE_INFO_INVALID)
+ if (ret ||
+ args.v.runlists.mthd == NV_DEVICE_INFO_INVALID || !args.v.runlists.data ||
+ args.v.channels.mthd == NV_DEVICE_INFO_INVALID)
return -ENODEV;
- drm->chan.nr = args.v.channels.data;
- drm->chan.context_base = dma_fence_context_alloc(drm->chan.nr);
+ drm->chan_nr = drm->chan_total = args.v.channels.data;
+ drm->runl_nr = fls64(args.v.runlists.data);
+ drm->runl = kcalloc(drm->runl_nr, sizeof(*drm->runl), GFP_KERNEL);
+ if (!drm->runl)
+ return -ENOMEM;
+
+ if (drm->chan_nr == 0) {
+ for (i = 0; i < drm->runl_nr; i++) {
+ if (!(args.v.runlists.data & BIT(i)))
+ continue;
+
+ args.v.channels.mthd = NV_DEVICE_HOST_RUNLIST_CHANNELS;
+ args.v.channels.data = i;
+
+ ret = nvif_object_mthd(device, NV_DEVICE_V0_INFO, &args, sizeof(args));
+ if (ret || args.v.channels.mthd == NV_DEVICE_INFO_INVALID)
+ return -ENODEV;
+
+ drm->runl[i].chan_nr = args.v.channels.data;
+ drm->runl[i].chan_id_base = drm->chan_total;
+ drm->runl[i].context_base = dma_fence_context_alloc(drm->runl[i].chan_nr);
+
+ drm->chan_total += drm->runl[i].chan_nr;
+ }
+ } else {
+ drm->runl[0].context_base = dma_fence_context_alloc(drm->chan_nr);
+ for (i = 1; i < drm->runl_nr; i++)
+ drm->runl[i].context_base = drm->runl[0].context_base;
+
+ }
+
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_chan.h b/drivers/gpu/drm/nouveau/nouveau_chan.h
index 98ba9d27..e06a8ff 100644
--- a/drivers/gpu/drm/nouveau/nouveau_chan.h
+++ b/drivers/gpu/drm/nouveau/nouveau_chan.h
@@ -2,7 +2,7 @@
#ifndef __NOUVEAU_CHAN_H__
#define __NOUVEAU_CHAN_H__
#include <nvif/object.h>
-#include <nvif/notify.h>
+#include <nvif/event.h>
#include <nvif/push.h>
struct nvif_device;
@@ -16,6 +16,10 @@ struct nouveau_channel {
struct nouveau_drm *drm;
struct nouveau_vmm *vmm;
+ struct nvif_mem mem_userd;
+ struct nvif_object *userd;
+
+ int runlist;
int chid;
u64 inst;
u32 token;
@@ -50,15 +54,15 @@ struct nouveau_channel {
struct nvif_object user;
- struct nvif_notify kill;
+ struct nvif_event kill;
atomic_t killed;
};
int nouveau_channels_init(struct nouveau_drm *);
+void nouveau_channels_fini(struct nouveau_drm *);
-int nouveau_channel_new(struct nouveau_drm *, struct nvif_device *,
- u32 arg0, u32 arg1, bool priv,
- struct nouveau_channel **);
+int nouveau_channel_new(struct nouveau_drm *, struct nvif_device *, bool priv, u64 runm,
+ u32 vram, u32 gart, struct nouveau_channel **);
void nouveau_channel_del(struct nouveau_channel **);
int nouveau_channel_idle(struct nouveau_channel *);
diff --git a/drivers/gpu/drm/nouveau/nouveau_connector.c b/drivers/gpu/drm/nouveau/nouveau_connector.c
index 1991bbb..086b66b 100644
--- a/drivers/gpu/drm/nouveau/nouveau_connector.c
+++ b/drivers/gpu/drm/nouveau/nouveau_connector.c
@@ -47,8 +47,7 @@
#include "nouveau_crtc.h"
#include <nvif/class.h>
-#include <nvif/cl0046.h>
-#include <nvif/event.h>
+#include <nvif/if0011.h>
struct drm_display_mode *
nouveau_conn_native_mode(struct drm_connector *connector)
@@ -396,7 +395,8 @@ static void
nouveau_connector_destroy(struct drm_connector *connector)
{
struct nouveau_connector *nv_connector = nouveau_connector(connector);
- nvif_notify_dtor(&nv_connector->hpd);
+ nvif_event_dtor(&nv_connector->irq);
+ nvif_event_dtor(&nv_connector->hpd);
kfree(nv_connector->edid);
drm_connector_unregister(connector);
drm_connector_cleanup(connector);
@@ -1162,39 +1162,38 @@ nouveau_connector_funcs_lvds = {
};
void
-nouveau_connector_hpd(struct drm_connector *connector)
+nouveau_connector_hpd(struct nouveau_connector *nv_connector, u64 bits)
{
- struct nouveau_drm *drm = nouveau_drm(connector->dev);
- u32 mask = drm_connector_mask(connector);
+ struct nouveau_drm *drm = nouveau_drm(nv_connector->base.dev);
+ u32 mask = drm_connector_mask(&nv_connector->base);
+ unsigned long flags;
- mutex_lock(&drm->hpd_lock);
+ spin_lock_irqsave(&drm->hpd_lock, flags);
if (!(drm->hpd_pending & mask)) {
+ nv_connector->hpd_pending |= bits;
drm->hpd_pending |= mask;
schedule_work(&drm->hpd_work);
}
- mutex_unlock(&drm->hpd_lock);
+ spin_unlock_irqrestore(&drm->hpd_lock, flags);
}
static int
-nouveau_connector_hotplug(struct nvif_notify *notify)
+nouveau_connector_irq(struct nvif_event *event, void *repv, u32 repc)
{
- struct nouveau_connector *nv_connector =
- container_of(notify, typeof(*nv_connector), hpd);
- struct drm_connector *connector = &nv_connector->base;
- struct drm_device *dev = connector->dev;
- struct nouveau_drm *drm = nouveau_drm(dev);
- const struct nvif_notify_conn_rep_v0 *rep = notify->data;
- bool plugged = (rep->mask != NVIF_NOTIFY_CONN_V0_UNPLUG);
+ struct nouveau_connector *nv_connector = container_of(event, typeof(*nv_connector), irq);
- if (rep->mask & NVIF_NOTIFY_CONN_V0_IRQ) {
- nouveau_dp_irq(drm, nv_connector);
- return NVIF_NOTIFY_KEEP;
- }
+ schedule_work(&nv_connector->irq_work);
+ return NVIF_EVENT_KEEP;
+}
- NV_DEBUG(drm, "%splugged %s\n", plugged ? "" : "un", connector->name);
- nouveau_connector_hpd(connector);
+static int
+nouveau_connector_hotplug(struct nvif_event *event, void *repv, u32 repc)
+{
+ struct nouveau_connector *nv_connector = container_of(event, typeof(*nv_connector), hpd);
+ struct nvif_conn_event_v0 *rep = repv;
- return NVIF_NOTIFY_KEEP;
+ nouveau_connector_hpd(nv_connector, rep->types);
+ return NVIF_EVENT_KEEP;
}
static ssize_t
@@ -1290,6 +1289,7 @@ nouveau_connector_create(struct drm_device *dev,
connector = &nv_connector->base;
nv_connector->index = index;
+ INIT_WORK(&nv_connector->irq_work, nouveau_dp_irq);
/* attempt to parse vbios connector type and hotplug gpio */
nv_connector->dcb = olddcb_conn(dev, index);
@@ -1401,6 +1401,7 @@ nouveau_connector_create(struct drm_device *dev,
drm_connector_init(dev, connector, funcs, type);
drm_connector_helper_add(connector, &nouveau_connector_helper_funcs);
+ connector->polled = DRM_CONNECTOR_POLL_CONNECT;
if (nv_connector->dcb && (disp->disp.conn_mask & BIT(nv_connector->index))) {
ret = nvif_conn_ctor(&disp->disp, nv_connector->base.name, nv_connector->index,
@@ -1409,6 +1410,25 @@ nouveau_connector_create(struct drm_device *dev,
kfree(nv_connector);
return ERR_PTR(ret);
}
+
+ ret = nvif_conn_event_ctor(&nv_connector->conn, "kmsHotplug",
+ nouveau_connector_hotplug,
+ NVIF_CONN_EVENT_V0_PLUG | NVIF_CONN_EVENT_V0_UNPLUG,
+ &nv_connector->hpd);
+ if (ret == 0)
+ connector->polled = DRM_CONNECTOR_POLL_HPD;
+
+ if (nv_connector->aux.transfer) {
+ ret = nvif_conn_event_ctor(&nv_connector->conn, "kmsDpIrq",
+ nouveau_connector_irq, NVIF_CONN_EVENT_V0_IRQ,
+ &nv_connector->irq);
+ if (ret) {
+ nvif_event_dtor(&nv_connector->hpd);
+ nvif_conn_dtor(&nv_connector->conn);
+ kfree(nv_connector);
+ return ERR_PTR(ret);
+ }
+ }
}
connector->funcs->reset(connector);
@@ -1452,21 +1472,6 @@ nouveau_connector_create(struct drm_device *dev,
break;
}
- ret = nvif_notify_ctor(&disp->disp.object, "kmsHotplug",
- nouveau_connector_hotplug,
- true, NV04_DISP_NTFY_CONN,
- &(struct nvif_notify_conn_req_v0) {
- .mask = NVIF_NOTIFY_CONN_V0_ANY,
- .conn = index,
- },
- sizeof(struct nvif_notify_conn_req_v0),
- sizeof(struct nvif_notify_conn_rep_v0),
- &nv_connector->hpd);
- if (ret)
- connector->polled = DRM_CONNECTOR_POLL_CONNECT;
- else
- connector->polled = DRM_CONNECTOR_POLL_HPD;
-
drm_connector_register(connector);
return connector;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_connector.h b/drivers/gpu/drm/nouveau/nouveau_connector.h
index f4e17ff..35bcb54 100644
--- a/drivers/gpu/drm/nouveau/nouveau_connector.h
+++ b/drivers/gpu/drm/nouveau/nouveau_connector.h
@@ -27,7 +27,7 @@
#ifndef __NOUVEAU_CONNECTOR_H__
#define __NOUVEAU_CONNECTOR_H__
#include <nvif/conn.h>
-#include <nvif/notify.h>
+#include <nvif/event.h>
#include <nvhw/class/cl507d.h>
#include <nvhw/class/cl907d.h>
@@ -124,7 +124,10 @@ struct nouveau_connector {
u8 *dcb;
struct nvif_conn conn;
- struct nvif_notify hpd;
+ u64 hpd_pending;
+ struct nvif_event hpd;
+ struct nvif_event irq;
+ struct work_struct irq_work;
struct drm_dp_aux aux;
@@ -198,7 +201,7 @@ nouveau_crtc_connector_get(struct nouveau_crtc *nv_crtc)
struct drm_connector *
nouveau_connector_create(struct drm_device *, const struct dcb_output *);
-void nouveau_connector_hpd(struct drm_connector *connector);
+void nouveau_connector_hpd(struct nouveau_connector *, u64 bits);
extern int nouveau_tv_disable;
extern int nouveau_ignorelid;
diff --git a/drivers/gpu/drm/nouveau/nouveau_crtc.h b/drivers/gpu/drm/nouveau/nouveau_crtc.h
index 7f63be2..c717f66 100644
--- a/drivers/gpu/drm/nouveau/nouveau_crtc.h
+++ b/drivers/gpu/drm/nouveau/nouveau_crtc.h
@@ -26,16 +26,17 @@
#ifndef __NOUVEAU_CRTC_H__
#define __NOUVEAU_CRTC_H__
-
#include <drm/drm_crtc.h>
-#include <nvif/notify.h>
+#include <nvif/head.h>
+#include <nvif/event.h>
struct nouveau_crtc {
struct drm_crtc base;
+ struct nvif_head head;
int index;
- struct nvif_notify vblank;
+ struct nvif_event vblank;
uint32_t dpms_saved_fp_control;
uint32_t fp_users;
diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c
index a2f5df56..ec3ffff 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.c
+++ b/drivers/gpu/drm/nouveau/nouveau_display.c
@@ -35,15 +35,14 @@
#include <drm/drm_probe_helper.h>
#include <drm/drm_vblank.h>
-#include "nouveau_fbcon.h"
#include "nouveau_crtc.h"
#include "nouveau_gem.h"
#include "nouveau_connector.h"
#include "nv50_display.h"
#include <nvif/class.h>
-#include <nvif/cl0046.h>
-#include <nvif/event.h>
+#include <nvif/if0011.h>
+#include <nvif/if0013.h>
#include <dispnv50/crc.h>
int
@@ -52,7 +51,7 @@ nouveau_display_vblank_enable(struct drm_crtc *crtc)
struct nouveau_crtc *nv_crtc;
nv_crtc = nouveau_crtc(crtc);
- nvif_notify_get(&nv_crtc->vblank);
+ nvif_event_allow(&nv_crtc->vblank);
return 0;
}
@@ -63,7 +62,7 @@ nouveau_display_vblank_disable(struct drm_crtc *crtc)
struct nouveau_crtc *nv_crtc;
nv_crtc = nouveau_crtc(crtc);
- nvif_notify_put(&nv_crtc->vblank);
+ nvif_event_block(&nv_crtc->vblank);
}
static inline int
@@ -84,24 +83,20 @@ static bool
nouveau_display_scanoutpos_head(struct drm_crtc *crtc, int *vpos, int *hpos,
ktime_t *stime, ktime_t *etime)
{
- struct {
- struct nv04_disp_mthd_v0 base;
- struct nv04_disp_scanoutpos_v0 scan;
- } args = {
- .base.method = NV04_DISP_SCANOUTPOS,
- .base.head = nouveau_crtc(crtc)->index,
- };
- struct nouveau_display *disp = nouveau_display(crtc->dev);
struct drm_vblank_crtc *vblank = &crtc->dev->vblank[drm_crtc_index(crtc)];
+ struct nvif_head *head = &nouveau_crtc(crtc)->head;
+ struct nvif_head_scanoutpos_v0 args;
int retry = 20;
bool ret = false;
+ args.version = 0;
+
do {
- ret = nvif_mthd(&disp->disp.object, 0, &args, sizeof(args));
+ ret = nvif_mthd(&head->object, NVIF_HEAD_V0_SCANOUTPOS, &args, sizeof(args));
if (ret != 0)
return false;
- if (args.scan.vline) {
+ if (args.vline) {
ret = true;
break;
}
@@ -109,11 +104,10 @@ nouveau_display_scanoutpos_head(struct drm_crtc *crtc, int *vpos, int *hpos,
if (retry) ndelay(vblank->linedur_ns);
} while (retry--);
- *hpos = args.scan.hline;
- *vpos = calc(args.scan.vblanks, args.scan.vblanke,
- args.scan.vtotal, args.scan.vline);
- if (stime) *stime = ns_to_ktime(args.scan.time[0]);
- if (etime) *etime = ns_to_ktime(args.scan.time[1]);
+ *hpos = args.hline;
+ *vpos = calc(args.vblanks, args.vblanke, args.vtotal, args.vline);
+ if (stime) *stime = ns_to_ktime(args.time[0]);
+ if (etime) *etime = ns_to_ktime(args.time[1]);
return ret;
}
@@ -397,7 +391,7 @@ nouveau_user_framebuffer_create(struct drm_device *dev,
static const struct drm_mode_config_funcs nouveau_mode_config_funcs = {
.fb_create = nouveau_user_framebuffer_create,
- .output_poll_changed = nouveau_fbcon_output_poll_changed,
+ .output_poll_changed = drm_fb_helper_output_poll_changed,
};
@@ -456,9 +450,9 @@ nouveau_display_hpd_resume(struct drm_device *dev)
{
struct nouveau_drm *drm = nouveau_drm(dev);
- mutex_lock(&drm->hpd_lock);
+ spin_lock_irq(&drm->hpd_lock);
drm->hpd_pending = ~0;
- mutex_unlock(&drm->hpd_lock);
+ spin_unlock_irq(&drm->hpd_lock);
schedule_work(&drm->hpd_work);
}
@@ -475,10 +469,10 @@ nouveau_display_hpd_work(struct work_struct *work)
pm_runtime_get_sync(dev->dev);
- mutex_lock(&drm->hpd_lock);
+ spin_lock_irq(&drm->hpd_lock);
pending = drm->hpd_pending;
drm->hpd_pending = 0;
- mutex_unlock(&drm->hpd_lock);
+ spin_unlock_irq(&drm->hpd_lock);
/* Nothing to do, exit early without updating the last busy counter */
if (!pending)
@@ -488,14 +482,30 @@ nouveau_display_hpd_work(struct work_struct *work)
drm_connector_list_iter_begin(dev, &conn_iter);
nouveau_for_each_non_mst_connector_iter(connector, &conn_iter) {
+ struct nouveau_connector *nv_connector = nouveau_connector(connector);
enum drm_connector_status old_status = connector->status;
- u64 old_epoch_counter = connector->epoch_counter;
+ u64 bits, old_epoch_counter = connector->epoch_counter;
if (!(pending & drm_connector_mask(connector)))
continue;
- connector->status = drm_helper_probe_detect(connector, NULL,
- false);
+ spin_lock_irq(&drm->hpd_lock);
+ bits = nv_connector->hpd_pending;
+ nv_connector->hpd_pending = 0;
+ spin_unlock_irq(&drm->hpd_lock);
+
+ drm_dbg_kms(dev, "[CONNECTOR:%d:%s] plug:%d unplug:%d irq:%d\n",
+ connector->base.id, connector->name,
+ !!(bits & NVIF_CONN_EVENT_V0_PLUG),
+ !!(bits & NVIF_CONN_EVENT_V0_UNPLUG),
+ !!(bits & NVIF_CONN_EVENT_V0_IRQ));
+
+ if (bits & NVIF_CONN_EVENT_V0_IRQ) {
+ if (nouveau_dp_link_check(nv_connector))
+ continue;
+ }
+
+ connector->status = drm_helper_probe_detect(connector, NULL, false);
if (old_epoch_counter == connector->epoch_counter)
continue;
@@ -573,7 +583,8 @@ nouveau_display_init(struct drm_device *dev, bool resume, bool runtime)
drm_connector_list_iter_begin(dev, &conn_iter);
nouveau_for_each_non_mst_connector_iter(connector, &conn_iter) {
struct nouveau_connector *conn = nouveau_connector(connector);
- nvif_notify_get(&conn->hpd);
+ nvif_event_allow(&conn->hpd);
+ nvif_event_allow(&conn->irq);
}
drm_connector_list_iter_end(&conn_iter);
@@ -608,7 +619,8 @@ nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime)
drm_connector_list_iter_begin(dev, &conn_iter);
nouveau_for_each_non_mst_connector_iter(connector, &conn_iter) {
struct nouveau_connector *conn = nouveau_connector(connector);
- nvif_notify_put(&conn->hpd);
+ nvif_event_block(&conn->irq);
+ nvif_event_block(&conn->hpd);
}
drm_connector_list_iter_end(&conn_iter);
@@ -659,7 +671,6 @@ int
nouveau_display_create(struct drm_device *dev)
{
struct nouveau_drm *drm = nouveau_drm(dev);
- struct nvkm_device *device = nvxx_device(&drm->client.device);
struct nouveau_display *disp;
int ret;
@@ -672,7 +683,6 @@ nouveau_display_create(struct drm_device *dev)
drm_mode_create_dvi_i_properties(dev);
dev->mode_config.funcs = &nouveau_mode_config_funcs;
- dev->mode_config.fb_base = device->func->resource_addr(device, 1);
dev->mode_config.min_width = 0;
dev->mode_config.min_height = 0;
@@ -734,7 +744,7 @@ nouveau_display_create(struct drm_device *dev)
}
INIT_WORK(&drm->hpd_work, nouveau_display_hpd_work);
- mutex_init(&drm->hpd_lock);
+ spin_lock_init(&drm->hpd_lock);
#ifdef CONFIG_ACPI
drm->acpi_nb.notifier_call = nouveau_display_acpi_ntfy;
register_acpi_notifier(&drm->acpi_nb);
@@ -768,8 +778,7 @@ nouveau_display_destroy(struct drm_device *dev)
nvif_disp_dtor(&disp->disp);
- nouveau_drm(dev)->display = NULL;
- mutex_destroy(&drm->hpd_lock);
+ drm->display = NULL;
kfree(disp);
}
@@ -778,6 +787,9 @@ nouveau_display_suspend(struct drm_device *dev, bool runtime)
{
struct nouveau_display *disp = nouveau_display(dev);
+ /* Disable console. */
+ drm_fb_helper_set_suspend_unlocked(dev->fb_helper, true);
+
if (drm_drv_uses_atomic_modeset(dev)) {
if (!runtime) {
disp->suspend = drm_atomic_helper_suspend(dev);
@@ -805,8 +817,10 @@ nouveau_display_resume(struct drm_device *dev, bool runtime)
drm_atomic_helper_resume(dev, disp->suspend);
disp->suspend = NULL;
}
- return;
}
+
+ /* Enable console. */
+ drm_fb_helper_set_suspend_unlocked(dev->fb_helper, false);
}
int
diff --git a/drivers/gpu/drm/nouveau/nouveau_dma.c b/drivers/gpu/drm/nouveau/nouveau_dma.c
index ddb75d8..b90cac6 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dma.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dma.c
@@ -42,9 +42,9 @@ READ_GET(struct nouveau_channel *chan, uint64_t *prev_get, int *timeout)
{
uint64_t val;
- val = nvif_rd32(&chan->user, chan->user_get);
+ val = nvif_rd32(chan->userd, chan->user_get);
if (chan->user_get_hi)
- val |= (uint64_t)nvif_rd32(&chan->user, chan->user_get_hi) << 32;
+ val |= (uint64_t)nvif_rd32(chan->userd, chan->user_get_hi) << 32;
/* reset counter as long as GET is still advancing, this is
* to avoid misdetecting a GPU lockup if the GPU happens to
@@ -86,7 +86,7 @@ nv50_dma_push(struct nouveau_channel *chan, u64 offset, int length)
/* Flush writes. */
nouveau_bo_rd32(pb, 0);
- nvif_wr32(&chan->user, 0x8c, chan->dma.ib_put);
+ nvif_wr32(chan->userd, 0x8c, chan->dma.ib_put);
if (user->func && user->func->doorbell)
user->func->doorbell(user, chan->token);
chan->dma.ib_free--;
@@ -98,7 +98,7 @@ nv50_dma_push_wait(struct nouveau_channel *chan, int count)
uint32_t cnt = 0, prev_get = 0;
while (chan->dma.ib_free < count) {
- uint32_t get = nvif_rd32(&chan->user, 0x88);
+ uint32_t get = nvif_rd32(chan->userd, 0x88);
if (get != prev_get) {
prev_get = get;
cnt = 0;
diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c b/drivers/gpu/drm/nouveau/nouveau_dmem.c
index 20fe538..789857f 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dmem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dmem.c
@@ -33,7 +33,6 @@
#include <nvif/if000c.h>
#include <nvif/if500b.h>
#include <nvif/if900b.h>
-#include <nvif/if000c.h>
#include <nvhw/class/cla0b5.h>
diff --git a/drivers/gpu/drm/nouveau/nouveau_dp.c b/drivers/gpu/drm/nouveau/nouveau_dp.c
index 20db8ea..e00876f 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dp.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dp.c
@@ -29,8 +29,7 @@
#include "nouveau_encoder.h"
#include "nouveau_crtc.h"
-#include <nvif/class.h>
-#include <nvif/cl5070.h>
+#include <nvif/if0011.h>
MODULE_PARM_DESC(mst, "Enable DisplayPort multi-stream (default: enabled)");
static int nouveau_mst = 1;
@@ -140,12 +139,17 @@ nouveau_dp_detect(struct nouveau_connector *nv_connector,
* TODO: look into checking this before probing I2C to detect DVI/HDMI
*/
hpd = nvif_conn_hpd_status(&nv_connector->conn);
- if (hpd == NVIF_CONN_HPD_STATUS_NOT_PRESENT)
+ if (hpd == NVIF_CONN_HPD_STATUS_NOT_PRESENT) {
+ nvif_outp_dp_aux_pwr(&nv_encoder->outp, false);
goto out;
+ }
+ nvif_outp_dp_aux_pwr(&nv_encoder->outp, true);
status = nouveau_dp_probe_dpcd(nv_connector, nv_encoder);
- if (status == connector_status_disconnected)
+ if (status == connector_status_disconnected) {
+ nvif_outp_dp_aux_pwr(&nv_encoder->outp, false);
goto out;
+ }
/* If we're in MST mode, we're done here */
if (mstm && mstm->can_mst && mstm->is_mst) {
@@ -193,6 +197,7 @@ nouveau_dp_detect(struct nouveau_connector *nv_connector,
ret = NOUVEAU_DP_MST;
goto out;
} else if (ret != 0) {
+ nvif_outp_dp_aux_pwr(&nv_encoder->outp, false);
goto out;
}
}
@@ -206,14 +211,28 @@ nouveau_dp_detect(struct nouveau_connector *nv_connector,
return ret;
}
-void nouveau_dp_irq(struct nouveau_drm *drm,
- struct nouveau_connector *nv_connector)
+bool
+nouveau_dp_link_check(struct nouveau_connector *nv_connector)
{
+ struct nouveau_encoder *nv_encoder = find_encoder(&nv_connector->base, DCB_OUTPUT_DP);
+
+ if (!nv_encoder || nv_encoder->outp.or.id < 0)
+ return true;
+
+ return nvif_outp_dp_retrain(&nv_encoder->outp) == 0;
+}
+
+void
+nouveau_dp_irq(struct work_struct *work)
+{
+ struct nouveau_connector *nv_connector =
+ container_of(work, typeof(*nv_connector), irq_work);
struct drm_connector *connector = &nv_connector->base;
struct nouveau_encoder *outp = find_encoder(connector, DCB_OUTPUT_DP);
+ struct nouveau_drm *drm = nouveau_drm(outp->base.base.dev);
struct nv50_mstm *mstm;
+ u64 hpd = 0;
int ret;
- bool send_hpd = false;
if (!outp)
return;
@@ -225,14 +244,14 @@ void nouveau_dp_irq(struct nouveau_drm *drm,
if (mstm && mstm->is_mst) {
if (!nv50_mstm_service(drm, nv_connector, mstm))
- send_hpd = true;
+ hpd |= NVIF_CONN_EVENT_V0_UNPLUG;
} else {
drm_dp_cec_irq(&nv_connector->aux);
if (nouveau_dp_has_sink_count(connector, outp)) {
ret = drm_dp_read_sink_count(&nv_connector->aux);
if (ret != outp->dp.sink_count)
- send_hpd = true;
+ hpd |= NVIF_CONN_EVENT_V0_PLUG;
if (ret >= 0)
outp->dp.sink_count = ret;
}
@@ -240,8 +259,7 @@ void nouveau_dp_irq(struct nouveau_drm *drm,
mutex_unlock(&outp->dp.hpd_irq_lock);
- if (send_hpd)
- nouveau_connector_hpd(connector);
+ nouveau_connector_hpd(nv_connector, NVIF_CONN_EVENT_V0_IRQ | hpd);
}
/* TODO:
diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.c b/drivers/gpu/drm/nouveau/nouveau_drm.c
index fd99ec0..80f154b 100644
--- a/drivers/gpu/drm/nouveau/nouveau_drm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_drm.c
@@ -33,6 +33,8 @@
#include <drm/drm_aperture.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_ttm_helper.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_vblank.h>
@@ -49,7 +51,6 @@
#include <nvif/class.h>
#include <nvif/cl0002.h>
-#include <nvif/cla06f.h>
#include "nouveau_drv.h"
#include "nouveau_dma.h"
@@ -62,7 +63,6 @@
#include "nouveau_bios.h"
#include "nouveau_ioctl.h"
#include "nouveau_abi16.h"
-#include "nouveau_fbcon.h"
#include "nouveau_fence.h"
#include "nouveau_debugfs.h"
#include "nouveau_usif.h"
@@ -316,28 +316,19 @@ static void
nouveau_accel_ce_init(struct nouveau_drm *drm)
{
struct nvif_device *device = &drm->client.device;
+ u64 runm;
int ret = 0;
/* Allocate channel that has access to a (preferably async) copy
* engine, to use for TTM buffer moves.
*/
- if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) {
- ret = nouveau_channel_new(drm, device,
- nvif_fifo_runlist_ce(device), 0,
- true, &drm->cechan);
- } else
- if (device->info.chipset >= 0xa3 &&
- device->info.chipset != 0xaa &&
- device->info.chipset != 0xac) {
- /* Prior to Kepler, there's only a single runlist, so all
- * engines can be accessed from any channel.
- *
- * We still want to use a separate channel though.
- */
- ret = nouveau_channel_new(drm, device, NvDmaFB, NvDmaTT, false,
- &drm->cechan);
+ runm = nvif_fifo_runlist_ce(device);
+ if (!runm) {
+ NV_DEBUG(drm, "no ce runlist\n");
+ return;
}
+ ret = nouveau_channel_new(drm, device, false, runm, NvDmaFB, NvDmaTT, &drm->cechan);
if (ret)
NV_ERROR(drm, "failed to create ce channel, %d\n", ret);
}
@@ -355,23 +346,17 @@ static void
nouveau_accel_gr_init(struct nouveau_drm *drm)
{
struct nvif_device *device = &drm->client.device;
- u32 arg0, arg1;
+ u64 runm;
int ret;
- if (device->info.family >= NV_DEVICE_INFO_V0_AMPERE)
- return;
-
/* Allocate channel that has access to the graphics engine. */
- if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) {
- arg0 = nvif_fifo_runlist(device, NV_DEVICE_HOST_RUNLIST_ENGINES_GR);
- arg1 = 1;
- } else {
- arg0 = NvDmaFB;
- arg1 = NvDmaTT;
+ runm = nvif_fifo_runlist(device, NV_DEVICE_HOST_RUNLIST_ENGINES_GR);
+ if (!runm) {
+ NV_DEBUG(drm, "no gr runlist\n");
+ return;
}
- ret = nouveau_channel_new(drm, device, arg0, arg1, false,
- &drm->channel);
+ ret = nouveau_channel_new(drm, device, false, runm, NvDmaFB, NvDmaTT, &drm->channel);
if (ret) {
NV_ERROR(drm, "failed to create kernel channel, %d\n", ret);
nouveau_accel_gr_fini(drm);
@@ -436,6 +421,7 @@ nouveau_accel_fini(struct nouveau_drm *drm)
nouveau_accel_gr_fini(drm);
if (drm->fence)
nouveau_fence(drm)->dtor(drm);
+ nouveau_channels_fini(drm);
}
static void
@@ -485,6 +471,7 @@ nouveau_accel_init(struct nouveau_drm *drm)
case PASCAL_CHANNEL_GPFIFO_A:
case VOLTA_CHANNEL_GPFIFO_A:
case TURING_CHANNEL_GPFIFO_A:
+ case AMPERE_CHANNEL_GPFIFO_A:
case AMPERE_CHANNEL_GPFIFO_B:
ret = nvc0_fence_create(drm);
break;
@@ -611,7 +598,6 @@ nouveau_drm_device_init(struct drm_device *dev)
nouveau_hwmon_init(dev);
nouveau_svm_init(drm);
nouveau_dmem_init(drm);
- nouveau_fbcon_init(dev);
nouveau_led_init(dev);
if (nouveau_pmops_runtime()) {
@@ -655,7 +641,6 @@ nouveau_drm_device_fini(struct drm_device *dev)
}
nouveau_led_fini(dev);
- nouveau_fbcon_fini(dev);
nouveau_dmem_fini(drm);
nouveau_svm_fini(drm);
nouveau_hwmon_fini(dev);
@@ -809,6 +794,11 @@ static int nouveau_drm_probe(struct pci_dev *pdev,
if (ret)
goto fail_drm_dev_init;
+ if (nouveau_drm(drm_dev)->client.device.info.ram_size <= 32 * 1024 * 1024)
+ drm_fbdev_generic_setup(drm_dev, 8);
+ else
+ drm_fbdev_generic_setup(drm_dev, 32);
+
quirk_broken_nv_runpm(pdev);
return 0;
@@ -865,8 +855,6 @@ nouveau_do_suspend(struct drm_device *dev, bool runtime)
nouveau_led_suspend(dev);
if (dev->mode_config.num_crtc) {
- NV_DEBUG(drm, "suspending console...\n");
- nouveau_fbcon_set_suspend(dev, 1);
NV_DEBUG(drm, "suspending display...\n");
ret = nouveau_display_suspend(dev, runtime);
if (ret)
@@ -940,8 +928,6 @@ nouveau_do_resume(struct drm_device *dev, bool runtime)
if (dev->mode_config.num_crtc) {
NV_DEBUG(drm, "resuming display...\n");
nouveau_display_resume(dev, runtime);
- NV_DEBUG(drm, "resuming console...\n");
- nouveau_fbcon_set_suspend(dev, 0);
}
nouveau_led_resume(dev);
@@ -1296,7 +1282,6 @@ static void nouveau_display_options(void)
DRM_DEBUG_DRIVER("... tv_disable : %d\n", nouveau_tv_disable);
DRM_DEBUG_DRIVER("... ignorelid : %d\n", nouveau_ignorelid);
DRM_DEBUG_DRIVER("... duallink : %d\n", nouveau_duallink);
- DRM_DEBUG_DRIVER("... nofbaccel : %d\n", nouveau_nofbaccel);
DRM_DEBUG_DRIVER("... config : %s\n", nouveau_config);
DRM_DEBUG_DRIVER("... debug : %s\n", nouveau_debug);
DRM_DEBUG_DRIVER("... noaccel : %d\n", nouveau_noaccel);
diff --git a/drivers/gpu/drm/nouveau/nouveau_drv.h b/drivers/gpu/drm/nouveau/nouveau_drv.h
index 84df5dd..d6dd07bf 100644
--- a/drivers/gpu/drm/nouveau/nouveau_drv.h
+++ b/drivers/gpu/drm/nouveau/nouveau_drv.h
@@ -78,11 +78,6 @@ enum nouveau_drm_object_route {
NVDRM_OBJECT_ANY = NVIF_IOCTL_V0_OWNER_ANY,
};
-enum nouveau_drm_notify_route {
- NVDRM_NOTIFY_NVIF = 0,
- NVDRM_NOTIFY_USIF
-};
-
enum nouveau_drm_handle {
NVDRM_CHAN = 0xcccc0000, /* |= client chid */
NVDRM_NVSW = 0x55550000,
@@ -179,16 +174,19 @@ struct nouveau_drm {
void *fence;
/* Global channel management. */
+ int chan_total; /* Number of channels across all runlists. */
+ int chan_nr; /* 0 if per-runlist CHIDs. */
+ int runl_nr;
struct {
- int nr;
+ int chan_nr;
+ int chan_id_base;
u64 context_base;
- } chan;
+ } *runl;
/* context for accelerated drm-internal operations */
struct nouveau_channel *cechan;
struct nouveau_channel *channel;
struct nvkm_gpuobj *notify;
- struct nouveau_fbdev *fbcon;
struct nvif_object ntfy;
/* nv10-nv40 tiling regions */
@@ -201,10 +199,8 @@ struct nouveau_drm {
struct nvbios vbios;
struct nouveau_display *display;
struct work_struct hpd_work;
- struct mutex hpd_lock;
+ spinlock_t hpd_lock;
u32 hpd_pending;
- struct work_struct fbcon_work;
- int fbcon_new_state;
#ifdef CONFIG_ACPI
struct notifier_block acpi_nb;
#endif
diff --git a/drivers/gpu/drm/nouveau/nouveau_encoder.h b/drivers/gpu/drm/nouveau/nouveau_encoder.h
index b72e5783..70c1ad6 100644
--- a/drivers/gpu/drm/nouveau/nouveau_encoder.h
+++ b/drivers/gpu/drm/nouveau/nouveau_encoder.h
@@ -48,7 +48,6 @@ struct nouveau_encoder {
struct dcb_output *dcb;
struct nvif_outp outp;
int or;
- int link;
struct i2c_adapter *i2c;
struct nvkm_i2c_aux *aux;
@@ -142,8 +141,8 @@ enum nouveau_dp_status {
};
int nouveau_dp_detect(struct nouveau_connector *, struct nouveau_encoder *);
-void nouveau_dp_irq(struct nouveau_drm *drm,
- struct nouveau_connector *nv_connector);
+bool nouveau_dp_link_check(struct nouveau_connector *);
+void nouveau_dp_irq(struct work_struct *);
enum drm_mode_status nv50_dp_mode_valid(struct drm_connector *,
struct nouveau_encoder *,
const struct drm_display_mode *,
diff --git a/drivers/gpu/drm/nouveau/nouveau_fbcon.c b/drivers/gpu/drm/nouveau/nouveau_fbcon.c
index 3c7e0c9..e87de79 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fbcon.c
+++ b/drivers/gpu/drm/nouveau/nouveau_fbcon.c
@@ -231,9 +231,9 @@ void
nouveau_fbcon_accel_save_disable(struct drm_device *dev)
{
struct nouveau_drm *drm = nouveau_drm(dev);
- if (drm->fbcon && drm->fbcon->helper.fbdev) {
- drm->fbcon->saved_flags = drm->fbcon->helper.fbdev->flags;
- drm->fbcon->helper.fbdev->flags |= FBINFO_HWACCEL_DISABLED;
+ if (drm->fbcon && drm->fbcon->helper.info) {
+ drm->fbcon->saved_flags = drm->fbcon->helper.info->flags;
+ drm->fbcon->helper.info->flags |= FBINFO_HWACCEL_DISABLED;
}
}
@@ -241,9 +241,8 @@ void
nouveau_fbcon_accel_restore(struct drm_device *dev)
{
struct nouveau_drm *drm = nouveau_drm(dev);
- if (drm->fbcon && drm->fbcon->helper.fbdev) {
- drm->fbcon->helper.fbdev->flags = drm->fbcon->saved_flags;
- }
+ if (drm->fbcon && drm->fbcon->helper.info)
+ drm->fbcon->helper.info->flags = drm->fbcon->saved_flags;
}
static void
@@ -253,8 +252,8 @@ nouveau_fbcon_accel_fini(struct drm_device *dev)
struct nouveau_fbdev *fbcon = drm->fbcon;
if (fbcon && drm->channel) {
console_lock();
- if (fbcon->helper.fbdev)
- fbcon->helper.fbdev->flags |= FBINFO_HWACCEL_DISABLED;
+ if (fbcon->helper.info)
+ fbcon->helper.info->flags |= FBINFO_HWACCEL_DISABLED;
console_unlock();
nouveau_channel_idle(drm->channel);
nvif_object_dtor(&fbcon->twod);
@@ -272,7 +271,7 @@ nouveau_fbcon_accel_init(struct drm_device *dev)
{
struct nouveau_drm *drm = nouveau_drm(dev);
struct nouveau_fbdev *fbcon = drm->fbcon;
- struct fb_info *info = fbcon->helper.fbdev;
+ struct fb_info *info = fbcon->helper.info;
int ret;
if (drm->client.device.info.family < NV_DEVICE_INFO_V0_TESLA)
@@ -290,7 +289,7 @@ nouveau_fbcon_accel_init(struct drm_device *dev)
static void
nouveau_fbcon_zfill(struct drm_device *dev, struct nouveau_fbdev *fbcon)
{
- struct fb_info *info = fbcon->helper.fbdev;
+ struct fb_info *info = fbcon->helper.info;
struct fb_fillrect rect;
/* Clear the entire fbcon. The drm will program every connector
@@ -363,7 +362,7 @@ nouveau_fbcon_create(struct drm_fb_helper *helper,
}
}
- info = drm_fb_helper_alloc_fbi(helper);
+ info = drm_fb_helper_alloc_info(helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
goto out_unlock;
@@ -420,7 +419,7 @@ nouveau_fbcon_destroy(struct drm_device *dev, struct nouveau_fbdev *fbcon)
struct drm_framebuffer *fb = fbcon->helper.fb;
struct nouveau_bo *nvbo;
- drm_fb_helper_unregister_fbi(&fbcon->helper);
+ drm_fb_helper_unregister_info(&fbcon->helper);
drm_fb_helper_fini(&fbcon->helper);
if (fb && fb->obj[0]) {
@@ -586,8 +585,8 @@ nouveau_fbcon_init(struct drm_device *dev)
if (ret)
goto fini;
- if (fbcon->helper.fbdev)
- fbcon->helper.fbdev->pixmap.buf_align = 4;
+ if (fbcon->helper.info)
+ fbcon->helper.info->pixmap.buf_align = 4;
return 0;
fini:
diff --git a/drivers/gpu/drm/nouveau/nouveau_fbcon.h b/drivers/gpu/drm/nouveau/nouveau_fbcon.h
deleted file mode 100644
index 1796d88..0000000
--- a/drivers/gpu/drm/nouveau/nouveau_fbcon.h
+++ /dev/null
@@ -1,82 +0,0 @@
-/*
- * Copyright (C) 2008 Maarten Maathuis.
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining
- * a copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sublicense, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial
- * portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
- * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
- * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
- * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
- * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- *
- */
-
-#ifndef __NOUVEAU_FBCON_H__
-#define __NOUVEAU_FBCON_H__
-
-#include <drm/drm_fb_helper.h>
-
-#include "nouveau_display.h"
-
-struct nouveau_vma;
-
-struct nouveau_fbdev {
- struct drm_fb_helper helper; /* must be first */
- unsigned int saved_flags;
- struct nvif_object surf2d;
- struct nvif_object clip;
- struct nvif_object rop;
- struct nvif_object patt;
- struct nvif_object gdi;
- struct nvif_object blit;
- struct nvif_object twod;
- struct nouveau_vma *vma;
-
- struct mutex hotplug_lock;
- bool hotplug_waiting;
-};
-
-void nouveau_fbcon_restore(void);
-
-int nv04_fbcon_copyarea(struct fb_info *info, const struct fb_copyarea *region);
-int nv04_fbcon_fillrect(struct fb_info *info, const struct fb_fillrect *rect);
-int nv04_fbcon_imageblit(struct fb_info *info, const struct fb_image *image);
-int nv04_fbcon_accel_init(struct fb_info *info);
-
-int nv50_fbcon_fillrect(struct fb_info *info, const struct fb_fillrect *rect);
-int nv50_fbcon_copyarea(struct fb_info *info, const struct fb_copyarea *region);
-int nv50_fbcon_imageblit(struct fb_info *info, const struct fb_image *image);
-int nv50_fbcon_accel_init(struct fb_info *info);
-
-int nvc0_fbcon_fillrect(struct fb_info *info, const struct fb_fillrect *rect);
-int nvc0_fbcon_copyarea(struct fb_info *info, const struct fb_copyarea *region);
-int nvc0_fbcon_imageblit(struct fb_info *info, const struct fb_image *image);
-int nvc0_fbcon_accel_init(struct fb_info *info);
-
-void nouveau_fbcon_gpu_lockup(struct fb_info *info);
-
-int nouveau_fbcon_init(struct drm_device *dev);
-void nouveau_fbcon_fini(struct drm_device *dev);
-void nouveau_fbcon_set_suspend(struct drm_device *dev, int state);
-void nouveau_fbcon_accel_save_disable(struct drm_device *dev);
-void nouveau_fbcon_accel_restore(struct drm_device *dev);
-
-void nouveau_fbcon_output_poll_changed(struct drm_device *dev);
-void nouveau_fbcon_hotplug_resume(struct nouveau_fbdev *fbcon);
-extern int nouveau_nofbaccel;
-
-#endif /* __NV50_FBCON_H__ */
-
diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.c b/drivers/gpu/drm/nouveau/nouveau_fence.c
index abcac7d..ee5e9d4 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fence.c
+++ b/drivers/gpu/drm/nouveau/nouveau_fence.c
@@ -29,9 +29,7 @@
#include <linux/sched/signal.h>
#include <trace/events/dma_fence.h>
-#include <nvif/cl826e.h>
-#include <nvif/notify.h>
-#include <nvif/event.h>
+#include <nvif/if0020.h>
#include "nouveau_drv.h"
#include "nouveau_dma.h"
@@ -79,10 +77,6 @@ nouveau_local_fence(struct dma_fence *fence, struct nouveau_drm *drm)
fence->ops != &nouveau_fence_ops_uevent)
return NULL;
- if (fence->context < drm->chan.context_base ||
- fence->context >= drm->chan.context_base + drm->chan.nr)
- return NULL;
-
return from_fence(fence);
}
@@ -90,8 +84,9 @@ void
nouveau_fence_context_kill(struct nouveau_fence_chan *fctx, int error)
{
struct nouveau_fence *fence;
+ unsigned long flags;
- spin_lock_irq(&fctx->lock);
+ spin_lock_irqsave(&fctx->lock, flags);
while (!list_empty(&fctx->pending)) {
fence = list_entry(fctx->pending.next, typeof(*fence), head);
@@ -99,16 +94,16 @@ nouveau_fence_context_kill(struct nouveau_fence_chan *fctx, int error)
dma_fence_set_error(&fence->base, error);
if (nouveau_fence_signal(fence))
- nvif_notify_put(&fctx->notify);
+ nvif_event_block(&fctx->event);
}
- spin_unlock_irq(&fctx->lock);
+ spin_unlock_irqrestore(&fctx->lock, flags);
}
void
nouveau_fence_context_del(struct nouveau_fence_chan *fctx)
{
nouveau_fence_context_kill(fctx, 0);
- nvif_notify_dtor(&fctx->notify);
+ nvif_event_dtor(&fctx->event);
fctx->dead = 1;
/*
@@ -150,12 +145,11 @@ nouveau_fence_update(struct nouveau_channel *chan, struct nouveau_fence_chan *fc
}
static int
-nouveau_fence_wait_uevent_handler(struct nvif_notify *notify)
+nouveau_fence_wait_uevent_handler(struct nvif_event *event, void *repv, u32 repc)
{
- struct nouveau_fence_chan *fctx =
- container_of(notify, typeof(*fctx), notify);
+ struct nouveau_fence_chan *fctx = container_of(event, typeof(*fctx), event);
unsigned long flags;
- int ret = NVIF_NOTIFY_KEEP;
+ int ret = NVIF_EVENT_KEEP;
spin_lock_irqsave(&fctx->lock, flags);
if (!list_empty(&fctx->pending)) {
@@ -165,7 +159,7 @@ nouveau_fence_wait_uevent_handler(struct nvif_notify *notify)
fence = list_entry(fctx->pending.next, typeof(*fence), head);
chan = rcu_dereference_protected(fence->channel, lockdep_is_held(&fctx->lock));
if (nouveau_fence_update(chan, fctx))
- ret = NVIF_NOTIFY_DROP;
+ ret = NVIF_EVENT_DROP;
}
spin_unlock_irqrestore(&fctx->lock, flags);
@@ -177,12 +171,16 @@ nouveau_fence_context_new(struct nouveau_channel *chan, struct nouveau_fence_cha
{
struct nouveau_fence_priv *priv = (void*)chan->drm->fence;
struct nouveau_cli *cli = (void *)chan->user.client;
+ struct {
+ struct nvif_event_v0 base;
+ struct nvif_chan_event_v0 host;
+ } args;
int ret;
INIT_LIST_HEAD(&fctx->flip);
INIT_LIST_HEAD(&fctx->pending);
spin_lock_init(&fctx->lock);
- fctx->context = chan->drm->chan.context_base + chan->chid;
+ fctx->context = chan->drm->runl[chan->runlist].context_base + chan->chid;
if (chan == chan->drm->cechan)
strcpy(fctx->name, "copy engine channel");
@@ -195,13 +193,12 @@ nouveau_fence_context_new(struct nouveau_channel *chan, struct nouveau_fence_cha
if (!priv->uevent)
return;
- ret = nvif_notify_ctor(&chan->user, "fenceNonStallIntr",
- nouveau_fence_wait_uevent_handler,
- false, NV826E_V0_NTFY_NON_STALL_INTERRUPT,
- &(struct nvif_notify_uevent_req) { },
- sizeof(struct nvif_notify_uevent_req),
- sizeof(struct nvif_notify_uevent_rep),
- &fctx->notify);
+ args.host.version = 0;
+ args.host.type = NVIF_CHAN_EVENT_V0_NON_STALL_INTR;
+
+ ret = nvif_event_ctor(&chan->user, "fenceNonStallIntr", (chan->runlist << 16) | chan->chid,
+ nouveau_fence_wait_uevent_handler, false,
+ &args.base, sizeof(args), &fctx->event);
WARN_ON(ret);
}
@@ -230,7 +227,7 @@ nouveau_fence_emit(struct nouveau_fence *fence, struct nouveau_channel *chan)
spin_lock_irq(&fctx->lock);
if (nouveau_fence_update(chan, fctx))
- nvif_notify_put(&fctx->notify);
+ nvif_event_block(&fctx->event);
list_add_tail(&fence->head, &fctx->pending);
spin_unlock_irq(&fctx->lock);
@@ -254,7 +251,7 @@ nouveau_fence_done(struct nouveau_fence *fence)
spin_lock_irqsave(&fctx->lock, flags);
chan = rcu_dereference_protected(fence->channel, lockdep_is_held(&fctx->lock));
if (chan && nouveau_fence_update(chan, fctx))
- nvif_notify_put(&fctx->notify);
+ nvif_event_block(&fctx->event);
spin_unlock_irqrestore(&fctx->lock, flags);
}
return dma_fence_is_signaled(&fence->base);
@@ -505,13 +502,13 @@ static bool nouveau_fence_enable_signaling(struct dma_fence *f)
bool ret;
if (!fctx->notify_ref++)
- nvif_notify_get(&fctx->notify);
+ nvif_event_allow(&fctx->event);
ret = nouveau_fence_no_signaling(f);
if (ret)
set_bit(DMA_FENCE_FLAG_USER_BITS, &fence->base.flags);
else if (!--fctx->notify_ref)
- nvif_notify_put(&fctx->notify);
+ nvif_event_block(&fctx->event);
return ret;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.h b/drivers/gpu/drm/nouveau/nouveau_fence.h
index 4887caa..0ca2bc85 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fence.h
+++ b/drivers/gpu/drm/nouveau/nouveau_fence.h
@@ -3,7 +3,7 @@
#define __NOUVEAU_FENCE_H__
#include <linux/dma-fence.h>
-#include <nvif/notify.h>
+#include <nvif/event.h>
struct nouveau_drm;
struct nouveau_bo;
@@ -44,7 +44,7 @@ struct nouveau_fence_chan {
u32 context;
char name[32];
- struct nvif_notify notify;
+ struct nvif_event event;
int notify_ref, dead;
};
diff --git a/drivers/gpu/drm/nouveau/nouveau_gem.c b/drivers/gpu/drm/nouveau/nouveau_gem.c
index fab542a..ac5793c9 100644
--- a/drivers/gpu/drm/nouveau/nouveau_gem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_gem.c
@@ -679,7 +679,7 @@ nouveau_gem_pushbuf_reloc_apply(struct nouveau_cli *cli,
}
if (!nvbo->kmap.virtual) {
- ret = ttm_bo_kmap(&nvbo->bo, 0, nvbo->bo.resource->num_pages,
+ ret = ttm_bo_kmap(&nvbo->bo, 0, PFN_UP(nvbo->bo.base.size),
&nvbo->kmap);
if (ret) {
NV_PRINTK(err, cli, "failed kmap for reloc\n");
@@ -868,8 +868,7 @@ nouveau_gem_ioctl_pushbuf(struct drm_device *dev, void *data,
if (unlikely(cmd != req->suffix0)) {
if (!nvbo->kmap.virtual) {
ret = ttm_bo_kmap(&nvbo->bo, 0,
- nvbo->bo.resource->
- num_pages,
+ PFN_UP(nvbo->bo.base.size),
&nvbo->kmap);
if (ret) {
WIND_RING(chan);
diff --git a/drivers/gpu/drm/nouveau/nouveau_mem.c b/drivers/gpu/drm/nouveau/nouveau_mem.c
index 76f8ede..1fde3a5 100644
--- a/drivers/gpu/drm/nouveau/nouveau_mem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_mem.c
@@ -115,7 +115,7 @@ nouveau_mem_host(struct ttm_resource *reg, struct ttm_tt *tt)
mutex_lock(&drm->master.lock);
ret = nvif_mem_ctor_type(mmu, "ttmHostMem", cli->mem->oclass, type, PAGE_SHIFT,
- reg->num_pages << PAGE_SHIFT,
+ reg->size,
&args, sizeof(args), &mem->mem);
mutex_unlock(&drm->master.lock);
return ret;
@@ -128,7 +128,7 @@ nouveau_mem_vram(struct ttm_resource *reg, bool contig, u8 page)
struct nouveau_cli *cli = mem->cli;
struct nouveau_drm *drm = cli->drm;
struct nvif_mmu *mmu = &cli->mmu;
- u64 size = ALIGN(reg->num_pages << PAGE_SHIFT, 1 << page);
+ u64 size = ALIGN(reg->size, 1 << page);
int ret;
mutex_lock(&drm->master.lock);
diff --git a/drivers/gpu/drm/nouveau/nouveau_nvif.c b/drivers/gpu/drm/nouveau/nouveau_nvif.c
index df0fe58..1d49ebd 100644
--- a/drivers/gpu/drm/nouveau/nouveau_nvif.c
+++ b/drivers/gpu/drm/nouveau/nouveau_nvif.c
@@ -27,12 +27,10 @@
******************************************************************************/
#include <core/client.h>
-#include <core/notify.h>
#include <core/ioctl.h>
#include <nvif/client.h>
#include <nvif/driver.h>
-#include <nvif/notify.h>
#include <nvif/event.h>
#include <nvif/ioctl.h>
@@ -72,10 +70,23 @@ nvkm_client_suspend(void *priv)
}
static int
+nvkm_client_event(u64 token, void *repv, u32 repc)
+{
+ struct nvif_object *object = (void *)(unsigned long)token;
+ struct nvif_event *event = container_of(object, typeof(*event), object);
+
+ if (event->func(event, repv, repc) == NVIF_EVENT_KEEP)
+ return NVKM_EVENT_KEEP;
+
+ return NVKM_EVENT_DROP;
+}
+
+static int
nvkm_client_driver_init(const char *name, u64 device, const char *cfg,
const char *dbg, void **ppriv)
{
- return nvkm_client_new(name, device, cfg, dbg, nvif_notify, (struct nvkm_client **)ppriv);
+ return nvkm_client_new(name, device, cfg, dbg, nvkm_client_event,
+ (struct nvkm_client **)ppriv);
}
const struct nvif_driver
diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c
index 31a5b81..a74ba8d 100644
--- a/drivers/gpu/drm/nouveau/nouveau_svm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_svm.c
@@ -24,7 +24,7 @@
#include "nouveau_chan.h"
#include "nouveau_dmem.h"
-#include <nvif/notify.h>
+#include <nvif/event.h>
#include <nvif/object.h>
#include <nvif/vmm.h>
@@ -51,7 +51,8 @@ struct nouveau_svm {
u32 putaddr;
u32 get;
u32 put;
- struct nvif_notify notify;
+ struct nvif_event notify;
+ struct work_struct work;
struct nouveau_svm_fault {
u64 inst;
@@ -711,13 +712,11 @@ static int nouveau_range_fault(struct nouveau_svmm *svmm,
return ret;
}
-static int
-nouveau_svm_fault(struct nvif_notify *notify)
+static void
+nouveau_svm_fault(struct work_struct *work)
{
- struct nouveau_svm_fault_buffer *buffer =
- container_of(notify, typeof(*buffer), notify);
- struct nouveau_svm *svm =
- container_of(buffer, typeof(*svm), buffer[buffer->id]);
+ struct nouveau_svm_fault_buffer *buffer = container_of(work, typeof(*buffer), work);
+ struct nouveau_svm *svm = container_of(buffer, typeof(*svm), buffer[buffer->id]);
struct nvif_object *device = &svm->drm->client.device.object;
struct nouveau_svmm *svmm;
struct {
@@ -737,7 +736,7 @@ nouveau_svm_fault(struct nvif_notify *notify)
buffer->put = nvif_rd32(device, buffer->putaddr);
buffer->get = nvif_rd32(device, buffer->getaddr);
if (buffer->get == buffer->put)
- return NVIF_NOTIFY_KEEP;
+ return;
}
buffer->fault_nr = 0;
@@ -881,7 +880,15 @@ nouveau_svm_fault(struct nvif_notify *notify)
/* Issue fault replay to the GPU. */
if (replay)
nouveau_svm_fault_replay(svm);
- return NVIF_NOTIFY_KEEP;
+}
+
+static int
+nouveau_svm_event(struct nvif_event *event, void *argv, u32 argc)
+{
+ struct nouveau_svm_fault_buffer *buffer = container_of(event, typeof(*buffer), notify);
+
+ schedule_work(&buffer->work);
+ return NVIF_EVENT_KEEP;
}
static struct nouveau_pfnmap_args *
@@ -936,7 +943,9 @@ static void
nouveau_svm_fault_buffer_fini(struct nouveau_svm *svm, int id)
{
struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id];
- nvif_notify_put(&buffer->notify);
+
+ nvif_event_block(&buffer->notify);
+ flush_work(&buffer->work);
}
static int
@@ -944,10 +953,12 @@ nouveau_svm_fault_buffer_init(struct nouveau_svm *svm, int id)
{
struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id];
struct nvif_object *device = &svm->drm->client.device.object;
+
buffer->get = nvif_rd32(device, buffer->getaddr);
buffer->put = nvif_rd32(device, buffer->putaddr);
SVM_DBG(svm, "get %08x put %08x (init)", buffer->get, buffer->put);
- return nvif_notify_get(&buffer->notify);
+
+ return nvif_event_allow(&buffer->notify);
}
static void
@@ -956,15 +967,18 @@ nouveau_svm_fault_buffer_dtor(struct nouveau_svm *svm, int id)
struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id];
int i;
+ if (!nvif_object_constructed(&buffer->object))
+ return;
+
+ nouveau_svm_fault_buffer_fini(svm, id);
+
if (buffer->fault) {
for (i = 0; buffer->fault[i] && i < buffer->entries; i++)
kfree(buffer->fault[i]);
kvfree(buffer->fault);
}
- nouveau_svm_fault_buffer_fini(svm, id);
-
- nvif_notify_dtor(&buffer->notify);
+ nvif_event_dtor(&buffer->notify);
nvif_object_dtor(&buffer->object);
}
@@ -990,10 +1004,10 @@ nouveau_svm_fault_buffer_ctor(struct nouveau_svm *svm, s32 oclass, int id)
buffer->entries = args.entries;
buffer->getaddr = args.get;
buffer->putaddr = args.put;
+ INIT_WORK(&buffer->work, nouveau_svm_fault);
- ret = nvif_notify_ctor(&buffer->object, "svmFault", nouveau_svm_fault,
- true, NVB069_V0_NTFY_FAULT, NULL, 0, 0,
- &buffer->notify);
+ ret = nvif_event_ctor(&buffer->object, "svmFault", id, nouveau_svm_event, true, NULL, 0,
+ &buffer->notify);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/nouveau/nouveau_ttm.c b/drivers/gpu/drm/nouveau/nouveau_ttm.c
index 9602c30..1469a88 100644
--- a/drivers/gpu/drm/nouveau/nouveau_ttm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_ttm.c
@@ -139,7 +139,7 @@ nv04_gart_manager_new(struct ttm_resource_manager *man,
mem = nouveau_mem(*res);
ttm_resource_init(bo, place, *res);
ret = nvif_vmm_get(&mem->cli->vmm.vmm, PTES, false, 12, 0,
- (long)(*res)->num_pages << PAGE_SHIFT, &mem->vma[0]);
+ (long)(*res)->size, &mem->vma[0]);
if (ret) {
nouveau_mem_del(man, *res);
return ret;
diff --git a/drivers/gpu/drm/nouveau/nouveau_usif.c b/drivers/gpu/drm/nouveau/nouveau_usif.c
index 36df6840c..002d147 100644
--- a/drivers/gpu/drm/nouveau/nouveau_usif.c
+++ b/drivers/gpu/drm/nouveau/nouveau_usif.c
@@ -151,12 +151,6 @@ usif_ioctl(struct drm_file *filp, void __user *user, u32 argc)
case NVIF_IOCTL_V0_NEW:
ret = usif_object_new(filp, data, size, argv, argc, abi16);
break;
- case NVIF_IOCTL_V0_NTFY_NEW:
- case NVIF_IOCTL_V0_NTFY_DEL:
- case NVIF_IOCTL_V0_NTFY_GET:
- case NVIF_IOCTL_V0_NTFY_PUT:
- ret = -ENOSYS;
- break;
default:
ret = nvif_client_ioctl(client, argv, argc);
break;
diff --git a/drivers/gpu/drm/nouveau/nouveau_vga.c b/drivers/gpu/drm/nouveau/nouveau_vga.c
index 60cd8c0..789393b 100644
--- a/drivers/gpu/drm/nouveau/nouveau_vga.c
+++ b/drivers/gpu/drm/nouveau/nouveau_vga.c
@@ -7,7 +7,6 @@
#include "nouveau_drv.h"
#include "nouveau_acpi.h"
-#include "nouveau_fbcon.h"
#include "nouveau_vga.h"
static unsigned int
diff --git a/drivers/gpu/drm/nouveau/nv04_fbcon.c b/drivers/gpu/drm/nouveau/nv04_fbcon.c
deleted file mode 100644
index 92f3fb6..0000000
--- a/drivers/gpu/drm/nouveau/nv04_fbcon.c
+++ /dev/null
@@ -1,257 +0,0 @@
-/*
- * Copyright 2009 Ben Skeggs
- * Copyright 2008 Stuart Bennett
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- * DEALINGS IN THE SOFTWARE.
- */
-#define NVIF_DEBUG_PRINT_DISABLE
-#include "nouveau_drv.h"
-#include "nouveau_dma.h"
-#include "nouveau_fbcon.h"
-
-#include <nvif/push006c.h>
-
-int
-nv04_fbcon_copyarea(struct fb_info *info, const struct fb_copyarea *region)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- int ret;
-
- ret = PUSH_WAIT(push, 4);
- if (ret)
- return ret;
-
- PUSH_NVSQ(push, NV05F, 0x0300, (region->sy << 16) | region->sx,
- 0x0304, (region->dy << 16) | region->dx,
- 0x0308, (region->height << 16) | region->width);
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nv04_fbcon_fillrect(struct fb_info *info, const struct fb_fillrect *rect)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- int ret;
-
- ret = PUSH_WAIT(push, 7);
- if (ret)
- return ret;
-
- PUSH_NVSQ(push, NV04A, 0x02fc, (rect->rop != ROP_COPY) ? 1 : 3);
- if (info->fix.visual == FB_VISUAL_TRUECOLOR ||
- info->fix.visual == FB_VISUAL_DIRECTCOLOR)
- PUSH_NVSQ(push, NV04A, 0x03fc, ((uint32_t *)info->pseudo_palette)[rect->color]);
- else
- PUSH_NVSQ(push, NV04A, 0x03fc, rect->color);
- PUSH_NVSQ(push, NV04A, 0x0400, (rect->dx << 16) | rect->dy,
- 0x0404, (rect->width << 16) | rect->height);
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nv04_fbcon_imageblit(struct fb_info *info, const struct fb_image *image)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- uint32_t fg;
- uint32_t bg;
- uint32_t dsize;
- uint32_t *data = (uint32_t *)image->data;
- int ret;
-
- if (image->depth != 1)
- return -ENODEV;
-
- ret = PUSH_WAIT(push, 8);
- if (ret)
- return ret;
-
- if (info->fix.visual == FB_VISUAL_TRUECOLOR ||
- info->fix.visual == FB_VISUAL_DIRECTCOLOR) {
- fg = ((uint32_t *) info->pseudo_palette)[image->fg_color];
- bg = ((uint32_t *) info->pseudo_palette)[image->bg_color];
- } else {
- fg = image->fg_color;
- bg = image->bg_color;
- }
-
- PUSH_NVSQ(push, NV04A, 0x0be4, (image->dy << 16) | (image->dx & 0xffff),
- 0x0be8, ((image->dy + image->height) << 16) |
- ((image->dx + image->width) & 0xffff),
- 0x0bec, bg,
- 0x0bf0, fg,
- 0x0bf4, (image->height << 16) | ALIGN(image->width, 8),
- 0x0bf8, (image->height << 16) | image->width,
- 0x0bfc, (image->dy << 16) | (image->dx & 0xffff));
-
- dsize = ALIGN(ALIGN(image->width, 8) * image->height, 32) >> 5;
- while (dsize) {
- int iter_len = dsize > 128 ? 128 : dsize;
-
- ret = PUSH_WAIT(push, iter_len + 1);
- if (ret)
- return ret;
-
- PUSH_NVSQ(push, NV04A, 0x0c00, data, iter_len);
- data += iter_len;
- dsize -= iter_len;
- }
-
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nv04_fbcon_accel_init(struct fb_info *info)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct drm_device *dev = nfbdev->helper.dev;
- struct nouveau_drm *drm = nouveau_drm(dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_device *device = &drm->client.device;
- struct nvif_push *push = chan->chan.push;
- int surface_fmt, pattern_fmt, rect_fmt;
- int ret;
-
- switch (info->var.bits_per_pixel) {
- case 8:
- surface_fmt = 1;
- pattern_fmt = 3;
- rect_fmt = 3;
- break;
- case 16:
- surface_fmt = 4;
- pattern_fmt = 1;
- rect_fmt = 1;
- break;
- case 32:
- switch (info->var.transp.length) {
- case 0: /* depth 24 */
- case 8: /* depth 32 */
- break;
- default:
- return -EINVAL;
- }
-
- surface_fmt = 6;
- pattern_fmt = 3;
- rect_fmt = 3;
- break;
- default:
- return -EINVAL;
- }
-
- ret = nvif_object_ctor(&chan->user, "fbconCtxSurf2d", 0x0062,
- device->info.family >= NV_DEVICE_INFO_V0_CELSIUS ?
- 0x0062 : 0x0042, NULL, 0, &nfbdev->surf2d);
- if (ret)
- return ret;
-
- ret = nvif_object_ctor(&chan->user, "fbconCtxClip", 0x0019, 0x0019,
- NULL, 0, &nfbdev->clip);
- if (ret)
- return ret;
-
- ret = nvif_object_ctor(&chan->user, "fbconCtxRop", 0x0043, 0x0043,
- NULL, 0, &nfbdev->rop);
- if (ret)
- return ret;
-
- ret = nvif_object_ctor(&chan->user, "fbconCtxPatt", 0x0044, 0x0044,
- NULL, 0, &nfbdev->patt);
- if (ret)
- return ret;
-
- ret = nvif_object_ctor(&chan->user, "fbconGdiRectText", 0x004a, 0x004a,
- NULL, 0, &nfbdev->gdi);
- if (ret)
- return ret;
-
- ret = nvif_object_ctor(&chan->user, "fbconImageBlit", 0x005f,
- device->info.chipset >= 0x11 ? 0x009f : 0x005f,
- NULL, 0, &nfbdev->blit);
- if (ret)
- return ret;
-
- if (PUSH_WAIT(push, 49 + (device->info.chipset >= 0x11 ? 4 : 0))) {
- nouveau_fbcon_gpu_lockup(info);
- return 0;
- }
-
- PUSH_NVSQ(push, NV042, 0x0000, nfbdev->surf2d.handle);
- PUSH_NVSQ(push, NV042, 0x0184, chan->vram.handle,
- 0x0188, chan->vram.handle);
- PUSH_NVSQ(push, NV042, 0x0300, surface_fmt,
- 0x0304, info->fix.line_length | (info->fix.line_length << 16),
- 0x0308, info->fix.smem_start - dev->mode_config.fb_base,
- 0x030c, info->fix.smem_start - dev->mode_config.fb_base);
-
- PUSH_NVSQ(push, NV043, 0x0000, nfbdev->rop.handle);
- PUSH_NVSQ(push, NV043, 0x0300, 0x55);
-
- PUSH_NVSQ(push, NV044, 0x0000, nfbdev->patt.handle);
- PUSH_NVSQ(push, NV044, 0x0300, pattern_fmt,
-#ifdef __BIG_ENDIAN
- 0x0304, 2,
-#else
- 0x0304, 1,
-#endif
- 0x0308, 0,
- 0x030c, 1,
- 0x0310, ~0,
- 0x0314, ~0,
- 0x0318, ~0,
- 0x031c, ~0);
-
- PUSH_NVSQ(push, NV019, 0x0000, nfbdev->clip.handle);
- PUSH_NVSQ(push, NV019, 0x0300, 0,
- 0x0304, (info->var.yres_virtual << 16) | info->var.xres_virtual);
-
- PUSH_NVSQ(push, NV05F, 0x0000, nfbdev->blit.handle);
- PUSH_NVSQ(push, NV05F, 0x019c, nfbdev->surf2d.handle);
- PUSH_NVSQ(push, NV05F, 0x02fc, 3);
- if (nfbdev->blit.oclass == 0x009f) {
- PUSH_NVSQ(push, NV09F, 0x0120, 0,
- 0x0124, 1,
- 0x0128, 2);
- }
-
- PUSH_NVSQ(push, NV04A, 0x0000, nfbdev->gdi.handle);
- PUSH_NVSQ(push, NV04A, 0x0198, nfbdev->surf2d.handle);
- PUSH_NVSQ(push, NV04A, 0x0188, nfbdev->patt.handle,
- 0x018c, nfbdev->rop.handle);
- PUSH_NVSQ(push, NV04A, 0x0304, 1);
- PUSH_NVSQ(push, NV04A, 0x0300, rect_fmt);
- PUSH_NVSQ(push, NV04A, 0x02fc, 3);
-
- PUSH_KICK(push);
- return 0;
-}
-
diff --git a/drivers/gpu/drm/nouveau/nv50_fbcon.c b/drivers/gpu/drm/nouveau/nv50_fbcon.c
deleted file mode 100644
index 71f92e4..0000000
--- a/drivers/gpu/drm/nouveau/nv50_fbcon.c
+++ /dev/null
@@ -1,299 +0,0 @@
-/*
- * Copyright 2010 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#define NVIF_DEBUG_PRINT_DISABLE
-#include "nouveau_drv.h"
-#include "nouveau_dma.h"
-#include "nouveau_fbcon.h"
-#include "nouveau_vmm.h"
-
-#include <nvif/push206e.h>
-
-#include <nvhw/class/cl502d.h>
-
-int
-nv50_fbcon_fillrect(struct fb_info *info, const struct fb_fillrect *rect)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- u32 colour;
- int ret;
-
- if (info->fix.visual == FB_VISUAL_TRUECOLOR ||
- info->fix.visual == FB_VISUAL_DIRECTCOLOR)
- colour = ((uint32_t *)info->pseudo_palette)[rect->color];
- else
- colour = rect->color;
-
- ret = PUSH_WAIT(push, rect->rop == ROP_COPY ? 7 : 11);
- if (ret)
- return ret;
-
- if (rect->rop != ROP_COPY) {
- PUSH_MTHD(push, NV502D, SET_OPERATION,
- NVDEF(NV502D, SET_OPERATION, V, ROP_AND));
- }
-
- PUSH_MTHD(push, NV502D, SET_RENDER_SOLID_PRIM_COLOR, colour);
-
- PUSH_MTHD(push, NV502D, RENDER_SOLID_PRIM_POINT_SET_X(0), rect->dx,
- RENDER_SOLID_PRIM_POINT_Y(0), rect->dy,
- RENDER_SOLID_PRIM_POINT_SET_X(1), rect->dx + rect->width,
- RENDER_SOLID_PRIM_POINT_Y(1), rect->dy + rect->height);
-
- if (rect->rop != ROP_COPY) {
- PUSH_MTHD(push, NV502D, SET_OPERATION,
- NVDEF(NV502D, SET_OPERATION, V, SRCCOPY));
- }
-
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nv50_fbcon_copyarea(struct fb_info *info, const struct fb_copyarea *region)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- int ret;
-
- ret = PUSH_WAIT(push, 12);
- if (ret)
- return ret;
-
- PUSH_MTHD(push, NV502D, WAIT_FOR_IDLE, 0);
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_MEMORY_DST_X0, region->dx,
- SET_PIXELS_FROM_MEMORY_DST_Y0, region->dy,
- SET_PIXELS_FROM_MEMORY_DST_WIDTH, region->width,
- SET_PIXELS_FROM_MEMORY_DST_HEIGHT, region->height);
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_MEMORY_SRC_X0_FRAC, 0,
- SET_PIXELS_FROM_MEMORY_SRC_X0_INT, region->sx,
- SET_PIXELS_FROM_MEMORY_SRC_Y0_FRAC, 0,
- PIXELS_FROM_MEMORY_SRC_Y0_INT, region->sy);
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nv50_fbcon_imageblit(struct fb_info *info, const struct fb_image *image)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- uint32_t dwords, *data = (uint32_t *)image->data;
- uint32_t mask = ~(~0 >> (32 - info->var.bits_per_pixel));
- uint32_t *palette = info->pseudo_palette, bg, fg;
- int ret;
-
- if (image->depth != 1)
- return -ENODEV;
-
- if (info->fix.visual == FB_VISUAL_TRUECOLOR ||
- info->fix.visual == FB_VISUAL_DIRECTCOLOR) {
- bg = palette[image->bg_color] | mask;
- fg = palette[image->fg_color] | mask;
- } else {
- bg = image->bg_color;
- fg = image->fg_color;
- }
-
- ret = PUSH_WAIT(push, 11);
- if (ret)
- return ret;
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_CPU_COLOR0, bg,
- SET_PIXELS_FROM_CPU_COLOR1, fg);
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_CPU_SRC_WIDTH, image->width,
- SET_PIXELS_FROM_CPU_SRC_HEIGHT, image->height);
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_CPU_DST_X0_FRAC, 0,
- SET_PIXELS_FROM_CPU_DST_X0_INT, image->dx,
- SET_PIXELS_FROM_CPU_DST_Y0_FRAC, 0,
- SET_PIXELS_FROM_CPU_DST_Y0_INT, image->dy);
-
- dwords = ALIGN(ALIGN(image->width, 8) * image->height, 32) >> 5;
- while (dwords) {
- int count = dwords > 2047 ? 2047 : dwords;
-
- ret = PUSH_WAIT(push, count + 1);
- if (ret)
- return ret;
-
- dwords -= count;
-
- PUSH_NINC(push, NV502D, PIXELS_FROM_CPU_DATA, data, count);
- data += count;
- }
-
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nv50_fbcon_accel_init(struct fb_info *info)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct drm_device *dev = nfbdev->helper.dev;
- struct nouveau_drm *drm = nouveau_drm(dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- int ret, format;
-
- switch (info->var.bits_per_pixel) {
- case 8:
- format = NV502D_SET_DST_FORMAT_V_Y8;
- break;
- case 15:
- format = NV502D_SET_DST_FORMAT_V_X1R5G5B5;
- break;
- case 16:
- format = NV502D_SET_DST_FORMAT_V_R5G6B5;
- break;
- case 32:
- switch (info->var.transp.length) {
- case 0: /* depth 24 */
- case 8: /* depth 32, just use 24.. */
- format = NV502D_SET_DST_FORMAT_V_X8R8G8B8;
- break;
- case 2: /* depth 30 */
- format = NV502D_SET_DST_FORMAT_V_A2B10G10R10;
- break;
- default:
- return -EINVAL;
- }
- break;
- default:
- return -EINVAL;
- }
-
- ret = nvif_object_ctor(&chan->user, "fbconTwoD", 0x502d, 0x502d,
- NULL, 0, &nfbdev->twod);
- if (ret)
- return ret;
-
- ret = PUSH_WAIT(push, 56);
- if (ret) {
- nouveau_fbcon_gpu_lockup(info);
- return ret;
- }
-
- PUSH_MTHD(push, NV502D, SET_OBJECT, nfbdev->twod.handle);
- PUSH_MTHD(push, NV502D, SET_DST_CONTEXT_DMA, chan->vram.handle,
- SET_SRC_CONTEXT_DMA, chan->vram.handle,
- SET_SEMAPHORE_CONTEXT_DMA, chan->vram.handle);
-
- PUSH_MTHD(push, NV502D, SET_DST_FORMAT,
- NVVAL(NV502D, SET_DST_FORMAT, V, format),
-
- SET_DST_MEMORY_LAYOUT,
- NVDEF(NV502D, SET_DST_MEMORY_LAYOUT, V, PITCH));
-
- PUSH_MTHD(push, NV502D, SET_DST_PITCH, info->fix.line_length,
- SET_DST_WIDTH, info->var.xres_virtual,
- SET_DST_HEIGHT, info->var.yres_virtual,
-
- SET_DST_OFFSET_UPPER,
- NVVAL(NV502D, SET_DST_OFFSET_UPPER, V, upper_32_bits(nfbdev->vma->addr)),
-
- SET_DST_OFFSET_LOWER,
- NVVAL(NV502D, SET_DST_OFFSET_LOWER, V, lower_32_bits(nfbdev->vma->addr)));
-
- PUSH_MTHD(push, NV502D, SET_SRC_FORMAT,
- NVVAL(NV502D, SET_SRC_FORMAT, V, format),
-
- SET_SRC_MEMORY_LAYOUT,
- NVDEF(NV502D, SET_SRC_MEMORY_LAYOUT, V, PITCH));
-
- PUSH_MTHD(push, NV502D, SET_SRC_PITCH, info->fix.line_length,
- SET_SRC_WIDTH, info->var.xres_virtual,
- SET_SRC_HEIGHT, info->var.yres_virtual,
-
- SET_SRC_OFFSET_UPPER,
- NVVAL(NV502D, SET_SRC_OFFSET_UPPER, V, upper_32_bits(nfbdev->vma->addr)),
-
- SET_SRC_OFFSET_LOWER,
- NVVAL(NV502D, SET_SRC_OFFSET_LOWER, V, lower_32_bits(nfbdev->vma->addr)));
-
- PUSH_MTHD(push, NV502D, SET_CLIP_ENABLE,
- NVDEF(NV502D, SET_CLIP_ENABLE, V, FALSE));
-
- PUSH_MTHD(push, NV502D, SET_ROP,
- NVVAL(NV502D, SET_ROP, V, 0x55));
-
- PUSH_MTHD(push, NV502D, SET_OPERATION,
- NVDEF(NV502D, SET_OPERATION, V, SRCCOPY));
-
- PUSH_MTHD(push, NV502D, SET_MONOCHROME_PATTERN_COLOR_FORMAT,
- NVDEF(NV502D, SET_MONOCHROME_PATTERN_COLOR_FORMAT, V, A8R8G8B8),
-
- SET_MONOCHROME_PATTERN_FORMAT,
- NVDEF(NV502D, SET_MONOCHROME_PATTERN_FORMAT, V, LE_M1));
-
- PUSH_MTHD(push, NV502D, RENDER_SOLID_PRIM_MODE,
- NVDEF(NV502D, RENDER_SOLID_PRIM_MODE, V, RECTS),
-
- SET_RENDER_SOLID_PRIM_COLOR_FORMAT,
- NVVAL(NV502D, SET_RENDER_SOLID_PRIM_COLOR_FORMAT, V, format));
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_CPU_DATA_TYPE,
- NVDEF(NV502D, SET_PIXELS_FROM_CPU_DATA_TYPE, V, INDEX),
-
- SET_PIXELS_FROM_CPU_COLOR_FORMAT,
- NVVAL(NV502D, SET_PIXELS_FROM_CPU_COLOR_FORMAT, V, format),
-
- SET_PIXELS_FROM_CPU_INDEX_FORMAT,
- NVDEF(NV502D, SET_PIXELS_FROM_CPU_INDEX_FORMAT, V, I1),
-
- SET_PIXELS_FROM_CPU_MONO_FORMAT,
- NVDEF(NV502D, SET_PIXELS_FROM_CPU_MONO_FORMAT, V, CGA6_M1),
-
- SET_PIXELS_FROM_CPU_WRAP,
- NVDEF(NV502D, SET_PIXELS_FROM_CPU_WRAP, V, WRAP_BYTE));
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_CPU_MONO_OPACITY,
- NVDEF(NV502D, SET_PIXELS_FROM_CPU_MONO_OPACITY, V, OPAQUE));
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_CPU_DX_DU_FRAC, 0,
- SET_PIXELS_FROM_CPU_DX_DU_INT, 1,
- SET_PIXELS_FROM_CPU_DY_DV_FRAC, 0,
- SET_PIXELS_FROM_CPU_DY_DV_INT, 1);
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_MEMORY_SAFE_OVERLAP,
- NVDEF(NV502D, SET_PIXELS_FROM_MEMORY_SAFE_OVERLAP, V, TRUE));
-
- PUSH_MTHD(push, NV502D, SET_PIXELS_FROM_MEMORY_DU_DX_FRAC, 0,
- SET_PIXELS_FROM_MEMORY_DU_DX_INT, 1,
- SET_PIXELS_FROM_MEMORY_DV_DY_FRAC, 0,
- SET_PIXELS_FROM_MEMORY_DV_DY_INT, 1);
- PUSH_KICK(push);
- return 0;
-}
-
diff --git a/drivers/gpu/drm/nouveau/nv84_fence.c b/drivers/gpu/drm/nouveau/nv84_fence.c
index c3526a8..812b8c6 100644
--- a/drivers/gpu/drm/nouveau/nv84_fence.c
+++ b/drivers/gpu/drm/nouveau/nv84_fence.c
@@ -76,12 +76,18 @@ nv84_fence_sync32(struct nouveau_channel *chan, u64 virtual, u32 sequence)
return ret;
}
+static inline u32
+nv84_fence_chid(struct nouveau_channel *chan)
+{
+ return chan->drm->runl[chan->runlist].chan_id_base + chan->chid;
+}
+
static int
nv84_fence_emit(struct nouveau_fence *fence)
{
struct nouveau_channel *chan = fence->channel;
struct nv84_fence_chan *fctx = chan->fence;
- u64 addr = fctx->vma->addr + chan->chid * 16;
+ u64 addr = fctx->vma->addr + nv84_fence_chid(chan) * 16;
return fctx->base.emit32(chan, addr, fence->base.seqno);
}
@@ -91,7 +97,7 @@ nv84_fence_sync(struct nouveau_fence *fence,
struct nouveau_channel *prev, struct nouveau_channel *chan)
{
struct nv84_fence_chan *fctx = chan->fence;
- u64 addr = fctx->vma->addr + prev->chid * 16;
+ u64 addr = fctx->vma->addr + nv84_fence_chid(prev) * 16;
return fctx->base.sync32(chan, addr, fence->base.seqno);
}
@@ -100,7 +106,7 @@ static u32
nv84_fence_read(struct nouveau_channel *chan)
{
struct nv84_fence_priv *priv = chan->drm->fence;
- return nouveau_bo_rd32(priv->bo, chan->chid * 16/4);
+ return nouveau_bo_rd32(priv->bo, nv84_fence_chid(chan) * 16/4);
}
static void
@@ -109,7 +115,7 @@ nv84_fence_context_del(struct nouveau_channel *chan)
struct nv84_fence_priv *priv = chan->drm->fence;
struct nv84_fence_chan *fctx = chan->fence;
- nouveau_bo_wr32(priv->bo, chan->chid * 16 / 4, fctx->base.sequence);
+ nouveau_bo_wr32(priv->bo, nv84_fence_chid(chan) * 16 / 4, fctx->base.sequence);
mutex_lock(&priv->mutex);
nouveau_vma_del(&fctx->vma);
mutex_unlock(&priv->mutex);
@@ -152,9 +158,9 @@ nv84_fence_suspend(struct nouveau_drm *drm)
struct nv84_fence_priv *priv = drm->fence;
int i;
- priv->suspend = vmalloc(array_size(sizeof(u32), drm->chan.nr));
+ priv->suspend = vmalloc(array_size(sizeof(u32), drm->chan_total));
if (priv->suspend) {
- for (i = 0; i < drm->chan.nr; i++)
+ for (i = 0; i < drm->chan_total; i++)
priv->suspend[i] = nouveau_bo_rd32(priv->bo, i*4);
}
@@ -168,7 +174,7 @@ nv84_fence_resume(struct nouveau_drm *drm)
int i;
if (priv->suspend) {
- for (i = 0; i < drm->chan.nr; i++)
+ for (i = 0; i < drm->chan_total; i++)
nouveau_bo_wr32(priv->bo, i*4, priv->suspend[i]);
vfree(priv->suspend);
priv->suspend = NULL;
@@ -204,7 +210,7 @@ nv84_fence_create(struct nouveau_drm *drm)
priv->base.context_new = nv84_fence_context_new;
priv->base.context_del = nv84_fence_context_del;
- priv->base.uevent = drm->client.device.info.family < NV_DEVICE_INFO_V0_AMPERE;
+ priv->base.uevent = true;
mutex_init(&priv->mutex);
@@ -216,7 +222,7 @@ nv84_fence_create(struct nouveau_drm *drm)
* will lose CPU/GPU coherency!
*/
NOUVEAU_GEM_DOMAIN_GART | NOUVEAU_GEM_DOMAIN_COHERENT;
- ret = nouveau_bo_new(&drm->client, 16 * drm->chan.nr, 0,
+ ret = nouveau_bo_new(&drm->client, 16 * drm->chan_total, 0,
domain, 0, 0, NULL, NULL, &priv->bo);
if (ret == 0) {
ret = nouveau_bo_pin(priv->bo, domain, false);
diff --git a/drivers/gpu/drm/nouveau/nvc0_fbcon.c b/drivers/gpu/drm/nouveau/nvc0_fbcon.c
deleted file mode 100644
index 7908a1a..0000000
--- a/drivers/gpu/drm/nouveau/nvc0_fbcon.c
+++ /dev/null
@@ -1,297 +0,0 @@
-/*
- * Copyright 2010 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#define NVIF_DEBUG_PRINT_DISABLE
-#include "nouveau_drv.h"
-#include "nouveau_dma.h"
-#include "nouveau_fbcon.h"
-#include "nouveau_vmm.h"
-
-#include <nvif/push906f.h>
-
-#include <nvhw/class/cl902d.h>
-
-int
-nvc0_fbcon_fillrect(struct fb_info *info, const struct fb_fillrect *rect)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- u32 colour;
- int ret;
-
- if (info->fix.visual == FB_VISUAL_TRUECOLOR ||
- info->fix.visual == FB_VISUAL_DIRECTCOLOR)
- colour = ((uint32_t *)info->pseudo_palette)[rect->color];
- else
- colour = rect->color;
-
- ret = PUSH_WAIT(push, rect->rop == ROP_COPY ? 7 : 9);
- if (ret)
- return ret;
-
- if (rect->rop != ROP_COPY) {
- PUSH_IMMD(push, NV902D, SET_OPERATION,
- NVDEF(NV902D, SET_OPERATION, V, ROP_AND));
- }
-
- PUSH_MTHD(push, NV902D, SET_RENDER_SOLID_PRIM_COLOR, colour);
-
- PUSH_MTHD(push, NV902D, RENDER_SOLID_PRIM_POINT_SET_X(0), rect->dx,
- RENDER_SOLID_PRIM_POINT_Y(0), rect->dy,
- RENDER_SOLID_PRIM_POINT_SET_X(1), rect->dx + rect->width,
- RENDER_SOLID_PRIM_POINT_Y(1), rect->dy + rect->height);
-
- if (rect->rop != ROP_COPY) {
- PUSH_IMMD(push, NV902D, SET_OPERATION,
- NVDEF(NV902D, SET_OPERATION, V, SRCCOPY));
- }
-
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nvc0_fbcon_copyarea(struct fb_info *info, const struct fb_copyarea *region)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- int ret;
-
- ret = PUSH_WAIT(push, 11);
- if (ret)
- return ret;
-
- PUSH_IMMD(push, NV902D, WAIT_FOR_IDLE, 0);
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_MEMORY_DST_X0, region->dx,
- SET_PIXELS_FROM_MEMORY_DST_Y0, region->dy,
- SET_PIXELS_FROM_MEMORY_DST_WIDTH, region->width,
- SET_PIXELS_FROM_MEMORY_DST_HEIGHT, region->height);
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_MEMORY_SRC_X0_FRAC, 0,
- SET_PIXELS_FROM_MEMORY_SRC_X0_INT, region->sx,
- SET_PIXELS_FROM_MEMORY_SRC_Y0_FRAC, 0,
- PIXELS_FROM_MEMORY_SRC_Y0_INT, region->sy);
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nvc0_fbcon_imageblit(struct fb_info *info, const struct fb_image *image)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct nouveau_drm *drm = nouveau_drm(nfbdev->helper.dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- uint32_t dwords, *data = (uint32_t *)image->data;
- uint32_t mask = ~(~0 >> (32 - info->var.bits_per_pixel));
- uint32_t *palette = info->pseudo_palette, bg, fg;
- int ret;
-
- if (image->depth != 1)
- return -ENODEV;
-
- if (info->fix.visual == FB_VISUAL_TRUECOLOR ||
- info->fix.visual == FB_VISUAL_DIRECTCOLOR) {
- bg = palette[image->bg_color] | mask;
- fg = palette[image->fg_color] | mask;
- } else {
- bg = image->bg_color;
- fg = image->fg_color;
- }
-
- ret = PUSH_WAIT(push, 11);
- if (ret)
- return ret;
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_CPU_COLOR0, bg,
- SET_PIXELS_FROM_CPU_COLOR1, fg);
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_CPU_SRC_WIDTH, image->width,
- SET_PIXELS_FROM_CPU_SRC_HEIGHT, image->height);
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_CPU_DST_X0_FRAC, 0,
- SET_PIXELS_FROM_CPU_DST_X0_INT, image->dx,
- SET_PIXELS_FROM_CPU_DST_Y0_FRAC, 0,
- SET_PIXELS_FROM_CPU_DST_Y0_INT, image->dy);
-
- dwords = ALIGN(ALIGN(image->width, 8) * image->height, 32) >> 5;
- while (dwords) {
- int count = dwords > 2047 ? 2047 : dwords;
-
- ret = PUSH_WAIT(push, count + 1);
- if (ret)
- return ret;
-
- dwords -= count;
-
- PUSH_NINC(push, NV902D, PIXELS_FROM_CPU_DATA, data, count);
- data += count;
- }
-
- PUSH_KICK(push);
- return 0;
-}
-
-int
-nvc0_fbcon_accel_init(struct fb_info *info)
-{
- struct nouveau_fbdev *nfbdev = info->par;
- struct drm_device *dev = nfbdev->helper.dev;
- struct nouveau_drm *drm = nouveau_drm(dev);
- struct nouveau_channel *chan = drm->channel;
- struct nvif_push *push = chan->chan.push;
- int ret, format;
-
- ret = nvif_object_ctor(&chan->user, "fbconTwoD", 0x902d, 0x902d,
- NULL, 0, &nfbdev->twod);
- if (ret)
- return ret;
-
- switch (info->var.bits_per_pixel) {
- case 8:
- format = NV902D_SET_DST_FORMAT_V_Y8;
- break;
- case 15:
- format = NV902D_SET_DST_FORMAT_V_X1R5G5B5;
- break;
- case 16:
- format = NV902D_SET_DST_FORMAT_V_R5G6B5;
- break;
- case 32:
- switch (info->var.transp.length) {
- case 0: /* depth 24 */
- case 8: /* depth 32, just use 24.. */
- format = NV902D_SET_DST_FORMAT_V_X8R8G8B8;
- break;
- case 2: /* depth 30 */
- format = NV902D_SET_DST_FORMAT_V_A2B10G10R10;
- break;
- default:
- return -EINVAL;
- }
- break;
- default:
- return -EINVAL;
- }
-
- ret = PUSH_WAIT(push, 52);
- if (ret) {
- WARN_ON(1);
- nouveau_fbcon_gpu_lockup(info);
- return ret;
- }
-
- PUSH_MTHD(push, NV902D, SET_OBJECT, nfbdev->twod.handle);
-
- PUSH_MTHD(push, NV902D, SET_DST_FORMAT,
- NVVAL(NV902D, SET_DST_FORMAT, V, format),
-
- SET_DST_MEMORY_LAYOUT,
- NVDEF(NV902D, SET_DST_MEMORY_LAYOUT, V, PITCH));
-
- PUSH_MTHD(push, NV902D, SET_DST_PITCH, info->fix.line_length,
- SET_DST_WIDTH, info->var.xres_virtual,
- SET_DST_HEIGHT, info->var.yres_virtual,
-
- SET_DST_OFFSET_UPPER,
- NVVAL(NV902D, SET_DST_OFFSET_UPPER, V, upper_32_bits(nfbdev->vma->addr)),
-
- SET_DST_OFFSET_LOWER,
- NVVAL(NV902D, SET_DST_OFFSET_LOWER, V, lower_32_bits(nfbdev->vma->addr)));
-
- PUSH_MTHD(push, NV902D, SET_SRC_FORMAT,
- NVVAL(NV902D, SET_SRC_FORMAT, V, format),
-
- SET_SRC_MEMORY_LAYOUT,
- NVDEF(NV902D, SET_SRC_MEMORY_LAYOUT, V, PITCH));
-
- PUSH_MTHD(push, NV902D, SET_SRC_PITCH, info->fix.line_length,
- SET_SRC_WIDTH, info->var.xres_virtual,
- SET_SRC_HEIGHT, info->var.yres_virtual,
-
- SET_SRC_OFFSET_UPPER,
- NVVAL(NV902D, SET_SRC_OFFSET_UPPER, V, upper_32_bits(nfbdev->vma->addr)),
-
- SET_SRC_OFFSET_LOWER,
- NVVAL(NV902D, SET_SRC_OFFSET_LOWER, V, lower_32_bits(nfbdev->vma->addr)));
-
- PUSH_IMMD(push, NV902D, SET_CLIP_ENABLE,
- NVDEF(NV902D, SET_CLIP_ENABLE, V, FALSE));
-
- PUSH_IMMD(push, NV902D, SET_ROP,
- NVVAL(NV902D, SET_ROP, V, 0x55));
-
- PUSH_IMMD(push, NV902D, SET_OPERATION,
- NVDEF(NV902D, SET_OPERATION, V, SRCCOPY));
-
- PUSH_MTHD(push, NV902D, SET_MONOCHROME_PATTERN_COLOR_FORMAT,
- NVDEF(NV902D, SET_MONOCHROME_PATTERN_COLOR_FORMAT, V, A8R8G8B8),
-
- SET_MONOCHROME_PATTERN_FORMAT,
- NVDEF(NV902D, SET_MONOCHROME_PATTERN_FORMAT, V, LE_M1));
-
- PUSH_MTHD(push, NV902D, RENDER_SOLID_PRIM_MODE,
- NVDEF(NV902D, RENDER_SOLID_PRIM_MODE, V, RECTS),
-
- SET_RENDER_SOLID_PRIM_COLOR_FORMAT,
- NVVAL(NV902D, SET_RENDER_SOLID_PRIM_COLOR_FORMAT, V, format));
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_CPU_DATA_TYPE,
- NVDEF(NV902D, SET_PIXELS_FROM_CPU_DATA_TYPE, V, INDEX),
-
- SET_PIXELS_FROM_CPU_COLOR_FORMAT,
- NVVAL(NV902D, SET_PIXELS_FROM_CPU_COLOR_FORMAT, V, format),
-
- SET_PIXELS_FROM_CPU_INDEX_FORMAT,
- NVDEF(NV902D, SET_PIXELS_FROM_CPU_INDEX_FORMAT, V, I1),
-
- SET_PIXELS_FROM_CPU_MONO_FORMAT,
- NVDEF(NV902D, SET_PIXELS_FROM_CPU_MONO_FORMAT, V, CGA6_M1),
-
- SET_PIXELS_FROM_CPU_WRAP,
- NVDEF(NV902D, SET_PIXELS_FROM_CPU_WRAP, V, WRAP_BYTE));
-
- PUSH_IMMD(push, NV902D, SET_PIXELS_FROM_CPU_MONO_OPACITY,
- NVDEF(NV902D, SET_PIXELS_FROM_CPU_MONO_OPACITY, V, OPAQUE));
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_CPU_DX_DU_FRAC, 0,
- SET_PIXELS_FROM_CPU_DX_DU_INT, 1,
- SET_PIXELS_FROM_CPU_DY_DV_FRAC, 0,
- SET_PIXELS_FROM_CPU_DY_DV_INT, 1);
-
- PUSH_IMMD(push, NV902D, SET_PIXELS_FROM_MEMORY_SAFE_OVERLAP,
- NVDEF(NV902D, SET_PIXELS_FROM_MEMORY_SAFE_OVERLAP, V, TRUE));
-
- PUSH_MTHD(push, NV902D, SET_PIXELS_FROM_MEMORY_DU_DX_FRAC, 0,
- SET_PIXELS_FROM_MEMORY_DU_DX_INT, 1,
- SET_PIXELS_FROM_MEMORY_DV_DY_FRAC, 0,
- SET_PIXELS_FROM_MEMORY_DV_DY_INT, 1);
- PUSH_KICK(push);
- return 0;
-}
-
diff --git a/drivers/gpu/drm/nouveau/nvif/Kbuild b/drivers/gpu/drm/nouveau/nvif/Kbuild
index 6abc4bc4..b7963a3 100644
--- a/drivers/gpu/drm/nouveau/nvif/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvif/Kbuild
@@ -5,10 +5,11 @@
nvif-y += nvif/device.o
nvif-y += nvif/disp.o
nvif-y += nvif/driver.o
+nvif-y += nvif/event.o
nvif-y += nvif/fifo.o
+nvif-y += nvif/head.o
nvif-y += nvif/mem.o
nvif-y += nvif/mmu.o
-nvif-y += nvif/notify.o
nvif-y += nvif/outp.o
nvif-y += nvif/timer.o
nvif-y += nvif/vmm.o
diff --git a/drivers/gpu/drm/nouveau/nvif/conn.c b/drivers/gpu/drm/nouveau/nvif/conn.c
index 4ce935d..a3cf91a 100644
--- a/drivers/gpu/drm/nouveau/nvif/conn.c
+++ b/drivers/gpu/drm/nouveau/nvif/conn.c
@@ -27,6 +27,25 @@
#include <nvif/if0011.h>
int
+nvif_conn_event_ctor(struct nvif_conn *conn, const char *name, nvif_event_func func, u8 types,
+ struct nvif_event *event)
+{
+ struct {
+ struct nvif_event_v0 base;
+ struct nvif_conn_event_v0 conn;
+ } args;
+ int ret;
+
+ args.conn.version = 0;
+ args.conn.types = types;
+
+ ret = nvif_event_ctor_(&conn->object, name ?: "nvifConnHpd", nvif_conn_id(conn),
+ func, true, &args.base, sizeof(args), false, event);
+ NVIF_DEBUG(&conn->object, "[NEW EVENT:HPD types:%02x]", types);
+ return ret;
+}
+
+int
nvif_conn_hpd_status(struct nvif_conn *conn)
{
struct nvif_conn_hpd_status_v0 args;
diff --git a/drivers/gpu/drm/nouveau/nvif/disp.c b/drivers/gpu/drm/nouveau/nvif/disp.c
index 926b0c0..09915f2 100644
--- a/drivers/gpu/drm/nouveau/nvif/disp.c
+++ b/drivers/gpu/drm/nouveau/nvif/disp.c
@@ -72,9 +72,10 @@ nvif_disp_ctor(struct nvif_device *device, const char *name, s32 oclass, struct
if (ret)
return ret;
- NVIF_DEBUG(&disp->object, "[NEW] conn_mask:%08x outp_mask:%08x",
- args.conn_mask, args.outp_mask);
+ NVIF_DEBUG(&disp->object, "[NEW] conn_mask:%08x outp_mask:%08x head_mask:%08x",
+ args.conn_mask, args.outp_mask, args.head_mask);
disp->conn_mask = args.conn_mask;
disp->outp_mask = args.outp_mask;
+ disp->head_mask = args.head_mask;
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvif/event.c b/drivers/gpu/drm/nouveau/nvif/event.c
new file mode 100644
index 0000000..61ff4d6
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvif/event.c
@@ -0,0 +1,81 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include <nvif/event.h>
+#include <nvif/printf.h>
+
+#include <nvif/class.h>
+#include <nvif/if000e.h>
+
+int
+nvif_event_block(struct nvif_event *event)
+{
+ if (nvif_event_constructed(event)) {
+ int ret = nvif_mthd(&event->object, NVIF_EVENT_V0_BLOCK, NULL, 0);
+ NVIF_ERRON(ret, &event->object, "[BLOCK]");
+ return ret;
+ }
+ return 0;
+}
+
+int
+nvif_event_allow(struct nvif_event *event)
+{
+ if (nvif_event_constructed(event)) {
+ int ret = nvif_mthd(&event->object, NVIF_EVENT_V0_ALLOW, NULL, 0);
+ NVIF_ERRON(ret, &event->object, "[ALLOW]");
+ return ret;
+ }
+ return 0;
+}
+
+void
+nvif_event_dtor(struct nvif_event *event)
+{
+ nvif_object_dtor(&event->object);
+}
+
+int
+nvif_event_ctor_(struct nvif_object *parent, const char *name, u32 handle, nvif_event_func func,
+ bool wait, struct nvif_event_v0 *args, u32 argc, bool warn,
+ struct nvif_event *event)
+{
+ struct nvif_event_v0 _args;
+ int ret;
+
+ if (!args) {
+ args = &_args;
+ argc = sizeof(_args);
+ }
+
+ args->version = 0;
+ args->wait = wait;
+
+ ret = nvif_object_ctor(parent, name ?: "nvifEvent", handle,
+ NVIF_CLASS_EVENT, args, argc, &event->object);
+ NVIF_ERRON(ret && warn, parent, "[NEW EVENT wait:%d size:%zd]",
+ args->wait, argc - sizeof(*args));
+ if (ret)
+ return ret;
+
+ event->func = func;
+ return 0;
+}
diff --git a/drivers/gpu/drm/nouveau/nvif/head.c b/drivers/gpu/drm/nouveau/nvif/head.c
new file mode 100644
index 0000000..f00e01d
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvif/head.c
@@ -0,0 +1,58 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include <nvif/head.h>
+#include <nvif/disp.h>
+#include <nvif/printf.h>
+
+#include <nvif/class.h>
+#include <nvif/if0013.h>
+
+int
+nvif_head_vblank_event_ctor(struct nvif_head *head, const char *name, nvif_event_func func,
+ bool wait, struct nvif_event *event)
+{
+ int ret = nvif_event_ctor(&head->object, name ?: "nvifHeadVBlank", nvif_head_id(head),
+ func, wait, NULL, 0, event);
+ NVIF_ERRON(ret, &head->object, "[NEW EVENT:VBLANK]");
+ return ret;
+}
+
+void
+nvif_head_dtor(struct nvif_head *head)
+{
+ nvif_object_dtor(&head->object);
+}
+
+int
+nvif_head_ctor(struct nvif_disp *disp, const char *name, int id, struct nvif_head *head)
+{
+ struct nvif_head_v0 args;
+ int ret;
+
+ args.version = 0;
+ args.id = id;
+
+ ret = nvif_object_ctor(&disp->object, name ? name : "nvifHead", id, NVIF_CLASS_HEAD,
+ &args, sizeof(args), &head->object);
+ NVIF_ERRON(ret, &disp->object, "[NEW head id:%d]", args.id);
+ return ret;
+}
diff --git a/drivers/gpu/drm/nouveau/nvif/notify.c b/drivers/gpu/drm/nouveau/nvif/notify.c
deleted file mode 100644
index 143c8dc..0000000
--- a/drivers/gpu/drm/nouveau/nvif/notify.c
+++ /dev/null
@@ -1,210 +0,0 @@
-/*
- * Copyright 2014 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs <bskeggs@redhat.com>
- */
-
-#include <nvif/client.h>
-#include <nvif/driver.h>
-#include <nvif/notify.h>
-#include <nvif/object.h>
-#include <nvif/ioctl.h>
-#include <nvif/event.h>
-
-static inline int
-nvif_notify_put_(struct nvif_notify *notify)
-{
- struct nvif_object *object = notify->object;
- struct {
- struct nvif_ioctl_v0 ioctl;
- struct nvif_ioctl_ntfy_put_v0 ntfy;
- } args = {
- .ioctl.type = NVIF_IOCTL_V0_NTFY_PUT,
- .ntfy.index = notify->index,
- };
-
- if (atomic_inc_return(¬ify->putcnt) != 1)
- return 0;
-
- return nvif_object_ioctl(object, &args, sizeof(args), NULL);
-}
-
-int
-nvif_notify_put(struct nvif_notify *notify)
-{
- if (likely(notify->object) &&
- test_and_clear_bit(NVIF_NOTIFY_USER, ¬ify->flags)) {
- int ret = nvif_notify_put_(notify);
- if (test_bit(NVIF_NOTIFY_WORK, ¬ify->flags))
- flush_work(¬ify->work);
- return ret;
- }
- return 0;
-}
-
-static inline int
-nvif_notify_get_(struct nvif_notify *notify)
-{
- struct nvif_object *object = notify->object;
- struct {
- struct nvif_ioctl_v0 ioctl;
- struct nvif_ioctl_ntfy_get_v0 ntfy;
- } args = {
- .ioctl.type = NVIF_IOCTL_V0_NTFY_GET,
- .ntfy.index = notify->index,
- };
-
- if (atomic_dec_return(¬ify->putcnt) != 0)
- return 0;
-
- return nvif_object_ioctl(object, &args, sizeof(args), NULL);
-}
-
-int
-nvif_notify_get(struct nvif_notify *notify)
-{
- if (likely(notify->object) &&
- !test_and_set_bit(NVIF_NOTIFY_USER, ¬ify->flags))
- return nvif_notify_get_(notify);
- return 0;
-}
-
-static inline int
-nvif_notify_func(struct nvif_notify *notify, bool keep)
-{
- int ret = notify->func(notify);
- if (ret == NVIF_NOTIFY_KEEP ||
- !test_and_clear_bit(NVIF_NOTIFY_USER, ¬ify->flags)) {
- if (!keep)
- atomic_dec(¬ify->putcnt);
- else
- nvif_notify_get_(notify);
- }
- return ret;
-}
-
-static void
-nvif_notify_work(struct work_struct *work)
-{
- struct nvif_notify *notify = container_of(work, typeof(*notify), work);
- nvif_notify_func(notify, true);
-}
-
-int
-nvif_notify(const void *header, u32 length, const void *data, u32 size)
-{
- struct nvif_notify *notify = NULL;
- const union {
- struct nvif_notify_rep_v0 v0;
- } *args = header;
- int ret = NVIF_NOTIFY_DROP;
-
- if (length == sizeof(args->v0) && args->v0.version == 0) {
- if (WARN_ON(args->v0.route))
- return NVIF_NOTIFY_DROP;
- notify = (void *)(unsigned long)args->v0.token;
- }
-
- if (!WARN_ON(notify == NULL)) {
- struct nvif_client *client = notify->object->client;
- if (!WARN_ON(notify->size != size)) {
- atomic_inc(¬ify->putcnt);
- if (test_bit(NVIF_NOTIFY_WORK, ¬ify->flags)) {
- memcpy((void *)notify->data, data, size);
- schedule_work(¬ify->work);
- return NVIF_NOTIFY_DROP;
- }
- notify->data = data;
- ret = nvif_notify_func(notify, client->driver->keep);
- notify->data = NULL;
- }
- }
-
- return ret;
-}
-
-int
-nvif_notify_dtor(struct nvif_notify *notify)
-{
- struct nvif_object *object = notify->object;
- struct {
- struct nvif_ioctl_v0 ioctl;
- struct nvif_ioctl_ntfy_del_v0 ntfy;
- } args = {
- .ioctl.type = NVIF_IOCTL_V0_NTFY_DEL,
- .ntfy.index = notify->index,
- };
- int ret = nvif_notify_put(notify);
- if (ret >= 0 && object) {
- ret = nvif_object_ioctl(object, &args, sizeof(args), NULL);
- notify->object = NULL;
- kfree((void *)notify->data);
- }
- return ret;
-}
-
-int
-nvif_notify_ctor(struct nvif_object *object, const char *name,
- int (*func)(struct nvif_notify *), bool work, u8 event,
- void *data, u32 size, u32 reply, struct nvif_notify *notify)
-{
- struct {
- struct nvif_ioctl_v0 ioctl;
- struct nvif_ioctl_ntfy_new_v0 ntfy;
- struct nvif_notify_req_v0 req;
- } *args;
- int ret = -ENOMEM;
-
- notify->object = object;
- notify->name = name ? name : "nvifNotify";
- notify->flags = 0;
- atomic_set(¬ify->putcnt, 1);
- notify->func = func;
- notify->data = NULL;
- notify->size = reply;
- if (work) {
- INIT_WORK(¬ify->work, nvif_notify_work);
- set_bit(NVIF_NOTIFY_WORK, ¬ify->flags);
- notify->data = kmalloc(notify->size, GFP_KERNEL);
- if (!notify->data)
- goto done;
- }
-
- if (!(args = kmalloc(sizeof(*args) + size, GFP_KERNEL)))
- goto done;
- args->ioctl.version = 0;
- args->ioctl.type = NVIF_IOCTL_V0_NTFY_NEW;
- args->ntfy.version = 0;
- args->ntfy.event = event;
- args->req.version = 0;
- args->req.reply = notify->size;
- args->req.route = 0;
- args->req.token = (unsigned long)(void *)notify;
-
- memcpy(args->req.data, data, size);
- ret = nvif_object_ioctl(object, args, sizeof(*args) + size, NULL);
- notify->index = args->ntfy.index;
- kfree(args);
-done:
- if (ret)
- nvif_notify_dtor(notify);
- return ret;
-}
diff --git a/drivers/gpu/drm/nouveau/nvif/outp.c b/drivers/gpu/drm/nouveau/nvif/outp.c
index 7bfe91a..7da39f1e 100644
--- a/drivers/gpu/drm/nouveau/nvif/outp.c
+++ b/drivers/gpu/drm/nouveau/nvif/outp.c
@@ -24,7 +24,177 @@
#include <nvif/printf.h>
#include <nvif/class.h>
-#include <nvif/if0012.h>
+
+int
+nvif_outp_dp_mst_vcpi(struct nvif_outp *outp, int head,
+ u8 start_slot, u8 num_slots, u16 pbn, u16 aligned_pbn)
+{
+ struct nvif_outp_dp_mst_vcpi_v0 args;
+ int ret;
+
+ args.version = 0;
+ args.head = head;
+ args.start_slot = start_slot;
+ args.num_slots = num_slots;
+ args.pbn = pbn;
+ args.aligned_pbn = aligned_pbn;
+
+ ret = nvif_object_mthd(&outp->object, NVIF_OUTP_V0_DP_MST_VCPI, &args, sizeof(args));
+ NVIF_ERRON(ret, &outp->object,
+ "[DP_MST_VCPI head:%d start_slot:%02x num_slots:%02x pbn:%04x aligned_pbn:%04x]",
+ args.head, args.start_slot, args.num_slots, args.pbn, args.aligned_pbn);
+ return ret;
+}
+
+int
+nvif_outp_dp_retrain(struct nvif_outp *outp)
+{
+ int ret = nvif_object_mthd(&outp->object, NVIF_OUTP_V0_DP_RETRAIN, NULL, 0);
+ NVIF_ERRON(ret, &outp->object, "[DP_RETRAIN]");
+ return ret;
+}
+
+int
+nvif_outp_dp_aux_pwr(struct nvif_outp *outp, bool enable)
+{
+ struct nvif_outp_dp_aux_pwr_v0 args;
+ int ret;
+
+ args.version = 0;
+ args.state = enable;
+
+ ret = nvif_object_mthd(&outp->object, NVIF_OUTP_V0_DP_AUX_PWR, &args, sizeof(args));
+ NVIF_ERRON(ret, &outp->object, "[DP_AUX_PWR state:%d]", args.state);
+ return ret;
+}
+
+int
+nvif_outp_hda_eld(struct nvif_outp *outp, int head, void *data, u32 size)
+{
+ struct {
+ struct nvif_outp_hda_eld_v0 mthd;
+ u8 data[128];
+ } args;
+ int ret;
+
+ if (WARN_ON(size > ARRAY_SIZE(args.data)))
+ return -EINVAL;
+
+ args.mthd.version = 0;
+ args.mthd.head = head;
+
+ memcpy(args.data, data, size);
+ ret = nvif_mthd(&outp->object, NVIF_OUTP_V0_HDA_ELD, &args, sizeof(args.mthd) + size);
+ NVIF_ERRON(ret, &outp->object, "[HDA_ELD head:%d size:%d]", head, size);
+ return ret;
+}
+
+int
+nvif_outp_infoframe(struct nvif_outp *outp, u8 type, struct nvif_outp_infoframe_v0 *args, u32 size)
+{
+ int ret;
+
+ args->type = type;
+
+ ret = nvif_mthd(&outp->object, NVIF_OUTP_V0_INFOFRAME, args, sizeof(*args) + size);
+ NVIF_ERRON(ret, &outp->object, "[INFOFRAME type:%d size:%d]", type, size);
+ return ret;
+}
+
+void
+nvif_outp_release(struct nvif_outp *outp)
+{
+ int ret = nvif_mthd(&outp->object, NVIF_OUTP_V0_RELEASE, NULL, 0);
+ NVIF_ERRON(ret, &outp->object, "[RELEASE]");
+ outp->or.id = -1;
+}
+
+static inline int
+nvif_outp_acquire(struct nvif_outp *outp, u8 proto, struct nvif_outp_acquire_v0 *args)
+{
+ int ret;
+
+ args->version = 0;
+ args->proto = proto;
+
+ ret = nvif_mthd(&outp->object, NVIF_OUTP_V0_ACQUIRE, args, sizeof(*args));
+ if (ret)
+ return ret;
+
+ outp->or.id = args->or;
+ outp->or.link = args->link;
+ return 0;
+}
+
+int
+nvif_outp_acquire_dp(struct nvif_outp *outp, u8 dpcd[16],
+ int link_nr, int link_bw, bool hda, bool mst)
+{
+ struct nvif_outp_acquire_v0 args;
+ int ret;
+
+ args.dp.link_nr = link_nr;
+ args.dp.link_bw = link_bw;
+ args.dp.hda = hda;
+ args.dp.mst = mst;
+ memcpy(args.dp.dpcd, dpcd, sizeof(args.dp.dpcd));
+
+ ret = nvif_outp_acquire(outp, NVIF_OUTP_ACQUIRE_V0_DP, &args);
+ NVIF_ERRON(ret, &outp->object,
+ "[ACQUIRE proto:DP link_nr:%d link_bw:%02x hda:%d mst:%d] or:%d link:%d",
+ args.dp.link_nr, args.dp.link_bw, args.dp.hda, args.dp.mst, args.or, args.link);
+ return ret;
+}
+
+int
+nvif_outp_acquire_lvds(struct nvif_outp *outp, bool dual, bool bpc8)
+{
+ struct nvif_outp_acquire_v0 args;
+ int ret;
+
+ args.lvds.dual = dual;
+ args.lvds.bpc8 = bpc8;
+
+ ret = nvif_outp_acquire(outp, NVIF_OUTP_ACQUIRE_V0_LVDS, &args);
+ NVIF_ERRON(ret, &outp->object,
+ "[ACQUIRE proto:LVDS dual:%d 8bpc:%d] or:%d link:%d",
+ args.lvds.dual, args.lvds.bpc8, args.or, args.link);
+ return ret;
+}
+
+int
+nvif_outp_acquire_tmds(struct nvif_outp *outp, int head,
+ bool hdmi, u8 max_ac_packet, u8 rekey, u8 scdc, bool hda)
+{
+ struct nvif_outp_acquire_v0 args;
+ int ret;
+
+ args.tmds.head = head;
+ args.tmds.hdmi = hdmi;
+ args.tmds.hdmi_max_ac_packet = max_ac_packet;
+ args.tmds.hdmi_rekey = rekey;
+ args.tmds.hdmi_scdc = scdc;
+ args.tmds.hdmi_hda = hda;
+
+ ret = nvif_outp_acquire(outp, NVIF_OUTP_ACQUIRE_V0_TMDS, &args);
+ NVIF_ERRON(ret, &outp->object,
+ "[ACQUIRE proto:TMDS head:%d hdmi:%d max_ac_packet:%d rekey:%d scdc:%d hda:%d]"
+ " or:%d link:%d", args.tmds.head, args.tmds.hdmi, args.tmds.hdmi_max_ac_packet,
+ args.tmds.hdmi_rekey, args.tmds.hdmi_scdc, args.tmds.hdmi_hda,
+ args.or, args.link);
+ return ret;
+}
+
+int
+nvif_outp_acquire_rgb_crt(struct nvif_outp *outp)
+{
+ struct nvif_outp_acquire_v0 args;
+ int ret;
+
+ ret = nvif_outp_acquire(outp, NVIF_OUTP_ACQUIRE_V0_RGB_CRT, &args);
+ NVIF_ERRON(ret, &outp->object, "[ACQUIRE proto:RGB_CRT] or:%d", args.or);
+ return ret;
+}
int
nvif_outp_load_detect(struct nvif_outp *outp, u32 loadval)
@@ -58,5 +228,9 @@ nvif_outp_ctor(struct nvif_disp *disp, const char *name, int id, struct nvif_out
ret = nvif_object_ctor(&disp->object, name ?: "nvifOutp", id, NVIF_CLASS_OUTP,
&args, sizeof(args), &outp->object);
NVIF_ERRON(ret, &disp->object, "[NEW outp id:%d]", id);
- return ret;
+ if (ret)
+ return ret;
+
+ outp->or.id = -1;
+ return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvif/user.c b/drivers/gpu/drm/nouveau/nvif/user.c
index d89f5b6..b648a5e 100644
--- a/drivers/gpu/drm/nouveau/nvif/user.c
+++ b/drivers/gpu/drm/nouveau/nvif/user.c
@@ -41,7 +41,9 @@ nvif_user_ctor(struct nvif_device *device, const char *name)
int version;
const struct nvif_user_func *func;
} users[] = {
- { VOLTA_USERMODE_A, -1, &nvif_userc361 },
+ { AMPERE_USERMODE_A, -1, &nvif_userc361 },
+ { TURING_USERMODE_A, -1, &nvif_userc361 },
+ { VOLTA_USERMODE_A, -1, &nvif_userc361 },
{}
};
int cid, ret;
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/Kbuild b/drivers/gpu/drm/nouveau/nvkm/core/Kbuild
index 2b471ab..e407120 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/core/Kbuild
@@ -5,12 +5,13 @@
nvkm-y += nvkm/core/event.o
nvkm-y += nvkm/core/firmware.o
nvkm-y += nvkm/core/gpuobj.o
+nvkm-y += nvkm/core/intr.o
nvkm-y += nvkm/core/ioctl.o
nvkm-y += nvkm/core/memory.o
nvkm-y += nvkm/core/mm.o
-nvkm-y += nvkm/core/notify.o
nvkm-y += nvkm/core/object.o
nvkm-y += nvkm/core/oproxy.o
nvkm-y += nvkm/core/option.o
nvkm-y += nvkm/core/ramht.o
nvkm-y += nvkm/core/subdev.o
+nvkm-y += nvkm/core/uevent.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/client.c b/drivers/gpu/drm/nouveau/nvkm/core/client.c
index 0c8c55c..ebdeb8e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/client.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/client.c
@@ -23,7 +23,6 @@
*/
#include <core/client.h>
#include <core/device.h>
-#include <core/notify.h>
#include <core/option.h>
#include <nvif/class.h>
@@ -44,7 +43,7 @@ nvkm_uclient_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))){
args->v0.name[sizeof(args->v0.name) - 1] = 0;
ret = nvkm_client_new(args->v0.name, args->v0.device, NULL,
- NULL, oclass->client->ntfy, &client);
+ NULL, oclass->client->event, &client);
if (ret)
return ret;
} else
@@ -68,113 +67,6 @@ nvkm_uclient_sclass = {
.ctor = nvkm_uclient_new,
};
-struct nvkm_client_notify {
- struct nvkm_client *client;
- struct nvkm_notify n;
- u8 version;
- u8 size;
- union {
- struct nvif_notify_rep_v0 v0;
- } rep;
-};
-
-static int
-nvkm_client_notify(struct nvkm_notify *n)
-{
- struct nvkm_client_notify *notify = container_of(n, typeof(*notify), n);
- struct nvkm_client *client = notify->client;
- return client->ntfy(¬ify->rep, notify->size, n->data, n->size);
-}
-
-int
-nvkm_client_notify_put(struct nvkm_client *client, int index)
-{
- if (index < ARRAY_SIZE(client->notify)) {
- if (client->notify[index]) {
- nvkm_notify_put(&client->notify[index]->n);
- return 0;
- }
- }
- return -ENOENT;
-}
-
-int
-nvkm_client_notify_get(struct nvkm_client *client, int index)
-{
- if (index < ARRAY_SIZE(client->notify)) {
- if (client->notify[index]) {
- nvkm_notify_get(&client->notify[index]->n);
- return 0;
- }
- }
- return -ENOENT;
-}
-
-int
-nvkm_client_notify_del(struct nvkm_client *client, int index)
-{
- if (index < ARRAY_SIZE(client->notify)) {
- if (client->notify[index]) {
- nvkm_notify_fini(&client->notify[index]->n);
- kfree(client->notify[index]);
- client->notify[index] = NULL;
- return 0;
- }
- }
- return -ENOENT;
-}
-
-int
-nvkm_client_notify_new(struct nvkm_object *object,
- struct nvkm_event *event, void *data, u32 size)
-{
- struct nvkm_client *client = object->client;
- struct nvkm_client_notify *notify;
- union {
- struct nvif_notify_req_v0 v0;
- } *req = data;
- u8 index, reply;
- int ret = -ENOSYS;
-
- for (index = 0; index < ARRAY_SIZE(client->notify); index++) {
- if (!client->notify[index])
- break;
- }
-
- if (index == ARRAY_SIZE(client->notify))
- return -ENOSPC;
-
- notify = kzalloc(sizeof(*notify), GFP_KERNEL);
- if (!notify)
- return -ENOMEM;
-
- nvif_ioctl(object, "notify new size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, req->v0, 0, 0, true))) {
- nvif_ioctl(object, "notify new vers %d reply %d route %02x "
- "token %llx\n", req->v0.version,
- req->v0.reply, req->v0.route, req->v0.token);
- notify->version = req->v0.version;
- notify->size = sizeof(notify->rep.v0);
- notify->rep.v0.version = req->v0.version;
- notify->rep.v0.route = req->v0.route;
- notify->rep.v0.token = req->v0.token;
- reply = req->v0.reply;
- }
-
- if (ret == 0) {
- ret = nvkm_notify_init(object, event, nvkm_client_notify,
- false, data, size, reply, ¬ify->n);
- if (ret == 0) {
- client->notify[index] = notify;
- notify->client = client;
- return index;
- }
- }
-
- kfree(notify);
- return ret;
-}
-
static const struct nvkm_object_func nvkm_client;
struct nvkm_client *
nvkm_client_search(struct nvkm_client *client, u64 handle)
@@ -255,23 +147,13 @@ nvkm_client_child_get(struct nvkm_object *object, int index,
static int
nvkm_client_fini(struct nvkm_object *object, bool suspend)
{
- struct nvkm_client *client = nvkm_client(object);
- const char *name[2] = { "fini", "suspend" };
- int i;
- nvif_debug(object, "%s notify\n", name[suspend]);
- for (i = 0; i < ARRAY_SIZE(client->notify); i++)
- nvkm_client_notify_put(client, i);
return 0;
}
static void *
nvkm_client_dtor(struct nvkm_object *object)
{
- struct nvkm_client *client = nvkm_client(object);
- int i;
- for (i = 0; i < ARRAY_SIZE(client->notify); i++)
- nvkm_client_notify_del(client, i);
- return client;
+ return nvkm_client(object);
}
static const struct nvkm_object_func
@@ -283,10 +165,8 @@ nvkm_client = {
};
int
-nvkm_client_new(const char *name, u64 device, const char *cfg,
- const char *dbg,
- int (*ntfy)(const void *, u32, const void *, u32),
- struct nvkm_client **pclient)
+nvkm_client_new(const char *name, u64 device, const char *cfg, const char *dbg,
+ int (*event)(u64, void *, u32), struct nvkm_client **pclient)
{
struct nvkm_oclass oclass = { .base = nvkm_uclient_sclass };
struct nvkm_client *client;
@@ -300,7 +180,7 @@ nvkm_client_new(const char *name, u64 device, const char *cfg,
client->device = device;
client->debug = nvkm_dbgopt(dbg, "CLIENT");
client->objroot = RB_ROOT;
- client->ntfy = ntfy;
+ client->event = event;
INIT_LIST_HEAD(&client->umem);
spin_lock_init(&client->lock);
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/engine.c b/drivers/gpu/drm/nouveau/nvkm/core/engine.c
index e41a39a..36a31e9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/engine.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/engine.c
@@ -35,16 +35,23 @@ nvkm_engine_chsw_load(struct nvkm_engine *engine)
return false;
}
+int
+nvkm_engine_reset(struct nvkm_engine *engine)
+{
+ if (engine->func->reset)
+ return engine->func->reset(engine);
+
+ nvkm_subdev_fini(&engine->subdev, false);
+ return nvkm_subdev_init(&engine->subdev);
+}
+
void
nvkm_engine_unref(struct nvkm_engine **pengine)
{
struct nvkm_engine *engine = *pengine;
+
if (engine) {
- if (refcount_dec_and_mutex_lock(&engine->use.refcount, &engine->use.mutex)) {
- nvkm_subdev_fini(&engine->subdev, false);
- engine->use.enabled = false;
- mutex_unlock(&engine->use.mutex);
- }
+ nvkm_subdev_unref(&engine->subdev);
*pengine = NULL;
}
}
@@ -53,21 +60,13 @@ struct nvkm_engine *
nvkm_engine_ref(struct nvkm_engine *engine)
{
int ret;
+
if (engine) {
- if (!refcount_inc_not_zero(&engine->use.refcount)) {
- mutex_lock(&engine->use.mutex);
- if (!refcount_inc_not_zero(&engine->use.refcount)) {
- engine->use.enabled = true;
- if ((ret = nvkm_subdev_init(&engine->subdev))) {
- engine->use.enabled = false;
- mutex_unlock(&engine->use.mutex);
- return ERR_PTR(ret);
- }
- refcount_set(&engine->use.refcount, 1);
- }
- mutex_unlock(&engine->use.mutex);
- }
+ ret = nvkm_subdev_ref(&engine->subdev);
+ if (ret)
+ return ERR_PTR(ret);
}
+
return engine;
}
@@ -91,14 +90,10 @@ static int
nvkm_engine_info(struct nvkm_subdev *subdev, u64 mthd, u64 *data)
{
struct nvkm_engine *engine = nvkm_engine(subdev);
- if (engine->func->info) {
- if (!IS_ERR((engine = nvkm_engine_ref(engine)))) {
- int ret = engine->func->info(engine, mthd, data);
- nvkm_engine_unref(&engine);
- return ret;
- }
- return PTR_ERR(engine);
- }
+
+ if (engine->func->info)
+ return engine->func->info(engine, mthd, data);
+
return -ENOSYS;
}
@@ -117,26 +112,6 @@ nvkm_engine_init(struct nvkm_subdev *subdev)
struct nvkm_engine *engine = nvkm_engine(subdev);
struct nvkm_fb *fb = subdev->device->fb;
int ret = 0, i;
- s64 time;
-
- if (!engine->use.enabled) {
- nvkm_trace(subdev, "init skipped, engine has no users\n");
- return ret;
- }
-
- if (engine->func->oneinit && !engine->subdev.oneinit) {
- nvkm_trace(subdev, "one-time init running...\n");
- time = ktime_to_us(ktime_get());
- ret = engine->func->oneinit(engine);
- if (ret) {
- nvkm_trace(subdev, "one-time init failed, %d\n", ret);
- return ret;
- }
-
- engine->subdev.oneinit = true;
- time = ktime_to_us(ktime_get()) - time;
- nvkm_trace(subdev, "one-time init completed in %lldus\n", time);
- }
if (engine->func->init)
ret = engine->func->init(engine);
@@ -147,6 +122,17 @@ nvkm_engine_init(struct nvkm_subdev *subdev)
}
static int
+nvkm_engine_oneinit(struct nvkm_subdev *subdev)
+{
+ struct nvkm_engine *engine = nvkm_engine(subdev);
+
+ if (engine->func->oneinit)
+ return engine->func->oneinit(engine);
+
+ return 0;
+}
+
+static int
nvkm_engine_preinit(struct nvkm_subdev *subdev)
{
struct nvkm_engine *engine = nvkm_engine(subdev);
@@ -161,7 +147,6 @@ nvkm_engine_dtor(struct nvkm_subdev *subdev)
struct nvkm_engine *engine = nvkm_engine(subdev);
if (engine->func->dtor)
return engine->func->dtor(engine);
- mutex_destroy(&engine->use.mutex);
return engine;
}
@@ -169,6 +154,7 @@ const struct nvkm_subdev_func
nvkm_engine = {
.dtor = nvkm_engine_dtor,
.preinit = nvkm_engine_preinit,
+ .oneinit = nvkm_engine_oneinit,
.init = nvkm_engine_init,
.fini = nvkm_engine_fini,
.info = nvkm_engine_info,
@@ -179,10 +165,9 @@ int
nvkm_engine_ctor(const struct nvkm_engine_func *func, struct nvkm_device *device,
enum nvkm_subdev_type type, int inst, bool enable, struct nvkm_engine *engine)
{
- nvkm_subdev_ctor(&nvkm_engine, device, type, inst, &engine->subdev);
engine->func = func;
- refcount_set(&engine->use.refcount, 0);
- mutex_init(&engine->use.mutex);
+ nvkm_subdev_ctor(&nvkm_engine, device, type, inst, &engine->subdev);
+ refcount_set(&engine->subdev.use.refcount, 0);
if (!nvkm_boolopt(device->cfgopt, engine->subdev.name, enable)) {
nvkm_debug(&engine->subdev, "disabled\n");
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/event.c b/drivers/gpu/drm/nouveau/nvkm/core/event.c
index 006618d..a6c8771 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/event.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/event.c
@@ -20,54 +20,171 @@
* OTHER DEALINGS IN THE SOFTWARE.
*/
#include <core/event.h>
-#include <core/notify.h>
+#include <core/subdev.h>
-void
+static void
nvkm_event_put(struct nvkm_event *event, u32 types, int index)
{
assert_spin_locked(&event->refs_lock);
+
+ nvkm_trace(event->subdev, "event: decr %08x on %d\n", types, index);
+
while (types) {
int type = __ffs(types); types &= ~(1 << type);
if (--event->refs[index * event->types_nr + type] == 0) {
+ nvkm_trace(event->subdev, "event: blocking %d on %d\n", type, index);
if (event->func->fini)
event->func->fini(event, 1 << type, index);
}
}
}
-void
+static void
nvkm_event_get(struct nvkm_event *event, u32 types, int index)
{
assert_spin_locked(&event->refs_lock);
+
+ nvkm_trace(event->subdev, "event: incr %08x on %d\n", types, index);
+
while (types) {
int type = __ffs(types); types &= ~(1 << type);
if (++event->refs[index * event->types_nr + type] == 1) {
+ nvkm_trace(event->subdev, "event: allowing %d on %d\n", type, index);
if (event->func->init)
event->func->init(event, 1 << type, index);
}
}
}
-void
-nvkm_event_send(struct nvkm_event *event, u32 types, int index,
- void *data, u32 size)
+static void
+nvkm_event_ntfy_state(struct nvkm_event_ntfy *ntfy)
{
- struct nvkm_notify *notify;
+ struct nvkm_event *event = ntfy->event;
unsigned long flags;
- if (!event->refs || WARN_ON(index >= event->index_nr))
- return;
+ nvkm_trace(event->subdev, "event: ntfy state changed\n");
+ spin_lock_irqsave(&event->refs_lock, flags);
- spin_lock_irqsave(&event->list_lock, flags);
- list_for_each_entry(notify, &event->list, head) {
- if (notify->index == index && (notify->types & types)) {
- if (event->func->send) {
- event->func->send(data, size, notify);
- continue;
- }
- nvkm_notify_send(notify, data, size);
+ if (atomic_read(&ntfy->allowed) != ntfy->running) {
+ if (ntfy->running) {
+ nvkm_event_put(ntfy->event, ntfy->bits, ntfy->id);
+ ntfy->running = false;
+ } else {
+ nvkm_event_get(ntfy->event, ntfy->bits, ntfy->id);
+ ntfy->running = true;
}
}
+
+ spin_unlock_irqrestore(&event->refs_lock, flags);
+}
+
+static void
+nvkm_event_ntfy_remove(struct nvkm_event_ntfy *ntfy)
+{
+ spin_lock_irq(&ntfy->event->list_lock);
+ list_del_init(&ntfy->head);
+ spin_unlock_irq(&ntfy->event->list_lock);
+}
+
+static void
+nvkm_event_ntfy_insert(struct nvkm_event_ntfy *ntfy)
+{
+ spin_lock_irq(&ntfy->event->list_lock);
+ list_add_tail(&ntfy->head, &ntfy->event->ntfy);
+ spin_unlock_irq(&ntfy->event->list_lock);
+}
+
+static void
+nvkm_event_ntfy_block_(struct nvkm_event_ntfy *ntfy, bool wait)
+{
+ struct nvkm_subdev *subdev = ntfy->event->subdev;
+
+ nvkm_trace(subdev, "event: ntfy block %08x on %d wait:%d\n", ntfy->bits, ntfy->id, wait);
+
+ if (atomic_xchg(&ntfy->allowed, 0) == 1) {
+ nvkm_event_ntfy_state(ntfy);
+ if (wait)
+ nvkm_event_ntfy_remove(ntfy);
+ }
+}
+
+void
+nvkm_event_ntfy_block(struct nvkm_event_ntfy *ntfy)
+{
+ if (ntfy->event)
+ nvkm_event_ntfy_block_(ntfy, ntfy->wait);
+}
+
+void
+nvkm_event_ntfy_allow(struct nvkm_event_ntfy *ntfy)
+{
+ nvkm_trace(ntfy->event->subdev, "event: ntfy allow %08x on %d\n", ntfy->bits, ntfy->id);
+
+ if (atomic_xchg(&ntfy->allowed, 1) == 0) {
+ nvkm_event_ntfy_state(ntfy);
+ if (ntfy->wait)
+ nvkm_event_ntfy_insert(ntfy);
+ }
+}
+
+void
+nvkm_event_ntfy_del(struct nvkm_event_ntfy *ntfy)
+{
+ struct nvkm_event *event = ntfy->event;
+
+ if (!event)
+ return;
+
+ nvkm_trace(event->subdev, "event: ntfy del %08x on %d\n", ntfy->bits, ntfy->id);
+
+ nvkm_event_ntfy_block_(ntfy, false);
+ nvkm_event_ntfy_remove(ntfy);
+ ntfy->event = NULL;
+}
+
+void
+nvkm_event_ntfy_add(struct nvkm_event *event, int id, u32 bits, bool wait, nvkm_event_func func,
+ struct nvkm_event_ntfy *ntfy)
+{
+ nvkm_trace(event->subdev, "event: ntfy add %08x on %d wait:%d\n", id, bits, wait);
+
+ ntfy->event = event;
+ ntfy->id = id;
+ ntfy->bits = bits;
+ ntfy->wait = wait;
+ ntfy->func = func;
+ atomic_set(&ntfy->allowed, 0);
+ ntfy->running = false;
+ INIT_LIST_HEAD(&ntfy->head);
+ if (!ntfy->wait)
+ nvkm_event_ntfy_insert(ntfy);
+}
+
+bool
+nvkm_event_ntfy_valid(struct nvkm_event *event, int id, u32 bits)
+{
+ return true;
+}
+
+void
+nvkm_event_ntfy(struct nvkm_event *event, int id, u32 bits)
+{
+ struct nvkm_event_ntfy *ntfy, *ntmp;
+ unsigned long flags;
+
+ if (!event->refs || WARN_ON(id >= event->index_nr))
+ return;
+
+ nvkm_trace(event->subdev, "event: ntfy %08x on %d\n", bits, id);
+ spin_lock_irqsave(&event->list_lock, flags);
+
+ list_for_each_entry_safe(ntfy, ntmp, &event->ntfy, head) {
+ if (ntfy->id == id && ntfy->bits & bits) {
+ if (atomic_read(&ntfy->allowed))
+ ntfy->func(ntfy, ntfy->bits & bits);
+ }
+ }
+
spin_unlock_irqrestore(&event->list_lock, flags);
}
@@ -81,20 +198,17 @@ nvkm_event_fini(struct nvkm_event *event)
}
int
-nvkm_event_init(const struct nvkm_event_func *func, int types_nr, int index_nr,
- struct nvkm_event *event)
+__nvkm_event_init(const struct nvkm_event_func *func, struct nvkm_subdev *subdev,
+ int types_nr, int index_nr, struct nvkm_event *event)
{
- event->refs = kzalloc(array3_size(index_nr, types_nr,
- sizeof(*event->refs)),
- GFP_KERNEL);
+ event->refs = kzalloc(array3_size(index_nr, types_nr, sizeof(*event->refs)), GFP_KERNEL);
if (!event->refs)
return -ENOMEM;
event->func = func;
+ event->subdev = subdev;
event->types_nr = types_nr;
event->index_nr = index_nr;
- spin_lock_init(&event->refs_lock);
- spin_lock_init(&event->list_lock);
- INIT_LIST_HEAD(&event->list);
+ INIT_LIST_HEAD(&event->ntfy);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/firmware.c b/drivers/gpu/drm/nouveau/nvkm/core/firmware.c
index ca1f846..fcf2a00 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/firmware.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/firmware.c
@@ -22,6 +22,9 @@
#include <core/device.h>
#include <core/firmware.h>
+#include <subdev/fb.h>
+#include <subdev/mmu.h>
+
int
nvkm_firmware_load_name(const struct nvkm_subdev *subdev, const char *base,
const char *name, int ver, const struct firmware **pfw)
@@ -107,3 +110,127 @@ nvkm_firmware_put(const struct firmware *fw)
{
release_firmware(fw);
}
+
+#define nvkm_firmware_mem(p) container_of((p), struct nvkm_firmware, mem.memory)
+
+static int
+nvkm_firmware_mem_map(struct nvkm_memory *memory, u64 offset, struct nvkm_vmm *vmm,
+ struct nvkm_vma *vma, void *argv, u32 argc)
+{
+ struct nvkm_firmware *fw = nvkm_firmware_mem(memory);
+ struct nvkm_vmm_map map = {
+ .memory = &fw->mem.memory,
+ .offset = offset,
+ .sgl = &fw->mem.sgl,
+ };
+
+ if (WARN_ON(fw->func->type != NVKM_FIRMWARE_IMG_DMA))
+ return -ENOSYS;
+
+ return nvkm_vmm_map(vmm, vma, argv, argc, &map);
+}
+
+static u64
+nvkm_firmware_mem_size(struct nvkm_memory *memory)
+{
+ return sg_dma_len(&nvkm_firmware_mem(memory)->mem.sgl);
+}
+
+static u64
+nvkm_firmware_mem_addr(struct nvkm_memory *memory)
+{
+ return nvkm_firmware_mem(memory)->phys;
+}
+
+static u8
+nvkm_firmware_mem_page(struct nvkm_memory *memory)
+{
+ return PAGE_SHIFT;
+}
+
+static enum nvkm_memory_target
+nvkm_firmware_mem_target(struct nvkm_memory *memory)
+{
+ return NVKM_MEM_TARGET_HOST;
+}
+
+static void *
+nvkm_firmware_mem_dtor(struct nvkm_memory *memory)
+{
+ return NULL;
+}
+
+static const struct nvkm_memory_func
+nvkm_firmware_mem = {
+ .dtor = nvkm_firmware_mem_dtor,
+ .target = nvkm_firmware_mem_target,
+ .page = nvkm_firmware_mem_page,
+ .addr = nvkm_firmware_mem_addr,
+ .size = nvkm_firmware_mem_size,
+ .map = nvkm_firmware_mem_map,
+};
+
+void
+nvkm_firmware_dtor(struct nvkm_firmware *fw)
+{
+ struct nvkm_memory *memory = &fw->mem.memory;
+
+ if (!fw->img)
+ return;
+
+ switch (fw->func->type) {
+ case NVKM_FIRMWARE_IMG_RAM:
+ kfree(fw->img);
+ break;
+ case NVKM_FIRMWARE_IMG_DMA:
+ nvkm_memory_unref(&memory);
+ dma_free_coherent(fw->device->dev, sg_dma_len(&fw->mem.sgl), fw->img, fw->phys);
+ break;
+ default:
+ WARN_ON(1);
+ break;
+ }
+
+ fw->img = NULL;
+}
+
+int
+nvkm_firmware_ctor(const struct nvkm_firmware_func *func, const char *name,
+ struct nvkm_device *device, const void *src, int len, struct nvkm_firmware *fw)
+{
+ fw->func = func;
+ fw->name = name;
+ fw->device = device;
+ fw->len = len;
+
+ switch (fw->func->type) {
+ case NVKM_FIRMWARE_IMG_RAM:
+ fw->img = kmemdup(src, fw->len, GFP_KERNEL);
+ break;
+ case NVKM_FIRMWARE_IMG_DMA: {
+ dma_addr_t addr;
+
+ len = ALIGN(fw->len, PAGE_SIZE);
+
+ fw->img = dma_alloc_coherent(fw->device->dev, len, &addr, GFP_KERNEL);
+ if (fw->img) {
+ memcpy(fw->img, src, fw->len);
+ fw->phys = addr;
+ }
+
+ sg_init_one(&fw->mem.sgl, fw->img, len);
+ sg_dma_address(&fw->mem.sgl) = fw->phys;
+ sg_dma_len(&fw->mem.sgl) = len;
+ }
+ break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ if (!fw->img)
+ return -ENOMEM;
+
+ nvkm_memory_ctor(&nvkm_firmware_mem, &fw->mem.memory);
+ return 0;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/intr.c b/drivers/gpu/drm/nouveau/nvkm/core/intr.c
new file mode 100644
index 0000000..e20b7ca
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/core/intr.c
@@ -0,0 +1,442 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include <core/intr.h>
+#include <core/device.h>
+#include <core/subdev.h>
+#include <subdev/pci.h>
+#include <subdev/top.h>
+
+static int
+nvkm_intr_xlat(struct nvkm_subdev *subdev, struct nvkm_intr *intr,
+ enum nvkm_intr_type type, int *leaf, u32 *mask)
+{
+ struct nvkm_device *device = subdev->device;
+
+ if (type < NVKM_INTR_VECTOR_0) {
+ if (type == NVKM_INTR_SUBDEV) {
+ const struct nvkm_intr_data *data = intr->data;
+ struct nvkm_top_device *tdev;
+
+ while (data && data->mask) {
+ if (data->type == NVKM_SUBDEV_TOP) {
+ list_for_each_entry(tdev, &device->top->device, head) {
+ if (tdev->intr >= 0 &&
+ tdev->type == subdev->type &&
+ tdev->inst == subdev->inst) {
+ if (data->mask & BIT(tdev->intr)) {
+ *leaf = data->leaf;
+ *mask = BIT(tdev->intr);
+ return 0;
+ }
+ }
+ }
+ } else
+ if (data->type == subdev->type && data->inst == subdev->inst) {
+ *leaf = data->leaf;
+ *mask = data->mask;
+ return 0;
+ }
+
+ data++;
+ }
+ } else {
+ return -ENOSYS;
+ }
+ } else {
+ if (type < intr->leaves * sizeof(*intr->stat) * 8) {
+ *leaf = type / 32;
+ *mask = BIT(type % 32);
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+
+static struct nvkm_intr *
+nvkm_intr_find(struct nvkm_subdev *subdev, enum nvkm_intr_type type, int *leaf, u32 *mask)
+{
+ struct nvkm_intr *intr;
+ int ret;
+
+ list_for_each_entry(intr, &subdev->device->intr.intr, head) {
+ ret = nvkm_intr_xlat(subdev, intr, type, leaf, mask);
+ if (ret == 0)
+ return intr;
+ }
+
+ return NULL;
+}
+
+static void
+nvkm_intr_allow_locked(struct nvkm_intr *intr, int leaf, u32 mask)
+{
+ intr->mask[leaf] |= mask;
+ if (intr->func->allow) {
+ if (intr->func->reset)
+ intr->func->reset(intr, leaf, mask);
+ intr->func->allow(intr, leaf, mask);
+ }
+}
+
+void
+nvkm_intr_allow(struct nvkm_subdev *subdev, enum nvkm_intr_type type)
+{
+ struct nvkm_device *device = subdev->device;
+ struct nvkm_intr *intr;
+ unsigned long flags;
+ int leaf;
+ u32 mask;
+
+ intr = nvkm_intr_find(subdev, type, &leaf, &mask);
+ if (intr) {
+ nvkm_debug(intr->subdev, "intr %d/%08x allowed by %s\n", leaf, mask, subdev->name);
+ spin_lock_irqsave(&device->intr.lock, flags);
+ nvkm_intr_allow_locked(intr, leaf, mask);
+ spin_unlock_irqrestore(&device->intr.lock, flags);
+ }
+}
+
+static void
+nvkm_intr_block_locked(struct nvkm_intr *intr, int leaf, u32 mask)
+{
+ intr->mask[leaf] &= ~mask;
+ if (intr->func->block)
+ intr->func->block(intr, leaf, mask);
+}
+
+void
+nvkm_intr_block(struct nvkm_subdev *subdev, enum nvkm_intr_type type)
+{
+ struct nvkm_device *device = subdev->device;
+ struct nvkm_intr *intr;
+ unsigned long flags;
+ int leaf;
+ u32 mask;
+
+ intr = nvkm_intr_find(subdev, type, &leaf, &mask);
+ if (intr) {
+ nvkm_debug(intr->subdev, "intr %d/%08x blocked by %s\n", leaf, mask, subdev->name);
+ spin_lock_irqsave(&device->intr.lock, flags);
+ nvkm_intr_block_locked(intr, leaf, mask);
+ spin_unlock_irqrestore(&device->intr.lock, flags);
+ }
+}
+
+static void
+nvkm_intr_rearm_locked(struct nvkm_device *device)
+{
+ struct nvkm_intr *intr;
+
+ list_for_each_entry(intr, &device->intr.intr, head)
+ intr->func->rearm(intr);
+}
+
+static void
+nvkm_intr_unarm_locked(struct nvkm_device *device)
+{
+ struct nvkm_intr *intr;
+
+ list_for_each_entry(intr, &device->intr.intr, head)
+ intr->func->unarm(intr);
+}
+
+static irqreturn_t
+nvkm_intr(int irq, void *arg)
+{
+ struct nvkm_device *device = arg;
+ struct nvkm_intr *intr;
+ struct nvkm_inth *inth;
+ irqreturn_t ret = IRQ_NONE;
+ bool pending = false;
+ int prio, leaf;
+
+ /* Disable all top-level interrupt sources, and re-arm MSI interrupts. */
+ spin_lock(&device->intr.lock);
+ if (!device->intr.armed)
+ goto done_unlock;
+
+ nvkm_intr_unarm_locked(device);
+ nvkm_pci_msi_rearm(device);
+
+ /* Fetch pending interrupt masks. */
+ list_for_each_entry(intr, &device->intr.intr, head) {
+ if (intr->func->pending(intr))
+ pending = true;
+ }
+
+ if (!pending)
+ goto done;
+
+ /* Check that GPU is still on the bus by reading NV_PMC_BOOT_0. */
+ if (WARN_ON(nvkm_rd32(device, 0x000000) == 0xffffffff))
+ goto done;
+
+ /* Execute handlers. */
+ for (prio = 0; prio < ARRAY_SIZE(device->intr.prio); prio++) {
+ list_for_each_entry(inth, &device->intr.prio[prio], head) {
+ struct nvkm_intr *intr = inth->intr;
+
+ if (intr->stat[inth->leaf] & inth->mask) {
+ if (atomic_read(&inth->allowed)) {
+ if (intr->func->reset)
+ intr->func->reset(intr, inth->leaf, inth->mask);
+ if (inth->func(inth) == IRQ_HANDLED)
+ ret = IRQ_HANDLED;
+ }
+ }
+ }
+ }
+
+ /* Nothing handled? Some debugging/protection from IRQ storms is in order... */
+ if (ret == IRQ_NONE) {
+ list_for_each_entry(intr, &device->intr.intr, head) {
+ for (leaf = 0; leaf < intr->leaves; leaf++) {
+ if (intr->stat[leaf]) {
+ nvkm_warn(intr->subdev, "intr%d: %08x\n",
+ leaf, intr->stat[leaf]);
+ nvkm_intr_block_locked(intr, leaf, intr->stat[leaf]);
+ }
+ }
+ }
+ }
+
+done:
+ /* Re-enable all top-level interrupt sources. */
+ nvkm_intr_rearm_locked(device);
+done_unlock:
+ spin_unlock(&device->intr.lock);
+ return ret;
+}
+
+int
+nvkm_intr_add(const struct nvkm_intr_func *func, const struct nvkm_intr_data *data,
+ struct nvkm_subdev *subdev, int leaves, struct nvkm_intr *intr)
+{
+ struct nvkm_device *device = subdev->device;
+ int i;
+
+ intr->func = func;
+ intr->data = data;
+ intr->subdev = subdev;
+ intr->leaves = leaves;
+ intr->stat = kcalloc(leaves, sizeof(*intr->stat), GFP_KERNEL);
+ intr->mask = kcalloc(leaves, sizeof(*intr->mask), GFP_KERNEL);
+ if (!intr->stat || !intr->mask) {
+ kfree(intr->stat);
+ return -ENOMEM;
+ }
+
+ if (intr->subdev->debug >= NV_DBG_DEBUG) {
+ for (i = 0; i < intr->leaves; i++)
+ intr->mask[i] = ~0;
+ }
+
+ spin_lock_irq(&device->intr.lock);
+ list_add_tail(&intr->head, &device->intr.intr);
+ spin_unlock_irq(&device->intr.lock);
+ return 0;
+}
+
+static irqreturn_t
+nvkm_intr_subdev(struct nvkm_inth *inth)
+{
+ struct nvkm_subdev *subdev = container_of(inth, typeof(*subdev), inth);
+
+ nvkm_subdev_intr(subdev);
+ return IRQ_HANDLED;
+}
+
+static void
+nvkm_intr_subdev_add_dev(struct nvkm_intr *intr, enum nvkm_subdev_type type, int inst)
+{
+ struct nvkm_subdev *subdev;
+ enum nvkm_intr_prio prio;
+ int ret;
+
+ subdev = nvkm_device_subdev(intr->subdev->device, type, inst);
+ if (!subdev || !subdev->func->intr)
+ return;
+
+ if (type == NVKM_ENGINE_DISP)
+ prio = NVKM_INTR_PRIO_VBLANK;
+ else
+ prio = NVKM_INTR_PRIO_NORMAL;
+
+ ret = nvkm_inth_add(intr, NVKM_INTR_SUBDEV, prio, subdev, nvkm_intr_subdev, &subdev->inth);
+ if (WARN_ON(ret))
+ return;
+
+ nvkm_inth_allow(&subdev->inth);
+}
+
+static void
+nvkm_intr_subdev_add(struct nvkm_intr *intr)
+{
+ const struct nvkm_intr_data *data;
+ struct nvkm_device *device = intr->subdev->device;
+ struct nvkm_top_device *tdev;
+
+ for (data = intr->data; data && data->mask; data++) {
+ if (data->legacy) {
+ if (data->type == NVKM_SUBDEV_TOP) {
+ list_for_each_entry(tdev, &device->top->device, head) {
+ if (tdev->intr < 0 || !(data->mask & BIT(tdev->intr)))
+ continue;
+
+ nvkm_intr_subdev_add_dev(intr, tdev->type, tdev->inst);
+ }
+ } else {
+ nvkm_intr_subdev_add_dev(intr, data->type, data->inst);
+ }
+ }
+ }
+}
+
+void
+nvkm_intr_rearm(struct nvkm_device *device)
+{
+ struct nvkm_intr *intr;
+ int i;
+
+ if (unlikely(!device->intr.legacy_done)) {
+ list_for_each_entry(intr, &device->intr.intr, head)
+ nvkm_intr_subdev_add(intr);
+ device->intr.legacy_done = true;
+ }
+
+ spin_lock_irq(&device->intr.lock);
+ list_for_each_entry(intr, &device->intr.intr, head) {
+ for (i = 0; intr->func->block && i < intr->leaves; i++) {
+ intr->func->block(intr, i, ~0);
+ intr->func->allow(intr, i, intr->mask[i]);
+ }
+ }
+
+ nvkm_intr_rearm_locked(device);
+ device->intr.armed = true;
+ spin_unlock_irq(&device->intr.lock);
+}
+
+void
+nvkm_intr_unarm(struct nvkm_device *device)
+{
+ spin_lock_irq(&device->intr.lock);
+ nvkm_intr_unarm_locked(device);
+ device->intr.armed = false;
+ spin_unlock_irq(&device->intr.lock);
+}
+
+int
+nvkm_intr_install(struct nvkm_device *device)
+{
+ int ret;
+
+ device->intr.irq = device->func->irq(device);
+ if (device->intr.irq < 0)
+ return device->intr.irq;
+
+ ret = request_irq(device->intr.irq, nvkm_intr, IRQF_SHARED, "nvkm", device);
+ if (ret)
+ return ret;
+
+ device->intr.alloc = true;
+ return 0;
+}
+
+void
+nvkm_intr_dtor(struct nvkm_device *device)
+{
+ struct nvkm_intr *intr, *intt;
+
+ list_for_each_entry_safe(intr, intt, &device->intr.intr, head) {
+ list_del(&intr->head);
+ kfree(intr->mask);
+ kfree(intr->stat);
+ }
+
+ if (device->intr.alloc)
+ free_irq(device->intr.irq, device);
+}
+
+void
+nvkm_intr_ctor(struct nvkm_device *device)
+{
+ int i;
+
+ INIT_LIST_HEAD(&device->intr.intr);
+ for (i = 0; i < ARRAY_SIZE(device->intr.prio); i++)
+ INIT_LIST_HEAD(&device->intr.prio[i]);
+
+ spin_lock_init(&device->intr.lock);
+ device->intr.armed = false;
+}
+
+void
+nvkm_inth_block(struct nvkm_inth *inth)
+{
+ if (unlikely(!inth->intr))
+ return;
+
+ atomic_set(&inth->allowed, 0);
+}
+
+void
+nvkm_inth_allow(struct nvkm_inth *inth)
+{
+ struct nvkm_intr *intr = inth->intr;
+ unsigned long flags;
+
+ if (unlikely(!inth->intr))
+ return;
+
+ spin_lock_irqsave(&intr->subdev->device->intr.lock, flags);
+ if (!atomic_xchg(&inth->allowed, 1)) {
+ if ((intr->mask[inth->leaf] & inth->mask) != inth->mask)
+ nvkm_intr_allow_locked(intr, inth->leaf, inth->mask);
+ }
+ spin_unlock_irqrestore(&intr->subdev->device->intr.lock, flags);
+}
+
+int
+nvkm_inth_add(struct nvkm_intr *intr, enum nvkm_intr_type type, enum nvkm_intr_prio prio,
+ struct nvkm_subdev *subdev, nvkm_inth_func func, struct nvkm_inth *inth)
+{
+ struct nvkm_device *device = subdev->device;
+ int ret;
+
+ if (WARN_ON(inth->mask))
+ return -EBUSY;
+
+ ret = nvkm_intr_xlat(subdev, intr, type, &inth->leaf, &inth->mask);
+ if (ret)
+ return ret;
+
+ nvkm_debug(intr->subdev, "intr %d/%08x requested by %s\n",
+ inth->leaf, inth->mask, subdev->name);
+
+ inth->intr = intr;
+ inth->func = func;
+ atomic_set(&inth->allowed, 0);
+ list_add_tail(&inth->head, &device->intr.prio[prio]);
+ return 0;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/ioctl.c b/drivers/gpu/drm/nouveau/nvkm/core/ioctl.c
index 45f920d..0b33287 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/ioctl.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/ioctl.c
@@ -47,6 +47,26 @@ nvkm_ioctl_nop(struct nvkm_client *client,
return ret;
}
+#include <nvif/class.h>
+
+static int
+nvkm_ioctl_sclass_(struct nvkm_object *object, int index, struct nvkm_oclass *oclass)
+{
+ if ( object->func->uevent &&
+ !object->func->uevent(object, NULL, 0, NULL) && index-- == 0) {
+ oclass->ctor = nvkm_uevent_new;
+ oclass->base.minver = 0;
+ oclass->base.maxver = 0;
+ oclass->base.oclass = NVIF_CLASS_EVENT;
+ return 0;
+ }
+
+ if (object->func->sclass)
+ return object->func->sclass(object, index, oclass);
+
+ return -ENOSYS;
+}
+
static int
nvkm_ioctl_sclass(struct nvkm_client *client,
struct nvkm_object *object, void *data, u32 size)
@@ -64,8 +84,7 @@ nvkm_ioctl_sclass(struct nvkm_client *client,
if (size != args->v0.count * sizeof(args->v0.oclass[0]))
return -EINVAL;
- while (object->func->sclass &&
- object->func->sclass(object, i, &oclass) >= 0) {
+ while (nvkm_ioctl_sclass_(object, i, &oclass) >= 0) {
if (i < args->v0.count) {
args->v0.oclass[i].oclass = oclass.base.oclass;
args->v0.oclass[i].minver = oclass.base.minver;
@@ -100,7 +119,7 @@ nvkm_ioctl_new(struct nvkm_client *client,
} else
return ret;
- if (!parent->func->sclass) {
+ if (!parent->func->sclass && !parent->func->uevent) {
nvif_ioctl(parent, "cannot have children\n");
return -EINVAL;
}
@@ -113,7 +132,7 @@ nvkm_ioctl_new(struct nvkm_client *client,
oclass.object = args->v0.object;
oclass.client = client;
oclass.parent = parent;
- ret = parent->func->sclass(parent, i++, &oclass);
+ ret = nvkm_ioctl_sclass_(parent, i++, &oclass);
if (ret)
return ret;
} while (oclass.base.oclass != args->v0.oclass);
@@ -294,90 +313,6 @@ nvkm_ioctl_unmap(struct nvkm_client *client,
return ret;
}
-static int
-nvkm_ioctl_ntfy_new(struct nvkm_client *client,
- struct nvkm_object *object, void *data, u32 size)
-{
- union {
- struct nvif_ioctl_ntfy_new_v0 v0;
- } *args = data;
- struct nvkm_event *event;
- int ret = -ENOSYS;
-
- nvif_ioctl(object, "ntfy new size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, true))) {
- nvif_ioctl(object, "ntfy new vers %d event %02x\n",
- args->v0.version, args->v0.event);
- ret = nvkm_object_ntfy(object, args->v0.event, &event);
- if (ret == 0) {
- ret = nvkm_client_notify_new(object, event, data, size);
- if (ret >= 0) {
- args->v0.index = ret;
- ret = 0;
- }
- }
- }
-
- return ret;
-}
-
-static int
-nvkm_ioctl_ntfy_del(struct nvkm_client *client,
- struct nvkm_object *object, void *data, u32 size)
-{
- union {
- struct nvif_ioctl_ntfy_del_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
-
- nvif_ioctl(object, "ntfy del size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(object, "ntfy del vers %d index %d\n",
- args->v0.version, args->v0.index);
- ret = nvkm_client_notify_del(client, args->v0.index);
- }
-
- return ret;
-}
-
-static int
-nvkm_ioctl_ntfy_get(struct nvkm_client *client,
- struct nvkm_object *object, void *data, u32 size)
-{
- union {
- struct nvif_ioctl_ntfy_get_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
-
- nvif_ioctl(object, "ntfy get size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(object, "ntfy get vers %d index %d\n",
- args->v0.version, args->v0.index);
- ret = nvkm_client_notify_get(client, args->v0.index);
- }
-
- return ret;
-}
-
-static int
-nvkm_ioctl_ntfy_put(struct nvkm_client *client,
- struct nvkm_object *object, void *data, u32 size)
-{
- union {
- struct nvif_ioctl_ntfy_put_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
-
- nvif_ioctl(object, "ntfy put size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(object, "ntfy put vers %d index %d\n",
- args->v0.version, args->v0.index);
- ret = nvkm_client_notify_put(client, args->v0.index);
- }
-
- return ret;
-}
-
static struct {
int version;
int (*func)(struct nvkm_client *, struct nvkm_object *, void *, u32);
@@ -392,10 +327,6 @@ nvkm_ioctl_v0[] = {
{ 0x00, nvkm_ioctl_wr },
{ 0x00, nvkm_ioctl_map },
{ 0x00, nvkm_ioctl_unmap },
- { 0x00, nvkm_ioctl_ntfy_new },
- { 0x00, nvkm_ioctl_ntfy_del },
- { 0x00, nvkm_ioctl_ntfy_get },
- { 0x00, nvkm_ioctl_ntfy_put },
};
static int
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/notify.c b/drivers/gpu/drm/nouveau/nvkm/core/notify.c
deleted file mode 100644
index 023610d..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/core/notify.c
+++ /dev/null
@@ -1,163 +0,0 @@
-/*
- * Copyright 2014 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs <bskeggs@redhat.com>
- */
-#include <core/notify.h>
-#include <core/event.h>
-
-static inline void
-nvkm_notify_put_locked(struct nvkm_notify *notify)
-{
- if (notify->block++ == 0)
- nvkm_event_put(notify->event, notify->types, notify->index);
-}
-
-void
-nvkm_notify_put(struct nvkm_notify *notify)
-{
- struct nvkm_event *event = notify->event;
- unsigned long flags;
- if (likely(event) &&
- test_and_clear_bit(NVKM_NOTIFY_USER, ¬ify->flags)) {
- spin_lock_irqsave(&event->refs_lock, flags);
- nvkm_notify_put_locked(notify);
- spin_unlock_irqrestore(&event->refs_lock, flags);
- if (test_bit(NVKM_NOTIFY_WORK, ¬ify->flags))
- flush_work(¬ify->work);
- }
-}
-
-static inline void
-nvkm_notify_get_locked(struct nvkm_notify *notify)
-{
- if (--notify->block == 0)
- nvkm_event_get(notify->event, notify->types, notify->index);
-}
-
-void
-nvkm_notify_get(struct nvkm_notify *notify)
-{
- struct nvkm_event *event = notify->event;
- unsigned long flags;
- if (likely(event) &&
- !test_and_set_bit(NVKM_NOTIFY_USER, ¬ify->flags)) {
- spin_lock_irqsave(&event->refs_lock, flags);
- nvkm_notify_get_locked(notify);
- spin_unlock_irqrestore(&event->refs_lock, flags);
- }
-}
-
-static inline void
-nvkm_notify_func(struct nvkm_notify *notify)
-{
- struct nvkm_event *event = notify->event;
- int ret = notify->func(notify);
- unsigned long flags;
- if ((ret == NVKM_NOTIFY_KEEP) ||
- !test_and_clear_bit(NVKM_NOTIFY_USER, ¬ify->flags)) {
- spin_lock_irqsave(&event->refs_lock, flags);
- nvkm_notify_get_locked(notify);
- spin_unlock_irqrestore(&event->refs_lock, flags);
- }
-}
-
-static void
-nvkm_notify_work(struct work_struct *work)
-{
- struct nvkm_notify *notify = container_of(work, typeof(*notify), work);
- nvkm_notify_func(notify);
-}
-
-void
-nvkm_notify_send(struct nvkm_notify *notify, void *data, u32 size)
-{
- struct nvkm_event *event = notify->event;
- unsigned long flags;
-
- assert_spin_locked(&event->list_lock);
- BUG_ON(size != notify->size);
-
- spin_lock_irqsave(&event->refs_lock, flags);
- if (notify->block) {
- spin_unlock_irqrestore(&event->refs_lock, flags);
- return;
- }
- nvkm_notify_put_locked(notify);
- spin_unlock_irqrestore(&event->refs_lock, flags);
-
- if (test_bit(NVKM_NOTIFY_WORK, ¬ify->flags)) {
- memcpy((void *)notify->data, data, size);
- schedule_work(¬ify->work);
- } else {
- notify->data = data;
- nvkm_notify_func(notify);
- notify->data = NULL;
- }
-}
-
-void
-nvkm_notify_fini(struct nvkm_notify *notify)
-{
- unsigned long flags;
- if (notify->event) {
- nvkm_notify_put(notify);
- spin_lock_irqsave(¬ify->event->list_lock, flags);
- list_del(¬ify->head);
- spin_unlock_irqrestore(¬ify->event->list_lock, flags);
- kfree((void *)notify->data);
- notify->event = NULL;
- }
-}
-
-int
-nvkm_notify_init(struct nvkm_object *object, struct nvkm_event *event,
- int (*func)(struct nvkm_notify *), bool work,
- void *data, u32 size, u32 reply,
- struct nvkm_notify *notify)
-{
- unsigned long flags;
- int ret = -ENODEV;
- if ((notify->event = event), event->refs) {
- ret = event->func->ctor(object, data, size, notify);
- if (ret == 0 && (ret = -EINVAL, notify->size == reply)) {
- notify->flags = 0;
- notify->block = 1;
- notify->func = func;
- notify->data = NULL;
- if (ret = 0, work) {
- INIT_WORK(¬ify->work, nvkm_notify_work);
- set_bit(NVKM_NOTIFY_WORK, ¬ify->flags);
- notify->data = kmalloc(reply, GFP_KERNEL);
- if (!notify->data)
- ret = -ENOMEM;
- }
- }
- if (ret == 0) {
- spin_lock_irqsave(&event->list_lock, flags);
- list_add_tail(¬ify->head, &event->list);
- spin_unlock_irqrestore(&event->list_lock, flags);
- }
- }
- if (ret)
- notify->event = NULL;
- return ret;
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/oproxy.c b/drivers/gpu/drm/nouveau/nvkm/core/oproxy.c
index 16299837..3385528 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/oproxy.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/oproxy.c
@@ -47,7 +47,12 @@ nvkm_oproxy_map(struct nvkm_object *object, void *argv, u32 argc,
static int
nvkm_oproxy_unmap(struct nvkm_object *object)
{
- return nvkm_object_unmap(nvkm_oproxy(object)->object);
+ struct nvkm_oproxy *oproxy = nvkm_oproxy(object);
+
+ if (unlikely(!oproxy->object))
+ return 0;
+
+ return nvkm_object_unmap(oproxy->object);
}
static int
@@ -106,6 +111,18 @@ nvkm_oproxy_sclass(struct nvkm_object *object, int index,
}
static int
+nvkm_oproxy_uevent(struct nvkm_object *object, void *argv, u32 argc,
+ struct nvkm_uevent *uevent)
+{
+ struct nvkm_oproxy *oproxy = nvkm_oproxy(object);
+
+ if (!oproxy->object->func->uevent)
+ return -ENOSYS;
+
+ return oproxy->object->func->uevent(oproxy->object, argv, argc, uevent);
+}
+
+static int
nvkm_oproxy_fini(struct nvkm_object *object, bool suspend)
{
struct nvkm_oproxy *oproxy = nvkm_oproxy(object);
@@ -188,6 +205,7 @@ nvkm_oproxy_func = {
.wr32 = nvkm_oproxy_wr32,
.bind = nvkm_oproxy_bind,
.sclass = nvkm_oproxy_sclass,
+ .uevent = nvkm_oproxy_uevent,
};
void
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
index a74b7ac..6c20e82 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
@@ -54,7 +54,7 @@ int
nvkm_subdev_fini(struct nvkm_subdev *subdev, bool suspend)
{
struct nvkm_device *device = subdev->device;
- const char *action = suspend ? "suspend" : "fini";
+ const char *action = suspend ? "suspend" : subdev->use.enabled ? "fini" : "reset";
s64 time;
nvkm_trace(subdev, "%s running...\n", action);
@@ -68,6 +68,7 @@ nvkm_subdev_fini(struct nvkm_subdev *subdev, bool suspend)
return ret;
}
}
+ subdev->use.enabled = false;
nvkm_mc_reset(device, subdev->type, subdev->inst);
@@ -97,29 +98,48 @@ nvkm_subdev_preinit(struct nvkm_subdev *subdev)
return 0;
}
-int
-nvkm_subdev_init(struct nvkm_subdev *subdev)
+static int
+nvkm_subdev_oneinit_(struct nvkm_subdev *subdev)
{
s64 time;
int ret;
+ if (!subdev->func->oneinit || subdev->oneinit)
+ return 0;
+
+ nvkm_trace(subdev, "one-time init running...\n");
+ time = ktime_to_us(ktime_get());
+ ret = subdev->func->oneinit(subdev);
+ if (ret) {
+ nvkm_error(subdev, "one-time init failed, %d\n", ret);
+ return ret;
+ }
+
+ subdev->oneinit = true;
+ time = ktime_to_us(ktime_get()) - time;
+ nvkm_trace(subdev, "one-time init completed in %lldus\n", time);
+ return 0;
+}
+
+static int
+nvkm_subdev_init_(struct nvkm_subdev *subdev)
+{
+ s64 time;
+ int ret;
+
+ if (subdev->use.enabled) {
+ nvkm_trace(subdev, "init skipped, already running\n");
+ return 0;
+ }
+
nvkm_trace(subdev, "init running...\n");
time = ktime_to_us(ktime_get());
- if (subdev->func->oneinit && !subdev->oneinit) {
- s64 time;
- nvkm_trace(subdev, "one-time init running...\n");
- time = ktime_to_us(ktime_get());
- ret = subdev->func->oneinit(subdev);
- if (ret) {
- nvkm_error(subdev, "one-time init failed, %d\n", ret);
- return ret;
- }
+ ret = nvkm_subdev_oneinit_(subdev);
+ if (ret)
+ return ret;
- subdev->oneinit = true;
- time = ktime_to_us(ktime_get()) - time;
- nvkm_trace(subdev, "one-time init completed in %lldus\n", time);
- }
+ subdev->use.enabled = true;
if (subdev->func->init) {
ret = subdev->func->init(subdev);
@@ -134,6 +154,64 @@ nvkm_subdev_init(struct nvkm_subdev *subdev)
return 0;
}
+int
+nvkm_subdev_init(struct nvkm_subdev *subdev)
+{
+ int ret;
+
+ mutex_lock(&subdev->use.mutex);
+ if (refcount_read(&subdev->use.refcount) == 0) {
+ nvkm_trace(subdev, "init skipped, no users\n");
+ mutex_unlock(&subdev->use.mutex);
+ return 0;
+ }
+
+ ret = nvkm_subdev_init_(subdev);
+ mutex_unlock(&subdev->use.mutex);
+ return ret;
+}
+
+int
+nvkm_subdev_oneinit(struct nvkm_subdev *subdev)
+{
+ int ret;
+
+ mutex_lock(&subdev->use.mutex);
+ ret = nvkm_subdev_oneinit_(subdev);
+ mutex_unlock(&subdev->use.mutex);
+ return ret;
+}
+
+void
+nvkm_subdev_unref(struct nvkm_subdev *subdev)
+{
+ if (refcount_dec_and_mutex_lock(&subdev->use.refcount, &subdev->use.mutex)) {
+ nvkm_subdev_fini(subdev, false);
+ mutex_unlock(&subdev->use.mutex);
+ }
+}
+
+int
+nvkm_subdev_ref(struct nvkm_subdev *subdev)
+{
+ int ret;
+
+ if (subdev && !refcount_inc_not_zero(&subdev->use.refcount)) {
+ mutex_lock(&subdev->use.mutex);
+ if (!refcount_inc_not_zero(&subdev->use.refcount)) {
+ if ((ret = nvkm_subdev_init_(subdev))) {
+ mutex_unlock(&subdev->use.mutex);
+ return ret;
+ }
+
+ refcount_set(&subdev->use.refcount, 1);
+ }
+ mutex_unlock(&subdev->use.mutex);
+ }
+
+ return 0;
+}
+
void
nvkm_subdev_del(struct nvkm_subdev **psubdev)
{
@@ -146,6 +224,7 @@ nvkm_subdev_del(struct nvkm_subdev **psubdev)
list_del(&subdev->head);
if (subdev->func->dtor)
*psubdev = subdev->func->dtor(subdev);
+ mutex_destroy(&subdev->use.mutex);
time = ktime_to_us(ktime_get()) - time;
nvkm_trace(subdev, "destroy completed in %lldus\n", time);
kfree(*psubdev);
@@ -167,8 +246,8 @@ nvkm_subdev_disable(struct nvkm_device *device, enum nvkm_subdev_type type, int
}
void
-nvkm_subdev_ctor(const struct nvkm_subdev_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_subdev *subdev)
+__nvkm_subdev_ctor(const struct nvkm_subdev_func *func, struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_subdev *subdev)
{
subdev->func = func;
subdev->device = device;
@@ -180,6 +259,8 @@ nvkm_subdev_ctor(const struct nvkm_subdev_func *func, struct nvkm_device *device
else
strscpy(subdev->name, nvkm_subdev_type[type], sizeof(subdev->name));
subdev->debug = nvkm_dbgopt(device->dbgopt, subdev->name);
+
+ refcount_set(&subdev->use.refcount, 1);
list_add_tail(&subdev->head, &device->subdev);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/uevent.c b/drivers/gpu/drm/nouveau/nvkm/core/uevent.c
new file mode 100644
index 0000000..ba9d9ed
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/core/uevent.c
@@ -0,0 +1,157 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#define nvkm_uevent(p) container_of((p), struct nvkm_uevent, object)
+#include <core/event.h>
+#include <core/client.h>
+
+#include <nvif/if000e.h>
+
+struct nvkm_uevent {
+ struct nvkm_object object;
+ struct nvkm_object *parent;
+ nvkm_uevent_func func;
+ bool wait;
+
+ struct nvkm_event_ntfy ntfy;
+ atomic_t allowed;
+};
+
+static int
+nvkm_uevent_mthd_block(struct nvkm_uevent *uevent, union nvif_event_block_args *args, u32 argc)
+{
+ if (argc != sizeof(args->vn))
+ return -ENOSYS;
+
+ nvkm_event_ntfy_block(&uevent->ntfy);
+ atomic_set(&uevent->allowed, 0);
+ return 0;
+}
+
+static int
+nvkm_uevent_mthd_allow(struct nvkm_uevent *uevent, union nvif_event_allow_args *args, u32 argc)
+{
+ if (argc != sizeof(args->vn))
+ return -ENOSYS;
+
+ nvkm_event_ntfy_allow(&uevent->ntfy);
+ atomic_set(&uevent->allowed, 1);
+ return 0;
+}
+
+static int
+nvkm_uevent_mthd(struct nvkm_object *object, u32 mthd, void *argv, u32 argc)
+{
+ struct nvkm_uevent *uevent = nvkm_uevent(object);
+
+ switch (mthd) {
+ case NVIF_EVENT_V0_ALLOW: return nvkm_uevent_mthd_allow(uevent, argv, argc);
+ case NVIF_EVENT_V0_BLOCK: return nvkm_uevent_mthd_block(uevent, argv, argc);
+ default:
+ break;
+ }
+
+ return -EINVAL;
+}
+
+static int
+nvkm_uevent_fini(struct nvkm_object *object, bool suspend)
+{
+ struct nvkm_uevent *uevent = nvkm_uevent(object);
+
+ nvkm_event_ntfy_block(&uevent->ntfy);
+ return 0;
+}
+
+static int
+nvkm_uevent_init(struct nvkm_object *object)
+{
+ struct nvkm_uevent *uevent = nvkm_uevent(object);
+
+ if (atomic_read(&uevent->allowed))
+ nvkm_event_ntfy_allow(&uevent->ntfy);
+
+ return 0;
+}
+
+static void *
+nvkm_uevent_dtor(struct nvkm_object *object)
+{
+ struct nvkm_uevent *uevent = nvkm_uevent(object);
+
+ nvkm_event_ntfy_del(&uevent->ntfy);
+ return uevent;
+}
+
+static const struct nvkm_object_func
+nvkm_uevent = {
+ .dtor = nvkm_uevent_dtor,
+ .init = nvkm_uevent_init,
+ .fini = nvkm_uevent_fini,
+ .mthd = nvkm_uevent_mthd,
+};
+
+static int
+nvkm_uevent_ntfy(struct nvkm_event_ntfy *ntfy, u32 bits)
+{
+ struct nvkm_uevent *uevent = container_of(ntfy, typeof(*uevent), ntfy);
+ struct nvkm_client *client = uevent->object.client;
+
+ if (uevent->func)
+ return uevent->func(uevent->parent, uevent->object.token, bits);
+
+ return client->event(uevent->object.token, NULL, 0);
+}
+
+int
+nvkm_uevent_add(struct nvkm_uevent *uevent, struct nvkm_event *event, int id, u32 bits,
+ nvkm_uevent_func func)
+{
+ if (WARN_ON(uevent->func))
+ return -EBUSY;
+
+ nvkm_event_ntfy_add(event, id, bits, uevent->wait, nvkm_uevent_ntfy, &uevent->ntfy);
+ uevent->func = func;
+ return 0;
+}
+
+int
+nvkm_uevent_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
+ struct nvkm_object **pobject)
+{
+ struct nvkm_object *parent = oclass->parent;
+ struct nvkm_uevent *uevent;
+ union nvif_event_args *args = argv;
+
+ if (argc < sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+
+ if (!(uevent = kzalloc(sizeof(*uevent), GFP_KERNEL)))
+ return -ENOMEM;
+ *pobject = &uevent->object;
+
+ nvkm_object_ctor(&nvkm_uevent, oclass, &uevent->object);
+ uevent->parent = parent;
+ uevent->func = NULL;
+ uevent->wait = args->v0.wait;
+ uevent->ntfy.event = NULL;
+ return parent->func->uevent(parent, &args->v0.data, argc - sizeof(args->v0), uevent);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/Kbuild b/drivers/gpu/drm/nouveau/nvkm/engine/ce/Kbuild
index ba88613..8bf1635 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/Kbuild
@@ -8,3 +8,5 @@
nvkm-y += nvkm/engine/ce/gp102.o
nvkm-y += nvkm/engine/ce/gv100.o
nvkm-y += nvkm/engine/ce/tu102.o
+nvkm-y += nvkm/engine/ce/ga100.o
+nvkm-y += nvkm/engine/ce/ga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/ga100.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/ga100.c
new file mode 100644
index 0000000..6648ed6
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/ga100.c
@@ -0,0 +1,82 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <subdev/vfn.h>
+
+#include <nvif/class.h>
+
+static irqreturn_t
+ga100_ce_intr(struct nvkm_inth *inth)
+{
+ struct nvkm_subdev *subdev = container_of(inth, typeof(*subdev), inth);
+
+ /*TODO*/
+ nvkm_error(subdev, "intr\n");
+ return IRQ_NONE;
+}
+
+int
+ga100_ce_fini(struct nvkm_engine *engine, bool suspend)
+{
+ nvkm_inth_block(&engine->subdev.inth);
+ return 0;
+}
+
+int
+ga100_ce_init(struct nvkm_engine *engine)
+{
+ nvkm_inth_allow(&engine->subdev.inth);
+ return 0;
+}
+
+int
+ga100_ce_oneinit(struct nvkm_engine *engine)
+{
+ struct nvkm_subdev *subdev = &engine->subdev;
+ struct nvkm_device *device = subdev->device;
+ u32 vector;
+
+ vector = nvkm_rd32(device, 0x10442c + (subdev->inst * 0x80)) & 0x00000fff;
+
+ return nvkm_inth_add(&device->vfn->intr, vector, NVKM_INTR_PRIO_NORMAL,
+ subdev, ga100_ce_intr, &subdev->inth);
+}
+
+static const struct nvkm_engine_func
+ga100_ce = {
+ .oneinit = ga100_ce_oneinit,
+ .init = ga100_ce_init,
+ .fini = ga100_ce_fini,
+ .cclass = &gv100_ce_cclass,
+ .sclass = {
+ { -1, -1, AMPERE_DMA_COPY_A },
+ {}
+ }
+};
+
+int
+ga100_ce_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_engine **pengine)
+{
+ return nvkm_engine_new_(&ga100_ce, device, type, inst, true, pengine);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/ga102.c
similarity index 63%
rename from drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
rename to drivers/gpu/drm/nouveau/nvkm/engine/ce/ga102.c
index 217268f..9f3448a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/ga102.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2018 Red Hat Inc.
+ * Copyright 2021 Red Hat Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -19,27 +19,26 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include "user.h"
+#include "priv.h"
-static int
-tu102_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc,
- enum nvkm_object_map *type, u64 *addr, u64 *size)
-{
- struct nvkm_device *device = object->engine->subdev.device;
- *addr = 0xbb0000 + device->func->resource_addr(device, 0);
- *size = 0x010000;
- *type = NVKM_OBJECT_MAP_IO;
- return 0;
-}
+#include <nvif/class.h>
-static const struct nvkm_object_func
-tu102_fifo_user = {
- .map = tu102_fifo_user_map,
+static const struct nvkm_engine_func
+ga102_ce = {
+ .oneinit = ga100_ce_oneinit,
+ .init = ga100_ce_init,
+ .fini = ga100_ce_fini,
+ .cclass = &gv100_ce_cclass,
+ .sclass = {
+ { -1, -1, AMPERE_DMA_COPY_A },
+ { -1, -1, AMPERE_DMA_COPY_B },
+ {}
+ }
};
int
-tu102_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
- struct nvkm_object **pobject)
+ga102_ce_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_engine **pengine)
{
- return nvkm_object_new_(&tu102_fifo_user, oclass, argv, argc, pobject);
+ return nvkm_engine_new_(&ga102_ce, device, type, inst, true, pengine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c
index 09a112a..c9bf630 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c
@@ -40,7 +40,7 @@ gt215_ce_isr_error_name[] = {
};
void
-gt215_ce_intr(struct nvkm_falcon *ce, struct nvkm_fifo_chan *chan)
+gt215_ce_intr(struct nvkm_falcon *ce, struct nvkm_chan *chan)
{
struct nvkm_subdev *subdev = &ce->engine.subdev;
struct nvkm_device *device = subdev->device;
@@ -55,9 +55,9 @@ gt215_ce_intr(struct nvkm_falcon *ce, struct nvkm_fifo_chan *chan)
nvkm_error(subdev, "DISPATCH_ERROR %04x [%s] ch %d [%010llx %s] "
"subc %d mthd %04x data %08x\n", ssta,
- en ? en->name : "", chan ? chan->chid : -1,
+ en ? en->name : "", chan ? chan->id : -1,
chan ? chan->inst->addr : 0,
- chan ? chan->object.client->name : "unknown",
+ chan ? chan->name : "unknown",
subc, mthd, data);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/ce/priv.h
index cd53b936..c4c0469 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/priv.h
@@ -8,4 +8,8 @@ void gk104_ce_intr(struct nvkm_engine *);
void gp100_ce_intr(struct nvkm_engine *);
extern const struct nvkm_object_func gv100_ce_cclass;
+
+int ga100_ce_oneinit(struct nvkm_engine *);
+int ga100_ce_init(struct nvkm_engine *);
+int ga100_ce_fini(struct nvkm_engine *, bool);
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c b/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c
index be2a718..caca4f6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c
@@ -81,8 +81,7 @@ g84_cipher_intr(struct nvkm_engine *cipher)
{
struct nvkm_subdev *subdev = &cipher->subdev;
struct nvkm_device *device = subdev->device;
- struct nvkm_fifo *fifo = device->fifo;
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
u32 stat = nvkm_rd32(device, 0x102130);
u32 mthd = nvkm_rd32(device, 0x102190);
u32 data = nvkm_rd32(device, 0x102194);
@@ -90,16 +89,16 @@ g84_cipher_intr(struct nvkm_engine *cipher)
unsigned long flags;
char msg[128];
- chan = nvkm_fifo_chan_inst(fifo, (u64)inst << 12, &flags);
+ chan = nvkm_chan_get_inst(cipher, (u64)inst << 12, &flags);
if (stat) {
nvkm_snprintbf(msg, sizeof(msg), g84_cipher_intr_mask, stat);
nvkm_error(subdev, "%08x [%s] ch %d [%010llx %s] "
"mthd %04x data %08x\n", stat, msg,
- chan ? chan->chid : -1, (u64)inst << 12,
- chan ? chan->object.client->name : "unknown",
+ chan ? chan->id : -1, (u64)inst << 12,
+ chan ? chan->name : "unknown",
mthd, data);
}
- nvkm_fifo_chan_put(fifo, flags, &chan);
+ nvkm_chan_put(&chan, flags);
nvkm_wr32(device, 0x102130, stat);
nvkm_wr32(device, 0x10200c, 0x10);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
index d8cf71f..364fea3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
@@ -1095,7 +1095,7 @@ nv98_chipset = {
.volt = { 0x00000001, nv40_volt_new },
.disp = { 0x00000001, g94_disp_new },
.dma = { 0x00000001, nv50_dma_new },
- .fifo = { 0x00000001, g84_fifo_new },
+ .fifo = { 0x00000001, g98_fifo_new },
.gr = { 0x00000001, g84_gr_new },
.mspdec = { 0x00000001, g98_mspdec_new },
.msppp = { 0x00000001, g98_msppp_new },
@@ -1161,7 +1161,7 @@ nva3_chipset = {
.ce = { 0x00000001, gt215_ce_new },
.disp = { 0x00000001, gt215_disp_new },
.dma = { 0x00000001, nv50_dma_new },
- .fifo = { 0x00000001, g84_fifo_new },
+ .fifo = { 0x00000001, g98_fifo_new },
.gr = { 0x00000001, gt215_gr_new },
.mpeg = { 0x00000001, g84_mpeg_new },
.mspdec = { 0x00000001, gt215_mspdec_new },
@@ -1195,7 +1195,7 @@ nva5_chipset = {
.ce = { 0x00000001, gt215_ce_new },
.disp = { 0x00000001, gt215_disp_new },
.dma = { 0x00000001, nv50_dma_new },
- .fifo = { 0x00000001, g84_fifo_new },
+ .fifo = { 0x00000001, g98_fifo_new },
.gr = { 0x00000001, gt215_gr_new },
.mspdec = { 0x00000001, gt215_mspdec_new },
.msppp = { 0x00000001, gt215_msppp_new },
@@ -1228,7 +1228,7 @@ nva8_chipset = {
.ce = { 0x00000001, gt215_ce_new },
.disp = { 0x00000001, gt215_disp_new },
.dma = { 0x00000001, nv50_dma_new },
- .fifo = { 0x00000001, g84_fifo_new },
+ .fifo = { 0x00000001, g98_fifo_new },
.gr = { 0x00000001, gt215_gr_new },
.mspdec = { 0x00000001, gt215_mspdec_new },
.msppp = { 0x00000001, gt215_msppp_new },
@@ -1259,7 +1259,7 @@ nvaa_chipset = {
.volt = { 0x00000001, nv40_volt_new },
.disp = { 0x00000001, mcp77_disp_new },
.dma = { 0x00000001, nv50_dma_new },
- .fifo = { 0x00000001, g84_fifo_new },
+ .fifo = { 0x00000001, g98_fifo_new },
.gr = { 0x00000001, gt200_gr_new },
.mspdec = { 0x00000001, g98_mspdec_new },
.msppp = { 0x00000001, g98_msppp_new },
@@ -1291,7 +1291,7 @@ nvac_chipset = {
.volt = { 0x00000001, nv40_volt_new },
.disp = { 0x00000001, mcp77_disp_new },
.dma = { 0x00000001, nv50_dma_new },
- .fifo = { 0x00000001, g84_fifo_new },
+ .fifo = { 0x00000001, g98_fifo_new },
.gr = { 0x00000001, mcp79_gr_new },
.mspdec = { 0x00000001, g98_mspdec_new },
.msppp = { 0x00000001, g98_msppp_new },
@@ -1325,7 +1325,7 @@ nvaf_chipset = {
.ce = { 0x00000001, gt215_ce_new },
.disp = { 0x00000001, mcp89_disp_new },
.dma = { 0x00000001, nv50_dma_new },
- .fifo = { 0x00000001, g84_fifo_new },
+ .fifo = { 0x00000001, g98_fifo_new },
.gr = { 0x00000001, mcp89_gr_new },
.mspdec = { 0x00000001, gt215_mspdec_new },
.msppp = { 0x00000001, gt215_msppp_new },
@@ -2130,7 +2130,7 @@ nv12b_chipset = {
.volt = { 0x00000001, gm20b_volt_new },
.ce = { 0x00000004, gm200_ce_new },
.dma = { 0x00000001, gf119_dma_new },
- .fifo = { 0x00000001, gm20b_fifo_new },
+ .fifo = { 0x00000001, gm200_fifo_new },
.gr = { 0x00000001, gm20b_gr_new },
.sw = { 0x00000001, gf100_sw_new },
};
@@ -2356,7 +2356,7 @@ nv13b_chipset = {
.top = { 0x00000001, gk104_top_new },
.ce = { 0x00000001, gp100_ce_new },
.dma = { 0x00000001, gf119_dma_new },
- .fifo = { 0x00000001, gp10b_fifo_new },
+ .fifo = { 0x00000001, gp100_fifo_new },
.gr = { 0x00000001, gp10b_gr_new },
.sw = { 0x00000001, gf100_sw_new },
};
@@ -2364,7 +2364,7 @@ nv13b_chipset = {
static const struct nvkm_device_chip
nv140_chipset = {
.name = "GV100",
- .acr = { 0x00000001, gp108_acr_new },
+ .acr = { 0x00000001, gv100_acr_new },
.bar = { 0x00000001, gm107_bar_new },
.bios = { 0x00000001, nvkm_bios_new },
.bus = { 0x00000001, gf100_bus_new },
@@ -2385,6 +2385,7 @@ nv140_chipset = {
.therm = { 0x00000001, gp100_therm_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, gk104_top_new },
+ .vfn = { 0x00000001, gv100_vfn_new },
.ce = { 0x000001ff, gv100_ce_new },
.disp = { 0x00000001, gv100_disp_new },
.dma = { 0x00000001, gv100_dma_new },
@@ -2411,7 +2412,7 @@ nv162_chipset = {
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
.ltc = { 0x00000001, gp102_ltc_new },
- .mc = { 0x00000001, tu102_mc_new },
+ .mc = { 0x00000001, gp100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.pmu = { 0x00000001, gp102_pmu_new },
@@ -2419,6 +2420,7 @@ nv162_chipset = {
.therm = { 0x00000001, gp100_therm_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, gk104_top_new },
+ .vfn = { 0x00000001, tu102_vfn_new },
.ce = { 0x0000001f, tu102_ce_new },
.disp = { 0x00000001, tu102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
@@ -2445,7 +2447,7 @@ nv164_chipset = {
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
.ltc = { 0x00000001, gp102_ltc_new },
- .mc = { 0x00000001, tu102_mc_new },
+ .mc = { 0x00000001, gp100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.pmu = { 0x00000001, gp102_pmu_new },
@@ -2453,6 +2455,7 @@ nv164_chipset = {
.therm = { 0x00000001, gp100_therm_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, gk104_top_new },
+ .vfn = { 0x00000001, tu102_vfn_new },
.ce = { 0x0000001f, tu102_ce_new },
.disp = { 0x00000001, tu102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
@@ -2479,7 +2482,7 @@ nv166_chipset = {
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
.ltc = { 0x00000001, gp102_ltc_new },
- .mc = { 0x00000001, tu102_mc_new },
+ .mc = { 0x00000001, gp100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.pmu = { 0x00000001, gp102_pmu_new },
@@ -2487,6 +2490,7 @@ nv166_chipset = {
.therm = { 0x00000001, gp100_therm_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, gk104_top_new },
+ .vfn = { 0x00000001, tu102_vfn_new },
.ce = { 0x0000001f, tu102_ce_new },
.disp = { 0x00000001, tu102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
@@ -2513,7 +2517,7 @@ nv167_chipset = {
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
.ltc = { 0x00000001, gp102_ltc_new },
- .mc = { 0x00000001, tu102_mc_new },
+ .mc = { 0x00000001, gp100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.pmu = { 0x00000001, gp102_pmu_new },
@@ -2521,6 +2525,7 @@ nv167_chipset = {
.therm = { 0x00000001, gp100_therm_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, gk104_top_new },
+ .vfn = { 0x00000001, tu102_vfn_new },
.ce = { 0x0000001f, tu102_ce_new },
.disp = { 0x00000001, tu102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
@@ -2547,7 +2552,7 @@ nv168_chipset = {
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
.ltc = { 0x00000001, gp102_ltc_new },
- .mc = { 0x00000001, tu102_mc_new },
+ .mc = { 0x00000001, gp100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.pmu = { 0x00000001, gp102_pmu_new },
@@ -2555,6 +2560,7 @@ nv168_chipset = {
.therm = { 0x00000001, gp100_therm_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, gk104_top_new },
+ .vfn = { 0x00000001, tu102_vfn_new },
.ce = { 0x0000001f, tu102_ce_new },
.disp = { 0x00000001, tu102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
@@ -2571,6 +2577,7 @@ nv170_chipset = {
.bar = { 0x00000001, tu102_bar_new },
.bios = { 0x00000001, nvkm_bios_new },
.devinit = { 0x00000001, ga100_devinit_new },
+ .fault = { 0x00000001, tu102_fault_new },
.fb = { 0x00000001, ga100_fb_new },
.gpio = { 0x00000001, gk104_gpio_new },
.i2c = { 0x00000001, gm200_i2c_new },
@@ -2581,111 +2588,159 @@ nv170_chipset = {
.privring = { 0x00000001, gm200_privring_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, ga100_top_new },
+ .vfn = { 0x00000001, ga100_vfn_new },
+ .ce = { 0x000003ff, ga100_ce_new },
+ .fifo = { 0x00000001, ga100_fifo_new },
};
static const struct nvkm_device_chip
nv172_chipset = {
.name = "GA102",
+ .acr = { 0x00000001, ga102_acr_new },
.bar = { 0x00000001, tu102_bar_new },
.bios = { 0x00000001, nvkm_bios_new },
.devinit = { 0x00000001, ga100_devinit_new },
+ .fault = { 0x00000001, tu102_fault_new },
.fb = { 0x00000001, ga102_fb_new },
.gpio = { 0x00000001, ga102_gpio_new },
+ .gsp = { 0x00000001, ga102_gsp_new },
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
+ .ltc = { 0x00000001, ga102_ltc_new },
.mc = { 0x00000001, ga100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.privring = { 0x00000001, gm200_privring_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, ga100_top_new },
+ .vfn = { 0x00000001, ga100_vfn_new },
+ .ce = { 0x0000001f, ga102_ce_new },
.disp = { 0x00000001, ga102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
.fifo = { 0x00000001, ga102_fifo_new },
+ .gr = { 0x00000001, ga102_gr_new },
+ .nvdec = { 0x00000001, ga102_nvdec_new },
+ .sec2 = { 0x00000001, ga102_sec2_new },
};
static const struct nvkm_device_chip
nv173_chipset = {
.name = "GA103",
+ .acr = { 0x00000001, ga102_acr_new },
.bar = { 0x00000001, tu102_bar_new },
.bios = { 0x00000001, nvkm_bios_new },
.devinit = { 0x00000001, ga100_devinit_new },
+ .fault = { 0x00000001, tu102_fault_new },
.fb = { 0x00000001, ga102_fb_new },
.gpio = { 0x00000001, ga102_gpio_new },
+ .gsp = { 0x00000001, ga102_gsp_new },
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
+ .ltc = { 0x00000001, ga102_ltc_new },
.mc = { 0x00000001, ga100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.privring = { 0x00000001, gm200_privring_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, ga100_top_new },
+ .vfn = { 0x00000001, ga100_vfn_new },
+ .ce = { 0x0000001f, ga102_ce_new },
.disp = { 0x00000001, ga102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
.fifo = { 0x00000001, ga102_fifo_new },
+ .gr = { 0x00000001, ga102_gr_new },
+ .nvdec = { 0x00000001, ga102_nvdec_new },
+ .sec2 = { 0x00000001, ga102_sec2_new },
};
static const struct nvkm_device_chip
nv174_chipset = {
.name = "GA104",
+ .acr = { 0x00000001, ga102_acr_new },
.bar = { 0x00000001, tu102_bar_new },
.bios = { 0x00000001, nvkm_bios_new },
.devinit = { 0x00000001, ga100_devinit_new },
+ .fault = { 0x00000001, tu102_fault_new },
.fb = { 0x00000001, ga102_fb_new },
.gpio = { 0x00000001, ga102_gpio_new },
+ .gsp = { 0x00000001, ga102_gsp_new },
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
+ .ltc = { 0x00000001, ga102_ltc_new },
.mc = { 0x00000001, ga100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.privring = { 0x00000001, gm200_privring_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, ga100_top_new },
+ .vfn = { 0x00000001, ga100_vfn_new },
+ .ce = { 0x0000001f, ga102_ce_new },
.disp = { 0x00000001, ga102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
.fifo = { 0x00000001, ga102_fifo_new },
+ .gr = { 0x00000001, ga102_gr_new },
+ .nvdec = { 0x00000001, ga102_nvdec_new },
+ .sec2 = { 0x00000001, ga102_sec2_new },
};
static const struct nvkm_device_chip
nv176_chipset = {
.name = "GA106",
+ .acr = { 0x00000001, ga102_acr_new },
.bar = { 0x00000001, tu102_bar_new },
.bios = { 0x00000001, nvkm_bios_new },
.devinit = { 0x00000001, ga100_devinit_new },
+ .fault = { 0x00000001, tu102_fault_new },
.fb = { 0x00000001, ga102_fb_new },
.gpio = { 0x00000001, ga102_gpio_new },
+ .gsp = { 0x00000001, ga102_gsp_new },
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
+ .ltc = { 0x00000001, ga102_ltc_new },
.mc = { 0x00000001, ga100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.privring = { 0x00000001, gm200_privring_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, ga100_top_new },
+ .vfn = { 0x00000001, ga100_vfn_new },
+ .ce = { 0x0000001f, ga102_ce_new },
.disp = { 0x00000001, ga102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
.fifo = { 0x00000001, ga102_fifo_new },
+ .gr = { 0x00000001, ga102_gr_new },
+ .nvdec = { 0x00000001, ga102_nvdec_new },
+ .sec2 = { 0x00000001, ga102_sec2_new },
};
static const struct nvkm_device_chip
nv177_chipset = {
.name = "GA107",
+ .acr = { 0x00000001, ga102_acr_new },
.bar = { 0x00000001, tu102_bar_new },
.bios = { 0x00000001, nvkm_bios_new },
.devinit = { 0x00000001, ga100_devinit_new },
+ .fault = { 0x00000001, tu102_fault_new },
.fb = { 0x00000001, ga102_fb_new },
.gpio = { 0x00000001, ga102_gpio_new },
+ .gsp = { 0x00000001, ga102_gsp_new },
.i2c = { 0x00000001, gm200_i2c_new },
.imem = { 0x00000001, nv50_instmem_new },
+ .ltc = { 0x00000001, ga102_ltc_new },
.mc = { 0x00000001, ga100_mc_new },
.mmu = { 0x00000001, tu102_mmu_new },
.pci = { 0x00000001, gp100_pci_new },
.privring = { 0x00000001, gm200_privring_new },
.timer = { 0x00000001, gk20a_timer_new },
.top = { 0x00000001, ga100_top_new },
+ .vfn = { 0x00000001, ga100_vfn_new },
+ .ce = { 0x0000001f, ga102_ce_new },
.disp = { 0x00000001, ga102_disp_new },
.dma = { 0x00000001, gv100_dma_new },
.fifo = { 0x00000001, ga102_fifo_new },
+ .gr = { 0x00000001, ga102_gr_new },
+ .nvdec = { 0x00000001, ga102_nvdec_new },
+ .sec2 = { 0x00000001, ga102_sec2_new },
};
struct nvkm_subdev *
@@ -2734,6 +2789,8 @@ nvkm_device_fini(struct nvkm_device *device, bool suspend)
if (device->func->fini)
device->func->fini(device, suspend);
+ nvkm_intr_unarm(device);
+
time = ktime_to_us(ktime_get()) - time;
nvdev_trace(device, "%s completed in %lldus...\n", action, time);
return 0;
@@ -2759,6 +2816,8 @@ nvkm_device_preinit(struct nvkm_device *device)
nvdev_trace(device, "preinit running...\n");
time = ktime_to_us(ktime_get());
+ nvkm_intr_unarm(device);
+
if (device->func->preinit) {
ret = device->func->preinit(device);
if (ret)
@@ -2775,6 +2834,14 @@ nvkm_device_preinit(struct nvkm_device *device)
if (ret)
goto fail;
+ ret = nvkm_top_parse(device);
+ if (ret)
+ goto fail;
+
+ ret = nvkm_fb_mem_unlock(device->fb);
+ if (ret)
+ goto fail;
+
time = ktime_to_us(ktime_get()) - time;
nvdev_trace(device, "preinit completed in %lldus\n", time);
return 0;
@@ -2800,6 +2867,8 @@ nvkm_device_init(struct nvkm_device *device)
nvdev_trace(device, "init running...\n");
time = ktime_to_us(ktime_get());
+ nvkm_intr_rearm(device);
+
if (device->func->init) {
ret = device->func->init(device);
if (ret)
@@ -2837,6 +2906,8 @@ nvkm_device_del(struct nvkm_device **pdevice)
if (device) {
mutex_lock(&nv_devices_mutex);
+ nvkm_intr_dtor(device);
+
list_for_each_entry_safe_reverse(subdev, subtmp, &device->subdev, head)
nvkm_subdev_del(&subdev);
@@ -3144,6 +3215,7 @@ nvkm_device_ctor(const struct nvkm_device_func *func,
device->name = device->chip->name;
mutex_init(&device->mutex);
+ nvkm_intr_ctor(device);
#define NVKM_LAYOUT_ONCE(type,data,ptr) \
if (device->chip->ptr.inst && (subdev_mask & (BIT_ULL(type)))) { \
@@ -3185,7 +3257,7 @@ nvkm_device_ctor(const struct nvkm_device_func *func,
#undef NVKM_LAYOUT_INST
#undef NVKM_LAYOUT_ONCE
- ret = 0;
+ ret = nvkm_intr_install(device);
done:
if (device->pri && (!mmio || ret)) {
iounmap(device->pri);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/pci.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/pci.c
index f302d2b..abccb2b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/pci.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/pci.c
@@ -1574,6 +1574,12 @@ nvkm_device_pci_resource_size(struct nvkm_device *device, unsigned bar)
return pci_resource_len(pdev->pdev, bar);
}
+static int
+nvkm_device_pci_irq(struct nvkm_device *device)
+{
+ return nvkm_device_pci(device)->pdev->irq;
+}
+
static void
nvkm_device_pci_fini(struct nvkm_device *device, bool suspend)
{
@@ -1612,6 +1618,7 @@ nvkm_device_pci_func = {
.dtor = nvkm_device_pci_dtor,
.preinit = nvkm_device_pci_preinit,
.fini = nvkm_device_pci_fini,
+ .irq = nvkm_device_pci_irq,
.resource_addr = nvkm_device_pci_resource_addr,
.resource_size = nvkm_device_pci_resource_size,
.cpu_coherent = !IS_ENABLED(CONFIG_ARM),
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h
index 93949b3..24faaac 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h
@@ -27,6 +27,7 @@
#include <subdev/therm.h>
#include <subdev/timer.h>
#include <subdev/top.h>
+#include <subdev/vfn.h>
#include <subdev/volt.h>
#include <engine/bsp.h>
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/tegra.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/tegra.c
index ac9e122..87caa4a7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/tegra.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/tegra.c
@@ -206,45 +206,12 @@ nvkm_device_tegra_resource_size(struct nvkm_device *device, unsigned bar)
return res ? resource_size(res) : 0;
}
-static irqreturn_t
-nvkm_device_tegra_intr(int irq, void *arg)
-{
- struct nvkm_device_tegra *tdev = arg;
- struct nvkm_device *device = &tdev->device;
- bool handled = false;
- nvkm_mc_intr_unarm(device);
- nvkm_mc_intr(device, &handled);
- nvkm_mc_intr_rearm(device);
- return handled ? IRQ_HANDLED : IRQ_NONE;
-}
-
-static void
-nvkm_device_tegra_fini(struct nvkm_device *device, bool suspend)
-{
- struct nvkm_device_tegra *tdev = nvkm_device_tegra(device);
- if (tdev->irq) {
- free_irq(tdev->irq, tdev);
- tdev->irq = 0;
- }
-}
-
static int
-nvkm_device_tegra_init(struct nvkm_device *device)
+nvkm_device_tegra_irq(struct nvkm_device *device)
{
struct nvkm_device_tegra *tdev = nvkm_device_tegra(device);
- int irq, ret;
- irq = platform_get_irq_byname(tdev->pdev, "stall");
- if (irq < 0)
- return irq;
-
- ret = request_irq(irq, nvkm_device_tegra_intr,
- IRQF_SHARED, "nvkm", tdev);
- if (ret)
- return ret;
-
- tdev->irq = irq;
- return 0;
+ return platform_get_irq_byname(tdev->pdev, "stall");
}
static void *
@@ -260,8 +227,7 @@ static const struct nvkm_device_func
nvkm_device_tegra_func = {
.tegra = nvkm_device_tegra,
.dtor = nvkm_device_tegra_dtor,
- .init = nvkm_device_tegra_init,
- .fini = nvkm_device_tegra_fini,
+ .irq = nvkm_device_tegra_irq,
.resource_addr = nvkm_device_tegra_resource_addr,
.resource_size = nvkm_device_tegra_resource_size,
.cpu_coherent = false,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c
index 45f509c..9b39ec3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c
@@ -342,6 +342,8 @@ nvkm_udevice_child_get(struct nvkm_object *object, int index,
sclass = &device->mmu->user;
else if (device->fault && index-- == 0)
sclass = &device->fault->user;
+ else if (device->vfn && index-- == 0)
+ sclass = &device->vfn->user;
else
return -EINVAL;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild b/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild
index 600072a..e1aecd3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild
@@ -28,9 +28,7 @@
nvkm-y += nvkm/engine/disp/tu102.o
nvkm-y += nvkm/engine/disp/ga102.o
-nvkm-y += nvkm/engine/disp/rootnv04.o
-nvkm-y += nvkm/engine/disp/rootnv50.o
-
nvkm-y += nvkm/engine/disp/udisp.o
nvkm-y += nvkm/engine/disp/uconn.o
nvkm-y += nvkm/engine/disp/uoutp.o
+nvkm-y += nvkm/engine/disp/uhead.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c
index 65c99d9..73104b59 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c
@@ -29,7 +29,6 @@
#include "outp.h"
#include <core/client.h>
-#include <core/notify.h>
#include <core/ramht.h>
#include <subdev/bios.h>
#include <subdev/bios/dcb.h>
@@ -57,32 +56,8 @@ nvkm_disp_vblank_init(struct nvkm_event *event, int type, int id)
head->func->vblank_get(head);
}
-static int
-nvkm_disp_vblank_ctor(struct nvkm_object *object, void *data, u32 size,
- struct nvkm_notify *notify)
-{
- struct nvkm_disp *disp =
- container_of(notify->event, typeof(*disp), vblank);
- union {
- struct nvif_notify_head_req_v0 v0;
- } *req = data;
- int ret = -ENOSYS;
-
- if (!(ret = nvif_unpack(ret, &data, &size, req->v0, 0, 0, false))) {
- notify->size = sizeof(struct nvif_notify_head_rep_v0);
- if (ret = -ENXIO, req->v0.head <= disp->vblank.index_nr) {
- notify->types = 1;
- notify->index = req->v0.head;
- return 0;
- }
- }
-
- return ret;
-}
-
static const struct nvkm_event_func
nvkm_disp_vblank_func = {
- .ctor = nvkm_disp_vblank_ctor,
.init = nvkm_disp_vblank_init,
.fini = nvkm_disp_vblank_fini,
};
@@ -90,59 +65,7 @@ nvkm_disp_vblank_func = {
void
nvkm_disp_vblank(struct nvkm_disp *disp, int head)
{
- struct nvif_notify_head_rep_v0 rep = {};
- nvkm_event_send(&disp->vblank, 1, head, &rep, sizeof(rep));
-}
-
-static int
-nvkm_disp_hpd_ctor(struct nvkm_object *object, void *data, u32 size,
- struct nvkm_notify *notify)
-{
- struct nvkm_disp *disp =
- container_of(notify->event, typeof(*disp), hpd);
- union {
- struct nvif_notify_conn_req_v0 v0;
- } *req = data;
- struct nvkm_outp *outp;
- int ret = -ENOSYS;
-
- if (!(ret = nvif_unpack(ret, &data, &size, req->v0, 0, 0, false))) {
- notify->size = sizeof(struct nvif_notify_conn_rep_v0);
- list_for_each_entry(outp, &disp->outps, head) {
- if (ret = -ENXIO, outp->conn->index == req->v0.conn) {
- if (ret = -ENODEV, outp->conn->hpd.event) {
- notify->types = req->v0.mask;
- notify->index = req->v0.conn;
- ret = 0;
- }
- break;
- }
- }
- }
-
- return ret;
-}
-
-static const struct nvkm_event_func
-nvkm_disp_hpd_func = {
- .ctor = nvkm_disp_hpd_ctor
-};
-
-int
-nvkm_disp_ntfy(struct nvkm_object *object, u32 type, struct nvkm_event **event)
-{
- struct nvkm_disp *disp = nvkm_disp(object->engine);
- switch (type) {
- case NV04_DISP_NTFY_VBLANK:
- *event = &disp->vblank;
- return 0;
- case NV04_DISP_NTFY_CONN:
- *event = &disp->hpd;
- return 0;
- default:
- break;
- }
- return -EINVAL;
+ nvkm_event_ntfy(&disp->vblank, head, NVKM_DISP_HEAD_EVENT_VBLANK);
}
static int
@@ -343,9 +266,7 @@ nvkm_disp_oneinit(struct nvkm_engine *engine)
/* Apparently we need to create a new one! */
ret = nvkm_conn_new(disp, i, &connE, &outp->conn);
if (ret) {
- nvkm_error(&disp->engine.subdev,
- "failed to create outp %d conn: %d\n",
- outp->index, ret);
+ nvkm_error(subdev, "failed to create outp %d conn: %d\n", outp->index, ret);
nvkm_conn_del(&outp->conn);
list_del(&outp->head);
nvkm_outp_del(&outp);
@@ -355,10 +276,6 @@ nvkm_disp_oneinit(struct nvkm_engine *engine)
list_add_tail(&outp->conn->head, &disp->conns);
}
- ret = nvkm_event_init(&nvkm_disp_hpd_func, 3, hpd, &disp->hpd);
- if (ret)
- return ret;
-
if (disp->func->oneinit) {
ret = disp->func->oneinit(disp);
if (ret)
@@ -382,7 +299,7 @@ nvkm_disp_oneinit(struct nvkm_engine *engine)
list_for_each_entry(head, &disp->heads, head)
i = max(i, head->id + 1);
- return nvkm_event_init(&nvkm_disp_vblank_func, 1, i, &disp->vblank);
+ return nvkm_event_init(&nvkm_disp_vblank_func, subdev, 1, i, &disp->vblank);
}
static void *
@@ -406,7 +323,6 @@ nvkm_disp_dtor(struct nvkm_engine *engine)
}
nvkm_event_fini(&disp->vblank);
- nvkm_event_fini(&disp->hpd);
while (!list_empty(&disp->conns)) {
conn = list_first_entry(&disp->conns, typeof(*conn), head);
@@ -473,5 +389,6 @@ nvkm_disp_new_(const struct nvkm_disp_func *func, struct nvkm_device *device,
mutex_init(&disp->super.mutex);
}
- return nvkm_event_init(func->uevent, 1, ARRAY_SIZE(disp->chan), &disp->uevent);
+ return nvkm_event_init(func->uevent, &disp->engine.subdev, 1, ARRAY_SIZE(disp->chan),
+ &disp->uevent);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.c
index 7ed1180..fbdae11 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.c
@@ -29,38 +29,14 @@
#include <nvif/event.h>
-static int
-nvkm_conn_hpd(struct nvkm_notify *notify)
-{
- struct nvkm_conn *conn = container_of(notify, typeof(*conn), hpd);
- struct nvkm_disp *disp = conn->disp;
- struct nvkm_gpio *gpio = disp->engine.subdev.device->gpio;
- const struct nvkm_gpio_ntfy_rep *line = notify->data;
- struct nvif_notify_conn_rep_v0 rep;
- int index = conn->index;
-
- CONN_DBG(conn, "HPD: %d", line->mask);
-
- if (!nvkm_gpio_get(gpio, 0, DCB_GPIO_UNUSED, conn->hpd.index))
- rep.mask = NVIF_NOTIFY_CONN_V0_UNPLUG;
- else
- rep.mask = NVIF_NOTIFY_CONN_V0_PLUG;
- rep.version = 0;
-
- nvkm_event_send(&disp->hpd, rep.mask, index, &rep, sizeof(rep));
- return NVKM_NOTIFY_KEEP;
-}
-
void
nvkm_conn_fini(struct nvkm_conn *conn)
{
- nvkm_notify_put(&conn->hpd);
}
void
nvkm_conn_init(struct nvkm_conn *conn)
{
- nvkm_notify_get(&conn->hpd);
}
void
@@ -68,7 +44,6 @@ nvkm_conn_del(struct nvkm_conn **pconn)
{
struct nvkm_conn *conn = *pconn;
if (conn) {
- nvkm_notify_fini(&conn->hpd);
kfree(*pconn);
*pconn = NULL;
}
@@ -106,20 +81,6 @@ nvkm_conn_ctor(struct nvkm_disp *disp, int index, struct nvbios_connE *info,
}
conn->info.hpd = func.line;
-
- ret = nvkm_notify_init(NULL, &gpio->event, nvkm_conn_hpd,
- true, &(struct nvkm_gpio_ntfy_req) {
- .mask = NVKM_GPIO_TOGGLED,
- .line = func.line,
- },
- sizeof(struct nvkm_gpio_ntfy_req),
- sizeof(struct nvkm_gpio_ntfy_rep),
- &conn->hpd);
- if (ret) {
- CONN_ERR(conn, "func %02x failed, %d", info->hpd, ret);
- } else {
- CONN_DBG(conn, "func %02x (HPD)", info->hpd);
- }
}
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.h
index f109634..a0600e7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.h
@@ -3,7 +3,6 @@
#define __NVKM_DISP_CONN_H__
#include "priv.h"
-#include <core/notify.h>
#include <subdev/bios.h>
#include <subdev/bios/conn.h>
@@ -12,8 +11,6 @@ struct nvkm_conn {
int index;
struct nvbios_connE info;
- struct nvkm_notify hpd;
-
struct list_head head;
struct nvkm_object object;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c
index c1b3206..40c8ea4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c
@@ -274,70 +274,17 @@ nvkm_dp_train_cr(struct lt_state *lt)
}
static int
-nvkm_dp_train_links(struct nvkm_outp *outp, int rate)
+nvkm_dp_train_link(struct nvkm_outp *outp, int rate)
{
struct nvkm_ior *ior = outp->ior;
- struct nvkm_disp *disp = outp->disp;
- struct nvkm_subdev *subdev = &disp->engine.subdev;
- struct nvkm_bios *bios = subdev->device->bios;
struct lt_state lt = {
.outp = outp,
+ .pc2 = outp->dp.dpcd[DPCD_RC02] & DPCD_RC02_TPS3_SUPPORTED,
};
- u32 lnkcmp;
u8 sink[2], data;
int ret;
- OUTP_DBG(outp, "training %d x %d MB/s", ior->dp.nr, ior->dp.bw * 27);
-
- /* Intersect misc. capabilities of the OR and sink. */
- if (disp->engine.subdev.device->chipset < 0x110)
- outp->dp.dpcd[DPCD_RC03] &= ~DPCD_RC03_TPS4_SUPPORTED;
- if (disp->engine.subdev.device->chipset < 0xd0)
- outp->dp.dpcd[DPCD_RC02] &= ~DPCD_RC02_TPS3_SUPPORTED;
- lt.pc2 = outp->dp.dpcd[DPCD_RC02] & DPCD_RC02_TPS3_SUPPORTED;
-
- if (AMPERE_IED_HACK(disp) && (lnkcmp = lt.outp->dp.info.script[0])) {
- /* Execute BeforeLinkTraining script from DP Info table. */
- while (ior->dp.bw < nvbios_rd08(bios, lnkcmp))
- lnkcmp += 3;
- lnkcmp = nvbios_rd16(bios, lnkcmp + 1);
-
- nvbios_init(&outp->disp->engine.subdev, lnkcmp,
- init.outp = &outp->info;
- init.or = ior->id;
- init.link = ior->asy.link;
- );
- }
-
- /* Set desired link configuration on the source. */
- if ((lnkcmp = lt.outp->dp.info.lnkcmp)) {
- if (outp->dp.version < 0x30) {
- while ((ior->dp.bw * 2700) < nvbios_rd16(bios, lnkcmp))
- lnkcmp += 4;
- lnkcmp = nvbios_rd16(bios, lnkcmp + 2);
- } else {
- while (ior->dp.bw < nvbios_rd08(bios, lnkcmp))
- lnkcmp += 3;
- lnkcmp = nvbios_rd16(bios, lnkcmp + 1);
- }
-
- nvbios_init(subdev, lnkcmp,
- init.outp = &outp->info;
- init.or = ior->id;
- init.link = ior->asy.link;
- );
- }
-
- ret = ior->func->dp->links(ior, outp->dp.aux);
- if (ret) {
- if (ret < 0) {
- OUTP_ERR(outp, "train failed with %d", ret);
- return ret;
- }
- return 0;
- }
-
- ior->func->dp->power(ior, ior->dp.nr);
+ OUTP_DBG(outp, "training %dx%02x", ior->dp.nr, ior->dp.bw);
/* Select LTTPR non-transparent mode if we have a valid configuration,
* use transparent mode otherwise.
@@ -393,6 +340,71 @@ nvkm_dp_train_links(struct nvkm_outp *outp, int rate)
return ret;
}
+static int
+nvkm_dp_train_links(struct nvkm_outp *outp, int rate)
+{
+ struct nvkm_ior *ior = outp->ior;
+ struct nvkm_disp *disp = outp->disp;
+ struct nvkm_subdev *subdev = &disp->engine.subdev;
+ struct nvkm_bios *bios = subdev->device->bios;
+ u32 lnkcmp;
+ int ret;
+
+ OUTP_DBG(outp, "programming link for %dx%02x", ior->dp.nr, ior->dp.bw);
+
+ /* Intersect misc. capabilities of the OR and sink. */
+ if (disp->engine.subdev.device->chipset < 0x110)
+ outp->dp.dpcd[DPCD_RC03] &= ~DPCD_RC03_TPS4_SUPPORTED;
+ if (disp->engine.subdev.device->chipset < 0xd0)
+ outp->dp.dpcd[DPCD_RC02] &= ~DPCD_RC02_TPS3_SUPPORTED;
+
+ if (AMPERE_IED_HACK(disp) && (lnkcmp = outp->dp.info.script[0])) {
+ /* Execute BeforeLinkTraining script from DP Info table. */
+ while (ior->dp.bw < nvbios_rd08(bios, lnkcmp))
+ lnkcmp += 3;
+ lnkcmp = nvbios_rd16(bios, lnkcmp + 1);
+
+ nvbios_init(&outp->disp->engine.subdev, lnkcmp,
+ init.outp = &outp->info;
+ init.or = ior->id;
+ init.link = ior->asy.link;
+ );
+ }
+
+ /* Set desired link configuration on the source. */
+ if ((lnkcmp = outp->dp.info.lnkcmp)) {
+ if (outp->dp.version < 0x30) {
+ while ((ior->dp.bw * 2700) < nvbios_rd16(bios, lnkcmp))
+ lnkcmp += 4;
+ lnkcmp = nvbios_rd16(bios, lnkcmp + 2);
+ } else {
+ while (ior->dp.bw < nvbios_rd08(bios, lnkcmp))
+ lnkcmp += 3;
+ lnkcmp = nvbios_rd16(bios, lnkcmp + 1);
+ }
+
+ nvbios_init(subdev, lnkcmp,
+ init.outp = &outp->info;
+ init.or = ior->id;
+ init.link = ior->asy.link;
+ );
+ }
+
+ ret = ior->func->dp->links(ior, outp->dp.aux);
+ if (ret) {
+ if (ret < 0) {
+ OUTP_ERR(outp, "train failed with %d", ret);
+ return ret;
+ }
+ return 0;
+ }
+
+ ior->func->dp->power(ior, ior->dp.nr);
+
+ /* Attempt to train the link in this configuration. */
+ return nvkm_dp_train_link(outp, rate);
+}
+
static void
nvkm_dp_train_fini(struct nvkm_outp *outp)
{
@@ -439,6 +451,16 @@ nvkm_dp_train(struct nvkm_outp *outp, u32 dataKBps)
int ret = -EINVAL, nr, rate;
u8 pwr;
+ /* Retraining link? Skip source configuration, it can mess up the active modeset. */
+ if (atomic_read(&outp->dp.lt.done)) {
+ for (rate = 0; rate < outp->dp.rates; rate++) {
+ if (outp->dp.rate[rate].rate == ior->dp.bw * 27000)
+ return nvkm_dp_train_link(outp, ret);
+ }
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
/* Ensure sink is not in a low-power state. */
if (!nvkm_rdaux(outp->dp.aux, DPCD_SC00, &pwr, 1)) {
if ((pwr & DPCD_SC00_SET_POWER) != DPCD_SC00_SET_POWER_D0) {
@@ -455,6 +477,21 @@ nvkm_dp_train(struct nvkm_outp *outp, u32 dataKBps)
/* Link training. */
OUTP_DBG(outp, "training");
nvkm_dp_train_init(outp);
+
+ /* Validate and train at configuration requested (if any) on ACQUIRE. */
+ if (outp->dp.lt.nr) {
+ for (nr = outp->dp.links; ret < 0 && nr; nr >>= 1) {
+ for (rate = 0; nr == outp->dp.lt.nr && rate < outp->dp.rates; rate++) {
+ if (outp->dp.rate[rate].rate / 27000 == outp->dp.lt.bw) {
+ ior->dp.bw = outp->dp.rate[rate].rate / 27000;
+ ior->dp.nr = nr;
+ ret = nvkm_dp_train_links(outp, rate);
+ }
+ }
+ }
+ }
+
+ /* Otherwise, loop through all valid link configurations that support the data rate. */
for (nr = outp->dp.links; ret < 0 && nr; nr >>= 1) {
for (rate = 0; ret < 0 && rate < outp->dp.rates; rate++) {
if (outp->dp.rate[rate].rate * nr >= dataKBps || WARN_ON(!ior->dp.nr)) {
@@ -465,6 +502,8 @@ nvkm_dp_train(struct nvkm_outp *outp, u32 dataKBps)
}
}
}
+
+ /* Finish up. */
nvkm_dp_train_fini(outp);
if (ret < 0)
OUTP_ERR(outp, "training failed");
@@ -595,18 +634,38 @@ nvkm_dp_enable_supported_link_rates(struct nvkm_outp *outp)
return outp->dp.rates != 0;
}
-static bool
-nvkm_dp_enable(struct nvkm_outp *outp, bool enable)
+void
+nvkm_dp_enable(struct nvkm_outp *outp, bool auxpwr)
{
+ struct nvkm_gpio *gpio = outp->disp->engine.subdev.device->gpio;
struct nvkm_i2c_aux *aux = outp->dp.aux;
- if (enable) {
- if (!outp->dp.present) {
- OUTP_DBG(outp, "aux power -> always");
- nvkm_i2c_aux_monitor(aux, true);
- outp->dp.present = true;
+ if (auxpwr && !outp->dp.aux_pwr) {
+ /* eDP panels need powering on by us (if the VBIOS doesn't default it
+ * to on) before doing any AUX channel transactions. LVDS panel power
+ * is handled by the SOR itself, and not required for LVDS DDC.
+ */
+ if (outp->conn->info.type == DCB_CONNECTOR_eDP) {
+ int power = nvkm_gpio_get(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff);
+ if (power == 0) {
+ nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, 1);
+ outp->dp.aux_pwr_pu = true;
+ }
+
+ /* We delay here unconditionally, even if already powered,
+ * because some laptop panels having a significant resume
+ * delay before the panel begins responding.
+ *
+ * This is likely a bit of a hack, but no better idea for
+ * handling this at the moment.
+ */
+ msleep(300);
}
+ OUTP_DBG(outp, "aux power -> always");
+ nvkm_i2c_aux_monitor(aux, true);
+ outp->dp.aux_pwr = true;
+
/* Detect any LTTPRs before reading DPCD receiver caps. */
if (!nvkm_rdaux(aux, DPCD_LTTPR_REV, outp->dp.lttpr, sizeof(outp->dp.lttpr)) &&
outp->dp.lttpr[0] >= 0x14 && outp->dp.lttpr[2]) {
@@ -659,96 +718,41 @@ nvkm_dp_enable(struct nvkm_outp *outp, bool enable)
outp->dp.rates++;
}
}
-
- return true;
}
- }
-
- if (outp->dp.present) {
+ } else
+ if (!auxpwr && outp->dp.aux_pwr) {
OUTP_DBG(outp, "aux power -> demand");
nvkm_i2c_aux_monitor(aux, false);
- outp->dp.present = false;
+ outp->dp.aux_pwr = false;
+ atomic_set(&outp->dp.lt.done, 0);
+
+ /* Restore eDP panel GPIO to its prior state if we changed it, as
+ * it could potentially interfere with other outputs.
+ */
+ if (outp->conn->info.type == DCB_CONNECTOR_eDP) {
+ if (outp->dp.aux_pwr_pu) {
+ nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, 0);
+ outp->dp.aux_pwr_pu = false;
+ }
+ }
}
-
- atomic_set(&outp->dp.lt.done, 0);
- return false;
-}
-
-static int
-nvkm_dp_hpd(struct nvkm_notify *notify)
-{
- const struct nvkm_i2c_ntfy_rep *line = notify->data;
- struct nvkm_outp *outp = container_of(notify, typeof(*outp), dp.hpd);
- struct nvkm_conn *conn = outp->conn;
- struct nvkm_disp *disp = outp->disp;
- struct nvif_notify_conn_rep_v0 rep = {};
-
- OUTP_DBG(outp, "HPD: %d", line->mask);
- if (line->mask & NVKM_I2C_IRQ) {
- if (atomic_read(&outp->dp.lt.done))
- outp->func->acquire(outp);
- rep.mask |= NVIF_NOTIFY_CONN_V0_IRQ;
- } else {
- nvkm_dp_enable(outp, true);
- }
-
- if (line->mask & NVKM_I2C_UNPLUG)
- rep.mask |= NVIF_NOTIFY_CONN_V0_UNPLUG;
- if (line->mask & NVKM_I2C_PLUG)
- rep.mask |= NVIF_NOTIFY_CONN_V0_PLUG;
-
- nvkm_event_send(&disp->hpd, rep.mask, conn->index, &rep, sizeof(rep));
- return NVKM_NOTIFY_KEEP;
}
static void
nvkm_dp_fini(struct nvkm_outp *outp)
{
- nvkm_notify_put(&outp->dp.hpd);
nvkm_dp_enable(outp, false);
}
static void
nvkm_dp_init(struct nvkm_outp *outp)
{
- struct nvkm_gpio *gpio = outp->disp->engine.subdev.device->gpio;
-
- nvkm_notify_put(&outp->conn->hpd);
-
- /* eDP panels need powering on by us (if the VBIOS doesn't default it
- * to on) before doing any AUX channel transactions. LVDS panel power
- * is handled by the SOR itself, and not required for LVDS DDC.
- */
- if (outp->conn->info.type == DCB_CONNECTOR_eDP) {
- int power = nvkm_gpio_get(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff);
- if (power == 0)
- nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, 1);
-
- /* We delay here unconditionally, even if already powered,
- * because some laptop panels having a significant resume
- * delay before the panel begins responding.
- *
- * This is likely a bit of a hack, but no better idea for
- * handling this at the moment.
- */
- msleep(300);
-
- /* If the eDP panel can't be detected, we need to restore
- * the panel power GPIO to avoid breaking another output.
- */
- if (!nvkm_dp_enable(outp, true) && power == 0)
- nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, 0);
- } else {
- nvkm_dp_enable(outp, true);
- }
-
- nvkm_notify_get(&outp->dp.hpd);
+ nvkm_dp_enable(outp, outp->dp.enabled);
}
static void *
nvkm_dp_dtor(struct nvkm_outp *outp)
{
- nvkm_notify_fini(&outp->dp.hpd);
return outp;
}
@@ -797,21 +801,6 @@ nvkm_dp_new(struct nvkm_disp *disp, int index, struct dcb_output *dcbE, struct n
OUTP_DBG(outp, "bios dp %02x %02x %02x %02x", outp->dp.version, hdr, cnt, len);
- /* hotplug detect, replaces gpio-based mechanism with aux events */
- ret = nvkm_notify_init(NULL, &i2c->event, nvkm_dp_hpd, true,
- &(struct nvkm_i2c_ntfy_req) {
- .mask = NVKM_I2C_PLUG | NVKM_I2C_UNPLUG |
- NVKM_I2C_IRQ,
- .port = outp->dp.aux->id,
- },
- sizeof(struct nvkm_i2c_ntfy_req),
- sizeof(struct nvkm_i2c_ntfy_rep),
- &outp->dp.hpd);
- if (ret) {
- OUTP_ERR(outp, "error monitoring aux hpd: %d", ret);
- return ret;
- }
-
mutex_init(&outp->dp.mutex);
atomic_set(&outp->dp.lt.done, 0);
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.h
index 1d86baa..9a6be43 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.h
@@ -6,6 +6,7 @@
int nvkm_dp_new(struct nvkm_disp *, int index, struct dcb_output *,
struct nvkm_outp **);
void nvkm_dp_disable(struct nvkm_outp *, struct nvkm_ior *);
+void nvkm_dp_enable(struct nvkm_outp *, bool auxpwr);
/* DPCD Receiver Capabilities */
#define DPCD_RC00_DPCD_REV 0x00000
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/g84.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/g84.c
index 4966a51..23ae451 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/g84.c
@@ -29,9 +29,54 @@
#include <nvif/class.h>
-void
-g84_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
- u8 rekey, u8 *avi, u8 avi_size, u8 *vendor, u8 vendor_size)
+static void
+g84_sor_hdmi_infoframe_vsi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe vsi;
+ const u32 hoff = head * 0x800;
+
+ nvkm_mask(device, 0x61653c + hoff, 0x00010001, 0x00010000);
+ if (!size)
+ return;
+
+ pack_hdmi_infoframe(&vsi, data, size);
+
+ nvkm_wr32(device, 0x616544 + hoff, vsi.header);
+ nvkm_wr32(device, 0x616548 + hoff, vsi.subpack0_low);
+ nvkm_wr32(device, 0x61654c + hoff, vsi.subpack0_high);
+ /* Is there a second (or up to fourth?) set of subpack registers here? */
+ /* nvkm_wr32(device, 0x616550 + hoff, vsi.subpack1_low); */
+ /* nvkm_wr32(device, 0x616554 + hoff, vsi.subpack1_high); */
+
+ nvkm_mask(device, 0x61653c + hoff, 0x00010001, 0x00010001);
+}
+
+static void
+g84_sor_hdmi_infoframe_avi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe avi;
+ const u32 hoff = head * 0x800;
+
+ pack_hdmi_infoframe(&avi, data, size);
+
+ nvkm_mask(device, 0x616520 + hoff, 0x00000001, 0x00000000);
+ if (!size)
+ return;
+
+ nvkm_wr32(device, 0x616528 + hoff, avi.header);
+ nvkm_wr32(device, 0x61652c + hoff, avi.subpack0_low);
+ nvkm_wr32(device, 0x616530 + hoff, avi.subpack0_high);
+ nvkm_wr32(device, 0x616534 + hoff, avi.subpack1_low);
+ nvkm_wr32(device, 0x616538 + hoff, avi.subpack1_high);
+
+ nvkm_mask(device, 0x616520 + hoff, 0x00000001, 0x00000001);
+}
+
+
+static void
+g84_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet, u8 rekey)
{
struct nvkm_device *device = ior->disp->engine.subdev.device;
const u32 ctrl = 0x40000000 * enable |
@@ -39,31 +84,13 @@ g84_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
max_ac_packet << 16 |
rekey;
const u32 hoff = head * 0x800;
- struct packed_hdmi_infoframe avi_infoframe;
- struct packed_hdmi_infoframe vendor_infoframe;
-
- pack_hdmi_infoframe(&avi_infoframe, avi, avi_size);
- pack_hdmi_infoframe(&vendor_infoframe, vendor, vendor_size);
if (!(ctrl & 0x40000000)) {
nvkm_mask(device, 0x6165a4 + hoff, 0x40000000, 0x00000000);
- nvkm_mask(device, 0x61653c + hoff, 0x00000001, 0x00000000);
- nvkm_mask(device, 0x616520 + hoff, 0x00000001, 0x00000000);
nvkm_mask(device, 0x616500 + hoff, 0x00000001, 0x00000000);
return;
}
- /* AVI InfoFrame */
- nvkm_mask(device, 0x616520 + hoff, 0x00000001, 0x00000000);
- if (avi_size) {
- nvkm_wr32(device, 0x616528 + hoff, avi_infoframe.header);
- nvkm_wr32(device, 0x61652c + hoff, avi_infoframe.subpack0_low);
- nvkm_wr32(device, 0x616530 + hoff, avi_infoframe.subpack0_high);
- nvkm_wr32(device, 0x616534 + hoff, avi_infoframe.subpack1_low);
- nvkm_wr32(device, 0x616538 + hoff, avi_infoframe.subpack1_high);
- nvkm_mask(device, 0x616520 + hoff, 0x00000001, 0x00000001);
- }
-
/* Audio InfoFrame */
nvkm_mask(device, 0x616500 + hoff, 0x00000001, 0x00000000);
nvkm_wr32(device, 0x616508 + hoff, 0x000a0184);
@@ -71,17 +98,6 @@ g84_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
nvkm_wr32(device, 0x616510 + hoff, 0x00000000);
nvkm_mask(device, 0x616500 + hoff, 0x00000001, 0x00000001);
- /* Vendor InfoFrame */
- nvkm_mask(device, 0x61653c + hoff, 0x00010001, 0x00010000);
- if (vendor_size) {
- nvkm_wr32(device, 0x616544 + hoff, vendor_infoframe.header);
- nvkm_wr32(device, 0x616548 + hoff, vendor_infoframe.subpack0_low);
- nvkm_wr32(device, 0x61654c + hoff, vendor_infoframe.subpack0_high);
- /* Is there a second (or up to fourth?) set of subpack registers here? */
- /* nvkm_wr32(device, 0x616550 + hoff, vendor_infoframe->subpack1_low); */
- /* nvkm_wr32(device, 0x616554 + hoff, vendor_infoframe->subpack1_high); */
- nvkm_mask(device, 0x61653c + hoff, 0x00010001, 0x00010001);
- }
nvkm_mask(device, 0x6165d0 + hoff, 0x00070001, 0x00010001); /* SPARE, HW_CTS */
nvkm_mask(device, 0x616568 + hoff, 0x00010101, 0x00000000); /* ACR_CTRL, ?? */
@@ -96,14 +112,19 @@ g84_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
nvkm_mask(device, 0x6165a4 + hoff, 0x5f1f007f, ctrl);
}
+const struct nvkm_ior_func_hdmi
+g84_sor_hdmi = {
+ .ctrl = g84_sor_hdmi_ctrl,
+ .infoframe_avi = g84_sor_hdmi_infoframe_avi,
+ .infoframe_vsi = g84_sor_hdmi_infoframe_vsi,
+};
+
static const struct nvkm_ior_func
g84_sor = {
.state = nv50_sor_state,
.power = nv50_sor_power,
.clock = nv50_sor_clock,
- .hdmi = {
- .ctrl = g84_sor_hdmi_ctrl,
- },
+ .hdmi = &g84_sor_hdmi,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/ga102.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/ga102.c
index 7489d0d..52099b7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/ga102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/ga102.c
@@ -105,10 +105,7 @@ ga102_sor = {
.state = gv100_sor_state,
.power = nv50_sor_power,
.clock = ga102_sor_clock,
- .hdmi = {
- .ctrl = gv100_sor_hdmi_ctrl,
- .scdc = gm200_sor_hdmi_scdc,
- },
+ .hdmi = &gv100_sor_hdmi,
.dp = &ga102_sor_dp,
.hda = &gv100_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gf119.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gf119.c
index 39822f1..a48e9bd 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gf119.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gf119.c
@@ -202,19 +202,61 @@ gf119_sor_dp = {
};
static void
-gf119_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
- u8 rekey, u8 *avi, u8 avi_size, u8 *vendor, u8 vendor_size)
+gf119_sor_hdmi_infoframe_vsi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe vsi;
+ const u32 hoff = head * 0x800;
+
+ pack_hdmi_infoframe(&vsi, data, size);
+
+ nvkm_mask(device, 0x616730 + hoff, 0x00010001, 0x00010000);
+ if (!size)
+ return;
+
+ /*
+ * These appear to be the audio infoframe registers,
+ * but no other set of infoframe registers has yet
+ * been found.
+ */
+ nvkm_wr32(device, 0x616738 + hoff, vsi.header);
+ nvkm_wr32(device, 0x61673c + hoff, vsi.subpack0_low);
+ nvkm_wr32(device, 0x616740 + hoff, vsi.subpack0_high);
+ /* Is there a second (or further?) set of subpack registers here? */
+
+ nvkm_mask(device, 0x616730 + hoff, 0x00000001, 0x00000001);
+}
+
+static void
+gf119_sor_hdmi_infoframe_avi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe avi;
+ const u32 hoff = head * 0x800;
+
+ pack_hdmi_infoframe(&avi, data, size);
+
+ nvkm_mask(device, 0x616714 + hoff, 0x00000001, 0x00000000);
+ if (!size)
+ return;
+
+ nvkm_wr32(device, 0x61671c + hoff, avi.header);
+ nvkm_wr32(device, 0x616720 + hoff, avi.subpack0_low);
+ nvkm_wr32(device, 0x616724 + hoff, avi.subpack0_high);
+ nvkm_wr32(device, 0x616728 + hoff, avi.subpack1_low);
+ nvkm_wr32(device, 0x61672c + hoff, avi.subpack1_high);
+
+ nvkm_mask(device, 0x616714 + hoff, 0x00000001, 0x00000001);
+}
+
+static void
+gf119_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet, u8 rekey)
{
struct nvkm_device *device = ior->disp->engine.subdev.device;
const u32 ctrl = 0x40000000 * enable |
max_ac_packet << 16 |
rekey;
const u32 hoff = head * 0x800;
- struct packed_hdmi_infoframe avi_infoframe;
- struct packed_hdmi_infoframe vendor_infoframe;
-
- pack_hdmi_infoframe(&avi_infoframe, avi, avi_size);
- pack_hdmi_infoframe(&vendor_infoframe, vendor, vendor_size);
if (!(ctrl & 0x40000000)) {
nvkm_mask(device, 0x616798 + hoff, 0x40000000, 0x00000000);
@@ -224,32 +266,6 @@ gf119_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
return;
}
- /* AVI InfoFrame */
- nvkm_mask(device, 0x616714 + hoff, 0x00000001, 0x00000000);
- if (avi_size) {
- nvkm_wr32(device, 0x61671c + hoff, avi_infoframe.header);
- nvkm_wr32(device, 0x616720 + hoff, avi_infoframe.subpack0_low);
- nvkm_wr32(device, 0x616724 + hoff, avi_infoframe.subpack0_high);
- nvkm_wr32(device, 0x616728 + hoff, avi_infoframe.subpack1_low);
- nvkm_wr32(device, 0x61672c + hoff, avi_infoframe.subpack1_high);
- nvkm_mask(device, 0x616714 + hoff, 0x00000001, 0x00000001);
- }
-
- /* GENERIC(?) / Vendor InfoFrame? */
- nvkm_mask(device, 0x616730 + hoff, 0x00010001, 0x00010000);
- if (vendor_size) {
- /*
- * These appear to be the audio infoframe registers,
- * but no other set of infoframe registers has yet
- * been found.
- */
- nvkm_wr32(device, 0x616738 + hoff, vendor_infoframe.header);
- nvkm_wr32(device, 0x61673c + hoff, vendor_infoframe.subpack0_low);
- nvkm_wr32(device, 0x616740 + hoff, vendor_infoframe.subpack0_high);
- /* Is there a second (or further?) set of subpack registers here? */
- nvkm_mask(device, 0x616730 + hoff, 0x00000001, 0x00000001);
- }
-
/* ??? InfoFrame? */
nvkm_mask(device, 0x6167a4 + hoff, 0x00000001, 0x00000000);
nvkm_wr32(device, 0x6167ac + hoff, 0x00000010);
@@ -259,6 +275,13 @@ gf119_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
nvkm_mask(device, 0x616798 + hoff, 0x401f007f, ctrl);
}
+static const struct nvkm_ior_func_hdmi
+gf119_sor_hdmi = {
+ .ctrl = gf119_sor_hdmi_ctrl,
+ .infoframe_avi = gf119_sor_hdmi_infoframe_avi,
+ .infoframe_vsi = gf119_sor_hdmi_infoframe_vsi,
+};
+
void
gf119_sor_clock(struct nvkm_ior *sor)
{
@@ -305,9 +328,7 @@ gf119_sor = {
.state = gf119_sor_state,
.power = nv50_sor_power,
.clock = gf119_sor_clock,
- .hdmi = {
- .ctrl = gf119_sor_hdmi_ctrl,
- },
+ .hdmi = &gf119_sor_hdmi,
.dp = &gf119_sor_dp,
.hda = &gf119_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gk104.c
index 7248e9e..876a21a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gk104.c
@@ -30,8 +30,51 @@
#include <nvif/class.h>
void
-gk104_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
- u8 rekey, u8 *avi, u8 avi_size, u8 *vendor, u8 vendor_size)
+gk104_sor_hdmi_infoframe_vsi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe vsi;
+ const u32 hoff = head * 0x400;
+
+ pack_hdmi_infoframe(&vsi, data, size);
+
+ /* GENERIC(?) / Vendor InfoFrame? */
+ nvkm_mask(device, 0x690100 + hoff, 0x00010001, 0x00000000);
+ if (!size)
+ return;
+
+ nvkm_wr32(device, 0x690108 + hoff, vsi.header);
+ nvkm_wr32(device, 0x69010c + hoff, vsi.subpack0_low);
+ nvkm_wr32(device, 0x690110 + hoff, vsi.subpack0_high);
+ /* Is there a second (or further?) set of subpack registers here? */
+ nvkm_mask(device, 0x690100 + hoff, 0x00000001, 0x00000001);
+}
+
+void
+gk104_sor_hdmi_infoframe_avi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe avi;
+ const u32 hoff = head * 0x400;
+
+ pack_hdmi_infoframe(&avi, data, size);
+
+ /* AVI InfoFrame */
+ nvkm_mask(device, 0x690000 + hoff, 0x00000001, 0x00000000);
+ if (!size)
+ return;
+
+ nvkm_wr32(device, 0x690008 + hoff, avi.header);
+ nvkm_wr32(device, 0x69000c + hoff, avi.subpack0_low);
+ nvkm_wr32(device, 0x690010 + hoff, avi.subpack0_high);
+ nvkm_wr32(device, 0x690014 + hoff, avi.subpack1_low);
+ nvkm_wr32(device, 0x690018 + hoff, avi.subpack1_high);
+
+ nvkm_mask(device, 0x690000 + hoff, 0x00000001, 0x00000001);
+}
+
+void
+gk104_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet, u8 rekey)
{
struct nvkm_device *device = ior->disp->engine.subdev.device;
const u32 ctrl = 0x40000000 * enable |
@@ -39,11 +82,6 @@ gk104_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
rekey;
const u32 hoff = head * 0x800;
const u32 hdmi = head * 0x400;
- struct packed_hdmi_infoframe avi_infoframe;
- struct packed_hdmi_infoframe vendor_infoframe;
-
- pack_hdmi_infoframe(&avi_infoframe, avi, avi_size);
- pack_hdmi_infoframe(&vendor_infoframe, vendor, vendor_size);
if (!(ctrl & 0x40000000)) {
nvkm_mask(device, 0x616798 + hoff, 0x40000000, 0x00000000);
@@ -53,28 +91,6 @@ gk104_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
return;
}
- /* AVI InfoFrame */
- nvkm_mask(device, 0x690000 + hdmi, 0x00000001, 0x00000000);
- if (avi_size) {
- nvkm_wr32(device, 0x690008 + hdmi, avi_infoframe.header);
- nvkm_wr32(device, 0x69000c + hdmi, avi_infoframe.subpack0_low);
- nvkm_wr32(device, 0x690010 + hdmi, avi_infoframe.subpack0_high);
- nvkm_wr32(device, 0x690014 + hdmi, avi_infoframe.subpack1_low);
- nvkm_wr32(device, 0x690018 + hdmi, avi_infoframe.subpack1_high);
- nvkm_mask(device, 0x690000 + hdmi, 0x00000001, 0x00000001);
- }
-
- /* GENERIC(?) / Vendor InfoFrame? */
- nvkm_mask(device, 0x690100 + hdmi, 0x00010001, 0x00000000);
- if (vendor_size) {
- nvkm_wr32(device, 0x690108 + hdmi, vendor_infoframe.header);
- nvkm_wr32(device, 0x69010c + hdmi, vendor_infoframe.subpack0_low);
- nvkm_wr32(device, 0x690110 + hdmi, vendor_infoframe.subpack0_high);
- /* Is there a second (or further?) set of subpack registers here? */
- nvkm_mask(device, 0x690100 + hdmi, 0x00000001, 0x00000001);
- }
-
-
/* ??? InfoFrame? */
nvkm_mask(device, 0x6900c0 + hdmi, 0x00000001, 0x00000000);
nvkm_wr32(device, 0x6900cc + hdmi, 0x00000010);
@@ -87,14 +103,19 @@ gk104_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
nvkm_mask(device, 0x616798 + hoff, 0x401f007f, ctrl);
}
+const struct nvkm_ior_func_hdmi
+gk104_sor_hdmi = {
+ .ctrl = gk104_sor_hdmi_ctrl,
+ .infoframe_avi = gk104_sor_hdmi_infoframe_avi,
+ .infoframe_vsi = gk104_sor_hdmi_infoframe_vsi,
+};
+
static const struct nvkm_ior_func
gk104_sor = {
.state = gf119_sor_state,
.power = nv50_sor_power,
.clock = gf119_sor_clock,
- .hdmi = {
- .ctrl = gk104_sor_hdmi_ctrl,
- },
+ .hdmi = &gk104_sor_hdmi,
.dp = &gf119_sor_dp,
.hda = &gf119_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm107.c
index 9e9ef49..b4d8e86 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm107.c
@@ -70,9 +70,7 @@ gm107_sor = {
.state = gf119_sor_state,
.power = nv50_sor_power,
.clock = gf119_sor_clock,
- .hdmi = {
- .ctrl = gk104_sor_hdmi_ctrl,
- },
+ .hdmi = &gk104_sor_hdmi,
.dp = &gm107_sor_dp,
.hda = &gf119_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm200.c
index 4ecc8f9..562ebae5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gm200.c
@@ -79,6 +79,14 @@ gm200_sor_hdmi_scdc(struct nvkm_ior *ior, u8 scdc)
ior->tmds.high_speed = !!(scdc & 0x2);
}
+const struct nvkm_ior_func_hdmi
+gm200_sor_hdmi = {
+ .ctrl = gk104_sor_hdmi_ctrl,
+ .scdc = gm200_sor_hdmi_scdc,
+ .infoframe_avi = gk104_sor_hdmi_infoframe_avi,
+ .infoframe_vsi = gk104_sor_hdmi_infoframe_vsi,
+};
+
void
gm200_sor_route_set(struct nvkm_outp *outp, struct nvkm_ior *ior)
{
@@ -131,10 +139,7 @@ gm200_sor = {
.state = gf119_sor_state,
.power = nv50_sor_power,
.clock = gf119_sor_clock,
- .hdmi = {
- .ctrl = gk104_sor_hdmi_ctrl,
- .scdc = gm200_sor_hdmi_scdc,
- },
+ .hdmi = &gm200_sor_hdmi,
.dp = &gm200_sor_dp,
.hda = &gf119_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gp100.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gp100.c
index 7172a9d..7f1eb43 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gp100.c
@@ -37,10 +37,7 @@ gp100_sor = {
.state = gf119_sor_state,
.power = nv50_sor_power,
.clock = gf119_sor_clock,
- .hdmi = {
- .ctrl = gk104_sor_hdmi_ctrl,
- .scdc = gm200_sor_hdmi_scdc,
- },
+ .hdmi = &gm200_sor_hdmi,
.dp = &gm200_sor_dp,
.hda = &gf119_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gt215.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gt215.c
index 70c49e7..a2c7c6f8 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gt215.c
@@ -92,9 +92,53 @@ gt215_sor_dp = {
.watermark = g94_sor_dp_watermark,
};
-void
-gt215_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
- u8 rekey, u8 *avi, u8 avi_size, u8 *vendor, u8 vendor_size)
+static void
+gt215_sor_hdmi_infoframe_vsi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe vsi;
+ const u32 soff = nv50_ior_base(ior);
+
+ pack_hdmi_infoframe(&vsi, data, size);
+
+ nvkm_mask(device, 0x61c53c + soff, 0x00010001, 0x00010000);
+ if (!size)
+ return;
+
+ nvkm_wr32(device, 0x61c544 + soff, vsi.header);
+ nvkm_wr32(device, 0x61c548 + soff, vsi.subpack0_low);
+ nvkm_wr32(device, 0x61c54c + soff, vsi.subpack0_high);
+ /* Is there a second (or up to fourth?) set of subpack registers here? */
+ /* nvkm_wr32(device, 0x61c550 + soff, vsi.subpack1_low); */
+ /* nvkm_wr32(device, 0x61c554 + soff, vsi.subpack1_high); */
+
+ nvkm_mask(device, 0x61c53c + soff, 0x00010001, 0x00010001);
+}
+
+static void
+gt215_sor_hdmi_infoframe_avi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe avi;
+ const u32 soff = nv50_ior_base(ior);
+
+ pack_hdmi_infoframe(&avi, data, size);
+
+ nvkm_mask(device, 0x61c520 + soff, 0x00000001, 0x00000000);
+ if (size)
+ return;
+
+ nvkm_wr32(device, 0x61c528 + soff, avi.header);
+ nvkm_wr32(device, 0x61c52c + soff, avi.subpack0_low);
+ nvkm_wr32(device, 0x61c530 + soff, avi.subpack0_high);
+ nvkm_wr32(device, 0x61c534 + soff, avi.subpack1_low);
+ nvkm_wr32(device, 0x61c538 + soff, avi.subpack1_high);
+
+ nvkm_mask(device, 0x61c520 + soff, 0x00000001, 0x00000001);
+}
+
+static void
+gt215_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet, u8 rekey)
{
struct nvkm_device *device = ior->disp->engine.subdev.device;
const u32 ctrl = 0x40000000 * enable |
@@ -102,11 +146,6 @@ gt215_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
max_ac_packet << 16 |
rekey;
const u32 soff = nv50_ior_base(ior);
- struct packed_hdmi_infoframe avi_infoframe;
- struct packed_hdmi_infoframe vendor_infoframe;
-
- pack_hdmi_infoframe(&avi_infoframe, avi, avi_size);
- pack_hdmi_infoframe(&vendor_infoframe, vendor, vendor_size);
if (!(ctrl & 0x40000000)) {
nvkm_mask(device, 0x61c5a4 + soff, 0x40000000, 0x00000000);
@@ -116,17 +155,6 @@ gt215_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
return;
}
- /* AVI InfoFrame */
- nvkm_mask(device, 0x61c520 + soff, 0x00000001, 0x00000000);
- if (avi_size) {
- nvkm_wr32(device, 0x61c528 + soff, avi_infoframe.header);
- nvkm_wr32(device, 0x61c52c + soff, avi_infoframe.subpack0_low);
- nvkm_wr32(device, 0x61c530 + soff, avi_infoframe.subpack0_high);
- nvkm_wr32(device, 0x61c534 + soff, avi_infoframe.subpack1_low);
- nvkm_wr32(device, 0x61c538 + soff, avi_infoframe.subpack1_high);
- nvkm_mask(device, 0x61c520 + soff, 0x00000001, 0x00000001);
- }
-
/* Audio InfoFrame */
nvkm_mask(device, 0x61c500 + soff, 0x00000001, 0x00000000);
nvkm_wr32(device, 0x61c508 + soff, 0x000a0184);
@@ -134,18 +162,6 @@ gt215_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
nvkm_wr32(device, 0x61c510 + soff, 0x00000000);
nvkm_mask(device, 0x61c500 + soff, 0x00000001, 0x00000001);
- /* Vendor InfoFrame */
- nvkm_mask(device, 0x61c53c + soff, 0x00010001, 0x00010000);
- if (vendor_size) {
- nvkm_wr32(device, 0x61c544 + soff, vendor_infoframe.header);
- nvkm_wr32(device, 0x61c548 + soff, vendor_infoframe.subpack0_low);
- nvkm_wr32(device, 0x61c54c + soff, vendor_infoframe.subpack0_high);
- /* Is there a second (or up to fourth?) set of subpack registers here? */
- /* nvkm_wr32(device, 0x61c550 + soff, vendor_infoframe.subpack1_low); */
- /* nvkm_wr32(device, 0x61c554 + soff, vendor_infoframe.subpack1_high); */
- nvkm_mask(device, 0x61c53c + soff, 0x00010001, 0x00010001);
- }
-
nvkm_mask(device, 0x61c5d0 + soff, 0x00070001, 0x00010001); /* SPARE, HW_CTS */
nvkm_mask(device, 0x61c568 + soff, 0x00010101, 0x00000000); /* ACR_CTRL, ?? */
nvkm_mask(device, 0x61c578 + soff, 0x80000000, 0x80000000); /* ACR_0441_ENABLE */
@@ -159,14 +175,19 @@ gt215_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
nvkm_mask(device, 0x61c5a4 + soff, 0x5f1f007f, ctrl);
}
+const struct nvkm_ior_func_hdmi
+gt215_sor_hdmi = {
+ .ctrl = gt215_sor_hdmi_ctrl,
+ .infoframe_avi = gt215_sor_hdmi_infoframe_avi,
+ .infoframe_vsi = gt215_sor_hdmi_infoframe_vsi,
+};
+
static const struct nvkm_ior_func
gt215_sor = {
.state = g94_sor_state,
.power = nv50_sor_power,
.clock = nv50_sor_clock,
- .hdmi = {
- .ctrl = gt215_sor_hdmi_ctrl,
- },
+ .hdmi = >215_sor_hdmi,
.dp = >215_sor_dp,
.hda = >215_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gv100.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gv100.c
index 6b9d492..115d099 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/gv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/gv100.c
@@ -96,9 +96,54 @@ gv100_sor_dp = {
.watermark = gv100_sor_dp_watermark,
};
-void
-gv100_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet,
- u8 rekey, u8 *avi, u8 avi_size, u8 *vendor, u8 vendor_size)
+static void
+gv100_sor_hdmi_infoframe_vsi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe vsi;
+ const u32 hoff = head * 0x400;
+
+ pack_hdmi_infoframe(&vsi, data, size);
+
+ nvkm_mask(device, 0x6f0100 + hoff, 0x00010001, 0x00000000);
+ if (!size)
+ return;
+
+ nvkm_wr32(device, 0x6f0108 + hoff, vsi.header);
+ nvkm_wr32(device, 0x6f010c + hoff, vsi.subpack0_low);
+ nvkm_wr32(device, 0x6f0110 + hoff, vsi.subpack0_high);
+ nvkm_wr32(device, 0x6f0114 + hoff, 0x00000000);
+ nvkm_wr32(device, 0x6f0118 + hoff, 0x00000000);
+ nvkm_wr32(device, 0x6f011c + hoff, 0x00000000);
+ nvkm_wr32(device, 0x6f0120 + hoff, 0x00000000);
+ nvkm_wr32(device, 0x6f0124 + hoff, 0x00000000);
+ nvkm_mask(device, 0x6f0100 + hoff, 0x00000001, 0x00000001);
+}
+
+static void
+gv100_sor_hdmi_infoframe_avi(struct nvkm_ior *ior, int head, void *data, u32 size)
+{
+ struct nvkm_device *device = ior->disp->engine.subdev.device;
+ struct packed_hdmi_infoframe avi;
+ const u32 hoff = head * 0x400;
+
+ pack_hdmi_infoframe(&avi, data, size);
+
+ nvkm_mask(device, 0x6f0000 + hoff, 0x00000001, 0x00000000);
+ if (!size)
+ return;
+
+ nvkm_wr32(device, 0x6f0008 + hoff, avi.header);
+ nvkm_wr32(device, 0x6f000c + hoff, avi.subpack0_low);
+ nvkm_wr32(device, 0x6f0010 + hoff, avi.subpack0_high);
+ nvkm_wr32(device, 0x6f0014 + hoff, avi.subpack1_low);
+ nvkm_wr32(device, 0x6f0018 + hoff, avi.subpack1_high);
+
+ nvkm_mask(device, 0x6f0000 + hoff, 0x00000001, 0x00000001);
+}
+
+static void
+gv100_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packet, u8 rekey)
{
struct nvkm_device *device = ior->disp->engine.subdev.device;
const u32 ctrl = 0x40000000 * enable |
@@ -106,11 +151,6 @@ gv100_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
rekey;
const u32 hoff = head * 0x800;
const u32 hdmi = head * 0x400;
- struct packed_hdmi_infoframe avi_infoframe;
- struct packed_hdmi_infoframe vendor_infoframe;
-
- pack_hdmi_infoframe(&avi_infoframe, avi, avi_size);
- pack_hdmi_infoframe(&vendor_infoframe, vendor, vendor_size);
if (!(ctrl & 0x40000000)) {
nvkm_mask(device, 0x6165c0 + hoff, 0x40000000, 0x00000000);
@@ -120,32 +160,6 @@ gv100_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
return;
}
- /* AVI InfoFrame (AVI). */
- nvkm_mask(device, 0x6f0000 + hdmi, 0x00000001, 0x00000000);
- if (avi_size) {
- nvkm_wr32(device, 0x6f0008 + hdmi, avi_infoframe.header);
- nvkm_wr32(device, 0x6f000c + hdmi, avi_infoframe.subpack0_low);
- nvkm_wr32(device, 0x6f0010 + hdmi, avi_infoframe.subpack0_high);
- nvkm_wr32(device, 0x6f0014 + hdmi, avi_infoframe.subpack1_low);
- nvkm_wr32(device, 0x6f0018 + hdmi, avi_infoframe.subpack1_high);
- nvkm_mask(device, 0x6f0000 + hdmi, 0x00000001, 0x00000001);
- }
-
- /* Vendor-specific InfoFrame (VSI). */
- nvkm_mask(device, 0x6f0100 + hdmi, 0x00010001, 0x00000000);
- if (vendor_size) {
- nvkm_wr32(device, 0x6f0108 + hdmi, vendor_infoframe.header);
- nvkm_wr32(device, 0x6f010c + hdmi, vendor_infoframe.subpack0_low);
- nvkm_wr32(device, 0x6f0110 + hdmi, vendor_infoframe.subpack0_high);
- nvkm_wr32(device, 0x6f0114 + hdmi, 0x00000000);
- nvkm_wr32(device, 0x6f0118 + hdmi, 0x00000000);
- nvkm_wr32(device, 0x6f011c + hdmi, 0x00000000);
- nvkm_wr32(device, 0x6f0120 + hdmi, 0x00000000);
- nvkm_wr32(device, 0x6f0124 + hdmi, 0x00000000);
- nvkm_mask(device, 0x6f0100 + hdmi, 0x00000001, 0x00000001);
- }
-
-
/* General Control (GCP). */
nvkm_mask(device, 0x6f00c0 + hdmi, 0x00000001, 0x00000000);
nvkm_wr32(device, 0x6f00cc + hdmi, 0x00000010);
@@ -158,6 +172,14 @@ gv100_sor_hdmi_ctrl(struct nvkm_ior *ior, int head, bool enable, u8 max_ac_packe
nvkm_mask(device, 0x6165c0 + hoff, 0x401f007f, ctrl);
}
+const struct nvkm_ior_func_hdmi
+gv100_sor_hdmi = {
+ .ctrl = gv100_sor_hdmi_ctrl,
+ .scdc = gm200_sor_hdmi_scdc,
+ .infoframe_avi = gv100_sor_hdmi_infoframe_avi,
+ .infoframe_vsi = gv100_sor_hdmi_infoframe_vsi,
+};
+
void
gv100_sor_state(struct nvkm_ior *sor, struct nvkm_ior_state *state)
{
@@ -190,10 +212,7 @@ gv100_sor = {
.state = gv100_sor_state,
.power = nv50_sor_power,
.clock = gf119_sor_clock,
- .hdmi = {
- .ctrl = gv100_sor_hdmi_ctrl,
- .scdc = gm200_sor_hdmi_scdc,
- },
+ .hdmi = &gv100_sor_hdmi,
.dp = &gv100_sor_dp,
.hda = &gv100_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.c
index 83152c2..7f5d13d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.c
@@ -39,44 +39,6 @@ nvkm_head_find(struct nvkm_disp *disp, int id)
return NULL;
}
-int
-nvkm_head_mthd_scanoutpos(struct nvkm_object *object,
- struct nvkm_head *head, void *data, u32 size)
-{
- union {
- struct nv04_disp_scanoutpos_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
-
- nvif_ioctl(object, "head scanoutpos size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(object, "head scanoutpos vers %d\n",
- args->v0.version);
-
- head->func->state(head, &head->arm);
- args->v0.vtotal = head->arm.vtotal;
- args->v0.vblanks = head->arm.vblanks;
- args->v0.vblanke = head->arm.vblanke;
- args->v0.htotal = head->arm.htotal;
- args->v0.hblanks = head->arm.hblanks;
- args->v0.hblanke = head->arm.hblanke;
-
- /* We don't support reading htotal/vtotal on pre-NV50 VGA,
- * so we have to give up and trigger the timestamping
- * fallback in the drm core.
- */
- if (!args->v0.vtotal || !args->v0.htotal)
- return -ENOTSUPP;
-
- args->v0.time[0] = ktime_to_ns(ktime_get());
- head->func->rgpos(head, &args->v0.hline, &args->v0.vline);
- args->v0.time[1] = ktime_to_ns(ktime_get());
- } else
- return ret;
-
- return 0;
-}
-
void
nvkm_head_del(struct nvkm_head **phead)
{
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.h
index 84a2989..856252b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/head.h
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: MIT */
#ifndef __NVKM_DISP_HEAD_H__
#define __NVKM_DISP_HEAD_H__
+#include <nvif/object.h>
#include "priv.h"
struct nvkm_head {
@@ -26,12 +27,12 @@ struct nvkm_head {
u8 depth;
} or;
} arm, asy;
+
+ struct nvkm_object object;
};
int nvkm_head_new_(const struct nvkm_head_func *, struct nvkm_disp *, int id);
void nvkm_head_del(struct nvkm_head **);
-int nvkm_head_mthd_scanoutpos(struct nvkm_object *,
- struct nvkm_head *, void *, u32);
struct nvkm_head *nvkm_head_find(struct nvkm_disp *, int id);
struct nvkm_head_func {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h
index 671c467..da1b1a6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h
@@ -63,12 +63,12 @@ struct nvkm_ior_func {
void (*war_2)(struct nvkm_ior *);
void (*war_3)(struct nvkm_ior *);
- struct {
- void (*ctrl)(struct nvkm_ior *, int head, bool enable,
- u8 max_ac_packet, u8 rekey, u8 *avi, u8 avi_size,
- u8 *vendor, u8 vendor_size);
+ const struct nvkm_ior_func_hdmi {
+ void (*ctrl)(struct nvkm_ior *, int head, bool enable, u8 max_ac_packet, u8 rekey);
void (*scdc)(struct nvkm_ior *, u8 scdc);
- } hdmi;
+ void (*infoframe_avi)(struct nvkm_ior *, int head, void *data, u32 size);
+ void (*infoframe_vsi)(struct nvkm_ior *, int head, void *data, u32 size);
+ } *hdmi;
const struct nvkm_ior_func_dp {
u8 lanes[4];
@@ -124,9 +124,10 @@ void nv50_sor_power(struct nvkm_ior *, bool, bool, bool, bool, bool);
void nv50_sor_clock(struct nvkm_ior *);
int g84_sor_new(struct nvkm_disp *, int);
-void g84_sor_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8, u8 *, u8 , u8 *, u8);
+extern const struct nvkm_ior_func_hdmi g84_sor_hdmi;
int g94_sor_cnt(struct nvkm_disp *, unsigned long *);
+
void g94_sor_state(struct nvkm_ior *, struct nvkm_ior_state *);
extern const struct nvkm_ior_func_dp g94_sor_dp;
int g94_sor_dp_links(struct nvkm_ior *, struct nvkm_i2c_aux *);
@@ -137,7 +138,7 @@ void g94_sor_dp_audio_sym(struct nvkm_ior *, int, u16, u32);
void g94_sor_dp_activesym(struct nvkm_ior *, int, u8, u8, u8, u8);
void g94_sor_dp_watermark(struct nvkm_ior *, int, u8);
-void gt215_sor_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8, u8 *, u8 , u8 *, u8);
+extern const struct nvkm_ior_func_hdmi gt215_sor_hdmi;
void gt215_sor_dp_audio(struct nvkm_ior *, int, bool);
extern const struct nvkm_ior_func_hda gt215_sor_hda;
@@ -156,12 +157,16 @@ void gf119_sor_hda_hpd(struct nvkm_ior *, int, bool);
void gf119_sor_hda_eld(struct nvkm_ior *, int, u8 *, u8);
int gk104_sor_new(struct nvkm_disp *, int);
-void gk104_sor_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8, u8 *, u8 , u8 *, u8);
+extern const struct nvkm_ior_func_hdmi gk104_sor_hdmi;
+void gk104_sor_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8);
+void gk104_sor_hdmi_infoframe_avi(struct nvkm_ior *, int, void *, u32);
+void gk104_sor_hdmi_infoframe_vsi(struct nvkm_ior *, int, void *, u32);
void gm107_sor_dp_pattern(struct nvkm_ior *, int);
void gm200_sor_route_set(struct nvkm_outp *, struct nvkm_ior *);
int gm200_sor_route_get(struct nvkm_outp *, int *);
+extern const struct nvkm_ior_func_hdmi gm200_sor_hdmi;
void gm200_sor_hdmi_scdc(struct nvkm_ior *, u8);
extern const struct nvkm_ior_func_dp gm200_sor_dp;
void gm200_sor_dp_drive(struct nvkm_ior *, int, int, int, int, int);
@@ -170,7 +175,7 @@ int gp100_sor_new(struct nvkm_disp *, int);
int gv100_sor_cnt(struct nvkm_disp *, unsigned long *);
void gv100_sor_state(struct nvkm_ior *, struct nvkm_ior_state *);
-void gv100_sor_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8, u8 *, u8 , u8 *, u8);
+extern const struct nvkm_ior_func_hdmi gv100_sor_hdmi;
void gv100_sor_dp_audio(struct nvkm_ior *, int, bool);
void gv100_sor_dp_audio_sym(struct nvkm_ior *, int, u16, u32);
void gv100_sor_dp_watermark(struct nvkm_ior *, int, u8);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp77.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp77.c
index 916b1d4..841e3b6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp77.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp77.c
@@ -31,9 +31,7 @@ mcp77_sor = {
.state = g94_sor_state,
.power = nv50_sor_power,
.clock = nv50_sor_clock,
- .hdmi = {
- .ctrl = g84_sor_hdmi_ctrl,
- },
+ .hdmi = &g84_sor_hdmi,
.dp = &g94_sor_dp,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp89.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp89.c
index a5a0b943..f96ba47 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp89.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp89.c
@@ -44,9 +44,7 @@ mcp89_sor = {
.state = g94_sor_state,
.power = nv50_sor_power,
.clock = nv50_sor_clock,
- .hdmi = {
- .ctrl = gt215_sor_hdmi_ctrl,
- },
+ .hdmi = >215_sor_hdmi,
.dp = &mcp89_sor_dp,
.hda = >215_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c
index a46e13cc..be81168 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c
@@ -503,7 +503,7 @@ nv50_disp_chan_uevent_init(struct nvkm_event *event, int types, int index)
void
nv50_disp_chan_uevent_send(struct nvkm_disp *disp, int chid)
{
- nvkm_event_send(&disp->uevent, NVKM_DISP_EVENT_CHAN_AWAKEN, chid, NULL, 0);
+ nvkm_event_ntfy(&disp->uevent, chid, NVKM_DISP_EVENT_CHAN_AWAKEN);
}
const struct nvkm_event_func
@@ -1238,6 +1238,8 @@ nv50_disp_super_2_2(struct nvkm_disp *disp, struct nvkm_head *head)
if (!ior)
return;
+ outp = ior->asy.outp;
+
/* For some reason, NVIDIA decided not to:
*
* A) Give dual-link LVDS a separate EVO protocol, like for TMDS.
@@ -1247,13 +1249,13 @@ nv50_disp_super_2_2(struct nvkm_disp *disp, struct nvkm_head *head)
* Override the values we usually read from HW with the same
* data we pass though an ioctl instead.
*/
- if (ior->type == SOR && ior->asy.proto == LVDS) {
- head->asy.or.depth = (disp->sor.lvdsconf & 0x0200) ? 24 : 18;
- ior->asy.link = (disp->sor.lvdsconf & 0x0100) ? 3 : 1;
+ if (outp && ior->type == SOR && ior->asy.proto == LVDS) {
+ head->asy.or.depth = outp->lvds.bpc8 ? 24 : 18;
+ ior->asy.link = outp->lvds.dual ? 3 : 1;
}
/* Handle any link training, etc. */
- if ((outp = ior->asy.outp) && outp->func->acquire)
+ if (outp && outp->func->acquire)
outp->func->acquire(outp);
/* Execute OnInt2 IED script. */
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h
index 3f3924c..b7631c1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h
@@ -2,7 +2,6 @@
#ifndef __NVKM_DISP_OUTP_H__
#define __NVKM_DISP_OUTP_H__
#include "priv.h"
-#include <core/notify.h>
#include <subdev/bios.h>
#include <subdev/bios/dcb.h>
@@ -28,13 +27,19 @@ struct nvkm_outp {
union {
struct {
+ bool dual;
+ bool bpc8;
+ } lvds;
+
+ struct {
struct nvbios_dpout info;
u8 version;
struct nvkm_i2c_aux *aux;
- struct nvkm_notify hpd;
- bool present;
+ bool enabled;
+ bool aux_pwr;
+ bool aux_pwr_pu;
u8 lttpr[6];
u8 lttprs;
u8 dpcd[16];
@@ -49,12 +54,17 @@ struct nvkm_outp {
struct mutex mutex;
struct {
atomic_t done;
+ u8 nr;
+ u8 bw;
bool mst;
} lt;
} dp;
};
struct nvkm_object object;
+ struct {
+ struct nvkm_head *head;
+ } asy;
};
int nvkm_outp_new_(const struct nvkm_outp_func *, struct nvkm_disp *, int index,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/priv.h
index cb25dfe..ec5292a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/priv.h
@@ -42,10 +42,6 @@ struct nvkm_disp_func {
} user[];
};
-int nvkm_disp_ntfy(struct nvkm_object *, u32, struct nvkm_event **);
-int nv04_disp_mthd(struct nvkm_object *, u32, void *, u32);
-int nv50_disp_root_mthd_(struct nvkm_object *, u32, void *, u32);
-
int nv50_disp_oneinit(struct nvkm_disp *);
int nv50_disp_init(struct nvkm_disp *);
void nv50_disp_fini(struct nvkm_disp *);
@@ -86,4 +82,5 @@ extern const struct nvkm_event_func gv100_disp_chan_uevent;
int nvkm_udisp_new(const struct nvkm_oclass *, void *, u32, struct nvkm_object **);
int nvkm_uconn_new(const struct nvkm_oclass *, void *, u32, struct nvkm_object **);
int nvkm_uoutp_new(const struct nvkm_oclass *, void *, u32, struct nvkm_object **);
+int nvkm_uhead_new(const struct nvkm_oclass *, void *, u32, struct nvkm_object **);
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv04.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv04.c
deleted file mode 100644
index 9acaec5..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv04.c
+++ /dev/null
@@ -1,62 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "priv.h"
-#include "head.h"
-
-#include <core/client.h>
-
-#include <nvif/cl0046.h>
-#include <nvif/unpack.h>
-
-int
-nv04_disp_mthd(struct nvkm_object *object, u32 mthd, void *data, u32 size)
-{
- struct nvkm_disp *disp = nvkm_disp(object->engine);
- union {
- struct nv04_disp_mthd_v0 v0;
- } *args = data;
- struct nvkm_head *head;
- int id, ret = -ENOSYS;
-
- nvif_ioctl(object, "disp mthd size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, true))) {
- nvif_ioctl(object, "disp mthd vers %d mthd %02x head %d\n",
- args->v0.version, args->v0.method, args->v0.head);
- mthd = args->v0.method;
- id = args->v0.head;
- } else
- return ret;
-
- if (!(head = nvkm_head_find(disp, id)))
- return -ENXIO;
-
- switch (mthd) {
- case NV04_DISP_SCANOUTPOS:
- return nvkm_head_mthd_scanoutpos(object, head, data, size);
- default:
- break;
- }
-
- return -EINVAL;
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv50.c
deleted file mode 100644
index 0af45cc..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv50.c
+++ /dev/null
@@ -1,250 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "chan.h"
-#include "head.h"
-#include "ior.h"
-#include "outp.h"
-
-#include <core/client.h>
-
-#include <nvif/class.h>
-#include <nvif/cl5070.h>
-#include <nvif/unpack.h>
-
-int
-nv50_disp_root_mthd_(struct nvkm_object *object, u32 mthd, void *data, u32 size)
-{
- union {
- struct nv50_disp_mthd_v0 v0;
- struct nv50_disp_mthd_v1 v1;
- } *args = data;
- struct nvkm_disp *disp = nvkm_udisp(object);
- struct nvkm_outp *temp, *outp = NULL;
- struct nvkm_head *head;
- u16 type, mask = 0;
- int hidx, ret = -ENOSYS;
-
- if (mthd != NV50_DISP_MTHD)
- return -EINVAL;
-
- nvif_ioctl(object, "disp mthd size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, true))) {
- nvif_ioctl(object, "disp mthd vers %d mthd %02x head %d\n",
- args->v0.version, args->v0.method, args->v0.head);
- mthd = args->v0.method;
- hidx = args->v0.head;
- } else
- if (!(ret = nvif_unpack(ret, &data, &size, args->v1, 1, 1, true))) {
- nvif_ioctl(object, "disp mthd vers %d mthd %02x "
- "type %04x mask %04x\n",
- args->v1.version, args->v1.method,
- args->v1.hasht, args->v1.hashm);
- mthd = args->v1.method;
- type = args->v1.hasht;
- mask = args->v1.hashm;
- hidx = ffs((mask >> 8) & 0x0f) - 1;
- } else
- return ret;
-
- if (!(head = nvkm_head_find(disp, hidx)))
- return -ENXIO;
-
- if (mask) {
- list_for_each_entry(temp, &disp->outps, head) {
- if ((temp->info.hasht == type) &&
- (temp->info.hashm & mask) == mask) {
- outp = temp;
- break;
- }
- }
- if (outp == NULL)
- return -ENXIO;
- }
-
- switch (mthd) {
- case NV50_DISP_SCANOUTPOS: {
- return nvkm_head_mthd_scanoutpos(object, head, data, size);
- }
- default:
- break;
- }
-
- switch (mthd * !!outp) {
- case NV50_DISP_MTHD_V1_ACQUIRE: {
- union {
- struct nv50_disp_acquire_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- ret = nvkm_outp_acquire(outp, NVKM_OUTP_USER, args->v0.hda);
- if (ret == 0) {
- args->v0.or = outp->ior->id;
- args->v0.link = outp->ior->asy.link;
- }
- }
- return ret;
- }
- break;
- case NV50_DISP_MTHD_V1_RELEASE:
- nvkm_outp_release(outp, NVKM_OUTP_USER);
- return 0;
- case NV50_DISP_MTHD_V1_SOR_HDA_ELD: {
- union {
- struct nv50_disp_sor_hda_eld_v0 v0;
- } *args = data;
- struct nvkm_ior *ior = outp->ior;
- int ret = -ENOSYS;
-
- nvif_ioctl(object, "disp sor hda eld size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, true))) {
- nvif_ioctl(object, "disp sor hda eld vers %d\n",
- args->v0.version);
- if (size > 0x60)
- return -E2BIG;
- } else
- return ret;
-
- if (!ior->hda)
- return -ENODEV;
-
- if (size && args->v0.data[0]) {
- if (outp->info.type == DCB_OUTPUT_DP)
- ior->func->dp->audio(ior, hidx, true);
- ior->func->hda->hpd(ior, hidx, true);
- ior->func->hda->eld(ior, hidx, data, size);
- } else {
- if (outp->info.type == DCB_OUTPUT_DP)
- ior->func->dp->audio(ior, hidx, false);
- ior->func->hda->hpd(ior, hidx, false);
- }
-
- return 0;
- }
- break;
- case NV50_DISP_MTHD_V1_SOR_HDMI_PWR: {
- union {
- struct nv50_disp_sor_hdmi_pwr_v0 v0;
- } *args = data;
- u8 *vendor, vendor_size;
- u8 *avi, avi_size;
- int ret = -ENOSYS;
-
- nvif_ioctl(object, "disp sor hdmi ctrl size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, true))) {
- nvif_ioctl(object, "disp sor hdmi ctrl vers %d state %d "
- "max_ac_packet %d rekey %d scdc %d\n",
- args->v0.version, args->v0.state,
- args->v0.max_ac_packet, args->v0.rekey,
- args->v0.scdc);
- if (args->v0.max_ac_packet > 0x1f || args->v0.rekey > 0x7f)
- return -EINVAL;
- if ((args->v0.avi_infoframe_length
- + args->v0.vendor_infoframe_length) > size)
- return -EINVAL;
- else
- if ((args->v0.avi_infoframe_length
- + args->v0.vendor_infoframe_length) < size)
- return -E2BIG;
- avi = data;
- avi_size = args->v0.avi_infoframe_length;
- vendor = avi + avi_size;
- vendor_size = args->v0.vendor_infoframe_length;
- } else
- return ret;
-
- if (!outp->ior->func->hdmi.ctrl)
- return -ENODEV;
-
- outp->ior->func->hdmi.ctrl(outp->ior, hidx, args->v0.state,
- args->v0.max_ac_packet,
- args->v0.rekey, avi, avi_size,
- vendor, vendor_size);
-
- if (outp->ior->func->hdmi.scdc)
- outp->ior->func->hdmi.scdc(outp->ior, args->v0.scdc);
-
- return 0;
- }
- break;
- case NV50_DISP_MTHD_V1_SOR_LVDS_SCRIPT: {
- union {
- struct nv50_disp_sor_lvds_script_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
- nvif_ioctl(object, "disp sor lvds script size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(object, "disp sor lvds script "
- "vers %d name %04x\n",
- args->v0.version, args->v0.script);
- disp->sor.lvdsconf = args->v0.script;
- return 0;
- } else
- return ret;
- }
- break;
- case NV50_DISP_MTHD_V1_SOR_DP_MST_LINK: {
- union {
- struct nv50_disp_sor_dp_mst_link_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
- nvif_ioctl(object, "disp sor dp mst link size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(object, "disp sor dp mst link vers %d state %d\n",
- args->v0.version, args->v0.state);
- outp->dp.lt.mst = !!args->v0.state;
- return 0;
- } else
- return ret;
- }
- break;
- case NV50_DISP_MTHD_V1_SOR_DP_MST_VCPI: {
- union {
- struct nv50_disp_sor_dp_mst_vcpi_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
- nvif_ioctl(object, "disp sor dp mst vcpi size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(object, "disp sor dp mst vcpi vers %d "
- "slot %02x/%02x pbn %04x/%04x\n",
- args->v0.version, args->v0.start_slot,
- args->v0.num_slots, args->v0.pbn,
- args->v0.aligned_pbn);
- if (!outp->ior->func->dp->vcpi)
- return -ENODEV;
- outp->ior->func->dp->vcpi(outp->ior, hidx,
- args->v0.start_slot,
- args->v0.num_slots,
- args->v0.pbn,
- args->v0.aligned_pbn);
- return 0;
- } else
- return ret;
- }
- break;
- default:
- break;
- }
-
- return -EINVAL;
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/tu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/tu102.c
index e4ad1a6..f5242a6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/tu102.c
@@ -88,10 +88,7 @@ tu102_sor = {
.state = gv100_sor_state,
.power = nv50_sor_power,
.clock = gf119_sor_clock,
- .hdmi = {
- .ctrl = gv100_sor_hdmi_ctrl,
- .scdc = gm200_sor_hdmi_scdc,
- },
+ .hdmi = &gv100_sor_hdmi,
.dp = &tu102_sor_dp,
.hda = &gv100_sor_hda,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c
index fd9f181..dad942b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c
@@ -21,12 +21,86 @@
*/
#define nvkm_uconn(p) container_of((p), struct nvkm_conn, object)
#include "conn.h"
+#include "outp.h"
+#include <core/client.h>
+#include <core/event.h>
#include <subdev/gpio.h>
+#include <subdev/i2c.h>
#include <nvif/if0011.h>
static int
+nvkm_uconn_uevent_aux(struct nvkm_object *object, u64 token, u32 bits)
+{
+ union nvif_conn_event_args args;
+
+ args.v0.version = 0;
+ args.v0.types = 0;
+ if (bits & NVKM_I2C_PLUG)
+ args.v0.types |= NVIF_CONN_EVENT_V0_PLUG;
+ if (bits & NVKM_I2C_UNPLUG)
+ args.v0.types |= NVIF_CONN_EVENT_V0_UNPLUG;
+ if (bits & NVKM_I2C_IRQ)
+ args.v0.types |= NVIF_CONN_EVENT_V0_IRQ;
+
+ return object->client->event(token, &args, sizeof(args.v0));
+}
+
+static int
+nvkm_uconn_uevent_gpio(struct nvkm_object *object, u64 token, u32 bits)
+{
+ union nvif_conn_event_args args;
+
+ args.v0.version = 0;
+ args.v0.types = 0;
+ if (bits & NVKM_GPIO_HI)
+ args.v0.types |= NVIF_CONN_EVENT_V0_PLUG;
+ if (bits & NVKM_GPIO_LO)
+ args.v0.types |= NVIF_CONN_EVENT_V0_UNPLUG;
+
+ return object->client->event(token, &args, sizeof(args.v0));
+}
+
+static int
+nvkm_uconn_uevent(struct nvkm_object *object, void *argv, u32 argc, struct nvkm_uevent *uevent)
+{
+ struct nvkm_conn *conn = nvkm_uconn(object);
+ struct nvkm_device *device = conn->disp->engine.subdev.device;
+ struct nvkm_outp *outp;
+ union nvif_conn_event_args *args = argv;
+ u64 bits = 0;
+
+ if (!uevent) {
+ if (conn->info.hpd == DCB_GPIO_UNUSED)
+ return -ENOSYS;
+ return 0;
+ }
+
+ if (argc != sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+
+ list_for_each_entry(outp, &conn->disp->outps, head) {
+ if (outp->info.connector == conn->index && outp->dp.aux) {
+ if (args->v0.types & NVIF_CONN_EVENT_V0_PLUG ) bits |= NVKM_I2C_PLUG;
+ if (args->v0.types & NVIF_CONN_EVENT_V0_UNPLUG) bits |= NVKM_I2C_UNPLUG;
+ if (args->v0.types & NVIF_CONN_EVENT_V0_IRQ ) bits |= NVKM_I2C_IRQ;
+
+ return nvkm_uevent_add(uevent, &device->i2c->event, outp->dp.aux->id, bits,
+ nvkm_uconn_uevent_aux);
+ }
+ }
+
+ if (args->v0.types & NVIF_CONN_EVENT_V0_PLUG ) bits |= NVKM_GPIO_HI;
+ if (args->v0.types & NVIF_CONN_EVENT_V0_UNPLUG) bits |= NVKM_GPIO_LO;
+ if (args->v0.types & NVIF_CONN_EVENT_V0_IRQ)
+ return -EINVAL;
+
+ return nvkm_uevent_add(uevent, &device->gpio->event, conn->info.hpd, bits,
+ nvkm_uconn_uevent_gpio);
+}
+
+static int
nvkm_uconn_mthd_hpd_status(struct nvkm_conn *conn, void *argv, u32 argc)
{
struct nvkm_gpio *gpio = conn->disp->engine.subdev.device->gpio;
@@ -82,6 +156,7 @@ static const struct nvkm_object_func
nvkm_uconn = {
.dtor = nvkm_uconn_dtor,
.mthd = nvkm_uconn_mthd,
+ .uevent = nvkm_uconn_uevent,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/udisp.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/udisp.c
index 0841e7c..0268d1d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/udisp.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/udisp.c
@@ -21,6 +21,7 @@
*/
#include "priv.h"
#include "conn.h"
+#include "head.h"
#include "outp.h"
#include <nvif/class.h>
@@ -43,6 +44,12 @@ nvkm_udisp_sclass(struct nvkm_object *object, int index, struct nvkm_oclass *scl
return 0;
}
+ if (index-- == 0) {
+ sclass->base = (struct nvkm_sclass) { 0, 0, NVIF_CLASS_HEAD };
+ sclass->ctor = nvkm_uhead_new;
+ return 0;
+ }
+
if (disp->func->user[index].ctor) {
sclass->base = disp->func->user[index].base;
sclass->ctor = disp->func->user[index].ctor;
@@ -52,17 +59,6 @@ nvkm_udisp_sclass(struct nvkm_object *object, int index, struct nvkm_oclass *scl
return -EINVAL;
}
-static int
-nvkm_udisp_mthd(struct nvkm_object *object, u32 mthd, void *argv, u32 argc)
-{
- struct nvkm_disp *disp = nvkm_udisp(object);
-
- if (disp->engine.subdev.device->card_type >= NV_50)
- return nv50_disp_root_mthd_(object, mthd, argv, argc);
-
- return nv04_disp_mthd(object, mthd, argv, argc);
-}
-
static void *
nvkm_udisp_dtor(struct nvkm_object *object)
{
@@ -78,8 +74,6 @@ nvkm_udisp_dtor(struct nvkm_object *object)
static const struct nvkm_object_func
nvkm_udisp = {
.dtor = nvkm_udisp_dtor,
- .mthd = nvkm_udisp_mthd,
- .ntfy = nvkm_disp_ntfy,
.sclass = nvkm_udisp_sclass,
};
@@ -89,6 +83,7 @@ nvkm_udisp_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, struct nv
struct nvkm_disp *disp = nvkm_disp(oclass->engine);
struct nvkm_conn *conn;
struct nvkm_outp *outp;
+ struct nvkm_head *head;
union nvif_disp_args *args = argv;
if (argc != sizeof(args->v0) || args->v0.version != 0)
@@ -111,5 +106,9 @@ nvkm_udisp_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, struct nv
list_for_each_entry(outp, &disp->outps, head)
args->v0.outp_mask |= BIT(outp->index);
+ args->v0.head_mask = 0;
+ list_for_each_entry(head, &disp->heads, head)
+ args->v0.head_mask |= BIT(head->id);
+
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uhead.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uhead.c
new file mode 100644
index 0000000..f072cec
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uhead.c
@@ -0,0 +1,127 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#define nvkm_uhead(p) container_of((p), struct nvkm_head, object)
+#include "head.h"
+#include <core/event.h>
+
+#include <nvif/if0013.h>
+
+#include <nvif/event.h>
+
+static int
+nvkm_uhead_uevent(struct nvkm_object *object, void *argv, u32 argc, struct nvkm_uevent *uevent)
+{
+ struct nvkm_head *head = nvkm_uhead(object);
+ union nvif_head_event_args *args = argv;
+
+ if (!uevent)
+ return 0;
+ if (argc != sizeof(args->vn))
+ return -ENOSYS;
+
+ return nvkm_uevent_add(uevent, &head->disp->vblank, head->id,
+ NVKM_DISP_HEAD_EVENT_VBLANK, NULL);
+}
+
+static int
+nvkm_uhead_mthd_scanoutpos(struct nvkm_head *head, void *argv, u32 argc)
+{
+ union nvif_head_scanoutpos_args *args = argv;
+
+ if (argc != sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+
+ head->func->state(head, &head->arm);
+ args->v0.vtotal = head->arm.vtotal;
+ args->v0.vblanks = head->arm.vblanks;
+ args->v0.vblanke = head->arm.vblanke;
+ args->v0.htotal = head->arm.htotal;
+ args->v0.hblanks = head->arm.hblanks;
+ args->v0.hblanke = head->arm.hblanke;
+
+ /* We don't support reading htotal/vtotal on pre-NV50 VGA,
+ * so we have to give up and trigger the timestamping
+ * fallback in the drm core.
+ */
+ if (!args->v0.vtotal || !args->v0.htotal)
+ return -ENOTSUPP;
+
+ args->v0.time[0] = ktime_to_ns(ktime_get());
+ head->func->rgpos(head, &args->v0.hline, &args->v0.vline);
+ args->v0.time[1] = ktime_to_ns(ktime_get());
+ return 0;
+}
+
+static int
+nvkm_uhead_mthd(struct nvkm_object *object, u32 mthd, void *argv, u32 argc)
+{
+ struct nvkm_head *head = nvkm_uhead(object);
+
+ switch (mthd) {
+ case NVIF_HEAD_V0_SCANOUTPOS: return nvkm_uhead_mthd_scanoutpos(head, argv, argc);
+ default:
+ return -EINVAL;
+ }
+}
+
+static void *
+nvkm_uhead_dtor(struct nvkm_object *object)
+{
+ struct nvkm_head *head = nvkm_uhead(object);
+ struct nvkm_disp *disp = head->disp;
+
+ spin_lock(&disp->client.lock);
+ head->object.func = NULL;
+ spin_unlock(&disp->client.lock);
+ return NULL;
+}
+
+static const struct nvkm_object_func
+nvkm_uhead = {
+ .dtor = nvkm_uhead_dtor,
+ .mthd = nvkm_uhead_mthd,
+ .uevent = nvkm_uhead_uevent,
+};
+
+int
+nvkm_uhead_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, struct nvkm_object **pobject)
+{
+ struct nvkm_disp *disp = nvkm_udisp(oclass->parent);
+ struct nvkm_head *head;
+ union nvif_head_args *args = argv;
+ int ret;
+
+ if (argc != sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+ if (!(head = nvkm_head_find(disp, args->v0.id)))
+ return -EINVAL;
+
+ ret = -EBUSY;
+ spin_lock(&disp->client.lock);
+ if (!head->object.func) {
+ nvkm_object_ctor(&nvkm_uhead, oclass, &head->object);
+ *pobject = &head->object;
+ ret = 0;
+ }
+ spin_unlock(&disp->client.lock);
+ return ret;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c
index abedb3e..4f0ca709 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c
@@ -21,11 +21,238 @@
*/
#define nvkm_uoutp(p) container_of((p), struct nvkm_outp, object)
#include "outp.h"
+#include "dp.h"
+#include "head.h"
#include "ior.h"
#include <nvif/if0012.h>
static int
+nvkm_uoutp_mthd_dp_mst_vcpi(struct nvkm_outp *outp, void *argv, u32 argc)
+{
+ struct nvkm_ior *ior = outp->ior;
+ union nvif_outp_dp_mst_vcpi_args *args = argv;
+
+ if (argc != sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+ if (!ior->func->dp || !ior->func->dp->vcpi || !nvkm_head_find(outp->disp, args->v0.head))
+ return -EINVAL;
+
+ ior->func->dp->vcpi(ior, args->v0.head, args->v0.start_slot, args->v0.num_slots,
+ args->v0.pbn, args->v0.aligned_pbn);
+ return 0;
+}
+
+static int
+nvkm_uoutp_mthd_dp_retrain(struct nvkm_outp *outp, void *argv, u32 argc)
+{
+ union nvif_outp_dp_retrain_args *args = argv;
+
+ if (argc != sizeof(args->vn))
+ return -ENOSYS;
+
+ if (!atomic_read(&outp->dp.lt.done))
+ return 0;
+
+ return outp->func->acquire(outp);
+}
+
+static int
+nvkm_uoutp_mthd_dp_aux_pwr(struct nvkm_outp *outp, void *argv, u32 argc)
+{
+ union nvif_outp_dp_aux_pwr_args *args = argv;
+
+ if (argc != sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+
+ outp->dp.enabled = !!args->v0.state;
+ nvkm_dp_enable(outp, outp->dp.enabled);
+ return 0;
+}
+
+static int
+nvkm_uoutp_mthd_hda_eld(struct nvkm_outp *outp, void *argv, u32 argc)
+{
+ struct nvkm_ior *ior = outp->ior;
+ union nvif_outp_hda_eld_args *args = argv;
+
+ if (argc < sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+ argc -= sizeof(args->v0);
+
+ if (!ior->hda || !nvkm_head_find(outp->disp, args->v0.head))
+ return -EINVAL;
+ if (argc > 0x60)
+ return -E2BIG;
+
+ if (argc && args->v0.data[0]) {
+ if (outp->info.type == DCB_OUTPUT_DP)
+ ior->func->dp->audio(ior, args->v0.head, true);
+ ior->func->hda->hpd(ior, args->v0.head, true);
+ ior->func->hda->eld(ior, args->v0.head, args->v0.data, argc);
+ } else {
+ if (outp->info.type == DCB_OUTPUT_DP)
+ ior->func->dp->audio(ior, args->v0.head, false);
+ ior->func->hda->hpd(ior, args->v0.head, false);
+ }
+
+ return 0;
+}
+
+static int
+nvkm_uoutp_mthd_infoframe(struct nvkm_outp *outp, void *argv, u32 argc)
+{
+ struct nvkm_ior *ior = outp->ior;
+ union nvif_outp_infoframe_args *args = argv;
+ ssize_t size = argc - sizeof(*args);
+
+ if (argc < sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+ if (!nvkm_head_find(outp->disp, args->v0.head))
+ return -EINVAL;
+
+ switch (ior->func->hdmi ? args->v0.type : 0xff) {
+ case NVIF_OUTP_INFOFRAME_V0_AVI:
+ ior->func->hdmi->infoframe_avi(ior, args->v0.head, &args->v0.data, size);
+ return 0;
+ case NVIF_OUTP_INFOFRAME_V0_VSI:
+ ior->func->hdmi->infoframe_vsi(ior, args->v0.head, &args->v0.data, size);
+ return 0;
+ default:
+ break;
+ }
+
+ return -EINVAL;
+}
+
+static int
+nvkm_uoutp_mthd_release(struct nvkm_outp *outp, void *argv, u32 argc)
+{
+ struct nvkm_head *head = outp->asy.head;
+ struct nvkm_ior *ior = outp->ior;
+ union nvif_outp_release_args *args = argv;
+
+ if (argc != sizeof(args->vn))
+ return -ENOSYS;
+
+ if (ior->func->hdmi && head) {
+ ior->func->hdmi->infoframe_avi(ior, head->id, NULL, 0);
+ ior->func->hdmi->infoframe_vsi(ior, head->id, NULL, 0);
+ ior->func->hdmi->ctrl(ior, head->id, false, 0, 0);
+ }
+
+ nvkm_outp_release(outp, NVKM_OUTP_USER);
+ return 0;
+}
+
+static int
+nvkm_uoutp_mthd_acquire_dp(struct nvkm_outp *outp, u8 dpcd[16],
+ u8 link_nr, u8 link_bw, bool hda, bool mst)
+{
+ int ret;
+
+ ret = nvkm_outp_acquire(outp, NVKM_OUTP_USER, hda);
+ if (ret)
+ return ret;
+
+ memcpy(outp->dp.dpcd, dpcd, sizeof(outp->dp.dpcd));
+ outp->dp.lt.nr = link_nr;
+ outp->dp.lt.bw = link_bw;
+ outp->dp.lt.mst = mst;
+ return 0;
+}
+
+static int
+nvkm_uoutp_mthd_acquire_tmds(struct nvkm_outp *outp, u8 head, u8 hdmi, u8 hdmi_max_ac_packet,
+ u8 hdmi_rekey, u8 hdmi_scdc, u8 hdmi_hda)
+{
+ struct nvkm_ior *ior;
+ int ret;
+
+ if (!(outp->asy.head = nvkm_head_find(outp->disp, head)))
+ return -EINVAL;
+
+ ret = nvkm_outp_acquire(outp, NVKM_OUTP_USER, hdmi && hdmi_hda);
+ if (ret)
+ return ret;
+
+ ior = outp->ior;
+
+ if (hdmi) {
+ if (!ior->func->hdmi ||
+ hdmi_max_ac_packet > 0x1f || hdmi_rekey > 0x7f ||
+ (hdmi_scdc && !ior->func->hdmi->scdc)) {
+ nvkm_outp_release(outp, NVKM_OUTP_USER);
+ return -EINVAL;
+ }
+
+ ior->func->hdmi->ctrl(ior, head, hdmi, hdmi_max_ac_packet, hdmi_rekey);
+ if (ior->func->hdmi->scdc)
+ ior->func->hdmi->scdc(ior, hdmi_scdc);
+ }
+
+ return 0;
+}
+
+static int
+nvkm_uoutp_mthd_acquire_lvds(struct nvkm_outp *outp, bool dual, bool bpc8)
+{
+ if (outp->info.type != DCB_OUTPUT_LVDS)
+ return -EINVAL;
+
+ outp->lvds.dual = dual;
+ outp->lvds.bpc8 = bpc8;
+
+ return nvkm_outp_acquire(outp, NVKM_OUTP_USER, false);
+}
+
+static int
+nvkm_uoutp_mthd_acquire(struct nvkm_outp *outp, void *argv, u32 argc)
+{
+ union nvif_outp_acquire_args *args = argv;
+ int ret;
+
+ if (argc != sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+ if (outp->ior)
+ return -EBUSY;
+
+ switch (args->v0.proto) {
+ case NVIF_OUTP_ACQUIRE_V0_RGB_CRT:
+ ret = nvkm_outp_acquire(outp, NVKM_OUTP_USER, false);
+ break;
+ case NVIF_OUTP_ACQUIRE_V0_TMDS:
+ ret = nvkm_uoutp_mthd_acquire_tmds(outp, args->v0.tmds.head,
+ args->v0.tmds.hdmi,
+ args->v0.tmds.hdmi_max_ac_packet,
+ args->v0.tmds.hdmi_rekey,
+ args->v0.tmds.hdmi_scdc,
+ args->v0.tmds.hdmi_hda);
+ break;
+ case NVIF_OUTP_ACQUIRE_V0_LVDS:
+ ret = nvkm_uoutp_mthd_acquire_lvds(outp, args->v0.lvds.dual, args->v0.lvds.bpc8);
+ break;
+ case NVIF_OUTP_ACQUIRE_V0_DP:
+ ret = nvkm_uoutp_mthd_acquire_dp(outp, args->v0.dp.dpcd,
+ args->v0.dp.link_nr,
+ args->v0.dp.link_bw,
+ args->v0.dp.hda != 0,
+ args->v0.dp.mst != 0);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ if (ret)
+ return ret;
+
+ args->v0.or = outp->ior->id;
+ args->v0.link = outp->ior->asy.link;
+ return 0;
+}
+
+static int
nvkm_uoutp_mthd_load_detect(struct nvkm_outp *outp, void *argv, u32 argc)
{
union nvif_outp_load_detect_args *args = argv;
@@ -49,10 +276,28 @@ nvkm_uoutp_mthd_load_detect(struct nvkm_outp *outp, void *argv, u32 argc)
}
static int
+nvkm_uoutp_mthd_acquired(struct nvkm_outp *outp, u32 mthd, void *argv, u32 argc)
+{
+ switch (mthd) {
+ case NVIF_OUTP_V0_RELEASE : return nvkm_uoutp_mthd_release (outp, argv, argc);
+ case NVIF_OUTP_V0_INFOFRAME : return nvkm_uoutp_mthd_infoframe (outp, argv, argc);
+ case NVIF_OUTP_V0_HDA_ELD : return nvkm_uoutp_mthd_hda_eld (outp, argv, argc);
+ case NVIF_OUTP_V0_DP_RETRAIN : return nvkm_uoutp_mthd_dp_retrain (outp, argv, argc);
+ case NVIF_OUTP_V0_DP_MST_VCPI: return nvkm_uoutp_mthd_dp_mst_vcpi(outp, argv, argc);
+ default:
+ break;
+ }
+
+ return -EINVAL;
+}
+
+static int
nvkm_uoutp_mthd_noacquire(struct nvkm_outp *outp, u32 mthd, void *argv, u32 argc)
{
switch (mthd) {
case NVIF_OUTP_V0_LOAD_DETECT: return nvkm_uoutp_mthd_load_detect(outp, argv, argc);
+ case NVIF_OUTP_V0_ACQUIRE : return nvkm_uoutp_mthd_acquire (outp, argv, argc);
+ case NVIF_OUTP_V0_DP_AUX_PWR : return nvkm_uoutp_mthd_dp_aux_pwr (outp, argv, argc);
default:
break;
}
@@ -73,6 +318,11 @@ nvkm_uoutp_mthd(struct nvkm_object *object, u32 mthd, void *argv, u32 argc)
if (ret <= 0)
goto done;
+ if (outp->ior)
+ ret = nvkm_uoutp_mthd_acquired(outp, mthd, argv, argc);
+ else
+ ret = -EIO;
+
done:
mutex_unlock(&disp->super.mutex);
return ret;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c b/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c
index 43b7dec..d619b40 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c
@@ -65,10 +65,10 @@ nvkm_falcon_intr(struct nvkm_engine *engine)
u32 dest = nvkm_rd32(device, base + 0x01c);
u32 intr = nvkm_rd32(device, base + 0x008) & dest & ~(dest >> 16);
u32 inst = nvkm_rd32(device, base + 0x050) & 0x3fffffff;
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
unsigned long flags;
- chan = nvkm_fifo_chan_inst(device->fifo, (u64)inst << 12, &flags);
+ chan = nvkm_chan_get_inst(engine, (u64)inst << 12, &flags);
if (intr & 0x00000040) {
if (falcon->func->intr) {
@@ -89,7 +89,7 @@ nvkm_falcon_intr(struct nvkm_engine *engine)
nvkm_wr32(device, base + 0x004, intr);
}
- nvkm_fifo_chan_put(device->fifo, flags, &chan);
+ nvkm_chan_put(&chan, flags);
}
static int
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/Kbuild b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/Kbuild
index 5e831d3..5a074b9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/Kbuild
@@ -1,11 +1,18 @@
# SPDX-License-Identifier: MIT
nvkm-y += nvkm/engine/fifo/base.o
+nvkm-y += nvkm/engine/fifo/cgrp.o
+nvkm-y += nvkm/engine/fifo/chan.o
+nvkm-y += nvkm/engine/fifo/chid.o
+nvkm-y += nvkm/engine/fifo/runl.o
+nvkm-y += nvkm/engine/fifo/runq.o
+
nvkm-y += nvkm/engine/fifo/nv04.o
nvkm-y += nvkm/engine/fifo/nv10.o
nvkm-y += nvkm/engine/fifo/nv17.o
nvkm-y += nvkm/engine/fifo/nv40.o
nvkm-y += nvkm/engine/fifo/nv50.o
nvkm-y += nvkm/engine/fifo/g84.o
+nvkm-y += nvkm/engine/fifo/g98.o
nvkm-y += nvkm/engine/fifo/gf100.o
nvkm-y += nvkm/engine/fifo/gk104.o
nvkm-y += nvkm/engine/fifo/gk110.o
@@ -13,28 +20,11 @@
nvkm-y += nvkm/engine/fifo/gk20a.o
nvkm-y += nvkm/engine/fifo/gm107.o
nvkm-y += nvkm/engine/fifo/gm200.o
-nvkm-y += nvkm/engine/fifo/gm20b.o
nvkm-y += nvkm/engine/fifo/gp100.o
-nvkm-y += nvkm/engine/fifo/gp10b.o
nvkm-y += nvkm/engine/fifo/gv100.o
nvkm-y += nvkm/engine/fifo/tu102.o
+nvkm-y += nvkm/engine/fifo/ga100.o
nvkm-y += nvkm/engine/fifo/ga102.o
-nvkm-y += nvkm/engine/fifo/chan.o
-nvkm-y += nvkm/engine/fifo/channv50.o
-nvkm-y += nvkm/engine/fifo/chang84.o
-
-nvkm-y += nvkm/engine/fifo/dmanv04.o
-nvkm-y += nvkm/engine/fifo/dmanv10.o
-nvkm-y += nvkm/engine/fifo/dmanv17.o
-nvkm-y += nvkm/engine/fifo/dmanv40.o
-
-nvkm-y += nvkm/engine/fifo/gpfifonv50.o
-nvkm-y += nvkm/engine/fifo/gpfifog84.o
-nvkm-y += nvkm/engine/fifo/gpfifogf100.o
-nvkm-y += nvkm/engine/fifo/gpfifogk104.o
-nvkm-y += nvkm/engine/fifo/gpfifogv100.o
-nvkm-y += nvkm/engine/fifo/gpfifotu102.o
-
-nvkm-y += nvkm/engine/fifo/usergv100.o
-nvkm-y += nvkm/engine/fifo/usertu102.o
+nvkm-y += nvkm/engine/fifo/ucgrp.o
+nvkm-y += nvkm/engine/fifo/uchan.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c
index 58b8df7..5ea9a2f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c
@@ -23,25 +23,32 @@
*/
#include "priv.h"
#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+#include "runq.h"
-#include <core/client.h>
#include <core/gpuobj.h>
-#include <core/notify.h>
+#include <subdev/bar.h>
#include <subdev/mc.h>
+#include <subdev/mmu.h>
-#include <nvif/event.h>
#include <nvif/cl0080.h>
#include <nvif/unpack.h>
-void
-nvkm_fifo_recover_chan(struct nvkm_fifo *fifo, int chid)
+bool
+nvkm_fifo_ctxsw_in_progress(struct nvkm_engine *engine)
{
- unsigned long flags;
- if (WARN_ON(!fifo->func->recover_chan))
- return;
- spin_lock_irqsave(&fifo->lock, flags);
- fifo->func->recover_chan(fifo, chid);
- spin_unlock_irqrestore(&fifo->lock, flags);
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+
+ nvkm_runl_foreach(runl, engine->subdev.device->fifo) {
+ nvkm_runl_foreach_engn(engn, runl) {
+ if (engn->engine == engine)
+ return engn->func->chsw ? engn->func->chsw(engn) : false;
+ }
+ }
+
+ return false;
}
void
@@ -59,186 +66,53 @@ nvkm_fifo_start(struct nvkm_fifo *fifo, unsigned long *flags)
void
nvkm_fifo_fault(struct nvkm_fifo *fifo, struct nvkm_fault_data *info)
{
- return fifo->func->fault(fifo, info);
-}
-
-void
-nvkm_fifo_chan_put(struct nvkm_fifo *fifo, unsigned long flags,
- struct nvkm_fifo_chan **pchan)
-{
- struct nvkm_fifo_chan *chan = *pchan;
- if (likely(chan)) {
- *pchan = NULL;
- spin_unlock_irqrestore(&fifo->lock, flags);
- }
-}
-
-struct nvkm_fifo_chan *
-nvkm_fifo_chan_inst_locked(struct nvkm_fifo *fifo, u64 inst)
-{
- struct nvkm_fifo_chan *chan;
- list_for_each_entry(chan, &fifo->chan, head) {
- if (chan->inst->addr == inst) {
- list_del(&chan->head);
- list_add(&chan->head, &fifo->chan);
- return chan;
- }
- }
- return NULL;
-}
-
-struct nvkm_fifo_chan *
-nvkm_fifo_chan_inst(struct nvkm_fifo *fifo, u64 inst, unsigned long *rflags)
-{
- struct nvkm_fifo_chan *chan;
- unsigned long flags;
- spin_lock_irqsave(&fifo->lock, flags);
- if ((chan = nvkm_fifo_chan_inst_locked(fifo, inst))) {
- *rflags = flags;
- return chan;
- }
- spin_unlock_irqrestore(&fifo->lock, flags);
- return NULL;
-}
-
-struct nvkm_fifo_chan *
-nvkm_fifo_chan_chid(struct nvkm_fifo *fifo, int chid, unsigned long *rflags)
-{
- struct nvkm_fifo_chan *chan;
- unsigned long flags;
- spin_lock_irqsave(&fifo->lock, flags);
- list_for_each_entry(chan, &fifo->chan, head) {
- if (chan->chid == chid) {
- list_del(&chan->head);
- list_add(&chan->head, &fifo->chan);
- *rflags = flags;
- return chan;
- }
- }
- spin_unlock_irqrestore(&fifo->lock, flags);
- return NULL;
-}
-
-void
-nvkm_fifo_kevent(struct nvkm_fifo *fifo, int chid)
-{
- nvkm_event_send(&fifo->kevent, 1, chid, NULL, 0);
+ return fifo->func->mmu_fault->recover(fifo, info);
}
static int
-nvkm_fifo_kevent_ctor(struct nvkm_object *object, void *data, u32 size,
- struct nvkm_notify *notify)
+nvkm_fifo_class_new(struct nvkm_device *device, const struct nvkm_oclass *oclass,
+ void *argv, u32 argc, struct nvkm_object **pobject)
{
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(object);
- if (size == 0) {
- notify->size = 0;
- notify->types = 1;
- notify->index = chan->chid;
- return 0;
- }
+ struct nvkm_fifo *fifo = nvkm_fifo(oclass->engine);
+
+ if (oclass->engn == &fifo->func->cgrp.user)
+ return nvkm_ucgrp_new(fifo, oclass, argv, argc, pobject);
+
+ if (oclass->engn == &fifo->func->chan.user)
+ return nvkm_uchan_new(fifo, NULL, oclass, argv, argc, pobject);
+
+ WARN_ON(1);
return -ENOSYS;
}
-static const struct nvkm_event_func
-nvkm_fifo_kevent_func = {
- .ctor = nvkm_fifo_kevent_ctor,
-};
-
-static void
-nvkm_fifo_uevent_fini(struct nvkm_event *event, int type, int index)
-{
- struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), uevent);
- fifo->func->uevent_fini(fifo);
-}
-
-static void
-nvkm_fifo_uevent_init(struct nvkm_event *event, int type, int index)
-{
- struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), uevent);
- fifo->func->uevent_init(fifo);
-}
-
-static int
-nvkm_fifo_uevent_ctor(struct nvkm_object *object, void *data, u32 size,
- struct nvkm_notify *notify)
-{
- union {
- struct nvif_notify_uevent_req none;
- } *req = data;
- int ret = -ENOSYS;
-
- if (!(ret = nvif_unvers(ret, &data, &size, req->none))) {
- notify->size = sizeof(struct nvif_notify_uevent_rep);
- notify->types = 1;
- notify->index = 0;
- }
-
- return ret;
-}
-
-static const struct nvkm_event_func
-nvkm_fifo_uevent_func = {
- .ctor = nvkm_fifo_uevent_ctor,
- .init = nvkm_fifo_uevent_init,
- .fini = nvkm_fifo_uevent_fini,
-};
-
-void
-nvkm_fifo_uevent(struct nvkm_fifo *fifo)
-{
- struct nvif_notify_uevent_rep rep = {
- };
- nvkm_event_send(&fifo->uevent, 1, 0, &rep, sizeof(rep));
-}
-
-static int
-nvkm_fifo_class_new_(struct nvkm_device *device,
- const struct nvkm_oclass *oclass, void *data, u32 size,
- struct nvkm_object **pobject)
-{
- struct nvkm_fifo *fifo = nvkm_fifo(oclass->engine);
- return fifo->func->class_new(fifo, oclass, data, size, pobject);
-}
-
-static const struct nvkm_device_oclass
-nvkm_fifo_class_ = {
- .ctor = nvkm_fifo_class_new_,
-};
-
-static int
-nvkm_fifo_class_new(struct nvkm_device *device,
- const struct nvkm_oclass *oclass, void *data, u32 size,
- struct nvkm_object **pobject)
-{
- const struct nvkm_fifo_chan_oclass *sclass = oclass->engn;
- struct nvkm_fifo *fifo = nvkm_fifo(oclass->engine);
- return sclass->ctor(fifo, oclass, data, size, pobject);
-}
-
static const struct nvkm_device_oclass
nvkm_fifo_class = {
.ctor = nvkm_fifo_class_new,
};
static int
-nvkm_fifo_class_get(struct nvkm_oclass *oclass, int index,
- const struct nvkm_device_oclass **class)
+nvkm_fifo_class_get(struct nvkm_oclass *oclass, int index, const struct nvkm_device_oclass **class)
{
struct nvkm_fifo *fifo = nvkm_fifo(oclass->engine);
- const struct nvkm_fifo_chan_oclass *sclass;
+ const struct nvkm_fifo_func_cgrp *cgrp = &fifo->func->cgrp;
+ const struct nvkm_fifo_func_chan *chan = &fifo->func->chan;
int c = 0;
- if (fifo->func->class_get) {
- int ret = fifo->func->class_get(fifo, index, oclass);
- if (ret == 0)
- *class = &nvkm_fifo_class_;
- return ret;
+ /* *_CHANNEL_GROUP_* */
+ if (cgrp->user.oclass) {
+ if (c++ == index) {
+ oclass->base = cgrp->user;
+ oclass->engn = &fifo->func->cgrp.user;
+ *class = &nvkm_fifo_class;
+ return 0;
+ }
}
- while ((sclass = fifo->func->chan[c])) {
+ /* *_CHANNEL_DMA, *_CHANNEL_GPFIFO_* */
+ if (chan->user.oclass) {
if (c++ == index) {
- oclass->base = sclass->base;
- oclass->engn = sclass;
+ oclass->base = chan->user;
+ oclass->engn = &fifo->func->chan.user;
*class = &nvkm_fifo_class;
return 0;
}
@@ -247,19 +121,47 @@ nvkm_fifo_class_get(struct nvkm_oclass *oclass, int index,
return c;
}
-static void
-nvkm_fifo_intr(struct nvkm_engine *engine)
-{
- struct nvkm_fifo *fifo = nvkm_fifo(engine);
- fifo->func->intr(fifo);
-}
-
static int
nvkm_fifo_fini(struct nvkm_engine *engine, bool suspend)
{
struct nvkm_fifo *fifo = nvkm_fifo(engine);
- if (fifo->func->fini)
- fifo->func->fini(fifo);
+ struct nvkm_runl *runl;
+
+ nvkm_inth_block(&fifo->engine.subdev.inth);
+
+ nvkm_runl_foreach(runl, fifo)
+ nvkm_runl_fini(runl);
+
+ return 0;
+}
+
+static int
+nvkm_fifo_init(struct nvkm_engine *engine)
+{
+ struct nvkm_fifo *fifo = nvkm_fifo(engine);
+ struct nvkm_runq *runq;
+ struct nvkm_runl *runl;
+ u32 mask = 0;
+
+ if (fifo->func->init_pbdmas) {
+ nvkm_runq_foreach(runq, fifo)
+ mask |= BIT(runq->id);
+
+ fifo->func->init_pbdmas(fifo, mask);
+
+ nvkm_runq_foreach(runq, fifo)
+ runq->func->init(runq);
+ }
+
+ nvkm_runl_foreach(runl, fifo) {
+ if (runl->func->init)
+ runl->func->init(runl);
+ }
+
+ if (fifo->func->init)
+ fifo->func->init(fifo);
+
+ nvkm_inth_allow(&fifo->engine.subdev.inth);
return 0;
}
@@ -267,22 +169,146 @@ static int
nvkm_fifo_info(struct nvkm_engine *engine, u64 mthd, u64 *data)
{
struct nvkm_fifo *fifo = nvkm_fifo(engine);
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+ int ret;
+
+ ret = nvkm_subdev_oneinit(&fifo->engine.subdev);
+ if (ret)
+ return ret;
+
switch (mthd) {
- case NV_DEVICE_HOST_CHANNELS: *data = fifo->nr; return 0;
+ case NV_DEVICE_HOST_CHANNELS: *data = fifo->chid ? fifo->chid->nr : 0; return 0;
+ case NV_DEVICE_HOST_RUNLISTS:
+ *data = 0;
+ nvkm_runl_foreach(runl, fifo)
+ *data |= BIT(runl->id);
+ return 0;
+ case NV_DEVICE_HOST_RUNLIST_ENGINES:
+ runl = nvkm_runl_get(fifo, *data, 0);
+ if (runl) {
+ *data = 0;
+ nvkm_runl_foreach_engn(engn, runl) {
+#define CASE(n) case NVKM_ENGINE_##n: *data |= NV_DEVICE_HOST_RUNLIST_ENGINES_##n; break
+ switch (engn->engine->subdev.type) {
+ case NVKM_ENGINE_DMAOBJ:
+ break;
+ CASE(SW );
+ CASE(GR );
+ CASE(MPEG );
+ CASE(ME );
+ CASE(CIPHER);
+ CASE(BSP );
+ CASE(VP );
+ CASE(CE );
+ CASE(SEC );
+ CASE(MSVLD );
+ CASE(MSPDEC);
+ CASE(MSPPP );
+ CASE(MSENC );
+ CASE(VIC );
+ CASE(SEC2 );
+ CASE(NVDEC );
+ CASE(NVENC );
+ default:
+ WARN_ON(1);
+ break;
+ }
+#undef CASE
+ }
+ return 0;
+ }
+ return -EINVAL;
+ case NV_DEVICE_HOST_RUNLIST_CHANNELS:
+ if (!fifo->chid) {
+ runl = nvkm_runl_get(fifo, *data, 0);
+ if (runl) {
+ *data = runl->chid->nr;
+ return 0;
+ }
+ }
+ return -EINVAL;
default:
- if (fifo->func->info)
- return fifo->func->info(fifo, mthd, data);
break;
}
+
return -ENOSYS;
}
static int
nvkm_fifo_oneinit(struct nvkm_engine *engine)
{
+ struct nvkm_subdev *subdev = &engine->subdev;
+ struct nvkm_device *device = subdev->device;
struct nvkm_fifo *fifo = nvkm_fifo(engine);
- if (fifo->func->oneinit)
- return fifo->func->oneinit(fifo);
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+ int ret, nr, i;
+
+ /* Initialise CHID/CGID allocator(s) on GPUs where they aren't per-runlist. */
+ if (fifo->func->chid_nr) {
+ ret = fifo->func->chid_ctor(fifo, fifo->func->chid_nr(fifo));
+ if (ret)
+ return ret;
+ }
+
+ /* Create runqueues for each PBDMA. */
+ if (fifo->func->runq_nr) {
+ for (nr = fifo->func->runq_nr(fifo), i = 0; i < nr; i++) {
+ if (!nvkm_runq_new(fifo, i))
+ return -ENOMEM;
+ }
+ }
+
+ /* Create runlists. */
+ ret = fifo->func->runl_ctor(fifo);
+ if (ret)
+ return ret;
+
+ nvkm_runl_foreach(runl, fifo) {
+ RUNL_DEBUG(runl, "chan:%06x", runl->chan);
+ nvkm_runl_foreach_engn(engn, runl) {
+ ENGN_DEBUG(engn, "");
+ }
+ }
+
+ /* Register interrupt handler. */
+ if (fifo->func->intr) {
+ ret = nvkm_inth_add(&device->mc->intr, NVKM_INTR_SUBDEV, NVKM_INTR_PRIO_NORMAL,
+ subdev, fifo->func->intr, &subdev->inth);
+ if (ret) {
+ nvkm_error(subdev, "intr %d\n", ret);
+ return ret;
+ }
+ }
+
+ /* Initialise non-stall intr handling. */
+ if (fifo->func->nonstall_ctor) {
+ ret = fifo->func->nonstall_ctor(fifo);
+ if (ret) {
+ nvkm_error(subdev, "nonstall %d\n", ret);
+ }
+ }
+
+ /* Allocate USERD + BAR1 polling area. */
+ if (fifo->func->chan.func->userd->bar == 1) {
+ struct nvkm_vmm *bar1 = nvkm_bar_bar1_vmm(device);
+
+ ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, fifo->chid->nr *
+ fifo->func->chan.func->userd->size, 0, true,
+ &fifo->userd.mem);
+ if (ret)
+ return ret;
+
+ ret = nvkm_vmm_get(bar1, 12, nvkm_memory_size(fifo->userd.mem), &fifo->userd.bar1);
+ if (ret)
+ return ret;
+
+ ret = nvkm_memory_map(fifo->userd.mem, 0, bar1, fifo->userd.bar1, NULL, 0);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
@@ -292,25 +318,28 @@ nvkm_fifo_preinit(struct nvkm_engine *engine)
nvkm_mc_reset(engine->subdev.device, NVKM_ENGINE_FIFO, 0);
}
-static int
-nvkm_fifo_init(struct nvkm_engine *engine)
-{
- struct nvkm_fifo *fifo = nvkm_fifo(engine);
- fifo->func->init(fifo);
- return 0;
-}
-
static void *
nvkm_fifo_dtor(struct nvkm_engine *engine)
{
struct nvkm_fifo *fifo = nvkm_fifo(engine);
- void *data = fifo;
- if (fifo->func->dtor)
- data = fifo->func->dtor(fifo);
- nvkm_event_fini(&fifo->kevent);
- nvkm_event_fini(&fifo->uevent);
+ struct nvkm_runl *runl, *runt;
+ struct nvkm_runq *runq, *rtmp;
+
+ if (fifo->userd.bar1)
+ nvkm_vmm_put(nvkm_bar_bar1_vmm(engine->subdev.device), &fifo->userd.bar1);
+ nvkm_memory_unref(&fifo->userd.mem);
+
+ list_for_each_entry_safe(runl, runt, &fifo->runls, head)
+ nvkm_runl_del(runl);
+ list_for_each_entry_safe(runq, rtmp, &fifo->runqs, head)
+ nvkm_runq_del(runq);
+
+ nvkm_chid_unref(&fifo->cgid);
+ nvkm_chid_unref(&fifo->chid);
+
+ nvkm_event_fini(&fifo->nonstall.event);
mutex_destroy(&fifo->mutex);
- return data;
+ return fifo;
}
static const struct nvkm_engine_func
@@ -321,37 +350,40 @@ nvkm_fifo = {
.info = nvkm_fifo_info,
.init = nvkm_fifo_init,
.fini = nvkm_fifo_fini,
- .intr = nvkm_fifo_intr,
.base.sclass = nvkm_fifo_class_get,
};
int
-nvkm_fifo_ctor(const struct nvkm_fifo_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, int nr, struct nvkm_fifo *fifo)
+nvkm_fifo_new_(const struct nvkm_fifo_func *func, struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_fifo **pfifo)
{
+ struct nvkm_fifo *fifo;
int ret;
+ if (!(fifo = *pfifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
+ return -ENOMEM;
+
fifo->func = func;
- INIT_LIST_HEAD(&fifo->chan);
+ INIT_LIST_HEAD(&fifo->runqs);
+ INIT_LIST_HEAD(&fifo->runls);
+ /*TODO: Needs to be >CTXSW_TIMEOUT, so RC can recover before this is hit.
+ * CTXSW_TIMEOUT HW default seems to differ between GPUs, so just a
+ * large number for now until we support changing it.
+ */
+ fifo->timeout.chan_msec = 10000;
spin_lock_init(&fifo->lock);
mutex_init(&fifo->mutex);
- if (WARN_ON(fifo->nr > NVKM_FIFO_CHID_NR))
- fifo->nr = NVKM_FIFO_CHID_NR;
- else
- fifo->nr = nr;
- bitmap_clear(fifo->mask, 0, fifo->nr);
-
ret = nvkm_engine_ctor(&nvkm_fifo, device, type, inst, true, &fifo->engine);
if (ret)
return ret;
- if (func->uevent_init) {
- ret = nvkm_event_init(&nvkm_fifo_uevent_func, 1, 1,
- &fifo->uevent);
+ if (func->nonstall) {
+ ret = nvkm_event_init(func->nonstall, &fifo->engine.subdev, 1, 1,
+ &fifo->nonstall.event);
if (ret)
return ret;
}
- return nvkm_event_init(&nvkm_fifo_kevent_func, 1, nr, &fifo->kevent);
+ return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.c
new file mode 100644
index 0000000..ea53fb3
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.c
@@ -0,0 +1,252 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+#include "priv.h"
+
+#include <core/gpuobj.h>
+#include <subdev/mmu.h>
+
+static void
+nvkm_cgrp_ectx_put(struct nvkm_cgrp *cgrp, struct nvkm_ectx **pectx)
+{
+ struct nvkm_ectx *ectx = *pectx;
+
+ if (ectx) {
+ struct nvkm_engn *engn = ectx->engn;
+
+ if (refcount_dec_and_test(&ectx->refs)) {
+ CGRP_TRACE(cgrp, "dtor ectx %d[%s]", engn->id, engn->engine->subdev.name);
+ nvkm_object_del(&ectx->object);
+ list_del(&ectx->head);
+ kfree(ectx);
+ }
+
+ *pectx = NULL;
+ }
+}
+
+static int
+nvkm_cgrp_ectx_get(struct nvkm_cgrp *cgrp, struct nvkm_engn *engn, struct nvkm_ectx **pectx,
+ struct nvkm_chan *chan, struct nvkm_client *client)
+{
+ struct nvkm_engine *engine = engn->engine;
+ struct nvkm_oclass cclass = {
+ .client = client,
+ .engine = engine,
+ };
+ struct nvkm_ectx *ectx;
+ int ret = 0;
+
+ /* Look for an existing context for this engine in the channel group. */
+ ectx = nvkm_list_find(ectx, &cgrp->ectxs, head, ectx->engn == engn);
+ if (ectx) {
+ refcount_inc(&ectx->refs);
+ *pectx = ectx;
+ return 0;
+ }
+
+ /* Nope - create a fresh one. */
+ CGRP_TRACE(cgrp, "ctor ectx %d[%s]", engn->id, engn->engine->subdev.name);
+ if (!(ectx = *pectx = kzalloc(sizeof(*ectx), GFP_KERNEL)))
+ return -ENOMEM;
+
+ ectx->engn = engn;
+ refcount_set(&ectx->refs, 1);
+ refcount_set(&ectx->uses, 0);
+ list_add_tail(&ectx->head, &cgrp->ectxs);
+
+ /* Allocate the HW structures. */
+ if (engine->func->fifo.cclass)
+ ret = engine->func->fifo.cclass(chan, &cclass, &ectx->object);
+ else if (engine->func->cclass)
+ ret = nvkm_object_new_(engine->func->cclass, &cclass, NULL, 0, &ectx->object);
+
+ if (ret)
+ nvkm_cgrp_ectx_put(cgrp, pectx);
+
+ return ret;
+}
+
+void
+nvkm_cgrp_vctx_put(struct nvkm_cgrp *cgrp, struct nvkm_vctx **pvctx)
+{
+ struct nvkm_vctx *vctx = *pvctx;
+
+ if (vctx) {
+ struct nvkm_engn *engn = vctx->ectx->engn;
+
+ if (refcount_dec_and_test(&vctx->refs)) {
+ CGRP_TRACE(cgrp, "dtor vctx %d[%s]", engn->id, engn->engine->subdev.name);
+ nvkm_vmm_put(vctx->vmm, &vctx->vma);
+ nvkm_gpuobj_del(&vctx->inst);
+
+ nvkm_cgrp_ectx_put(cgrp, &vctx->ectx);
+ if (vctx->vmm) {
+ atomic_dec(&vctx->vmm->engref[engn->engine->subdev.type]);
+ nvkm_vmm_unref(&vctx->vmm);
+ }
+ list_del(&vctx->head);
+ kfree(vctx);
+ }
+
+ *pvctx = NULL;
+ }
+}
+
+int
+nvkm_cgrp_vctx_get(struct nvkm_cgrp *cgrp, struct nvkm_engn *engn, struct nvkm_chan *chan,
+ struct nvkm_vctx **pvctx, struct nvkm_client *client)
+{
+ struct nvkm_ectx *ectx;
+ struct nvkm_vctx *vctx;
+ int ret;
+
+ /* Look for an existing sub-context for this engine+VEID in the channel group. */
+ vctx = nvkm_list_find(vctx, &cgrp->vctxs, head,
+ vctx->ectx->engn == engn && vctx->vmm == chan->vmm);
+ if (vctx) {
+ refcount_inc(&vctx->refs);
+ *pvctx = vctx;
+ return 0;
+ }
+
+ /* Nope - create a fresh one. But, context first. */
+ ret = nvkm_cgrp_ectx_get(cgrp, engn, &ectx, chan, client);
+ if (ret) {
+ CGRP_ERROR(cgrp, "ectx %d[%s]: %d", engn->id, engn->engine->subdev.name, ret);
+ return ret;
+ }
+
+ /* Now, create the sub-context. */
+ CGRP_TRACE(cgrp, "ctor vctx %d[%s]", engn->id, engn->engine->subdev.name);
+ if (!(vctx = *pvctx = kzalloc(sizeof(*vctx), GFP_KERNEL))) {
+ nvkm_cgrp_ectx_put(cgrp, &ectx);
+ return -ENOMEM;
+ }
+
+ vctx->ectx = ectx;
+ vctx->vmm = nvkm_vmm_ref(chan->vmm);
+ refcount_set(&vctx->refs, 1);
+ list_add_tail(&vctx->head, &cgrp->vctxs);
+
+ /* MMU on some GPUs needs to know engine usage for TLB invalidation. */
+ if (vctx->vmm)
+ atomic_inc(&vctx->vmm->engref[engn->engine->subdev.type]);
+
+ /* Allocate the HW structures. */
+ if (engn->func->bind) {
+ ret = nvkm_object_bind(vctx->ectx->object, NULL, 0, &vctx->inst);
+ if (ret == 0 && engn->func->ctor)
+ ret = engn->func->ctor(engn, vctx);
+ }
+
+ if (ret)
+ nvkm_cgrp_vctx_put(cgrp, pvctx);
+
+ return ret;
+}
+
+static void
+nvkm_cgrp_del(struct kref *kref)
+{
+ struct nvkm_cgrp *cgrp = container_of(kref, typeof(*cgrp), kref);
+ struct nvkm_runl *runl = cgrp->runl;
+
+ if (runl->cgid)
+ nvkm_chid_put(runl->cgid, cgrp->id, &cgrp->lock);
+
+ mutex_destroy(&cgrp->mutex);
+ nvkm_vmm_unref(&cgrp->vmm);
+ kfree(cgrp);
+}
+
+void
+nvkm_cgrp_unref(struct nvkm_cgrp **pcgrp)
+{
+ struct nvkm_cgrp *cgrp = *pcgrp;
+
+ if (!cgrp)
+ return;
+
+ kref_put(&cgrp->kref, nvkm_cgrp_del);
+ *pcgrp = NULL;
+}
+
+struct nvkm_cgrp *
+nvkm_cgrp_ref(struct nvkm_cgrp *cgrp)
+{
+ if (cgrp)
+ kref_get(&cgrp->kref);
+
+ return cgrp;
+}
+
+void
+nvkm_cgrp_put(struct nvkm_cgrp **pcgrp, unsigned long irqflags)
+{
+ struct nvkm_cgrp *cgrp = *pcgrp;
+
+ if (!cgrp)
+ return;
+
+ *pcgrp = NULL;
+ spin_unlock_irqrestore(&cgrp->lock, irqflags);
+}
+
+int
+nvkm_cgrp_new(struct nvkm_runl *runl, const char *name, struct nvkm_vmm *vmm, bool hw,
+ struct nvkm_cgrp **pcgrp)
+{
+ struct nvkm_cgrp *cgrp;
+
+ if (!(cgrp = *pcgrp = kmalloc(sizeof(*cgrp), GFP_KERNEL)))
+ return -ENOMEM;
+
+ cgrp->func = runl->fifo->func->cgrp.func;
+ strscpy(cgrp->name, name, sizeof(cgrp->name));
+ cgrp->runl = runl;
+ cgrp->vmm = nvkm_vmm_ref(vmm);
+ cgrp->hw = hw;
+ cgrp->id = -1;
+ kref_init(&cgrp->kref);
+ INIT_LIST_HEAD(&cgrp->chans);
+ cgrp->chan_nr = 0;
+ spin_lock_init(&cgrp->lock);
+ INIT_LIST_HEAD(&cgrp->ectxs);
+ INIT_LIST_HEAD(&cgrp->vctxs);
+ mutex_init(&cgrp->mutex);
+ atomic_set(&cgrp->rc, NVKM_CGRP_RC_NONE);
+
+ if (runl->cgid) {
+ cgrp->id = nvkm_chid_get(runl->cgid, cgrp);
+ if (cgrp->id < 0) {
+ RUNL_ERROR(runl, "!cgids");
+ nvkm_cgrp_unref(pcgrp);
+ return -ENOSPC;
+ }
+ }
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.h
index d0ac60b..5f6abd5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.h
@@ -1,11 +1,75 @@
-#ifndef __NVKM_FIFO_CGRP_H__
-#define __NVKM_FIFO_CGRP_H__
-#include "priv.h"
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVKM_CGRP_H__
+#define __NVKM_CGRP_H__
+#include <core/os.h>
+struct nvkm_chan;
+struct nvkm_client;
-struct nvkm_fifo_cgrp {
- int id;
+struct nvkm_vctx {
+ struct nvkm_ectx *ectx;
+ struct nvkm_vmm *vmm;
+ refcount_t refs;
+
+ struct nvkm_gpuobj *inst;
+ struct nvkm_vma *vma;
+
struct list_head head;
- struct list_head chan;
- int chan_nr;
};
+
+struct nvkm_ectx {
+ struct nvkm_engn *engn;
+ refcount_t refs;
+ refcount_t uses;
+
+ struct nvkm_object *object;
+
+ struct list_head head;
+};
+
+struct nvkm_cgrp {
+ const struct nvkm_cgrp_func {
+ void (*preempt)(struct nvkm_cgrp *);
+ } *func;
+ char name[64];
+ struct nvkm_runl *runl;
+ struct nvkm_vmm *vmm;
+ bool hw;
+ int id;
+ struct kref kref;
+
+ struct list_head chans;
+ int chan_nr;
+
+ spinlock_t lock; /* protects irq handler channel (group) lookup */
+
+ struct list_head ectxs;
+ struct list_head vctxs;
+ struct mutex mutex;
+
+#define NVKM_CGRP_RC_NONE 0
+#define NVKM_CGRP_RC_PENDING 1
+#define NVKM_CGRP_RC_RUNNING 2
+ atomic_t rc;
+
+ struct list_head head;
+};
+
+int nvkm_cgrp_new(struct nvkm_runl *, const char *name, struct nvkm_vmm *, bool hw,
+ struct nvkm_cgrp **);
+struct nvkm_cgrp *nvkm_cgrp_ref(struct nvkm_cgrp *);
+void nvkm_cgrp_unref(struct nvkm_cgrp **);
+int nvkm_cgrp_vctx_get(struct nvkm_cgrp *, struct nvkm_engn *, struct nvkm_chan *,
+ struct nvkm_vctx **, struct nvkm_client *);
+void nvkm_cgrp_vctx_put(struct nvkm_cgrp *, struct nvkm_vctx **);
+
+void nvkm_cgrp_put(struct nvkm_cgrp **, unsigned long irqflags);
+
+#define nvkm_cgrp_foreach_chan(chan,cgrp) list_for_each_entry((chan), &(cgrp)->chans, head)
+#define nvkm_cgrp_foreach_chan_safe(chan,ctmp,cgrp) \
+ list_for_each_entry_safe((chan), (ctmp), &(cgrp)->chans, head)
+
+#define CGRP_PRCLI(c,l,p,f,a...) RUNL_PRINT((c)->runl, l, p, "%04x:[%s]"f, (c)->id, (c)->name, ##a)
+#define CGRP_PRINT(c,l,p,f,a...) RUNL_PRINT((c)->runl, l, p, "%04x:"f, (c)->id, ##a)
+#define CGRP_ERROR(c,f,a...) CGRP_PRCLI((c), ERROR, err, " "f"\n", ##a)
+#define CGRP_TRACE(c,f,a...) CGRP_PRINT((c), TRACE, info, " "f"\n", ##a)
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.c
index 2e7f32c..b7c9d61 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.c
@@ -22,285 +22,265 @@
* Authors: Ben Skeggs
*/
#include "chan.h"
+#include "chid.h"
+#include "cgrp.h"
+#include "chid.h"
+#include "runl.h"
+#include "priv.h"
-#include <core/client.h>
-#include <core/gpuobj.h>
-#include <core/oproxy.h>
+#include <core/ramht.h>
#include <subdev/mmu.h>
#include <engine/dma.h>
-struct nvkm_fifo_chan_object {
- struct nvkm_oproxy oproxy;
- struct nvkm_fifo_chan *chan;
- int hash;
+#include <nvif/if0020.h>
+
+const struct nvkm_event_func
+nvkm_chan_event = {
};
-static struct nvkm_fifo_engn *
-nvkm_fifo_chan_engn(struct nvkm_fifo_chan *chan, struct nvkm_engine *engine)
+void
+nvkm_chan_cctx_bind(struct nvkm_chan *chan, struct nvkm_engn *engn, struct nvkm_cctx *cctx)
{
- int engi = chan->fifo->func->engine_id(chan->fifo, engine);
- if (engi >= 0)
- return &chan->engn[engi];
- return NULL;
+ struct nvkm_cgrp *cgrp = chan->cgrp;
+ struct nvkm_runl *runl = cgrp->runl;
+ struct nvkm_engine *engine = engn->engine;
+
+ if (!engn->func->bind)
+ return;
+
+ CHAN_TRACE(chan, "%sbind cctx %d[%s]", cctx ? "" : "un", engn->id, engine->subdev.name);
+
+ /* Prevent any channel in channel group from being rescheduled, kick them
+ * off host and any engine(s) they're loaded on.
+ */
+ if (cgrp->hw)
+ nvkm_runl_block(runl);
+ else
+ nvkm_chan_block(chan);
+ nvkm_chan_preempt(chan, true);
+
+ /* Update context pointer. */
+ engn->func->bind(engn, cctx, chan);
+
+ /* Resume normal operation. */
+ if (cgrp->hw)
+ nvkm_runl_allow(runl);
+ else
+ nvkm_chan_allow(chan);
}
-static int
-nvkm_fifo_chan_child_fini(struct nvkm_oproxy *base, bool suspend)
+void
+nvkm_chan_cctx_put(struct nvkm_chan *chan, struct nvkm_cctx **pcctx)
{
- struct nvkm_fifo_chan_object *object =
- container_of(base, typeof(*object), oproxy);
- struct nvkm_engine *engine = object->oproxy.object->engine;
- struct nvkm_fifo_chan *chan = object->chan;
- struct nvkm_fifo_engn *engn = nvkm_fifo_chan_engn(chan, engine);
- const char *name = engine->subdev.name;
- int ret = 0;
+ struct nvkm_cctx *cctx = *pcctx;
- if (--engn->usecount)
- return 0;
+ if (cctx) {
+ struct nvkm_engn *engn = cctx->vctx->ectx->engn;
- if (chan->func->engine_fini) {
- ret = chan->func->engine_fini(chan, engine, suspend);
- if (ret) {
- nvif_error(&chan->object,
- "detach %s failed, %d\n", name, ret);
- return ret;
+ if (refcount_dec_and_mutex_lock(&cctx->refs, &chan->cgrp->mutex)) {
+ CHAN_TRACE(chan, "dtor cctx %d[%s]", engn->id, engn->engine->subdev.name);
+ nvkm_cgrp_vctx_put(chan->cgrp, &cctx->vctx);
+ list_del(&cctx->head);
+ kfree(cctx);
+ mutex_unlock(&chan->cgrp->mutex);
}
+
+ *pcctx = NULL;
+ }
+}
+
+int
+nvkm_chan_cctx_get(struct nvkm_chan *chan, struct nvkm_engn *engn, struct nvkm_cctx **pcctx,
+ struct nvkm_client *client)
+{
+ struct nvkm_cgrp *cgrp = chan->cgrp;
+ struct nvkm_vctx *vctx;
+ struct nvkm_cctx *cctx;
+ int ret;
+
+ /* Look for an existing channel context for this engine+VEID. */
+ mutex_lock(&cgrp->mutex);
+ cctx = nvkm_list_find(cctx, &chan->cctxs, head,
+ cctx->vctx->ectx->engn == engn && cctx->vctx->vmm == chan->vmm);
+ if (cctx) {
+ refcount_inc(&cctx->refs);
+ *pcctx = cctx;
+ mutex_unlock(&chan->cgrp->mutex);
+ return 0;
}
- if (engn->object) {
- ret = nvkm_object_fini(engn->object, suspend);
- if (ret && suspend)
- return ret;
+ /* Nope - create a fresh one. But, sub-context first. */
+ ret = nvkm_cgrp_vctx_get(cgrp, engn, chan, &vctx, client);
+ if (ret) {
+ CHAN_ERROR(chan, "vctx %d[%s]: %d", engn->id, engn->engine->subdev.name, ret);
+ goto done;
}
- nvif_trace(&chan->object, "detached %s\n", name);
+ /* Now, create the channel context - to track engine binding. */
+ CHAN_TRACE(chan, "ctor cctx %d[%s]", engn->id, engn->engine->subdev.name);
+ if (!(cctx = *pcctx = kzalloc(sizeof(*cctx), GFP_KERNEL))) {
+ nvkm_cgrp_vctx_put(cgrp, &vctx);
+ ret = -ENOMEM;
+ goto done;
+ }
+
+ cctx->vctx = vctx;
+ refcount_set(&cctx->refs, 1);
+ refcount_set(&cctx->uses, 0);
+ list_add_tail(&cctx->head, &chan->cctxs);
+done:
+ mutex_unlock(&cgrp->mutex);
return ret;
}
-static int
-nvkm_fifo_chan_child_init(struct nvkm_oproxy *base)
+int
+nvkm_chan_preempt_locked(struct nvkm_chan *chan, bool wait)
{
- struct nvkm_fifo_chan_object *object =
- container_of(base, typeof(*object), oproxy);
- struct nvkm_engine *engine = object->oproxy.object->engine;
- struct nvkm_fifo_chan *chan = object->chan;
- struct nvkm_fifo_engn *engn = nvkm_fifo_chan_engn(chan, engine);
- const char *name = engine->subdev.name;
- int ret;
+ struct nvkm_runl *runl = chan->cgrp->runl;
- if (engn->usecount++)
+ CHAN_TRACE(chan, "preempt");
+ chan->func->preempt(chan);
+ if (!wait)
return 0;
- if (engn->object) {
- ret = nvkm_object_init(engn->object);
- if (ret)
- return ret;
+ return nvkm_runl_preempt_wait(runl);
+}
+
+int
+nvkm_chan_preempt(struct nvkm_chan *chan, bool wait)
+{
+ int ret;
+
+ if (!chan->func->preempt)
+ return 0;
+
+ mutex_lock(&chan->cgrp->runl->mutex);
+ ret = nvkm_chan_preempt_locked(chan, wait);
+ mutex_unlock(&chan->cgrp->runl->mutex);
+ return ret;
+}
+
+void
+nvkm_chan_remove_locked(struct nvkm_chan *chan)
+{
+ struct nvkm_cgrp *cgrp = chan->cgrp;
+ struct nvkm_runl *runl = cgrp->runl;
+
+ if (list_empty(&chan->head))
+ return;
+
+ CHAN_TRACE(chan, "remove");
+ if (!--cgrp->chan_nr) {
+ runl->cgrp_nr--;
+ list_del(&cgrp->head);
+ }
+ runl->chan_nr--;
+ list_del_init(&chan->head);
+ atomic_set(&runl->changed, 1);
+}
+
+void
+nvkm_chan_remove(struct nvkm_chan *chan, bool preempt)
+{
+ struct nvkm_runl *runl = chan->cgrp->runl;
+
+ mutex_lock(&runl->mutex);
+ if (preempt && chan->func->preempt)
+ nvkm_chan_preempt_locked(chan, true);
+ nvkm_chan_remove_locked(chan);
+ nvkm_runl_update_locked(runl, true);
+ mutex_unlock(&runl->mutex);
+}
+
+void
+nvkm_chan_insert(struct nvkm_chan *chan)
+{
+ struct nvkm_cgrp *cgrp = chan->cgrp;
+ struct nvkm_runl *runl = cgrp->runl;
+
+ mutex_lock(&runl->mutex);
+ if (WARN_ON(!list_empty(&chan->head))) {
+ mutex_unlock(&runl->mutex);
+ return;
}
- if (chan->func->engine_init) {
- ret = chan->func->engine_init(chan, engine);
- if (ret) {
- nvif_error(&chan->object,
- "attach %s failed, %d\n", name, ret);
- return ret;
- }
+ CHAN_TRACE(chan, "insert");
+ list_add_tail(&chan->head, &cgrp->chans);
+ runl->chan_nr++;
+ if (!cgrp->chan_nr++) {
+ list_add_tail(&cgrp->head, &cgrp->runl->cgrps);
+ runl->cgrp_nr++;
}
-
- nvif_trace(&chan->object, "attached %s\n", name);
- return 0;
+ atomic_set(&runl->changed, 1);
+ nvkm_runl_update_locked(runl, true);
+ mutex_unlock(&runl->mutex);
}
static void
-nvkm_fifo_chan_child_del(struct nvkm_oproxy *base)
+nvkm_chan_block_locked(struct nvkm_chan *chan)
{
- struct nvkm_fifo_chan_object *object =
- container_of(base, typeof(*object), oproxy);
- struct nvkm_engine *engine = object->oproxy.base.engine;
- struct nvkm_fifo_chan *chan = object->chan;
- struct nvkm_fifo_engn *engn = nvkm_fifo_chan_engn(chan, engine);
-
- if (chan->func->object_dtor)
- chan->func->object_dtor(chan, object->hash);
-
- if (!--engn->refcount) {
- if (chan->func->engine_dtor)
- chan->func->engine_dtor(chan, engine);
- nvkm_object_del(&engn->object);
- if (chan->vmm)
- atomic_dec(&chan->vmm->engref[engine->subdev.type]);
- }
+ CHAN_TRACE(chan, "block %d", atomic_read(&chan->blocked));
+ if (atomic_inc_return(&chan->blocked) == 1)
+ chan->func->stop(chan);
}
-static const struct nvkm_oproxy_func
-nvkm_fifo_chan_child_func = {
- .dtor[0] = nvkm_fifo_chan_child_del,
- .init[0] = nvkm_fifo_chan_child_init,
- .fini[0] = nvkm_fifo_chan_child_fini,
-};
-
-static int
-nvkm_fifo_chan_child_new(const struct nvkm_oclass *oclass, void *data, u32 size,
- struct nvkm_object **pobject)
+void
+nvkm_chan_error(struct nvkm_chan *chan, bool preempt)
{
- struct nvkm_engine *engine = oclass->engine;
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(oclass->parent);
- struct nvkm_fifo_engn *engn = nvkm_fifo_chan_engn(chan, engine);
- struct nvkm_fifo_chan_object *object;
- int ret = 0;
-
- if (!(object = kzalloc(sizeof(*object), GFP_KERNEL)))
- return -ENOMEM;
- nvkm_oproxy_ctor(&nvkm_fifo_chan_child_func, oclass, &object->oproxy);
- object->chan = chan;
- *pobject = &object->oproxy.base;
-
- if (!engn->refcount++) {
- struct nvkm_oclass cclass = {
- .client = oclass->client,
- .engine = oclass->engine,
- };
-
- if (chan->vmm)
- atomic_inc(&chan->vmm->engref[engine->subdev.type]);
-
- if (engine->func->fifo.cclass) {
- ret = engine->func->fifo.cclass(chan, &cclass,
- &engn->object);
- } else
- if (engine->func->cclass) {
- ret = nvkm_object_new_(engine->func->cclass, &cclass,
- NULL, 0, &engn->object);
- }
- if (ret)
- return ret;
-
- if (chan->func->engine_ctor) {
- ret = chan->func->engine_ctor(chan, oclass->engine,
- engn->object);
- if (ret)
- return ret;
- }
- }
-
- ret = oclass->base.ctor(&(const struct nvkm_oclass) {
- .base = oclass->base,
- .engn = oclass->engn,
- .handle = oclass->handle,
- .object = oclass->object,
- .client = oclass->client,
- .parent = engn->object ?
- engn->object :
- oclass->parent,
- .engine = engine,
- }, data, size, &object->oproxy.object);
- if (ret)
- return ret;
-
- if (chan->func->object_ctor) {
- object->hash =
- chan->func->object_ctor(chan, object->oproxy.object);
- if (object->hash < 0)
- return object->hash;
- }
-
- return 0;
-}
-
-static int
-nvkm_fifo_chan_child_get(struct nvkm_object *object, int index,
- struct nvkm_oclass *oclass)
-{
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(object);
- struct nvkm_fifo *fifo = chan->fifo;
- struct nvkm_engine *engine;
- u32 engm = chan->engm;
- int engi, ret, c;
-
- for (; c = 0, engi = __ffs(engm), engm; engm &= ~(1ULL << engi)) {
- if (!(engine = fifo->func->id_engine(fifo, engi)))
- continue;
- oclass->engine = engine;
- oclass->base.oclass = 0;
-
- if (engine->func->fifo.sclass) {
- ret = engine->func->fifo.sclass(oclass, index);
- if (oclass->base.oclass) {
- if (!oclass->base.ctor)
- oclass->base.ctor = nvkm_object_new;
- oclass->ctor = nvkm_fifo_chan_child_new;
- return 0;
- }
-
- index -= ret;
- continue;
- }
-
- while (engine->func->sclass[c].oclass) {
- if (c++ == index) {
- oclass->base = engine->func->sclass[index];
- if (!oclass->base.ctor)
- oclass->base.ctor = nvkm_object_new;
- oclass->ctor = nvkm_fifo_chan_child_new;
- return 0;
- }
- }
- index -= c;
- }
-
- return -EINVAL;
-}
-
-static int
-nvkm_fifo_chan_ntfy(struct nvkm_object *object, u32 type,
- struct nvkm_event **pevent)
-{
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(object);
- if (chan->func->ntfy)
- return chan->func->ntfy(chan, type, pevent);
- return -ENODEV;
-}
-
-static int
-nvkm_fifo_chan_map(struct nvkm_object *object, void *argv, u32 argc,
- enum nvkm_object_map *type, u64 *addr, u64 *size)
-{
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(object);
- *type = NVKM_OBJECT_MAP_IO;
- *addr = chan->addr;
- *size = chan->size;
- return 0;
-}
-
-static int
-nvkm_fifo_chan_fini(struct nvkm_object *object, bool suspend)
-{
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(object);
- chan->func->fini(chan);
- return 0;
-}
-
-static int
-nvkm_fifo_chan_init(struct nvkm_object *object)
-{
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(object);
- chan->func->init(chan);
- return 0;
-}
-
-static void *
-nvkm_fifo_chan_dtor(struct nvkm_object *object)
-{
- struct nvkm_fifo_chan *chan = nvkm_fifo_chan(object);
- struct nvkm_fifo *fifo = chan->fifo;
- void *data = chan->func->dtor(chan);
unsigned long flags;
- spin_lock_irqsave(&fifo->lock, flags);
- if (!list_empty(&chan->head)) {
- __clear_bit(chan->chid, fifo->mask);
- list_del(&chan->head);
+ spin_lock_irqsave(&chan->lock, flags);
+ if (atomic_inc_return(&chan->errored) == 1) {
+ CHAN_ERROR(chan, "errored - disabling channel");
+ nvkm_chan_block_locked(chan);
+ if (preempt)
+ chan->func->preempt(chan);
+ nvkm_event_ntfy(&chan->cgrp->runl->chid->event, chan->id, NVKM_CHAN_EVENT_ERRORED);
}
- spin_unlock_irqrestore(&fifo->lock, flags);
+ spin_unlock_irqrestore(&chan->lock, flags);
+}
+
+void
+nvkm_chan_block(struct nvkm_chan *chan)
+{
+ spin_lock_irq(&chan->lock);
+ nvkm_chan_block_locked(chan);
+ spin_unlock_irq(&chan->lock);
+}
+
+void
+nvkm_chan_allow(struct nvkm_chan *chan)
+{
+ spin_lock_irq(&chan->lock);
+ CHAN_TRACE(chan, "allow %d", atomic_read(&chan->blocked));
+ if (atomic_dec_and_test(&chan->blocked))
+ chan->func->start(chan);
+ spin_unlock_irq(&chan->lock);
+}
+
+void
+nvkm_chan_del(struct nvkm_chan **pchan)
+{
+ struct nvkm_chan *chan = *pchan;
+
+ if (!chan)
+ return;
+
+ if (chan->func->ramfc->clear)
+ chan->func->ramfc->clear(chan);
+
+ nvkm_ramht_del(&chan->ramht);
+ nvkm_gpuobj_del(&chan->pgd);
+ nvkm_gpuobj_del(&chan->eng);
+ nvkm_gpuobj_del(&chan->cache);
+ nvkm_gpuobj_del(&chan->ramfc);
+
+ nvkm_memory_unref(&chan->userd.mem);
+
+ if (chan->cgrp) {
+ nvkm_chid_put(chan->cgrp->runl->chid, chan->id, &chan->cgrp->lock);
+ nvkm_cgrp_unref(&chan->cgrp);
+ }
if (chan->vmm) {
nvkm_vmm_part(chan->vmm, chan->inst->memory);
@@ -309,85 +289,192 @@ nvkm_fifo_chan_dtor(struct nvkm_object *object)
nvkm_gpuobj_del(&chan->push);
nvkm_gpuobj_del(&chan->inst);
- return data;
+ kfree(chan);
}
-static const struct nvkm_object_func
-nvkm_fifo_chan_func = {
- .dtor = nvkm_fifo_chan_dtor,
- .init = nvkm_fifo_chan_init,
- .fini = nvkm_fifo_chan_fini,
- .ntfy = nvkm_fifo_chan_ntfy,
- .map = nvkm_fifo_chan_map,
- .sclass = nvkm_fifo_chan_child_get,
-};
-
-int
-nvkm_fifo_chan_ctor(const struct nvkm_fifo_chan_func *func,
- struct nvkm_fifo *fifo, u32 size, u32 align, bool zero,
- u64 hvmm, u64 push, u32 engm, int bar, u32 base,
- u32 user, const struct nvkm_oclass *oclass,
- struct nvkm_fifo_chan *chan)
+void
+nvkm_chan_put(struct nvkm_chan **pchan, unsigned long irqflags)
{
- struct nvkm_client *client = oclass->client;
- struct nvkm_device *device = fifo->engine.subdev.device;
- struct nvkm_dmaobj *dmaobj;
- unsigned long flags;
- int ret;
+ struct nvkm_chan *chan = *pchan;
- nvkm_object_ctor(&nvkm_fifo_chan_func, oclass, &chan->object);
- chan->func = func;
- chan->fifo = fifo;
- chan->engm = engm;
- INIT_LIST_HEAD(&chan->head);
+ if (!chan)
+ return;
- /* instance memory */
- ret = nvkm_gpuobj_new(device, size, align, zero, NULL, &chan->inst);
- if (ret)
- return ret;
+ *pchan = NULL;
+ spin_unlock_irqrestore(&chan->cgrp->lock, irqflags);
+}
- /* allocate push buffer ctxdma instance */
- if (push) {
- dmaobj = nvkm_dmaobj_search(client, push);
- if (IS_ERR(dmaobj))
- return PTR_ERR(dmaobj);
+struct nvkm_chan *
+nvkm_chan_get_inst(struct nvkm_engine *engine, u64 inst, unsigned long *pirqflags)
+{
+ struct nvkm_fifo *fifo = engine->subdev.device->fifo;
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+ struct nvkm_chan *chan;
- ret = nvkm_object_bind(&dmaobj->object, chan->inst, -16,
- &chan->push);
- if (ret)
- return ret;
+ nvkm_runl_foreach(runl, fifo) {
+ nvkm_runl_foreach_engn(engn, runl) {
+ if (engine == &fifo->engine || engn->engine == engine) {
+ chan = nvkm_runl_chan_get_inst(runl, inst, pirqflags);
+ if (chan || engn->engine == engine)
+ return chan;
+ }
+ }
}
- /* channel address space */
- if (hvmm) {
- struct nvkm_vmm *vmm = nvkm_uvmm_search(client, hvmm);
- if (IS_ERR(vmm))
- return PTR_ERR(vmm);
+ return NULL;
+}
- if (vmm->mmu != device->mmu)
+struct nvkm_chan *
+nvkm_chan_get_chid(struct nvkm_engine *engine, int id, unsigned long *pirqflags)
+{
+ struct nvkm_fifo *fifo = engine->subdev.device->fifo;
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+
+ nvkm_runl_foreach(runl, fifo) {
+ nvkm_runl_foreach_engn(engn, runl) {
+ if (fifo->chid || engn->engine == engine)
+ return nvkm_runl_chan_get_chid(runl, id, pirqflags);
+ }
+ }
+
+ return NULL;
+}
+
+int
+nvkm_chan_new_(const struct nvkm_chan_func *func, struct nvkm_runl *runl, int runq,
+ struct nvkm_cgrp *cgrp, const char *name, bool priv, u32 devm, struct nvkm_vmm *vmm,
+ struct nvkm_dmaobj *dmaobj, u64 offset, u64 length,
+ struct nvkm_memory *userd, u64 ouserd, struct nvkm_chan **pchan)
+{
+ struct nvkm_fifo *fifo = runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_chan *chan;
+ int ret;
+
+ /* Validate arguments against class requirements. */
+ if ((runq && runq >= runl->func->runqs) ||
+ (!func->inst->vmm != !vmm) ||
+ ((func->userd->bar < 0) == !userd) ||
+ (!func->ramfc->ctxdma != !dmaobj) ||
+ ((func->ramfc->devm < devm) && devm != BIT(0)) ||
+ (!func->ramfc->priv && priv)) {
+ RUNL_DEBUG(runl, "args runq:%d:%d vmm:%d:%p userd:%d:%p "
+ "push:%d:%p devm:%08x:%08x priv:%d:%d",
+ runl->func->runqs, runq, func->inst->vmm, vmm,
+ func->userd->bar < 0, userd, func->ramfc->ctxdma, dmaobj,
+ func->ramfc->devm, devm, func->ramfc->priv, priv);
+ return -EINVAL;
+ }
+
+ if (!(chan = *pchan = kzalloc(sizeof(*chan), GFP_KERNEL)))
+ return -ENOMEM;
+
+ chan->func = func;
+ strscpy(chan->name, name, sizeof(chan->name));
+ chan->runq = runq;
+ chan->id = -1;
+ spin_lock_init(&chan->lock);
+ atomic_set(&chan->blocked, 1);
+ atomic_set(&chan->errored, 0);
+ INIT_LIST_HEAD(&chan->cctxs);
+ INIT_LIST_HEAD(&chan->head);
+
+ /* Join channel group.
+ *
+ * GK110 and newer support channel groups (aka TSGs), where individual channels
+ * share a timeslice, and, engine context(s).
+ *
+ * As such, engine contexts are tracked in nvkm_cgrp and we need them even when
+ * channels aren't in an API channel group, and on HW that doesn't support TSGs.
+ */
+ if (!cgrp) {
+ ret = nvkm_cgrp_new(runl, chan->name, vmm, fifo->func->cgrp.force, &chan->cgrp);
+ if (ret) {
+ RUNL_DEBUG(runl, "cgrp %d", ret);
+ return ret;
+ }
+
+ cgrp = chan->cgrp;
+ } else {
+ if (cgrp->runl != runl || cgrp->vmm != vmm) {
+ RUNL_DEBUG(runl, "cgrp %d %d", cgrp->runl != runl, cgrp->vmm != vmm);
+ return -EINVAL;
+ }
+
+ chan->cgrp = nvkm_cgrp_ref(cgrp);
+ }
+
+ /* Allocate instance block. */
+ ret = nvkm_gpuobj_new(device, func->inst->size, 0x1000, func->inst->zero, NULL,
+ &chan->inst);
+ if (ret) {
+ RUNL_DEBUG(runl, "inst %d", ret);
+ return ret;
+ }
+
+ /* Initialise virtual address-space. */
+ if (func->inst->vmm) {
+ if (WARN_ON(vmm->mmu != device->mmu))
return -EINVAL;
ret = nvkm_vmm_join(vmm, chan->inst->memory);
- if (ret)
+ if (ret) {
+ RUNL_DEBUG(runl, "vmm %d", ret);
return ret;
+ }
chan->vmm = nvkm_vmm_ref(vmm);
}
- /* allocate channel id */
- spin_lock_irqsave(&fifo->lock, flags);
- chan->chid = find_first_zero_bit(fifo->mask, NVKM_FIFO_CHID_NR);
- if (chan->chid >= NVKM_FIFO_CHID_NR) {
- spin_unlock_irqrestore(&fifo->lock, flags);
+ /* Allocate HW ctxdma for push buffer. */
+ if (func->ramfc->ctxdma) {
+ ret = nvkm_object_bind(&dmaobj->object, chan->inst, -16, &chan->push);
+ if (ret) {
+ RUNL_DEBUG(runl, "bind %d", ret);
+ return ret;
+ }
+ }
+
+ /* Allocate channel ID. */
+ chan->id = nvkm_chid_get(runl->chid, chan);
+ if (chan->id < 0) {
+ RUNL_ERROR(runl, "!chids");
return -ENOSPC;
}
- list_add(&chan->head, &fifo->chan);
- __set_bit(chan->chid, fifo->mask);
- spin_unlock_irqrestore(&fifo->lock, flags);
- /* determine address of this channel's user registers */
- chan->addr = device->func->resource_addr(device, bar) +
- base + user * chan->chid;
- chan->size = user;
+ if (cgrp->id < 0)
+ cgrp->id = chan->id;
+
+ /* Initialise USERD. */
+ if (func->userd->bar < 0) {
+ if (ouserd + chan->func->userd->size >= nvkm_memory_size(userd)) {
+ RUNL_DEBUG(runl, "ouserd %llx", ouserd);
+ return -EINVAL;
+ }
+
+ ret = nvkm_memory_kmap(userd, &chan->userd.mem);
+ if (ret) {
+ RUNL_DEBUG(runl, "userd %d", ret);
+ return ret;
+ }
+
+ chan->userd.base = ouserd;
+ } else {
+ chan->userd.mem = nvkm_memory_ref(fifo->userd.mem);
+ chan->userd.base = chan->id * chan->func->userd->size;
+ }
+
+ if (chan->func->userd->clear)
+ chan->func->userd->clear(chan);
+
+ /* Initialise RAMFC. */
+ ret = chan->func->ramfc->write(chan, offset, length, devm, priv);
+ if (ret) {
+ RUNL_DEBUG(runl, "ramfc %d", ret);
+ return ret;
+ }
+
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.h
index e535043..85b94f6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.h
@@ -1,35 +1,78 @@
/* SPDX-License-Identifier: MIT */
-#ifndef __NVKM_FIFO_CHAN_H__
-#define __NVKM_FIFO_CHAN_H__
-#define nvkm_fifo_chan(p) container_of((p), struct nvkm_fifo_chan, object)
-#include "priv.h"
+#ifndef __NVKM_CHAN_H__
+#define __NVKM_CHAN_H__
+#include <engine/fifo.h>
+struct nvkm_dmaobj;
+struct nvkm_engn;
+struct nvkm_runl;
-struct nvkm_fifo_chan_func {
- void *(*dtor)(struct nvkm_fifo_chan *);
- void (*init)(struct nvkm_fifo_chan *);
- void (*fini)(struct nvkm_fifo_chan *);
- int (*ntfy)(struct nvkm_fifo_chan *, u32 type, struct nvkm_event **);
- int (*engine_ctor)(struct nvkm_fifo_chan *, struct nvkm_engine *,
- struct nvkm_object *);
- void (*engine_dtor)(struct nvkm_fifo_chan *, struct nvkm_engine *);
- int (*engine_init)(struct nvkm_fifo_chan *, struct nvkm_engine *);
- int (*engine_fini)(struct nvkm_fifo_chan *, struct nvkm_engine *,
- bool suspend);
- int (*object_ctor)(struct nvkm_fifo_chan *, struct nvkm_object *);
- void (*object_dtor)(struct nvkm_fifo_chan *, int);
- u32 (*submit_token)(struct nvkm_fifo_chan *);
+extern const struct nvkm_event_func nvkm_chan_event;
+
+struct nvkm_cctx {
+ struct nvkm_vctx *vctx;
+ refcount_t refs;
+ refcount_t uses;
+
+ struct list_head head;
};
-int nvkm_fifo_chan_ctor(const struct nvkm_fifo_chan_func *, struct nvkm_fifo *,
- u32 size, u32 align, bool zero, u64 vm, u64 push,
- u32 engm, int bar, u32 base, u32 user,
- const struct nvkm_oclass *, struct nvkm_fifo_chan *);
+struct nvkm_chan_func {
+ const struct nvkm_chan_func_inst {
+ u32 size;
+ bool zero;
+ bool vmm;
+ } *inst;
-struct nvkm_fifo_chan_oclass {
- int (*ctor)(struct nvkm_fifo *, const struct nvkm_oclass *,
- void *data, u32 size, struct nvkm_object **);
- struct nvkm_sclass base;
+ const struct nvkm_chan_func_userd {
+ int bar;
+ u32 base;
+ u32 size;
+ void (*clear)(struct nvkm_chan *);
+ } *userd;
+
+ const struct nvkm_chan_func_ramfc {
+ const struct nvkm_ramfc_layout {
+ unsigned bits:6;
+ unsigned ctxs:5;
+ unsigned ctxp:8;
+ unsigned regs:5;
+ unsigned regp;
+ } *layout;
+ int (*write)(struct nvkm_chan *, u64 offset, u64 length, u32 devm, bool priv);
+ void (*clear)(struct nvkm_chan *);
+ bool ctxdma;
+ u32 devm;
+ bool priv;
+ } *ramfc;
+
+ void (*bind)(struct nvkm_chan *);
+ void (*unbind)(struct nvkm_chan *);
+ void (*start)(struct nvkm_chan *);
+ void (*stop)(struct nvkm_chan *);
+ void (*preempt)(struct nvkm_chan *);
+ u32 (*doorbell_handle)(struct nvkm_chan *);
};
-int gf100_fifo_chan_ntfy(struct nvkm_fifo_chan *, u32, struct nvkm_event **);
+int nvkm_chan_new_(const struct nvkm_chan_func *, struct nvkm_runl *, int runq, struct nvkm_cgrp *,
+ const char *name, bool priv, u32 devm, struct nvkm_vmm *, struct nvkm_dmaobj *,
+ u64 offset, u64 length, struct nvkm_memory *userd, u64 userd_bar1,
+ struct nvkm_chan **);
+void nvkm_chan_del(struct nvkm_chan **);
+void nvkm_chan_allow(struct nvkm_chan *);
+void nvkm_chan_block(struct nvkm_chan *);
+void nvkm_chan_error(struct nvkm_chan *, bool preempt);
+void nvkm_chan_insert(struct nvkm_chan *);
+void nvkm_chan_remove(struct nvkm_chan *, bool preempt);
+void nvkm_chan_remove_locked(struct nvkm_chan *);
+int nvkm_chan_preempt(struct nvkm_chan *, bool wait);
+int nvkm_chan_preempt_locked(struct nvkm_chan *, bool wait);
+int nvkm_chan_cctx_get(struct nvkm_chan *, struct nvkm_engn *, struct nvkm_cctx **,
+ struct nvkm_client * /*TODO: remove need for this */);
+void nvkm_chan_cctx_put(struct nvkm_chan *, struct nvkm_cctx **);
+void nvkm_chan_cctx_bind(struct nvkm_chan *, struct nvkm_engn *, struct nvkm_cctx *);
+
+#define CHAN_PRCLI(c,l,p,f,a...) CGRP_PRINT((c)->cgrp, l, p, "%04x:[%s]"f, (c)->id, (c)->name, ##a)
+#define CHAN_PRINT(c,l,p,f,a...) CGRP_PRINT((c)->cgrp, l, p, "%04x:"f, (c)->id, ##a)
+#define CHAN_ERROR(c,f,a...) CHAN_PRCLI((c), ERROR, err, " "f"\n", ##a)
+#define CHAN_TRACE(c,f,a...) CHAN_PRINT((c), TRACE, info, " "f"\n", ##a)
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chang84.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chang84.c
deleted file mode 100644
index 3492c56..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chang84.c
+++ /dev/null
@@ -1,263 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv50.h"
-
-#include <core/client.h>
-#include <core/ramht.h>
-#include <subdev/mmu.h>
-#include <subdev/timer.h>
-
-#include <nvif/cl826e.h>
-
-static int
-g84_fifo_chan_ntfy(struct nvkm_fifo_chan *chan, u32 type,
- struct nvkm_event **pevent)
-{
- switch (type) {
- case NV826E_V0_NTFY_NON_STALL_INTERRUPT:
- *pevent = &chan->fifo->uevent;
- return 0;
- default:
- break;
- }
- return -EINVAL;
-}
-
-static int
-g84_fifo_chan_engine_addr(struct nvkm_engine *engine)
-{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_DMAOBJ:
- case NVKM_ENGINE_SW : return -1;
- case NVKM_ENGINE_GR : return 0x0020;
- case NVKM_ENGINE_VP :
- case NVKM_ENGINE_MSPDEC: return 0x0040;
- case NVKM_ENGINE_MPEG :
- case NVKM_ENGINE_MSPPP : return 0x0060;
- case NVKM_ENGINE_BSP :
- case NVKM_ENGINE_MSVLD : return 0x0080;
- case NVKM_ENGINE_CIPHER:
- case NVKM_ENGINE_SEC : return 0x00a0;
- case NVKM_ENGINE_CE : return 0x00c0;
- default:
- WARN_ON(1);
- return -1;
- }
-}
-
-static int
-g84_fifo_chan_engine_fini(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine, bool suspend)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- struct nv50_fifo *fifo = chan->fifo;
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 engn, save;
- int offset;
- bool done;
-
- offset = g84_fifo_chan_engine_addr(engine);
- if (offset < 0)
- return 0;
-
- engn = fifo->base.func->engine_id(&fifo->base, engine) - 1;
- save = nvkm_mask(device, 0x002520, 0x0000003f, 1 << engn);
- nvkm_wr32(device, 0x0032fc, chan->base.inst->addr >> 12);
- done = nvkm_msec(device, 2000,
- if (nvkm_rd32(device, 0x0032fc) != 0xffffffff)
- break;
- ) >= 0;
- nvkm_wr32(device, 0x002520, save);
- if (!done) {
- nvkm_error(subdev, "channel %d [%s] unload timeout\n",
- chan->base.chid, chan->base.object.client->name);
- if (suspend)
- return -EBUSY;
- }
-
- nvkm_kmap(chan->eng);
- nvkm_wo32(chan->eng, offset + 0x00, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x04, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x08, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x0c, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x10, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x14, 0x00000000);
- nvkm_done(chan->eng);
- return 0;
-}
-
-
-static int
-g84_fifo_chan_engine_init(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- struct nvkm_gpuobj *engn = *nv50_fifo_chan_engine(chan, engine);
- u64 limit, start;
- int offset;
-
- offset = g84_fifo_chan_engine_addr(engine);
- if (offset < 0)
- return 0;
- limit = engn->addr + engn->size - 1;
- start = engn->addr;
-
- nvkm_kmap(chan->eng);
- nvkm_wo32(chan->eng, offset + 0x00, 0x00190000);
- nvkm_wo32(chan->eng, offset + 0x04, lower_32_bits(limit));
- nvkm_wo32(chan->eng, offset + 0x08, lower_32_bits(start));
- nvkm_wo32(chan->eng, offset + 0x0c, upper_32_bits(limit) << 24 |
- upper_32_bits(start));
- nvkm_wo32(chan->eng, offset + 0x10, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x14, 0x00000000);
- nvkm_done(chan->eng);
- return 0;
-}
-
-static int
-g84_fifo_chan_engine_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine,
- struct nvkm_object *object)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
-
- if (g84_fifo_chan_engine_addr(engine) < 0)
- return 0;
-
- return nvkm_object_bind(object, NULL, 0, nv50_fifo_chan_engine(chan, engine));
-}
-
-static int
-g84_fifo_chan_object_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_object *object)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- u32 handle = object->handle;
- u32 context;
-
- switch (object->engine->subdev.type) {
- case NVKM_ENGINE_DMAOBJ:
- case NVKM_ENGINE_SW : context = 0x00000000; break;
- case NVKM_ENGINE_GR : context = 0x00100000; break;
- case NVKM_ENGINE_MPEG :
- case NVKM_ENGINE_MSPPP : context = 0x00200000; break;
- case NVKM_ENGINE_ME :
- case NVKM_ENGINE_CE : context = 0x00300000; break;
- case NVKM_ENGINE_VP :
- case NVKM_ENGINE_MSPDEC: context = 0x00400000; break;
- case NVKM_ENGINE_CIPHER:
- case NVKM_ENGINE_SEC :
- case NVKM_ENGINE_VIC : context = 0x00500000; break;
- case NVKM_ENGINE_BSP :
- case NVKM_ENGINE_MSVLD : context = 0x00600000; break;
- default:
- WARN_ON(1);
- return -EINVAL;
- }
-
- return nvkm_ramht_insert(chan->ramht, object, 0, 4, handle, context);
-}
-
-static void
-g84_fifo_chan_init(struct nvkm_fifo_chan *base)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- struct nv50_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u64 addr = chan->ramfc->addr >> 8;
- u32 chid = chan->base.chid;
-
- nvkm_wr32(device, 0x002600 + (chid * 4), 0x80000000 | addr);
- nv50_fifo_runlist_update(fifo);
-}
-
-static const struct nvkm_fifo_chan_func
-g84_fifo_chan_func = {
- .dtor = nv50_fifo_chan_dtor,
- .init = g84_fifo_chan_init,
- .fini = nv50_fifo_chan_fini,
- .ntfy = g84_fifo_chan_ntfy,
- .engine_ctor = g84_fifo_chan_engine_ctor,
- .engine_dtor = nv50_fifo_chan_engine_dtor,
- .engine_init = g84_fifo_chan_engine_init,
- .engine_fini = g84_fifo_chan_engine_fini,
- .object_ctor = g84_fifo_chan_object_ctor,
- .object_dtor = nv50_fifo_chan_object_dtor,
-};
-
-int
-g84_fifo_chan_ctor(struct nv50_fifo *fifo, u64 vmm, u64 push,
- const struct nvkm_oclass *oclass,
- struct nv50_fifo_chan *chan)
-{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- int ret;
-
- if (!vmm)
- return -EINVAL;
-
- ret = nvkm_fifo_chan_ctor(&g84_fifo_chan_func, &fifo->base,
- 0x10000, 0x1000, false, vmm, push,
- BIT(G84_FIFO_ENGN_SW) |
- BIT(G84_FIFO_ENGN_GR) |
- BIT(G84_FIFO_ENGN_MPEG) |
- BIT(G84_FIFO_ENGN_MSPPP) |
- BIT(G84_FIFO_ENGN_ME) |
- BIT(G84_FIFO_ENGN_CE0) |
- BIT(G84_FIFO_ENGN_VP) |
- BIT(G84_FIFO_ENGN_MSPDEC) |
- BIT(G84_FIFO_ENGN_CIPHER) |
- BIT(G84_FIFO_ENGN_SEC) |
- BIT(G84_FIFO_ENGN_VIC) |
- BIT(G84_FIFO_ENGN_BSP) |
- BIT(G84_FIFO_ENGN_MSVLD) |
- BIT(G84_FIFO_ENGN_DMA),
- 0, 0xc00000, 0x2000, oclass, &chan->base);
- chan->fifo = fifo;
- if (ret)
- return ret;
-
- ret = nvkm_gpuobj_new(device, 0x0200, 0, true, chan->base.inst,
- &chan->eng);
- if (ret)
- return ret;
-
- ret = nvkm_gpuobj_new(device, 0x4000, 0, false, chan->base.inst,
- &chan->pgd);
- if (ret)
- return ret;
-
- ret = nvkm_gpuobj_new(device, 0x1000, 0x400, true, chan->base.inst,
- &chan->cache);
- if (ret)
- return ret;
-
- ret = nvkm_gpuobj_new(device, 0x100, 0x100, true, chan->base.inst,
- &chan->ramfc);
- if (ret)
- return ret;
-
- return nvkm_ramht_new(device, 0x8000, 16, chan->base.inst, &chan->ramht);
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/changf100.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/changf100.h
deleted file mode 100644
index f7ac106..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/changf100.h
+++ /dev/null
@@ -1,29 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __GF100_FIFO_CHAN_H__
-#define __GF100_FIFO_CHAN_H__
-#define gf100_fifo_chan(p) container_of((p), struct gf100_fifo_chan, base)
-#include "chan.h"
-#include "gf100.h"
-
-struct gf100_fifo_chan {
- struct nvkm_fifo_chan base;
- struct gf100_fifo *fifo;
-
- struct list_head head;
- bool killed;
-
-#define GF100_FIFO_ENGN_GR 0
-#define GF100_FIFO_ENGN_MSPDEC 1
-#define GF100_FIFO_ENGN_MSPPP 2
-#define GF100_FIFO_ENGN_MSVLD 3
-#define GF100_FIFO_ENGN_CE0 4
-#define GF100_FIFO_ENGN_CE1 5
-#define GF100_FIFO_ENGN_SW 15
- struct gf100_fifo_engn {
- struct nvkm_gpuobj *inst;
- struct nvkm_vma *vma;
- } engn[NVKM_FIFO_ENGN_NR];
-};
-
-extern const struct nvkm_fifo_chan_oclass gf100_fifo_gpfifo_oclass;
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/changk104.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/changk104.h
deleted file mode 100644
index 9713dae..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/changk104.h
+++ /dev/null
@@ -1,52 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __GK104_FIFO_CHAN_H__
-#define __GK104_FIFO_CHAN_H__
-#define gk104_fifo_chan(p) container_of((p), struct gk104_fifo_chan, base)
-#include "chan.h"
-#include "gk104.h"
-
-struct gk104_fifo_chan {
- struct nvkm_fifo_chan base;
- struct gk104_fifo *fifo;
- int runl;
-
- struct nvkm_fifo_cgrp *cgrp;
- struct list_head head;
- bool killed;
-
-#define GK104_FIFO_ENGN_SW 15
- struct gk104_fifo_engn {
- struct nvkm_gpuobj *inst;
- struct nvkm_vma *vma;
- } engn[NVKM_FIFO_ENGN_NR];
-};
-
-extern const struct nvkm_fifo_chan_func gk104_fifo_gpfifo_func;
-
-int gk104_fifo_gpfifo_new(struct gk104_fifo *, const struct nvkm_oclass *,
- void *data, u32 size, struct nvkm_object **);
-void *gk104_fifo_gpfifo_dtor(struct nvkm_fifo_chan *);
-void gk104_fifo_gpfifo_init(struct nvkm_fifo_chan *);
-void gk104_fifo_gpfifo_fini(struct nvkm_fifo_chan *);
-struct gk104_fifo_engn *gk104_fifo_gpfifo_engine(struct gk104_fifo_chan *, struct nvkm_engine *);
-int gk104_fifo_gpfifo_engine_ctor(struct nvkm_fifo_chan *, struct nvkm_engine *,
- struct nvkm_object *);
-void gk104_fifo_gpfifo_engine_dtor(struct nvkm_fifo_chan *,
- struct nvkm_engine *);
-int gk104_fifo_gpfifo_kick(struct gk104_fifo_chan *);
-int gk104_fifo_gpfifo_kick_locked(struct gk104_fifo_chan *);
-
-int gv100_fifo_gpfifo_new(struct gk104_fifo *, const struct nvkm_oclass *,
- void *data, u32 size, struct nvkm_object **);
-int gv100_fifo_gpfifo_new_(const struct nvkm_fifo_chan_func *,
- struct gk104_fifo *, u64 *, u16 *, u64, u64, u64,
- u64 *, bool, u32 *, const struct nvkm_oclass *,
- struct nvkm_object **);
-int gv100_fifo_gpfifo_engine_init(struct nvkm_fifo_chan *,
- struct nvkm_engine *);
-int gv100_fifo_gpfifo_engine_fini(struct nvkm_fifo_chan *,
- struct nvkm_engine *, bool);
-
-int tu102_fifo_gpfifo_new(struct gk104_fifo *, const struct nvkm_oclass *,
- void *data, u32 size, struct nvkm_object **);
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv04.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv04.h
deleted file mode 100644
index 727bc89..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv04.h
+++ /dev/null
@@ -1,29 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NV04_FIFO_CHAN_H__
-#define __NV04_FIFO_CHAN_H__
-#define nv04_fifo_chan(p) container_of((p), struct nv04_fifo_chan, base)
-#include "chan.h"
-#include "nv04.h"
-
-struct nv04_fifo_chan {
- struct nvkm_fifo_chan base;
- struct nv04_fifo *fifo;
- u32 ramfc;
-#define NV04_FIFO_ENGN_SW 0
-#define NV04_FIFO_ENGN_GR 1
-#define NV04_FIFO_ENGN_MPEG 2
-#define NV04_FIFO_ENGN_DMA 3
- struct nvkm_gpuobj *engn[NVKM_FIFO_ENGN_NR];
-};
-
-extern const struct nvkm_fifo_chan_func nv04_fifo_dma_func;
-void *nv04_fifo_dma_dtor(struct nvkm_fifo_chan *);
-void nv04_fifo_dma_init(struct nvkm_fifo_chan *);
-void nv04_fifo_dma_fini(struct nvkm_fifo_chan *);
-void nv04_fifo_dma_object_dtor(struct nvkm_fifo_chan *, int);
-
-extern const struct nvkm_fifo_chan_oclass nv04_fifo_dma_oclass;
-extern const struct nvkm_fifo_chan_oclass nv10_fifo_dma_oclass;
-extern const struct nvkm_fifo_chan_oclass nv17_fifo_dma_oclass;
-extern const struct nvkm_fifo_chan_oclass nv40_fifo_dma_oclass;
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv50.c
deleted file mode 100644
index c44d7c8..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv50.c
+++ /dev/null
@@ -1,276 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv50.h"
-
-#include <core/client.h>
-#include <core/ramht.h>
-#include <subdev/mmu.h>
-#include <subdev/timer.h>
-
-static int
-nv50_fifo_chan_engine_addr(struct nvkm_engine *engine)
-{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_DMAOBJ:
- case NVKM_ENGINE_SW : return -1;
- case NVKM_ENGINE_GR : return 0x0000;
- case NVKM_ENGINE_MPEG : return 0x0060;
- default:
- WARN_ON(1);
- return -1;
- }
-}
-
-struct nvkm_gpuobj **
-nv50_fifo_chan_engine(struct nv50_fifo_chan *chan, struct nvkm_engine *engine)
-{
- int engi = chan->base.fifo->func->engine_id(chan->base.fifo, engine);
- if (engi >= 0)
- return &chan->engn[engi];
- return NULL;
-}
-
-static int
-nv50_fifo_chan_engine_fini(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine, bool suspend)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- struct nv50_fifo *fifo = chan->fifo;
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- int offset, ret = 0;
- u32 me;
-
- offset = nv50_fifo_chan_engine_addr(engine);
- if (offset < 0)
- return 0;
-
- /* HW bug workaround:
- *
- * PFIFO will hang forever if the connected engines don't report
- * that they've processed the context switch request.
- *
- * In order for the kickoff to work, we need to ensure all the
- * connected engines are in a state where they can answer.
- *
- * Newer chipsets don't seem to suffer from this issue, and well,
- * there's also a "ignore these engines" bitmask reg we can use
- * if we hit the issue there..
- */
- me = nvkm_mask(device, 0x00b860, 0x00000001, 0x00000001);
-
- /* do the kickoff... */
- nvkm_wr32(device, 0x0032fc, chan->base.inst->addr >> 12);
- if (nvkm_msec(device, 2000,
- if (nvkm_rd32(device, 0x0032fc) != 0xffffffff)
- break;
- ) < 0) {
- nvkm_error(subdev, "channel %d [%s] unload timeout\n",
- chan->base.chid, chan->base.object.client->name);
- if (suspend)
- ret = -EBUSY;
- }
- nvkm_wr32(device, 0x00b860, me);
-
- if (ret == 0) {
- nvkm_kmap(chan->eng);
- nvkm_wo32(chan->eng, offset + 0x00, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x04, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x08, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x0c, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x10, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x14, 0x00000000);
- nvkm_done(chan->eng);
- }
-
- return ret;
-}
-
-static int
-nv50_fifo_chan_engine_init(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- struct nvkm_gpuobj *engn = *nv50_fifo_chan_engine(chan, engine);
- u64 limit, start;
- int offset;
-
- offset = nv50_fifo_chan_engine_addr(engine);
- if (offset < 0)
- return 0;
- limit = engn->addr + engn->size - 1;
- start = engn->addr;
-
- nvkm_kmap(chan->eng);
- nvkm_wo32(chan->eng, offset + 0x00, 0x00190000);
- nvkm_wo32(chan->eng, offset + 0x04, lower_32_bits(limit));
- nvkm_wo32(chan->eng, offset + 0x08, lower_32_bits(start));
- nvkm_wo32(chan->eng, offset + 0x0c, upper_32_bits(limit) << 24 |
- upper_32_bits(start));
- nvkm_wo32(chan->eng, offset + 0x10, 0x00000000);
- nvkm_wo32(chan->eng, offset + 0x14, 0x00000000);
- nvkm_done(chan->eng);
- return 0;
-}
-
-void
-nv50_fifo_chan_engine_dtor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- nvkm_gpuobj_del(nv50_fifo_chan_engine(chan, engine));
-}
-
-static int
-nv50_fifo_chan_engine_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine,
- struct nvkm_object *object)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
-
- if (nv50_fifo_chan_engine_addr(engine) < 0)
- return 0;
-
- return nvkm_object_bind(object, NULL, 0, nv50_fifo_chan_engine(chan, engine));
-}
-
-void
-nv50_fifo_chan_object_dtor(struct nvkm_fifo_chan *base, int cookie)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- nvkm_ramht_remove(chan->ramht, cookie);
-}
-
-static int
-nv50_fifo_chan_object_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_object *object)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- u32 handle = object->handle;
- u32 context;
-
- switch (object->engine->subdev.type) {
- case NVKM_ENGINE_DMAOBJ:
- case NVKM_ENGINE_SW : context = 0x00000000; break;
- case NVKM_ENGINE_GR : context = 0x00100000; break;
- case NVKM_ENGINE_MPEG : context = 0x00200000; break;
- default:
- WARN_ON(1);
- return -EINVAL;
- }
-
- return nvkm_ramht_insert(chan->ramht, object, 0, 4, handle, context);
-}
-
-void
-nv50_fifo_chan_fini(struct nvkm_fifo_chan *base)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- struct nv50_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u32 chid = chan->base.chid;
-
- /* remove channel from runlist, fifo will unload context */
- nvkm_mask(device, 0x002600 + (chid * 4), 0x80000000, 0x00000000);
- nv50_fifo_runlist_update(fifo);
- nvkm_wr32(device, 0x002600 + (chid * 4), 0x00000000);
-}
-
-static void
-nv50_fifo_chan_init(struct nvkm_fifo_chan *base)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- struct nv50_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u64 addr = chan->ramfc->addr >> 12;
- u32 chid = chan->base.chid;
-
- nvkm_wr32(device, 0x002600 + (chid * 4), 0x80000000 | addr);
- nv50_fifo_runlist_update(fifo);
-}
-
-void *
-nv50_fifo_chan_dtor(struct nvkm_fifo_chan *base)
-{
- struct nv50_fifo_chan *chan = nv50_fifo_chan(base);
- nvkm_ramht_del(&chan->ramht);
- nvkm_gpuobj_del(&chan->pgd);
- nvkm_gpuobj_del(&chan->eng);
- nvkm_gpuobj_del(&chan->cache);
- nvkm_gpuobj_del(&chan->ramfc);
- return chan;
-}
-
-static const struct nvkm_fifo_chan_func
-nv50_fifo_chan_func = {
- .dtor = nv50_fifo_chan_dtor,
- .init = nv50_fifo_chan_init,
- .fini = nv50_fifo_chan_fini,
- .engine_ctor = nv50_fifo_chan_engine_ctor,
- .engine_dtor = nv50_fifo_chan_engine_dtor,
- .engine_init = nv50_fifo_chan_engine_init,
- .engine_fini = nv50_fifo_chan_engine_fini,
- .object_ctor = nv50_fifo_chan_object_ctor,
- .object_dtor = nv50_fifo_chan_object_dtor,
-};
-
-int
-nv50_fifo_chan_ctor(struct nv50_fifo *fifo, u64 vmm, u64 push,
- const struct nvkm_oclass *oclass,
- struct nv50_fifo_chan *chan)
-{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- int ret;
-
- if (!vmm)
- return -EINVAL;
-
- ret = nvkm_fifo_chan_ctor(&nv50_fifo_chan_func, &fifo->base,
- 0x10000, 0x1000, false, vmm, push,
- BIT(NV50_FIFO_ENGN_SW) |
- BIT(NV50_FIFO_ENGN_GR) |
- BIT(NV50_FIFO_ENGN_MPEG) |
- BIT(NV50_FIFO_ENGN_DMA),
- 0, 0xc00000, 0x2000, oclass, &chan->base);
- chan->fifo = fifo;
- if (ret)
- return ret;
-
- ret = nvkm_gpuobj_new(device, 0x0200, 0x1000, true, chan->base.inst,
- &chan->ramfc);
- if (ret)
- return ret;
-
- ret = nvkm_gpuobj_new(device, 0x1200, 0, true, chan->base.inst,
- &chan->eng);
- if (ret)
- return ret;
-
- ret = nvkm_gpuobj_new(device, 0x4000, 0, false, chan->base.inst,
- &chan->pgd);
- if (ret)
- return ret;
-
- return nvkm_ramht_new(device, 0x8000, 16, chan->base.inst, &chan->ramht);
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv50.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv50.h
deleted file mode 100644
index 3a95730..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/channv50.h
+++ /dev/null
@@ -1,53 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NV50_FIFO_CHAN_H__
-#define __NV50_FIFO_CHAN_H__
-#define nv50_fifo_chan(p) container_of((p), struct nv50_fifo_chan, base)
-#include "chan.h"
-#include "nv50.h"
-
-struct nv50_fifo_chan {
- struct nv50_fifo *fifo;
- struct nvkm_fifo_chan base;
-
- struct nvkm_gpuobj *ramfc;
- struct nvkm_gpuobj *cache;
- struct nvkm_gpuobj *eng;
- struct nvkm_gpuobj *pgd;
- struct nvkm_ramht *ramht;
-
-#define NV50_FIFO_ENGN_SW 0
-#define NV50_FIFO_ENGN_GR 1
-#define NV50_FIFO_ENGN_MPEG 2
-#define NV50_FIFO_ENGN_DMA 3
-
-#define G84_FIFO_ENGN_SW 0
-#define G84_FIFO_ENGN_GR 1
-#define G84_FIFO_ENGN_MPEG 2
-#define G84_FIFO_ENGN_MSPPP 2
-#define G84_FIFO_ENGN_ME 3
-#define G84_FIFO_ENGN_CE0 3
-#define G84_FIFO_ENGN_VP 4
-#define G84_FIFO_ENGN_MSPDEC 4
-#define G84_FIFO_ENGN_CIPHER 5
-#define G84_FIFO_ENGN_SEC 5
-#define G84_FIFO_ENGN_VIC 5
-#define G84_FIFO_ENGN_BSP 6
-#define G84_FIFO_ENGN_MSVLD 6
-#define G84_FIFO_ENGN_DMA 7
- struct nvkm_gpuobj *engn[NVKM_FIFO_ENGN_NR];
-};
-
-int nv50_fifo_chan_ctor(struct nv50_fifo *, u64 vmm, u64 push,
- const struct nvkm_oclass *, struct nv50_fifo_chan *);
-void *nv50_fifo_chan_dtor(struct nvkm_fifo_chan *);
-void nv50_fifo_chan_fini(struct nvkm_fifo_chan *);
-struct nvkm_gpuobj **nv50_fifo_chan_engine(struct nv50_fifo_chan *, struct nvkm_engine *);
-void nv50_fifo_chan_engine_dtor(struct nvkm_fifo_chan *, struct nvkm_engine *);
-void nv50_fifo_chan_object_dtor(struct nvkm_fifo_chan *, int);
-
-int g84_fifo_chan_ctor(struct nv50_fifo *, u64 vmm, u64 push,
- const struct nvkm_oclass *, struct nv50_fifo_chan *);
-
-extern const struct nvkm_fifo_chan_oclass nv50_fifo_gpfifo_oclass;
-extern const struct nvkm_fifo_chan_oclass g84_fifo_gpfifo_oclass;
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chid.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chid.c
new file mode 100644
index 0000000..23944d9
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chid.c
@@ -0,0 +1,111 @@
+/*
+ * Copyright 2020 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "chid.h"
+
+void
+nvkm_chid_put(struct nvkm_chid *chid, int id, spinlock_t *data_lock)
+{
+ if (id >= 0) {
+ spin_lock_irq(&chid->lock);
+ spin_lock(data_lock);
+ chid->data[id] = NULL;
+ spin_unlock(data_lock);
+ clear_bit(id, chid->used);
+ spin_unlock_irq(&chid->lock);
+ }
+}
+
+int
+nvkm_chid_get(struct nvkm_chid *chid, void *data)
+{
+ int id = -1, cid;
+
+ spin_lock_irq(&chid->lock);
+ cid = find_first_zero_bit(chid->used, chid->nr);
+ if (cid < chid->nr) {
+ set_bit(cid, chid->used);
+ chid->data[cid] = data;
+ id = cid;
+ }
+ spin_unlock_irq(&chid->lock);
+ return id;
+}
+
+static void
+nvkm_chid_del(struct kref *kref)
+{
+ struct nvkm_chid *chid = container_of(kref, typeof(*chid), kref);
+
+ nvkm_event_fini(&chid->event);
+
+ kvfree(chid->data);
+ kfree(chid);
+}
+
+void
+nvkm_chid_unref(struct nvkm_chid **pchid)
+{
+ struct nvkm_chid *chid = *pchid;
+
+ if (!chid)
+ return;
+
+ kref_put(&chid->kref, nvkm_chid_del);
+ *pchid = NULL;
+}
+
+struct nvkm_chid *
+nvkm_chid_ref(struct nvkm_chid *chid)
+{
+ if (chid)
+ kref_get(&chid->kref);
+
+ return chid;
+}
+
+int
+nvkm_chid_new(const struct nvkm_event_func *func, struct nvkm_subdev *subdev,
+ int nr, int first, int count, struct nvkm_chid **pchid)
+{
+ struct nvkm_chid *chid;
+ int id;
+
+ if (!(chid = *pchid = kzalloc(struct_size(chid, used, nr), GFP_KERNEL)))
+ return -ENOMEM;
+
+ kref_init(&chid->kref);
+ chid->nr = nr;
+ chid->mask = chid->nr - 1;
+ spin_lock_init(&chid->lock);
+
+ if (!(chid->data = kvzalloc(sizeof(*chid->data) * nr, GFP_KERNEL))) {
+ nvkm_chid_unref(pchid);
+ return -ENOMEM;
+ }
+
+ for (id = 0; id < first; id++)
+ __set_bit(id, chid->used);
+ for (id = first + count; id < nr; id++)
+ __set_bit(id, chid->used);
+
+ return nvkm_event_init(func, subdev, 1, nr, &chid->event);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chid.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chid.h
new file mode 100644
index 0000000..2a42efb
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/chid.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVKM_CHID_H__
+#define __NVKM_CHID_H__
+#include <core/event.h>
+
+struct nvkm_chid {
+ struct kref kref;
+ int nr;
+ u32 mask;
+
+ struct nvkm_event event;
+
+ void **data;
+
+ spinlock_t lock;
+ unsigned long used[];
+};
+
+int nvkm_chid_new(const struct nvkm_event_func *, struct nvkm_subdev *,
+ int nr, int first, int count, struct nvkm_chid **pchid);
+struct nvkm_chid *nvkm_chid_ref(struct nvkm_chid *);
+void nvkm_chid_unref(struct nvkm_chid **);
+int nvkm_chid_get(struct nvkm_chid *, void *data);
+void nvkm_chid_put(struct nvkm_chid *, int id, spinlock_t *data_lock);
+#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv04.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv04.c
deleted file mode 100644
index dbcdc5f..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv04.c
+++ /dev/null
@@ -1,226 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv04.h"
-#include "regsnv04.h"
-
-#include <core/client.h>
-#include <core/ramht.h>
-#include <subdev/instmem.h>
-
-#include <nvif/class.h>
-#include <nvif/cl006b.h>
-#include <nvif/unpack.h>
-
-void
-nv04_fifo_dma_object_dtor(struct nvkm_fifo_chan *base, int cookie)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nvkm_instmem *imem = chan->fifo->base.engine.subdev.device->imem;
-
- mutex_lock(&chan->fifo->base.mutex);
- nvkm_ramht_remove(imem->ramht, cookie);
- mutex_unlock(&chan->fifo->base.mutex);
-}
-
-static int
-nv04_fifo_dma_object_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_object *object)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nvkm_instmem *imem = chan->fifo->base.engine.subdev.device->imem;
- u32 context = 0x80000000 | chan->base.chid << 24;
- u32 handle = object->handle;
- int hash;
-
- switch (object->engine->subdev.type) {
- case NVKM_ENGINE_DMAOBJ:
- case NVKM_ENGINE_SW : context |= 0x00000000; break;
- case NVKM_ENGINE_GR : context |= 0x00010000; break;
- case NVKM_ENGINE_MPEG : context |= 0x00020000; break;
- default:
- WARN_ON(1);
- return -EINVAL;
- }
-
- mutex_lock(&chan->fifo->base.mutex);
- hash = nvkm_ramht_insert(imem->ramht, object, chan->base.chid, 4,
- handle, context);
- mutex_unlock(&chan->fifo->base.mutex);
- return hash;
-}
-
-void
-nv04_fifo_dma_fini(struct nvkm_fifo_chan *base)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nv04_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_memory *fctx = device->imem->ramfc;
- const struct nv04_fifo_ramfc *c;
- unsigned long flags;
- u32 mask = fifo->base.nr - 1;
- u32 data = chan->ramfc;
- u32 chid;
-
- /* prevent fifo context switches */
- spin_lock_irqsave(&fifo->base.lock, flags);
- nvkm_wr32(device, NV03_PFIFO_CACHES, 0);
-
- /* if this channel is active, replace it with a null context */
- chid = nvkm_rd32(device, NV03_PFIFO_CACHE1_PUSH1) & mask;
- if (chid == chan->base.chid) {
- nvkm_mask(device, NV04_PFIFO_CACHE1_DMA_PUSH, 0x00000001, 0);
- nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH0, 0);
- nvkm_mask(device, NV04_PFIFO_CACHE1_PULL0, 0x00000001, 0);
-
- c = fifo->ramfc;
- nvkm_kmap(fctx);
- do {
- u32 rm = ((1ULL << c->bits) - 1) << c->regs;
- u32 cm = ((1ULL << c->bits) - 1) << c->ctxs;
- u32 rv = (nvkm_rd32(device, c->regp) & rm) >> c->regs;
- u32 cv = (nvkm_ro32(fctx, c->ctxp + data) & ~cm);
- nvkm_wo32(fctx, c->ctxp + data, cv | (rv << c->ctxs));
- } while ((++c)->bits);
- nvkm_done(fctx);
-
- c = fifo->ramfc;
- do {
- nvkm_wr32(device, c->regp, 0x00000000);
- } while ((++c)->bits);
-
- nvkm_wr32(device, NV03_PFIFO_CACHE1_GET, 0);
- nvkm_wr32(device, NV03_PFIFO_CACHE1_PUT, 0);
- nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, mask);
- nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH0, 1);
- nvkm_wr32(device, NV04_PFIFO_CACHE1_PULL0, 1);
- }
-
- /* restore normal operation, after disabling dma mode */
- nvkm_mask(device, NV04_PFIFO_MODE, 1 << chan->base.chid, 0);
- nvkm_wr32(device, NV03_PFIFO_CACHES, 1);
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-}
-
-void
-nv04_fifo_dma_init(struct nvkm_fifo_chan *base)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nv04_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u32 mask = 1 << chan->base.chid;
- unsigned long flags;
- spin_lock_irqsave(&fifo->base.lock, flags);
- nvkm_mask(device, NV04_PFIFO_MODE, mask, mask);
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-}
-
-void *
-nv04_fifo_dma_dtor(struct nvkm_fifo_chan *base)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nv04_fifo *fifo = chan->fifo;
- struct nvkm_instmem *imem = fifo->base.engine.subdev.device->imem;
- const struct nv04_fifo_ramfc *c = fifo->ramfc;
-
- nvkm_kmap(imem->ramfc);
- do {
- nvkm_wo32(imem->ramfc, chan->ramfc + c->ctxp, 0x00000000);
- } while ((++c)->bits);
- nvkm_done(imem->ramfc);
- return chan;
-}
-
-const struct nvkm_fifo_chan_func
-nv04_fifo_dma_func = {
- .dtor = nv04_fifo_dma_dtor,
- .init = nv04_fifo_dma_init,
- .fini = nv04_fifo_dma_fini,
- .object_ctor = nv04_fifo_dma_object_ctor,
- .object_dtor = nv04_fifo_dma_object_dtor,
-};
-
-static int
-nv04_fifo_dma_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct nv03_channel_dma_v0 v0;
- } *args = data;
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nv04_fifo_chan *chan = NULL;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_instmem *imem = device->imem;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel dma size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel dma vers %d pushbuf %llx "
- "offset %08x\n", args->v0.version,
- args->v0.pushbuf, args->v0.offset);
- if (!args->v0.pushbuf)
- return -EINVAL;
- } else
- return ret;
-
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
-
- ret = nvkm_fifo_chan_ctor(&nv04_fifo_dma_func, &fifo->base,
- 0x1000, 0x1000, false, 0, args->v0.pushbuf,
- BIT(NV04_FIFO_ENGN_SW) |
- BIT(NV04_FIFO_ENGN_GR) |
- BIT(NV04_FIFO_ENGN_DMA),
- 0, 0x800000, 0x10000, oclass, &chan->base);
- chan->fifo = fifo;
- if (ret)
- return ret;
-
- args->v0.chid = chan->base.chid;
- chan->ramfc = chan->base.chid * 32;
-
- nvkm_kmap(imem->ramfc);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x00, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x04, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x08, chan->base.push->addr >> 4);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x10,
- NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
- NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
-#ifdef __BIG_ENDIAN
- NV_PFIFO_CACHE1_BIG_ENDIAN |
-#endif
- NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
- nvkm_done(imem->ramfc);
- return 0;
-}
-
-const struct nvkm_fifo_chan_oclass
-nv04_fifo_dma_oclass = {
- .base.oclass = NV03_CHANNEL_DMA,
- .base.minver = 0,
- .base.maxver = 0,
- .ctor = nv04_fifo_dma_new,
-};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv10.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv10.c
deleted file mode 100644
index 07d80d5..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv10.c
+++ /dev/null
@@ -1,97 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv04.h"
-#include "regsnv04.h"
-
-#include <core/client.h>
-#include <core/gpuobj.h>
-#include <subdev/instmem.h>
-
-#include <nvif/class.h>
-#include <nvif/cl006b.h>
-#include <nvif/unpack.h>
-
-static int
-nv10_fifo_dma_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct nv03_channel_dma_v0 v0;
- } *args = data;
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nv04_fifo_chan *chan = NULL;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_instmem *imem = device->imem;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel dma size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel dma vers %d pushbuf %llx "
- "offset %08x\n", args->v0.version,
- args->v0.pushbuf, args->v0.offset);
- if (!args->v0.pushbuf)
- return -EINVAL;
- } else
- return ret;
-
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
-
- ret = nvkm_fifo_chan_ctor(&nv04_fifo_dma_func, &fifo->base,
- 0x1000, 0x1000, false, 0, args->v0.pushbuf,
- BIT(NV04_FIFO_ENGN_SW) |
- BIT(NV04_FIFO_ENGN_GR) |
- BIT(NV04_FIFO_ENGN_DMA),
- 0, 0x800000, 0x10000, oclass, &chan->base);
- chan->fifo = fifo;
- if (ret)
- return ret;
-
- args->v0.chid = chan->base.chid;
- chan->ramfc = chan->base.chid * 32;
-
- nvkm_kmap(imem->ramfc);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x00, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x04, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x0c, chan->base.push->addr >> 4);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x14,
- NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
- NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
-#ifdef __BIG_ENDIAN
- NV_PFIFO_CACHE1_BIG_ENDIAN |
-#endif
- NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
- nvkm_done(imem->ramfc);
- return 0;
-}
-
-const struct nvkm_fifo_chan_oclass
-nv10_fifo_dma_oclass = {
- .base.oclass = NV10_CHANNEL_DMA,
- .base.minver = 0,
- .base.maxver = 0,
- .ctor = nv10_fifo_dma_new,
-};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv17.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv17.c
deleted file mode 100644
index edd70a1..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv17.c
+++ /dev/null
@@ -1,98 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv04.h"
-#include "regsnv04.h"
-
-#include <core/client.h>
-#include <core/gpuobj.h>
-#include <subdev/instmem.h>
-
-#include <nvif/class.h>
-#include <nvif/cl006b.h>
-#include <nvif/unpack.h>
-
-static int
-nv17_fifo_dma_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct nv03_channel_dma_v0 v0;
- } *args = data;
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nv04_fifo_chan *chan = NULL;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_instmem *imem = device->imem;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel dma size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel dma vers %d pushbuf %llx "
- "offset %08x\n", args->v0.version,
- args->v0.pushbuf, args->v0.offset);
- if (!args->v0.pushbuf)
- return -EINVAL;
- } else
- return ret;
-
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
-
- ret = nvkm_fifo_chan_ctor(&nv04_fifo_dma_func, &fifo->base,
- 0x1000, 0x1000, false, 0, args->v0.pushbuf,
- BIT(NV04_FIFO_ENGN_SW) |
- BIT(NV04_FIFO_ENGN_GR) |
- BIT(NV04_FIFO_ENGN_MPEG) | /* NV31- */
- BIT(NV04_FIFO_ENGN_DMA),
- 0, 0x800000, 0x10000, oclass, &chan->base);
- chan->fifo = fifo;
- if (ret)
- return ret;
-
- args->v0.chid = chan->base.chid;
- chan->ramfc = chan->base.chid * 64;
-
- nvkm_kmap(imem->ramfc);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x00, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x04, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x0c, chan->base.push->addr >> 4);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x14,
- NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
- NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
-#ifdef __BIG_ENDIAN
- NV_PFIFO_CACHE1_BIG_ENDIAN |
-#endif
- NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
- nvkm_done(imem->ramfc);
- return 0;
-}
-
-const struct nvkm_fifo_chan_oclass
-nv17_fifo_dma_oclass = {
- .base.oclass = NV17_CHANNEL_DMA,
- .base.minver = 0,
- .base.maxver = 0,
- .ctor = nv17_fifo_dma_new,
-};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv40.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv40.c
deleted file mode 100644
index 0411fb9..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/dmanv40.c
+++ /dev/null
@@ -1,254 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv04.h"
-#include "regsnv04.h"
-
-#include <core/client.h>
-#include <core/ramht.h>
-#include <subdev/instmem.h>
-
-#include <nvif/class.h>
-#include <nvif/cl006b.h>
-#include <nvif/unpack.h>
-
-static bool
-nv40_fifo_dma_engine(struct nvkm_engine *engine, u32 *reg, u32 *ctx)
-{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_DMAOBJ:
- case NVKM_ENGINE_SW:
- return false;
- case NVKM_ENGINE_GR:
- *reg = 0x0032e0;
- *ctx = 0x38;
- return true;
- case NVKM_ENGINE_MPEG:
- if (engine->subdev.device->chipset < 0x44)
- return false;
- *reg = 0x00330c;
- *ctx = 0x54;
- return true;
- default:
- WARN_ON(1);
- return false;
- }
-}
-
-static struct nvkm_gpuobj **
-nv40_fifo_dma_engn(struct nv04_fifo_chan *chan, struct nvkm_engine *engine)
-{
- int engi = chan->base.fifo->func->engine_id(chan->base.fifo, engine);
- if (engi >= 0)
- return &chan->engn[engi];
- return NULL;
-}
-
-static int
-nv40_fifo_dma_engine_fini(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine, bool suspend)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nv04_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_instmem *imem = device->imem;
- unsigned long flags;
- u32 reg, ctx;
- int chid;
-
- if (!nv40_fifo_dma_engine(engine, ®, &ctx))
- return 0;
-
- spin_lock_irqsave(&fifo->base.lock, flags);
- nvkm_mask(device, 0x002500, 0x00000001, 0x00000000);
-
- chid = nvkm_rd32(device, 0x003204) & (fifo->base.nr - 1);
- if (chid == chan->base.chid)
- nvkm_wr32(device, reg, 0x00000000);
- nvkm_kmap(imem->ramfc);
- nvkm_wo32(imem->ramfc, chan->ramfc + ctx, 0x00000000);
- nvkm_done(imem->ramfc);
-
- nvkm_mask(device, 0x002500, 0x00000001, 0x00000001);
- spin_unlock_irqrestore(&fifo->base.lock, flags);
- return 0;
-}
-
-static int
-nv40_fifo_dma_engine_init(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nv04_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_instmem *imem = device->imem;
- unsigned long flags;
- u32 inst, reg, ctx;
- int chid;
-
- if (!nv40_fifo_dma_engine(engine, ®, &ctx))
- return 0;
- inst = (*nv40_fifo_dma_engn(chan, engine))->addr >> 4;
-
- spin_lock_irqsave(&fifo->base.lock, flags);
- nvkm_mask(device, 0x002500, 0x00000001, 0x00000000);
-
- chid = nvkm_rd32(device, 0x003204) & (fifo->base.nr - 1);
- if (chid == chan->base.chid)
- nvkm_wr32(device, reg, inst);
- nvkm_kmap(imem->ramfc);
- nvkm_wo32(imem->ramfc, chan->ramfc + ctx, inst);
- nvkm_done(imem->ramfc);
-
- nvkm_mask(device, 0x002500, 0x00000001, 0x00000001);
- spin_unlock_irqrestore(&fifo->base.lock, flags);
- return 0;
-}
-
-static void
-nv40_fifo_dma_engine_dtor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- nvkm_gpuobj_del(nv40_fifo_dma_engn(chan, engine));
-}
-
-static int
-nv40_fifo_dma_engine_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine,
- struct nvkm_object *object)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- u32 reg, ctx;
-
- if (!nv40_fifo_dma_engine(engine, ®, &ctx))
- return 0;
-
- return nvkm_object_bind(object, NULL, 0, nv40_fifo_dma_engn(chan, engine));
-}
-
-static int
-nv40_fifo_dma_object_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_object *object)
-{
- struct nv04_fifo_chan *chan = nv04_fifo_chan(base);
- struct nvkm_instmem *imem = chan->fifo->base.engine.subdev.device->imem;
- u32 context = chan->base.chid << 23;
- u32 handle = object->handle;
- int hash;
-
- switch (object->engine->subdev.type) {
- case NVKM_ENGINE_DMAOBJ:
- case NVKM_ENGINE_SW : context |= 0x00000000; break;
- case NVKM_ENGINE_GR : context |= 0x00100000; break;
- case NVKM_ENGINE_MPEG : context |= 0x00200000; break;
- default:
- WARN_ON(1);
- return -EINVAL;
- }
-
- mutex_lock(&chan->fifo->base.mutex);
- hash = nvkm_ramht_insert(imem->ramht, object, chan->base.chid, 4,
- handle, context);
- mutex_unlock(&chan->fifo->base.mutex);
- return hash;
-}
-
-static const struct nvkm_fifo_chan_func
-nv40_fifo_dma_func = {
- .dtor = nv04_fifo_dma_dtor,
- .init = nv04_fifo_dma_init,
- .fini = nv04_fifo_dma_fini,
- .engine_ctor = nv40_fifo_dma_engine_ctor,
- .engine_dtor = nv40_fifo_dma_engine_dtor,
- .engine_init = nv40_fifo_dma_engine_init,
- .engine_fini = nv40_fifo_dma_engine_fini,
- .object_ctor = nv40_fifo_dma_object_ctor,
- .object_dtor = nv04_fifo_dma_object_dtor,
-};
-
-static int
-nv40_fifo_dma_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct nv03_channel_dma_v0 v0;
- } *args = data;
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nv04_fifo_chan *chan = NULL;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_instmem *imem = device->imem;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel dma size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel dma vers %d pushbuf %llx "
- "offset %08x\n", args->v0.version,
- args->v0.pushbuf, args->v0.offset);
- if (!args->v0.pushbuf)
- return -EINVAL;
- } else
- return ret;
-
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
-
- ret = nvkm_fifo_chan_ctor(&nv40_fifo_dma_func, &fifo->base,
- 0x1000, 0x1000, false, 0, args->v0.pushbuf,
- BIT(NV04_FIFO_ENGN_SW) |
- BIT(NV04_FIFO_ENGN_GR) |
- BIT(NV04_FIFO_ENGN_MPEG) |
- BIT(NV04_FIFO_ENGN_DMA),
- 0, 0xc00000, 0x1000, oclass, &chan->base);
- chan->fifo = fifo;
- if (ret)
- return ret;
-
- args->v0.chid = chan->base.chid;
- chan->ramfc = chan->base.chid * 128;
-
- nvkm_kmap(imem->ramfc);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x00, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x04, args->v0.offset);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x0c, chan->base.push->addr >> 4);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x18, 0x30000000 |
- NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
- NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
-#ifdef __BIG_ENDIAN
- NV_PFIFO_CACHE1_BIG_ENDIAN |
-#endif
- NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
- nvkm_wo32(imem->ramfc, chan->ramfc + 0x3c, 0x0001ffff);
- nvkm_done(imem->ramfc);
- return 0;
-}
-
-const struct nvkm_fifo_chan_oclass
-nv40_fifo_dma_oclass = {
- .base.oclass = NV40_CHANNEL_DMA,
- .base.minver = 0,
- .base.maxver = 0,
- .ctor = nv40_fifo_dma_new,
-};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/g84.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/g84.c
index 3885c38..6b229a3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/g84.c
@@ -21,112 +21,211 @@
*
* Authors: Ben Skeggs
*/
-#include "nv50.h"
-#include "channv50.h"
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "runl.h"
+
+#include <core/ramht.h>
+#include <subdev/timer.h>
+
+#include <nvif/class.h>
static void
-g84_fifo_uevent_fini(struct nvkm_fifo *fifo)
+g84_chan_bind(struct nvkm_chan *chan)
{
- struct nvkm_device *device = fifo->engine.subdev.device;
- nvkm_mask(device, 0x002140, 0x40000000, 0x00000000);
-}
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
-static void
-g84_fifo_uevent_init(struct nvkm_fifo *fifo)
-{
- struct nvkm_device *device = fifo->engine.subdev.device;
- nvkm_mask(device, 0x002140, 0x40000000, 0x40000000);
-}
-
-static struct nvkm_engine *
-g84_fifo_id_engine(struct nvkm_fifo *fifo, int engi)
-{
- struct nvkm_device *device = fifo->engine.subdev.device;
- struct nvkm_engine *engine;
- enum nvkm_subdev_type type;
-
- switch (engi) {
- case G84_FIFO_ENGN_SW : type = NVKM_ENGINE_SW; break;
- case G84_FIFO_ENGN_GR : type = NVKM_ENGINE_GR; break;
- case G84_FIFO_ENGN_MPEG :
- if ((engine = nvkm_device_engine(device, NVKM_ENGINE_MSPPP, 0)))
- return engine;
- type = NVKM_ENGINE_MPEG;
- break;
- case G84_FIFO_ENGN_ME :
- if ((engine = nvkm_device_engine(device, NVKM_ENGINE_CE, 0)))
- return engine;
- type = NVKM_ENGINE_ME;
- break;
- case G84_FIFO_ENGN_VP :
- if ((engine = nvkm_device_engine(device, NVKM_ENGINE_MSPDEC, 0)))
- return engine;
- type = NVKM_ENGINE_VP;
- break;
- case G84_FIFO_ENGN_CIPHER:
- if ((engine = nvkm_device_engine(device, NVKM_ENGINE_VIC, 0)))
- return engine;
- if ((engine = nvkm_device_engine(device, NVKM_ENGINE_SEC, 0)))
- return engine;
- type = NVKM_ENGINE_CIPHER;
- break;
- case G84_FIFO_ENGN_BSP :
- if ((engine = nvkm_device_engine(device, NVKM_ENGINE_MSVLD, 0)))
- return engine;
- type = NVKM_ENGINE_BSP;
- break;
- case G84_FIFO_ENGN_DMA : type = NVKM_ENGINE_DMAOBJ; break;
- default:
- WARN_ON(1);
- return NULL;
- }
-
- return nvkm_device_engine(fifo->engine.subdev.device, type, 0);
+ nvkm_wr32(device, 0x002600 + (chan->id * 4), chan->ramfc->addr >> 8);
}
static int
-g84_fifo_engine_id(struct nvkm_fifo *base, struct nvkm_engine *engine)
+g84_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_SW : return G84_FIFO_ENGN_SW;
- case NVKM_ENGINE_GR : return G84_FIFO_ENGN_GR;
- case NVKM_ENGINE_MPEG :
- case NVKM_ENGINE_MSPPP : return G84_FIFO_ENGN_MPEG;
- case NVKM_ENGINE_CE : return G84_FIFO_ENGN_CE0;
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+ const u32 limit2 = ilog2(length / 8);
+ int ret;
+
+ ret = nvkm_gpuobj_new(device, 0x0200, 0, true, chan->inst, &chan->eng);
+ if (ret)
+ return ret;
+
+ ret = nvkm_gpuobj_new(device, 0x4000, 0, false, chan->inst, &chan->pgd);
+ if (ret)
+ return ret;
+
+ ret = nvkm_gpuobj_new(device, 0x1000, 0x400, true, chan->inst, &chan->cache);
+ if (ret)
+ return ret;
+
+ ret = nvkm_gpuobj_new(device, 0x100, 0x100, true, chan->inst, &chan->ramfc);
+ if (ret)
+ return ret;
+
+ ret = nvkm_ramht_new(device, 0x8000, 16, chan->inst, &chan->ramht);
+ if (ret)
+ return ret;
+
+ nvkm_kmap(chan->ramfc);
+ nvkm_wo32(chan->ramfc, 0x3c, 0x403f6078);
+ nvkm_wo32(chan->ramfc, 0x44, 0x01003fff);
+ nvkm_wo32(chan->ramfc, 0x48, chan->push->node->offset >> 4);
+ nvkm_wo32(chan->ramfc, 0x50, lower_32_bits(offset));
+ nvkm_wo32(chan->ramfc, 0x54, upper_32_bits(offset) | (limit2 << 16));
+ nvkm_wo32(chan->ramfc, 0x60, 0x7fffffff);
+ nvkm_wo32(chan->ramfc, 0x78, 0x00000000);
+ nvkm_wo32(chan->ramfc, 0x7c, 0x30000000 | devm);
+ nvkm_wo32(chan->ramfc, 0x80, ((chan->ramht->bits - 9) << 27) |
+ (4 << 24) /* SEARCH_FULL */ |
+ (chan->ramht->gpuobj->node->offset >> 4));
+ nvkm_wo32(chan->ramfc, 0x88, chan->cache->addr >> 10);
+ nvkm_wo32(chan->ramfc, 0x98, chan->inst->addr >> 12);
+ nvkm_done(chan->ramfc);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+g84_chan_ramfc = {
+ .write = g84_chan_ramfc_write,
+ .ctxdma = true,
+ .devm = 0xfff,
+};
+
+const struct nvkm_chan_func
+g84_chan = {
+ .inst = &nv50_chan_inst,
+ .userd = &nv50_chan_userd,
+ .ramfc = &g84_chan_ramfc,
+ .bind = g84_chan_bind,
+ .unbind = nv50_chan_unbind,
+ .start = nv50_chan_start,
+ .stop = nv50_chan_stop,
+};
+
+static void
+g84_ectx_bind(struct nvkm_engn *engn, struct nvkm_cctx *cctx, struct nvkm_chan *chan)
+{
+ struct nvkm_subdev *subdev = &chan->cgrp->runl->fifo->engine.subdev;
+ struct nvkm_device *device = subdev->device;
+ u64 start = 0, limit = 0;
+ u32 flags = 0, ptr0, save;
+
+ switch (engn->engine->subdev.type) {
+ case NVKM_ENGINE_GR : ptr0 = 0x0020; break;
case NVKM_ENGINE_VP :
- case NVKM_ENGINE_MSPDEC: return G84_FIFO_ENGN_VP;
- case NVKM_ENGINE_CIPHER:
- case NVKM_ENGINE_SEC : return G84_FIFO_ENGN_CIPHER;
+ case NVKM_ENGINE_MSPDEC: ptr0 = 0x0040; break;
+ case NVKM_ENGINE_MPEG :
+ case NVKM_ENGINE_MSPPP : ptr0 = 0x0060; break;
case NVKM_ENGINE_BSP :
- case NVKM_ENGINE_MSVLD : return G84_FIFO_ENGN_BSP;
- case NVKM_ENGINE_DMAOBJ: return G84_FIFO_ENGN_DMA;
+ case NVKM_ENGINE_MSVLD : ptr0 = 0x0080; break;
+ case NVKM_ENGINE_CIPHER:
+ case NVKM_ENGINE_SEC : ptr0 = 0x00a0; break;
+ case NVKM_ENGINE_CE : ptr0 = 0x00c0; break;
default:
WARN_ON(1);
- return -1;
+ return;
}
+
+ if (!cctx) {
+ save = nvkm_mask(device, 0x002520, 0x0000003f, BIT(engn->id - 1));
+ nvkm_wr32(device, 0x0032fc, chan->inst->addr >> 12);
+ nvkm_msec(device, 2000,
+ if (nvkm_rd32(device, 0x0032fc) != 0xffffffff)
+ break;
+ );
+ nvkm_wr32(device, 0x002520, save);
+ } else {
+ flags = 0x00190000;
+ start = cctx->vctx->inst->addr;
+ limit = start + cctx->vctx->inst->size - 1;
+ }
+
+ nvkm_kmap(chan->eng);
+ nvkm_wo32(chan->eng, ptr0 + 0x00, flags);
+ nvkm_wo32(chan->eng, ptr0 + 0x04, lower_32_bits(limit));
+ nvkm_wo32(chan->eng, ptr0 + 0x08, lower_32_bits(start));
+ nvkm_wo32(chan->eng, ptr0 + 0x0c, upper_32_bits(limit) << 24 |
+ lower_32_bits(start));
+ nvkm_wo32(chan->eng, ptr0 + 0x10, 0x00000000);
+ nvkm_wo32(chan->eng, ptr0 + 0x14, 0x00000000);
+ nvkm_done(chan->eng);
+}
+
+const struct nvkm_engn_func
+g84_engn = {
+ .bind = g84_ectx_bind,
+ .ramht_add = nv50_eobj_ramht_add,
+ .ramht_del = nv50_eobj_ramht_del,
+};
+
+static void
+g84_fifo_nonstall_block(struct nvkm_event *event, int type, int index)
+{
+ struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), nonstall.event);
+ unsigned long flags;
+
+ spin_lock_irqsave(&fifo->lock, flags);
+ nvkm_mask(fifo->engine.subdev.device, 0x002140, 0x40000000, 0x00000000);
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+static void
+g84_fifo_nonstall_allow(struct nvkm_event *event, int type, int index)
+{
+ struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), nonstall.event);
+ unsigned long flags;
+
+ spin_lock_irqsave(&fifo->lock, flags);
+ nvkm_mask(fifo->engine.subdev.device, 0x002140, 0x40000000, 0x40000000);
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+const struct nvkm_event_func
+g84_fifo_nonstall = {
+ .init = g84_fifo_nonstall_allow,
+ .fini = g84_fifo_nonstall_block,
+};
+
+static int
+g84_fifo_runl_ctor(struct nvkm_fifo *fifo)
+{
+ struct nvkm_runl *runl;
+
+ runl = nvkm_runl_new(fifo, 0, 0, 0);
+ if (IS_ERR(runl))
+ return PTR_ERR(runl);
+
+ nvkm_runl_add(runl, 0, fifo->func->engn_sw, NVKM_ENGINE_SW, 0);
+ nvkm_runl_add(runl, 0, fifo->func->engn_sw, NVKM_ENGINE_DMAOBJ, 0);
+ nvkm_runl_add(runl, 1, fifo->func->engn, NVKM_ENGINE_GR, 0);
+ nvkm_runl_add(runl, 2, fifo->func->engn, NVKM_ENGINE_MPEG, 0);
+ nvkm_runl_add(runl, 3, fifo->func->engn, NVKM_ENGINE_ME, 0);
+ nvkm_runl_add(runl, 4, fifo->func->engn, NVKM_ENGINE_VP, 0);
+ nvkm_runl_add(runl, 5, fifo->func->engn, NVKM_ENGINE_CIPHER, 0);
+ nvkm_runl_add(runl, 6, fifo->func->engn, NVKM_ENGINE_BSP, 0);
+ return 0;
}
static const struct nvkm_fifo_func
g84_fifo = {
- .dtor = nv50_fifo_dtor,
- .oneinit = nv50_fifo_oneinit,
+ .chid_nr = nv50_fifo_chid_nr,
+ .chid_ctor = nv50_fifo_chid_ctor,
+ .runl_ctor = g84_fifo_runl_ctor,
.init = nv50_fifo_init,
.intr = nv04_fifo_intr,
- .engine_id = g84_fifo_engine_id,
- .id_engine = g84_fifo_id_engine,
.pause = nv04_fifo_pause,
.start = nv04_fifo_start,
- .uevent_init = g84_fifo_uevent_init,
- .uevent_fini = g84_fifo_uevent_fini,
- .chan = {
- &g84_fifo_gpfifo_oclass,
- NULL
- },
+ .nonstall = &g84_fifo_nonstall,
+ .runl = &nv50_runl,
+ .engn = &g84_engn,
+ .engn_sw = &nv50_engn_sw,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, G82_CHANNEL_GPFIFO }, &g84_chan },
};
int
g84_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return nv50_fifo_new_(&g84_fifo, device, type, inst, pfifo);
+ return nvkm_fifo_new_(&g84_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/g98.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/g98.c
new file mode 100644
index 0000000..c6ca050
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/g98.c
@@ -0,0 +1,70 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+#include "chan.h"
+#include "runl.h"
+
+#include <nvif/class.h>
+
+static int
+g98_fifo_runl_ctor(struct nvkm_fifo *fifo)
+{
+ struct nvkm_runl *runl;
+
+ runl = nvkm_runl_new(fifo, 0, 0, 0);
+ if (IS_ERR(runl))
+ return PTR_ERR(runl);
+
+ nvkm_runl_add(runl, 0, fifo->func->engn_sw, NVKM_ENGINE_SW, 0);
+ nvkm_runl_add(runl, 0, fifo->func->engn_sw, NVKM_ENGINE_DMAOBJ, 0);
+ nvkm_runl_add(runl, 1, fifo->func->engn, NVKM_ENGINE_GR, 0);
+ nvkm_runl_add(runl, 2, fifo->func->engn, NVKM_ENGINE_MSPPP, 0);
+ nvkm_runl_add(runl, 3, fifo->func->engn, NVKM_ENGINE_CE, 0);
+ nvkm_runl_add(runl, 4, fifo->func->engn, NVKM_ENGINE_MSPDEC, 0);
+ nvkm_runl_add(runl, 5, fifo->func->engn, NVKM_ENGINE_SEC, 0);
+ nvkm_runl_add(runl, 6, fifo->func->engn, NVKM_ENGINE_MSVLD, 0);
+ return 0;
+}
+
+static const struct nvkm_fifo_func
+g98_fifo = {
+ .chid_nr = nv50_fifo_chid_nr,
+ .chid_ctor = nv50_fifo_chid_ctor,
+ .runl_ctor = g98_fifo_runl_ctor,
+ .init = nv50_fifo_init,
+ .intr = nv04_fifo_intr,
+ .pause = nv04_fifo_pause,
+ .start = nv04_fifo_start,
+ .nonstall = &g84_fifo_nonstall,
+ .runl = &nv50_runl,
+ .engn = &g84_engn,
+ .engn_sw = &nv50_engn_sw,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, G82_CHANNEL_GPFIFO }, &g84_chan },
+};
+
+int
+g98_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_fifo **pfifo)
+{
+ return nvkm_fifo_new_(&g98_fifo, device, type, inst, pfifo);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga100.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga100.c
new file mode 100644
index 0000000..12a5d99
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga100.c
@@ -0,0 +1,550 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+#include "runq.h"
+
+#include <core/gpuobj.h>
+#include <subdev/top.h>
+#include <subdev/vfn.h>
+
+#include <nvif/class.h>
+
+/*TODO: allocate? */
+#define GA100_FIFO_NONSTALL_VECTOR 0
+
+static u32
+ga100_chan_doorbell_handle(struct nvkm_chan *chan)
+{
+ return (chan->cgrp->runl->doorbell << 16) | chan->id;
+}
+
+static void
+ga100_chan_stop(struct nvkm_chan *chan)
+{
+ struct nvkm_runl *runl = chan->cgrp->runl;
+
+ nvkm_wr32(runl->fifo->engine.subdev.device, runl->chan + (chan->id * 4), 0x00000003);
+}
+
+static void
+ga100_chan_start(struct nvkm_chan *chan)
+{
+ struct nvkm_runl *runl = chan->cgrp->runl;
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+ const int gfid = 0;
+
+ nvkm_wr32(device, runl->chan + (chan->id * 4), 0x00000002);
+ nvkm_wr32(device, runl->addr + 0x0090, (gfid << 16) | chan->id); /* INTERNAL_DOORBELL. */
+}
+
+static void
+ga100_chan_unbind(struct nvkm_chan *chan)
+{
+ struct nvkm_runl *runl = chan->cgrp->runl;
+
+ nvkm_wr32(runl->fifo->engine.subdev.device, runl->chan + (chan->id * 4), 0xffffffff);
+}
+
+static int
+ga100_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ const u32 limit2 = ilog2(length / 8);
+
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, 0x010, 0x0000face);
+ nvkm_wo32(chan->inst, 0x030, 0x7ffff902);
+ nvkm_wo32(chan->inst, 0x048, lower_32_bits(offset));
+ nvkm_wo32(chan->inst, 0x04c, upper_32_bits(offset) | (limit2 << 16));
+ nvkm_wo32(chan->inst, 0x084, 0x20400000);
+ nvkm_wo32(chan->inst, 0x094, 0x30000000 | devm);
+ nvkm_wo32(chan->inst, 0x0e4, priv ? 0x00000020 : 0x00000000);
+ nvkm_wo32(chan->inst, 0x0e8, chan->id);
+ nvkm_wo32(chan->inst, 0x0f4, 0x00001000 | (priv ? 0x00000100 : 0x00000000));
+ nvkm_wo32(chan->inst, 0x0f8, 0x80000000 | GA100_FIFO_NONSTALL_VECTOR);
+ nvkm_mo32(chan->inst, 0x218, 0x00000000, 0x00000000);
+ nvkm_done(chan->inst);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+ga100_chan_ramfc = {
+ .write = ga100_chan_ramfc_write,
+ .devm = 0xfff,
+ .priv = true,
+};
+
+const struct nvkm_chan_func
+ga100_chan = {
+ .inst = &gf100_chan_inst,
+ .userd = &gv100_chan_userd,
+ .ramfc = &ga100_chan_ramfc,
+ .unbind = ga100_chan_unbind,
+ .start = ga100_chan_start,
+ .stop = ga100_chan_stop,
+ .preempt = gk110_chan_preempt,
+ .doorbell_handle = ga100_chan_doorbell_handle,
+};
+
+static void
+ga100_cgrp_preempt(struct nvkm_cgrp *cgrp)
+{
+ struct nvkm_runl *runl = cgrp->runl;
+
+ nvkm_wr32(runl->fifo->engine.subdev.device, runl->addr + 0x098, 0x01000000 | cgrp->id);
+}
+
+const struct nvkm_cgrp_func
+ga100_cgrp = {
+ .preempt = ga100_cgrp_preempt,
+};
+
+static int
+ga100_engn_cxid(struct nvkm_engn *engn, bool *cgid)
+{
+ struct nvkm_runl *runl = engn->runl;
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+ u32 stat = nvkm_rd32(device, runl->addr + 0x200 + engn->id * 0x40);
+
+ ENGN_DEBUG(engn, "status %08x", stat);
+ *cgid = true;
+
+ switch ((stat & 0x0000e000) >> 13) {
+ case 0 /* INVALID */: return -ENODEV;
+ case 1 /* VALID */:
+ case 5 /* SAVE */: return (stat & 0x00000fff);
+ case 6 /* LOAD */: return (stat & 0x0fff0000) >> 16;
+ case 7 /* SWITCH */:
+ if (nvkm_engine_chsw_load(engn->engine))
+ return (stat & 0x0fff0000) >> 16;
+ return (stat & 0x00000fff);
+ default:
+ WARN_ON(1);
+ break;
+ }
+
+ return -ENODEV;
+}
+
+const struct nvkm_engn_func
+ga100_engn = {
+ .cxid = ga100_engn_cxid,
+ .ctor = gk104_ectx_ctor,
+ .bind = gv100_ectx_bind,
+};
+
+const struct nvkm_engn_func
+ga100_engn_ce = {
+ .cxid = ga100_engn_cxid,
+ .ctor = gv100_ectx_ce_ctor,
+ .bind = gv100_ectx_ce_bind,
+};
+
+static bool
+ga100_runq_idle(struct nvkm_runq *runq)
+{
+ struct nvkm_device *device = runq->fifo->engine.subdev.device;
+
+ return !(nvkm_rd32(device, 0x04015c + (runq->id * 0x800)) & 0x0000e000);
+}
+
+static bool
+ga100_runq_intr_1(struct nvkm_runq *runq, struct nvkm_runl *runl)
+{
+ struct nvkm_device *device = runq->fifo->engine.subdev.device;
+ u32 inte = nvkm_rd32(device, 0x040180 + (runq->id * 0x800));
+ u32 intr = nvkm_rd32(device, 0x040148 + (runq->id * 0x800));
+ u32 stat = intr & inte;
+
+ if (!stat) {
+ RUNQ_DEBUG(runq, "inte1 %08x %08x", intr, inte);
+ return false;
+ }
+
+ if (stat & 0x80000000) {
+ u32 chid = nvkm_rd32(device, 0x040120 + (runq->id * 0x0800)) & runl->chid->mask;
+ struct nvkm_chan *chan;
+ unsigned long flags;
+
+ RUNQ_ERROR(runq, "CTXNOTVALID chid:%d", chid);
+ chan = nvkm_runl_chan_get_chid(runl, chid, &flags);
+ if (chan) {
+ nvkm_chan_error(chan, true);
+ nvkm_chan_put(&chan, flags);
+ }
+
+ nvkm_mask(device, 0x0400ac + (runq->id * 0x800), 0x00030000, 0x00030000);
+ stat &= ~0x80000000;
+ }
+
+ if (stat) {
+ RUNQ_ERROR(runq, "intr1 %08x", stat);
+ nvkm_wr32(device, 0x0401a0 + (runq->id * 0x800), stat);
+ }
+
+ nvkm_wr32(device, 0x040148 + (runq->id * 0x800), intr);
+ return true;
+}
+
+static bool
+ga100_runq_intr_0(struct nvkm_runq *runq, struct nvkm_runl *runl)
+{
+ struct nvkm_device *device = runq->fifo->engine.subdev.device;
+ u32 inte = nvkm_rd32(device, 0x040170 + (runq->id * 0x800));
+ u32 intr = nvkm_rd32(device, 0x040108 + (runq->id * 0x800));
+ u32 stat = intr & inte;
+
+ if (!stat) {
+ RUNQ_DEBUG(runq, "inte0 %08x %08x", intr, inte);
+ return false;
+ }
+
+ /*TODO: expand on this when fixing up gf100's version. */
+ if (stat & 0xc6afe000) {
+ u32 chid = nvkm_rd32(device, 0x040120 + (runq->id * 0x0800)) & runl->chid->mask;
+ struct nvkm_chan *chan;
+ unsigned long flags;
+
+ RUNQ_ERROR(runq, "intr0 %08x", stat);
+ chan = nvkm_runl_chan_get_chid(runl, chid, &flags);
+ if (chan) {
+ nvkm_chan_error(chan, true);
+ nvkm_chan_put(&chan, flags);
+ }
+
+ stat &= ~0xc6afe000;
+ }
+
+ if (stat) {
+ RUNQ_ERROR(runq, "intr0 %08x", stat);
+ nvkm_wr32(device, 0x040190 + (runq->id * 0x800), stat);
+ }
+
+ nvkm_wr32(device, 0x040108 + (runq->id * 0x800), intr);
+ return true;
+}
+
+static bool
+ga100_runq_intr(struct nvkm_runq *runq, struct nvkm_runl *runl)
+{
+ bool intr0 = ga100_runq_intr_0(runq, runl);
+ bool intr1 = ga100_runq_intr_1(runq, runl);
+
+ return intr0 || intr1;
+}
+
+static void
+ga100_runq_init(struct nvkm_runq *runq)
+{
+ struct nvkm_device *device = runq->fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x040108 + (runq->id * 0x800), 0xffffffff); /* INTR_0 */
+ nvkm_wr32(device, 0x040148 + (runq->id * 0x800), 0xffffffff); /* INTR_1 */
+ nvkm_wr32(device, 0x040170 + (runq->id * 0x800), 0xffffffff); /* INTR_0_EN_SET_TREE */
+ nvkm_wr32(device, 0x040180 + (runq->id * 0x800), 0xffffffff); /* INTR_1_EN_SET_TREE */
+}
+
+const struct nvkm_runq_func
+ga100_runq = {
+ .init = ga100_runq_init,
+ .intr = ga100_runq_intr,
+ .idle = ga100_runq_idle,
+};
+
+static bool
+ga100_runl_preempt_pending(struct nvkm_runl *runl)
+{
+ return nvkm_rd32(runl->fifo->engine.subdev.device, runl->addr + 0x098) & 0x00100000;
+}
+
+static void
+ga100_runl_preempt(struct nvkm_runl *runl)
+{
+ nvkm_wr32(runl->fifo->engine.subdev.device, runl->addr + 0x098, 0x00000000);
+}
+
+static void
+ga100_runl_allow(struct nvkm_runl *runl, u32 engm)
+{
+ nvkm_mask(runl->fifo->engine.subdev.device, runl->addr + 0x094, 0x00000001, 0x00000000);
+}
+
+static void
+ga100_runl_block(struct nvkm_runl *runl, u32 engm)
+{
+ nvkm_mask(runl->fifo->engine.subdev.device, runl->addr + 0x094, 0x00000001, 0x00000001);
+}
+
+static bool
+ga100_runl_pending(struct nvkm_runl *runl)
+{
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+
+ return nvkm_rd32(device, runl->addr + 0x08c) & 0x00008000;
+}
+
+static void
+ga100_runl_commit(struct nvkm_runl *runl, struct nvkm_memory *memory, u32 start, int count)
+{
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+ u64 addr = nvkm_memory_addr(memory) + start;
+
+ nvkm_wr32(device, runl->addr + 0x080, lower_32_bits(addr));
+ nvkm_wr32(device, runl->addr + 0x084, upper_32_bits(addr));
+ nvkm_wr32(device, runl->addr + 0x088, count);
+}
+
+static irqreturn_t
+ga100_runl_intr(struct nvkm_inth *inth)
+{
+ struct nvkm_runl *runl = container_of(inth, typeof(*runl), inth);
+ struct nvkm_engn *engn;
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+ u32 inte = nvkm_rd32(device, runl->addr + 0x120);
+ u32 intr = nvkm_rd32(device, runl->addr + 0x100);
+ u32 stat = intr & inte;
+ u32 info;
+
+ if (!stat) {
+ RUNL_DEBUG(runl, "inte %08x %08x", intr, inte);
+ return IRQ_NONE;
+ }
+
+ if (stat & 0x00000007) {
+ nvkm_runl_foreach_engn_cond(engn, runl, stat & BIT(engn->id)) {
+ info = nvkm_rd32(device, runl->addr + 0x224 + (engn->id * 0x40));
+
+ tu102_fifo_intr_ctxsw_timeout_info(engn, info);
+
+ nvkm_wr32(device, runl->addr + 0x100, BIT(engn->id));
+ stat &= ~BIT(engn->id);
+ }
+ }
+
+ if (stat & 0x00000300) {
+ nvkm_wr32(device, runl->addr + 0x100, stat & 0x00000300);
+ stat &= ~0x00000300;
+ }
+
+ if (stat & 0x00010000) {
+ if (runl->runq[0]) {
+ if (runl->runq[0]->func->intr(runl->runq[0], runl))
+ stat &= ~0x00010000;
+ }
+ }
+
+ if (stat & 0x00020000) {
+ if (runl->runq[1]) {
+ if (runl->runq[1]->func->intr(runl->runq[1], runl))
+ stat &= ~0x00020000;
+ }
+ }
+
+ if (stat) {
+ RUNL_ERROR(runl, "intr %08x", stat);
+ nvkm_wr32(device, runl->addr + 0x140, stat);
+ }
+
+ nvkm_wr32(device, runl->addr + 0x180, 0x00000001);
+ return IRQ_HANDLED;
+}
+
+static void
+ga100_runl_fini(struct nvkm_runl *runl)
+{
+ nvkm_mask(runl->fifo->engine.subdev.device, runl->addr + 0x300, 0x80000000, 0x00000000);
+ nvkm_inth_block(&runl->inth);
+}
+
+static void
+ga100_runl_init(struct nvkm_runl *runl)
+{
+ struct nvkm_fifo *fifo = runl->fifo;
+ struct nvkm_runq *runq;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ int i;
+
+ /* Submit NULL runlist and preempt. */
+ nvkm_wr32(device, runl->addr + 0x088, 0x00000000);
+ runl->func->preempt(runl);
+
+ /* Enable doorbell. */
+ nvkm_mask(device, runl->addr + 0x300, 0x80000000, 0x80000000);
+
+ nvkm_wr32(device, runl->addr + 0x100, 0xffffffff); /* INTR_0 */
+ nvkm_wr32(device, runl->addr + 0x140, 0xffffffff); /* INTR_0_EN_CLEAR_TREE(0) */
+ nvkm_wr32(device, runl->addr + 0x120, 0x000f1307); /* INTR_0_EN_SET_TREE(0) */
+ nvkm_wr32(device, runl->addr + 0x148, 0xffffffff); /* INTR_0_EN_CLEAR_TREE(1) */
+ nvkm_wr32(device, runl->addr + 0x128, 0x00000000); /* INTR_0_EN_SET_TREE(1) */
+
+ /* Init PBDMA(s). */
+ for (i = 0; i < runl->runq_nr; i++) {
+ runq = runl->runq[i];
+ runq->func->init(runq);
+ }
+
+ nvkm_inth_allow(&runl->inth);
+}
+
+const struct nvkm_runl_func
+ga100_runl = {
+ .init = ga100_runl_init,
+ .fini = ga100_runl_fini,
+ .size = 16,
+ .update = nv50_runl_update,
+ .insert_cgrp = gv100_runl_insert_cgrp,
+ .insert_chan = gv100_runl_insert_chan,
+ .commit = ga100_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = ga100_runl_pending,
+ .block = ga100_runl_block,
+ .allow = ga100_runl_allow,
+ .preempt = ga100_runl_preempt,
+ .preempt_pending = ga100_runl_preempt_pending,
+};
+
+static int
+ga100_runl_new(struct nvkm_fifo *fifo, int id, u32 addr, struct nvkm_runl **prunl)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_runl *runl;
+ u32 chcfg = nvkm_rd32(device, addr + 0x004);
+ u32 chnum = 1 << (chcfg & 0x0000000f);
+ u32 chaddr = (chcfg & 0xfffffff0);
+ u32 dbcfg = nvkm_rd32(device, addr + 0x008);
+ u32 vector = nvkm_rd32(device, addr + 0x160);
+ int i, ret;
+
+ runl = *prunl = nvkm_runl_new(fifo, id, addr, chnum);
+ if (IS_ERR(runl))
+ return PTR_ERR(runl);
+
+ for (i = 0; i < 2; i++) {
+ u32 pbcfg = nvkm_rd32(device, addr + 0x010 + (i * 0x04));
+ if (pbcfg & 0x80000000) {
+ runl->runq[runl->runq_nr] =
+ nvkm_runq_new(fifo, ((pbcfg & 0x03fffc00) - 0x040000) / 0x800);
+ if (!runl->runq[runl->runq_nr])
+ return -ENOMEM;
+
+ runl->runq_nr++;
+ }
+ }
+
+ ret = nvkm_inth_add(&device->vfn->intr, vector & 0x00000fff, NVKM_INTR_PRIO_NORMAL,
+ &fifo->engine.subdev, ga100_runl_intr, &runl->inth);
+ if (ret)
+ return ret;
+
+ runl->chan = chaddr;
+ runl->doorbell = dbcfg >> 16;
+ return 0;
+}
+
+static irqreturn_t
+ga100_fifo_nonstall_intr(struct nvkm_inth *inth)
+{
+ struct nvkm_fifo *fifo = container_of(inth, typeof(*fifo), nonstall.intr);
+
+ nvkm_event_ntfy(&fifo->nonstall.event, 0, NVKM_FIFO_NONSTALL_EVENT);
+ return IRQ_HANDLED;
+}
+
+static void
+ga100_fifo_nonstall_block(struct nvkm_event *event, int type, int index)
+{
+ struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), nonstall.event);
+
+ nvkm_inth_block(&fifo->nonstall.intr);
+}
+
+static void
+ga100_fifo_nonstall_allow(struct nvkm_event *event, int type, int index)
+{
+ struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), nonstall.event);
+
+ nvkm_inth_allow(&fifo->nonstall.intr);
+}
+
+const struct nvkm_event_func
+ga100_fifo_nonstall = {
+ .init = ga100_fifo_nonstall_allow,
+ .fini = ga100_fifo_nonstall_block,
+};
+
+int
+ga100_fifo_nonstall_ctor(struct nvkm_fifo *fifo)
+{
+ return nvkm_inth_add(&fifo->engine.subdev.device->vfn->intr, GA100_FIFO_NONSTALL_VECTOR,
+ NVKM_INTR_PRIO_NORMAL, &fifo->engine.subdev, ga100_fifo_nonstall_intr,
+ &fifo->nonstall.intr);
+}
+
+int
+ga100_fifo_runl_ctor(struct nvkm_fifo *fifo)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_top_device *tdev;
+ struct nvkm_runl *runl;
+ int id = 0, ret;
+
+ nvkm_list_foreach(tdev, &device->top->device, head, tdev->runlist >= 0) {
+ runl = nvkm_runl_get(fifo, -1, tdev->runlist);
+ if (!runl) {
+ ret = ga100_runl_new(fifo, id++, tdev->runlist, &runl);
+ if (ret)
+ return ret;
+ }
+
+ if (tdev->engine < 0)
+ continue;
+
+ nvkm_runl_add(runl, tdev->engine, (tdev->type == NVKM_ENGINE_CE) ?
+ fifo->func->engn_ce : fifo->func->engn, tdev->type, tdev->inst);
+ }
+
+ return 0;
+}
+
+static const struct nvkm_fifo_func
+ga100_fifo = {
+ .runl_ctor = ga100_fifo_runl_ctor,
+ .mmu_fault = &tu102_fifo_mmu_fault,
+ .nonstall_ctor = ga100_fifo_nonstall_ctor,
+ .nonstall = &ga100_fifo_nonstall,
+ .runl = &ga100_runl,
+ .runq = &ga100_runq,
+ .engn = &ga100_engn,
+ .engn_ce = &ga100_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &ga100_cgrp, .force = true },
+ .chan = {{ 0, 0, AMPERE_CHANNEL_GPFIFO_A }, &ga100_chan },
+};
+
+int
+ga100_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_fifo **pfifo)
+{
+ return nvkm_fifo_new_(&ga100_fifo, device, type, inst, pfifo);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga102.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga102.c
index c630dbd..2cdf5da 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga102.c
@@ -19,293 +19,27 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#define ga102_fifo(p) container_of((p), struct ga102_fifo, base.engine)
-#define ga102_chan(p) container_of((p), struct ga102_chan, object)
-#include <engine/fifo.h>
-#include "user.h"
+#include "priv.h"
-#include <core/memory.h>
-#include <subdev/mmu.h>
-#include <subdev/timer.h>
-#include <subdev/top.h>
-
-#include <nvif/cl0080.h>
-#include <nvif/clc36f.h>
#include <nvif/class.h>
-struct ga102_fifo {
- struct nvkm_fifo base;
-};
-
-struct ga102_chan {
- struct nvkm_object object;
-
- struct {
- u32 runl;
- u32 chan;
- } ctrl;
-
- struct nvkm_memory *mthd;
- struct nvkm_memory *inst;
- struct nvkm_memory *user;
- struct nvkm_memory *runl;
-
- struct nvkm_vmm *vmm;
-};
-
-static int
-ga102_chan_sclass(struct nvkm_object *object, int index, struct nvkm_oclass *oclass)
-{
- if (index == 0) {
- oclass->ctor = nvkm_object_new;
- oclass->base = (struct nvkm_sclass) { -1, -1, AMPERE_DMA_COPY_B };
- return 0;
- }
-
- return -EINVAL;
-}
-
-static int
-ga102_chan_map(struct nvkm_object *object, void *argv, u32 argc,
- enum nvkm_object_map *type, u64 *addr, u64 *size)
-{
- struct ga102_chan *chan = ga102_chan(object);
- struct nvkm_device *device = chan->object.engine->subdev.device;
- u64 bar2 = nvkm_memory_bar2(chan->user);
-
- if (bar2 == ~0ULL)
- return -EFAULT;
-
- *type = NVKM_OBJECT_MAP_IO;
- *addr = device->func->resource_addr(device, 3) + bar2;
- *size = 0x1000;
- return 0;
-}
-
-static int
-ga102_chan_fini(struct nvkm_object *object, bool suspend)
-{
- struct ga102_chan *chan = ga102_chan(object);
- struct nvkm_device *device = chan->object.engine->subdev.device;
-
- nvkm_wr32(device, chan->ctrl.chan, 0x00000003);
-
- nvkm_wr32(device, chan->ctrl.runl + 0x098, 0x01000000);
- nvkm_msec(device, 2000,
- if (!(nvkm_rd32(device, chan->ctrl.runl + 0x098) & 0x00100000))
- break;
- );
-
- nvkm_wr32(device, chan->ctrl.runl + 0x088, 0);
-
- nvkm_wr32(device, chan->ctrl.chan, 0xffffffff);
- return 0;
-}
-
-static int
-ga102_chan_init(struct nvkm_object *object)
-{
- struct ga102_chan *chan = ga102_chan(object);
- struct nvkm_device *device = chan->object.engine->subdev.device;
-
- nvkm_mask(device, chan->ctrl.runl + 0x300, 0x80000000, 0x80000000);
-
- nvkm_wr32(device, chan->ctrl.runl + 0x080, lower_32_bits(nvkm_memory_addr(chan->runl)));
- nvkm_wr32(device, chan->ctrl.runl + 0x084, upper_32_bits(nvkm_memory_addr(chan->runl)));
- nvkm_wr32(device, chan->ctrl.runl + 0x088, 2);
-
- nvkm_wr32(device, chan->ctrl.chan, 0x00000002);
- nvkm_wr32(device, chan->ctrl.runl + 0x0090, 0);
- return 0;
-}
-
-static void *
-ga102_chan_dtor(struct nvkm_object *object)
-{
- struct ga102_chan *chan = ga102_chan(object);
-
- if (chan->vmm) {
- nvkm_vmm_part(chan->vmm, chan->inst);
- nvkm_vmm_unref(&chan->vmm);
- }
-
- nvkm_memory_unref(&chan->runl);
- nvkm_memory_unref(&chan->user);
- nvkm_memory_unref(&chan->inst);
- nvkm_memory_unref(&chan->mthd);
- return chan;
-}
-
-static const struct nvkm_object_func
-ga102_chan = {
- .dtor = ga102_chan_dtor,
- .init = ga102_chan_init,
- .fini = ga102_chan_fini,
- .map = ga102_chan_map,
- .sclass = ga102_chan_sclass,
-};
-
-static int
-ga102_chan_new(struct nvkm_device *device,
- const struct nvkm_oclass *oclass, void *argv, u32 argc, struct nvkm_object **pobject)
-{
- struct volta_channel_gpfifo_a_v0 *args = argv;
- struct nvkm_top_device *tdev;
- struct nvkm_vmm *vmm;
- struct ga102_chan *chan;
- int ret;
-
- if (argc != sizeof(*args))
- return -ENOSYS;
-
- vmm = nvkm_uvmm_search(oclass->client, args->vmm);
- if (IS_ERR(vmm))
- return PTR_ERR(vmm);
-
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
-
- nvkm_object_ctor(&ga102_chan, oclass, &chan->object);
- *pobject = &chan->object;
-
- list_for_each_entry(tdev, &device->top->device, head) {
- if (tdev->type == NVKM_ENGINE_CE) {
- chan->ctrl.runl = tdev->runlist;
- break;
- }
- }
-
- if (!chan->ctrl.runl)
- return -ENODEV;
-
- chan->ctrl.chan = nvkm_rd32(device, chan->ctrl.runl + 0x004) & 0xfffffff0;
-
- args->chid = 0;
- args->inst = 0;
- args->token = nvkm_rd32(device, chan->ctrl.runl + 0x008) & 0xffff0000;
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 0x1000, true, &chan->mthd);
- if (ret)
- return ret;
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 0x1000, true, &chan->inst);
- if (ret)
- return ret;
-
- nvkm_kmap(chan->inst);
- nvkm_wo32(chan->inst, 0x010, 0x0000face);
- nvkm_wo32(chan->inst, 0x030, 0x7ffff902);
- nvkm_wo32(chan->inst, 0x048, lower_32_bits(args->ioffset));
- nvkm_wo32(chan->inst, 0x04c, upper_32_bits(args->ioffset) |
- (order_base_2(args->ilength / 8) << 16));
- nvkm_wo32(chan->inst, 0x084, 0x20400000);
- nvkm_wo32(chan->inst, 0x094, 0x30000001);
- nvkm_wo32(chan->inst, 0x0ac, 0x00020000);
- nvkm_wo32(chan->inst, 0x0e4, 0x00000000);
- nvkm_wo32(chan->inst, 0x0e8, 0);
- nvkm_wo32(chan->inst, 0x0f4, 0x00001000);
- nvkm_wo32(chan->inst, 0x0f8, 0x10003080);
- nvkm_mo32(chan->inst, 0x218, 0x00000000, 0x00000000);
- nvkm_wo32(chan->inst, 0x220, lower_32_bits(nvkm_memory_bar2(chan->mthd)));
- nvkm_wo32(chan->inst, 0x224, upper_32_bits(nvkm_memory_bar2(chan->mthd)));
- nvkm_done(chan->inst);
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 0x1000, true, &chan->user);
- if (ret)
- return ret;
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 0x1000, true, &chan->runl);
- if (ret)
- return ret;
-
- nvkm_kmap(chan->runl);
- nvkm_wo32(chan->runl, 0x00, 0x80030001);
- nvkm_wo32(chan->runl, 0x04, 1);
- nvkm_wo32(chan->runl, 0x08, 0);
- nvkm_wo32(chan->runl, 0x0c, 0x00000000);
- nvkm_wo32(chan->runl, 0x10, lower_32_bits(nvkm_memory_addr(chan->user)));
- nvkm_wo32(chan->runl, 0x14, upper_32_bits(nvkm_memory_addr(chan->user)));
- nvkm_wo32(chan->runl, 0x18, lower_32_bits(nvkm_memory_addr(chan->inst)));
- nvkm_wo32(chan->runl, 0x1c, upper_32_bits(nvkm_memory_addr(chan->inst)));
- nvkm_done(chan->runl);
-
- ret = nvkm_vmm_join(vmm, chan->inst);
- if (ret)
- return ret;
-
- chan->vmm = nvkm_vmm_ref(vmm);
- return 0;
-}
-
-static const struct nvkm_device_oclass
-ga102_chan_oclass = {
- .ctor = ga102_chan_new,
-};
-
-static int
-ga102_user_new(struct nvkm_device *device,
- const struct nvkm_oclass *oclass, void *argv, u32 argc, struct nvkm_object **pobject)
-{
- return tu102_fifo_user_new(oclass, argv, argc, pobject);
-}
-
-static const struct nvkm_device_oclass
-ga102_user_oclass = {
- .ctor = ga102_user_new,
-};
-
-static int
-ga102_fifo_sclass(struct nvkm_oclass *oclass, int index, const struct nvkm_device_oclass **class)
-{
- if (index == 0) {
- oclass->base = (struct nvkm_sclass) { -1, -1, VOLTA_USERMODE_A };
- *class = &ga102_user_oclass;
- return 0;
- } else
- if (index == 1) {
- oclass->base = (struct nvkm_sclass) { 0, 0, AMPERE_CHANNEL_GPFIFO_B };
- *class = &ga102_chan_oclass;
- return 0;
- }
-
- return 2;
-}
-
-static int
-ga102_fifo_info(struct nvkm_engine *engine, u64 mthd, u64 *data)
-{
- switch (mthd) {
- case NV_DEVICE_HOST_CHANNELS: *data = 1; return 0;
- default:
- break;
- }
-
- return -ENOSYS;
-}
-
-static void *
-ga102_fifo_dtor(struct nvkm_engine *engine)
-{
- return ga102_fifo(engine);
-}
-
-static const struct nvkm_engine_func
+static const struct nvkm_fifo_func
ga102_fifo = {
- .dtor = ga102_fifo_dtor,
- .info = ga102_fifo_info,
- .base.sclass = ga102_fifo_sclass,
+ .runl_ctor = ga100_fifo_runl_ctor,
+ .mmu_fault = &tu102_fifo_mmu_fault,
+ .nonstall_ctor = ga100_fifo_nonstall_ctor,
+ .nonstall = &ga100_fifo_nonstall,
+ .runl = &ga100_runl,
+ .runq = &ga100_runq,
+ .engn = &ga100_engn,
+ .engn_ce = &ga100_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &ga100_cgrp, .force = true },
+ .chan = {{ 0, 0, AMPERE_CHANNEL_GPFIFO_B }, &ga100_chan },
};
int
ga102_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- struct ga102_fifo *fifo;
-
- if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
- return -ENOMEM;
-
- nvkm_engine_ctor(&ga102_fifo, device, type, inst, true, &fifo->base.engine);
- *pfifo = &fifo->base;
- return 0;
+ return nvkm_fifo_new_(&ga102_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.c
index 8b4f36b..5bb6525 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.c
@@ -21,186 +21,456 @@
*
* Authors: Ben Skeggs
*/
-#include "gf100.h"
-#include "changf100.h"
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+#include "runq.h"
-#include <core/client.h>
-#include <core/enum.h>
#include <core/gpuobj.h>
#include <subdev/bar.h>
#include <subdev/fault.h>
+#include <subdev/mc.h>
+#include <subdev/mmu.h>
#include <engine/sw.h>
#include <nvif/class.h>
-static void
-gf100_fifo_uevent_init(struct nvkm_fifo *fifo)
+void
+gf100_chan_preempt(struct nvkm_chan *chan)
{
- struct nvkm_device *device = fifo->engine.subdev.device;
- nvkm_mask(device, 0x002140, 0x80000000, 0x80000000);
+ nvkm_wr32(chan->cgrp->runl->fifo->engine.subdev.device, 0x002634, chan->id);
}
static void
-gf100_fifo_uevent_fini(struct nvkm_fifo *fifo)
+gf100_chan_stop(struct nvkm_chan *chan)
{
- struct nvkm_device *device = fifo->engine.subdev.device;
- nvkm_mask(device, 0x002140, 0x80000000, 0x00000000);
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_mask(device, 0x003004 + (chan->id * 8), 0x00000001, 0x00000000);
}
+static void
+gf100_chan_start(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x003004 + (chan->id * 8), 0x001f0001);
+}
+
+static void gf100_fifo_intr_engine(struct nvkm_fifo *);
+
+static void
+gf100_chan_unbind(struct nvkm_chan *chan)
+{
+ struct nvkm_fifo *fifo = chan->cgrp->runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+
+ /*TODO: Is this cargo-culted, or necessary? RM does *something* here... Why? */
+ gf100_fifo_intr_engine(fifo);
+
+ nvkm_wr32(device, 0x003000 + (chan->id * 8), 0x00000000);
+}
+
+static void
+gf100_chan_bind(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x003000 + (chan->id * 8), 0xc0000000 | chan->inst->addr >> 12);
+}
+
+static int
+gf100_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ const u64 userd = nvkm_memory_addr(chan->userd.mem) + chan->userd.base;
+ const u32 limit2 = ilog2(length / 8);
+
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, 0x08, lower_32_bits(userd));
+ nvkm_wo32(chan->inst, 0x0c, upper_32_bits(userd));
+ nvkm_wo32(chan->inst, 0x10, 0x0000face);
+ nvkm_wo32(chan->inst, 0x30, 0xfffff902);
+ nvkm_wo32(chan->inst, 0x48, lower_32_bits(offset));
+ nvkm_wo32(chan->inst, 0x4c, upper_32_bits(offset) | (limit2 << 16));
+ nvkm_wo32(chan->inst, 0x54, 0x00000002);
+ nvkm_wo32(chan->inst, 0x84, 0x20400000);
+ nvkm_wo32(chan->inst, 0x94, 0x30000000 | devm);
+ nvkm_wo32(chan->inst, 0x9c, 0x00000100);
+ nvkm_wo32(chan->inst, 0xa4, 0x1f1f1f1f);
+ nvkm_wo32(chan->inst, 0xa8, 0x1f1f1f1f);
+ nvkm_wo32(chan->inst, 0xac, 0x0000001f);
+ nvkm_wo32(chan->inst, 0xb8, 0xf8000000);
+ nvkm_wo32(chan->inst, 0xf8, 0x10003080); /* 0x002310 */
+ nvkm_wo32(chan->inst, 0xfc, 0x10000010); /* 0x002350 */
+ nvkm_done(chan->inst);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+gf100_chan_ramfc = {
+ .write = gf100_chan_ramfc_write,
+ .devm = 0xfff,
+};
+
void
-gf100_fifo_runlist_commit(struct gf100_fifo *fifo)
+gf100_chan_userd_clear(struct nvkm_chan *chan)
{
- struct gf100_fifo_chan *chan;
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- struct nvkm_memory *cur;
- int nr = 0;
- int target;
+ nvkm_kmap(chan->userd.mem);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x040, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x044, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x048, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x04c, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x050, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x058, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x05c, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x060, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x088, 0x00000000);
+ nvkm_wo32(chan->userd.mem, chan->userd.base + 0x08c, 0x00000000);
+ nvkm_done(chan->userd.mem);
+}
- mutex_lock(&fifo->base.mutex);
- cur = fifo->runlist.mem[fifo->runlist.active];
- fifo->runlist.active = !fifo->runlist.active;
+static const struct nvkm_chan_func_userd
+gf100_chan_userd = {
+ .bar = 1,
+ .size = 0x1000,
+ .clear = gf100_chan_userd_clear,
+};
- nvkm_kmap(cur);
- list_for_each_entry(chan, &fifo->chan, head) {
- nvkm_wo32(cur, (nr * 8) + 0, chan->base.chid);
- nvkm_wo32(cur, (nr * 8) + 4, 0x00000004);
- nr++;
- }
- nvkm_done(cur);
+const struct nvkm_chan_func_inst
+gf100_chan_inst = {
+ .size = 0x1000,
+ .zero = true,
+ .vmm = true,
+};
- switch (nvkm_memory_target(cur)) {
- case NVKM_MEM_TARGET_VRAM: target = 0; break;
- case NVKM_MEM_TARGET_NCOH: target = 3; break;
+static const struct nvkm_chan_func
+gf100_chan = {
+ .inst = &gf100_chan_inst,
+ .userd = &gf100_chan_userd,
+ .ramfc = &gf100_chan_ramfc,
+ .bind = gf100_chan_bind,
+ .unbind = gf100_chan_unbind,
+ .start = gf100_chan_start,
+ .stop = gf100_chan_stop,
+ .preempt = gf100_chan_preempt,
+};
+
+static void
+gf100_ectx_bind(struct nvkm_engn *engn, struct nvkm_cctx *cctx, struct nvkm_chan *chan)
+{
+ u64 addr = 0ULL;
+ u32 ptr0;
+
+ switch (engn->engine->subdev.type) {
+ case NVKM_ENGINE_SW : return;
+ case NVKM_ENGINE_GR : ptr0 = 0x0210; break;
+ case NVKM_ENGINE_CE : ptr0 = 0x0230 + (engn->engine->subdev.inst * 0x10); break;
+ case NVKM_ENGINE_MSPDEC: ptr0 = 0x0250; break;
+ case NVKM_ENGINE_MSPPP : ptr0 = 0x0260; break;
+ case NVKM_ENGINE_MSVLD : ptr0 = 0x0270; break;
default:
- mutex_unlock(&fifo->base.mutex);
WARN_ON(1);
return;
}
- nvkm_wr32(device, 0x002270, (nvkm_memory_addr(cur) >> 12) |
- (target << 28));
- nvkm_wr32(device, 0x002274, 0x01f00000 | nr);
-
- if (wait_event_timeout(fifo->runlist.wait,
- !(nvkm_rd32(device, 0x00227c) & 0x00100000),
- msecs_to_jiffies(2000)) == 0)
- nvkm_error(subdev, "runlist update timeout\n");
- mutex_unlock(&fifo->base.mutex);
-}
-
-void
-gf100_fifo_runlist_remove(struct gf100_fifo *fifo, struct gf100_fifo_chan *chan)
-{
- mutex_lock(&fifo->base.mutex);
- list_del_init(&chan->head);
- mutex_unlock(&fifo->base.mutex);
-}
-
-void
-gf100_fifo_runlist_insert(struct gf100_fifo *fifo, struct gf100_fifo_chan *chan)
-{
- mutex_lock(&fifo->base.mutex);
- list_add_tail(&chan->head, &fifo->chan);
- mutex_unlock(&fifo->base.mutex);
-}
-
-static struct nvkm_engine *
-gf100_fifo_id_engine(struct nvkm_fifo *fifo, int engi)
-{
- enum nvkm_subdev_type type;
- int inst;
-
- switch (engi) {
- case GF100_FIFO_ENGN_GR : type = NVKM_ENGINE_GR ; inst = 0; break;
- case GF100_FIFO_ENGN_MSPDEC: type = NVKM_ENGINE_MSPDEC; inst = 0; break;
- case GF100_FIFO_ENGN_MSPPP : type = NVKM_ENGINE_MSPPP ; inst = 0; break;
- case GF100_FIFO_ENGN_MSVLD : type = NVKM_ENGINE_MSVLD ; inst = 0; break;
- case GF100_FIFO_ENGN_CE0 : type = NVKM_ENGINE_CE ; inst = 0; break;
- case GF100_FIFO_ENGN_CE1 : type = NVKM_ENGINE_CE ; inst = 1; break;
- case GF100_FIFO_ENGN_SW : type = NVKM_ENGINE_SW ; inst = 0; break;
- default:
- WARN_ON(1);
- return NULL;
+ if (cctx) {
+ addr = cctx->vctx->vma->addr;
+ addr |= 4ULL;
}
- return nvkm_device_engine(fifo->engine.subdev.device, type, inst);
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, ptr0 + 0, lower_32_bits(addr));
+ nvkm_wo32(chan->inst, ptr0 + 4, upper_32_bits(addr));
+ nvkm_done(chan->inst);
}
static int
-gf100_fifo_engine_id(struct nvkm_fifo *base, struct nvkm_engine *engine)
+gf100_ectx_ctor(struct nvkm_engn *engn, struct nvkm_vctx *vctx)
{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_GR : return GF100_FIFO_ENGN_GR;
- case NVKM_ENGINE_MSPDEC: return GF100_FIFO_ENGN_MSPDEC;
- case NVKM_ENGINE_MSPPP : return GF100_FIFO_ENGN_MSPPP;
- case NVKM_ENGINE_MSVLD : return GF100_FIFO_ENGN_MSVLD;
- case NVKM_ENGINE_CE : return GF100_FIFO_ENGN_CE0 + engine->subdev.inst;
- case NVKM_ENGINE_SW : return GF100_FIFO_ENGN_SW;
- default:
- WARN_ON(1);
- return -1;
- }
+ int ret;
+
+ ret = nvkm_vmm_get(vctx->vmm, 12, vctx->inst->size, &vctx->vma);
+ if (ret)
+ return ret;
+
+ return nvkm_memory_map(vctx->inst, 0, vctx->vmm, vctx->vma, NULL, 0);
}
-static void
-gf100_fifo_recover_work(struct work_struct *w)
+bool
+gf100_engn_mmu_fault_triggered(struct nvkm_engn *engn)
{
- struct gf100_fifo *fifo = container_of(w, typeof(*fifo), recover.work);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_engine *engine;
+ struct nvkm_runl *runl = engn->runl;
+ struct nvkm_fifo *fifo = runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ u32 data = nvkm_rd32(device, 0x002a30 + (engn->id * 4));
+
+ ENGN_DEBUG(engn, "%08x: mmu fault triggered", data);
+ if (!(data & 0x00000100))
+ return false;
+
+ spin_lock(&fifo->lock);
+ nvkm_mask(device, 0x002a30 + (engn->id * 4), 0x00000100, 0x00000000);
+ if (atomic_dec_and_test(&runl->rc_triggered))
+ nvkm_mask(device, 0x002140, 0x00000100, 0x00000100);
+ spin_unlock(&fifo->lock);
+ return true;
+}
+
+void
+gf100_engn_mmu_fault_trigger(struct nvkm_engn *engn)
+{
+ struct nvkm_runl *runl = engn->runl;
+ struct nvkm_fifo *fifo = runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+
+ ENGN_DEBUG(engn, "triggering mmu fault on 0x%02x", engn->fault);
+ spin_lock(&fifo->lock);
+ if (atomic_inc_return(&runl->rc_triggered) == 1)
+ nvkm_mask(device, 0x002140, 0x00000100, 0x00000000);
+ nvkm_wr32(device, 0x002100, 0x00000100);
+ nvkm_wr32(device, 0x002a30 + (engn->id * 4), 0x00000100 | engn->fault);
+ spin_unlock(&fifo->lock);
+}
+
+/*TODO: clean all this up. */
+struct gf100_engn_status {
+ bool busy;
+ bool save;
+ bool unk0;
+ bool unk1;
+ u8 chid;
+};
+
+static void
+gf100_engn_status(struct nvkm_engn *engn, struct gf100_engn_status *status)
+{
+ u32 stat = nvkm_rd32(engn->engine->subdev.device, 0x002640 + (engn->id * 4));
+
+ status->busy = (stat & 0x10000000);
+ status->save = (stat & 0x00100000);
+ status->unk0 = (stat & 0x00004000);
+ status->unk1 = (stat & 0x00001000);
+ status->chid = (stat & 0x0000007f);
+
+ ENGN_DEBUG(engn, "%08x: busy %d save %d unk0 %d unk1 %d chid %d",
+ stat, status->busy, status->save, status->unk0, status->unk1, status->chid);
+}
+
+static int
+gf100_engn_cxid(struct nvkm_engn *engn, bool *cgid)
+{
+ struct gf100_engn_status status;
+
+ gf100_engn_status(engn, &status);
+ if (status.busy) {
+ *cgid = false;
+ return status.chid;
+ }
+
+ return -ENODEV;
+}
+
+static bool
+gf100_engn_chsw(struct nvkm_engn *engn)
+{
+ struct gf100_engn_status status;
+
+ gf100_engn_status(engn, &status);
+ if (status.busy && (status.unk0 || status.unk1))
+ return true;
+
+ return false;
+}
+
+static const struct nvkm_engn_func
+gf100_engn = {
+ .chsw = gf100_engn_chsw,
+ .cxid = gf100_engn_cxid,
+ .mmu_fault_trigger = gf100_engn_mmu_fault_trigger,
+ .mmu_fault_triggered = gf100_engn_mmu_fault_triggered,
+ .ctor = gf100_ectx_ctor,
+ .bind = gf100_ectx_bind,
+};
+
+const struct nvkm_engn_func
+gf100_engn_sw = {
+};
+
+static const struct nvkm_bitfield
+gf100_runq_intr_0_names[] = {
+/* { 0x00008000, "" } seen with null ib push */
+ { 0x00200000, "ILLEGAL_MTHD" },
+ { 0x00800000, "EMPTY_SUBC" },
+ {}
+};
+
+bool
+gf100_runq_intr(struct nvkm_runq *runq, struct nvkm_runl *null)
+{
+ struct nvkm_subdev *subdev = &runq->fifo->engine.subdev;
+ struct nvkm_device *device = subdev->device;
+ u32 mask = nvkm_rd32(device, 0x04010c + (runq->id * 0x2000));
+ u32 stat = nvkm_rd32(device, 0x040108 + (runq->id * 0x2000)) & mask;
+ u32 addr = nvkm_rd32(device, 0x0400c0 + (runq->id * 0x2000));
+ u32 data = nvkm_rd32(device, 0x0400c4 + (runq->id * 0x2000));
+ u32 chid = nvkm_rd32(device, 0x040120 + (runq->id * 0x2000)) & runq->fifo->chid->mask;
+ u32 subc = (addr & 0x00070000) >> 16;
+ u32 mthd = (addr & 0x00003ffc);
+ u32 show = stat;
+ struct nvkm_chan *chan;
unsigned long flags;
- u32 engm, engn, todo;
+ char msg[128];
- spin_lock_irqsave(&fifo->base.lock, flags);
- engm = fifo->recover.mask;
- fifo->recover.mask = 0ULL;
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-
- nvkm_mask(device, 0x002630, engm, engm);
-
- for (todo = engm; engn = __ffs(todo), todo; todo &= ~BIT_ULL(engn)) {
- if ((engine = gf100_fifo_id_engine(&fifo->base, engn))) {
- nvkm_subdev_fini(&engine->subdev, false);
- WARN_ON(nvkm_subdev_init(&engine->subdev));
+ if (stat & 0x00800000) {
+ if (device->sw) {
+ if (nvkm_sw_mthd(device->sw, chid, subc, mthd, data))
+ show &= ~0x00800000;
}
}
- gf100_fifo_runlist_commit(fifo);
- nvkm_wr32(device, 0x00262c, engm);
- nvkm_mask(device, 0x002630, engm, 0x00000000);
+ if (show) {
+ nvkm_snprintbf(msg, sizeof(msg), runq->func->intr_0_names, show);
+ chan = nvkm_chan_get_chid(&runq->fifo->engine, chid, &flags);
+ nvkm_error(subdev, "PBDMA%d: %08x [%s] ch %d [%010llx %s] "
+ "subc %d mthd %04x data %08x\n",
+ runq->id, show, msg, chid, chan ? chan->inst->addr : 0,
+ chan ? chan->name : "unknown", subc, mthd, data);
+
+ /*TODO: use proper procedure for clearing each exception / debug output */
+ if ((stat & 0xc67fe000) && chan)
+ nvkm_chan_error(chan, true);
+ nvkm_chan_put(&chan, flags);
+ }
+
+ nvkm_wr32(device, 0x0400c0 + (runq->id * 0x2000), 0x80600008);
+ nvkm_wr32(device, 0x040108 + (runq->id * 0x2000), stat);
+ return true;
+}
+
+void
+gf100_runq_init(struct nvkm_runq *runq)
+{
+ struct nvkm_device *device = runq->fifo->engine.subdev.device;
+
+ nvkm_mask(device, 0x04013c + (runq->id * 0x2000), 0x10000100, 0x00000000);
+ nvkm_wr32(device, 0x040108 + (runq->id * 0x2000), 0xffffffff); /* INTR */
+ nvkm_wr32(device, 0x04010c + (runq->id * 0x2000), 0xfffffeff); /* INTREN */
+}
+
+static const struct nvkm_runq_func
+gf100_runq = {
+ .init = gf100_runq_init,
+ .intr = gf100_runq_intr,
+ .intr_0_names = gf100_runq_intr_0_names,
+};
+
+bool
+gf100_runl_preempt_pending(struct nvkm_runl *runl)
+{
+ return nvkm_rd32(runl->fifo->engine.subdev.device, 0x002634) & 0x00100000;
}
static void
-gf100_fifo_recover(struct gf100_fifo *fifo, struct nvkm_engine *engine,
- struct gf100_fifo_chan *chan)
+gf100_runl_fault_clear(struct nvkm_runl *runl)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 chid = chan->base.chid;
- int engi = gf100_fifo_engine_id(&fifo->base, engine);
-
- nvkm_error(subdev, "%s engine fault on channel %d, recovering...\n",
- engine->subdev.name, chid);
- assert_spin_locked(&fifo->base.lock);
-
- nvkm_mask(device, 0x003004 + (chid * 0x08), 0x00000001, 0x00000000);
- list_del_init(&chan->head);
- chan->killed = true;
-
- if (engi >= 0 && engi != GF100_FIFO_ENGN_SW)
- fifo->recover.mask |= BIT(engi);
- schedule_work(&fifo->recover.work);
- nvkm_fifo_kevent(&fifo->base, chid);
+ nvkm_mask(runl->fifo->engine.subdev.device, 0x00262c, 0x00000000, 0x00000000);
}
+static void
+gf100_runl_allow(struct nvkm_runl *runl, u32 engm)
+{
+ nvkm_mask(runl->fifo->engine.subdev.device, 0x002630, engm, 0x00000000);
+}
+
+static void
+gf100_runl_block(struct nvkm_runl *runl, u32 engm)
+{
+ nvkm_mask(runl->fifo->engine.subdev.device, 0x002630, engm, engm);
+}
+
+static bool
+gf100_runl_pending(struct nvkm_runl *runl)
+{
+ return nvkm_rd32(runl->fifo->engine.subdev.device, 0x00227c) & 0x00100000;
+}
+
+static void
+gf100_runl_commit(struct nvkm_runl *runl, struct nvkm_memory *memory, u32 start, int count)
+{
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+ u64 addr = nvkm_memory_addr(memory) + start;
+ int target;
+
+ switch (nvkm_memory_target(memory)) {
+ case NVKM_MEM_TARGET_VRAM: target = 0; break;
+ case NVKM_MEM_TARGET_NCOH: target = 3; break;
+ default:
+ WARN_ON(1);
+ return;
+ }
+
+ nvkm_wr32(device, 0x002270, (target << 28) | (addr >> 12));
+ nvkm_wr32(device, 0x002274, 0x01f00000 | count);
+}
+
+static void
+gf100_runl_insert_chan(struct nvkm_chan *chan, struct nvkm_memory *memory, u64 offset)
+{
+ nvkm_wo32(memory, offset + 0, chan->id);
+ nvkm_wo32(memory, offset + 4, 0x00000004);
+}
+
+static const struct nvkm_runl_func
+gf100_runl = {
+ .size = 8,
+ .update = nv50_runl_update,
+ .insert_chan = gf100_runl_insert_chan,
+ .commit = gf100_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = gf100_runl_pending,
+ .block = gf100_runl_block,
+ .allow = gf100_runl_allow,
+ .fault_clear = gf100_runl_fault_clear,
+ .preempt_pending = gf100_runl_preempt_pending,
+};
+
+static void
+gf100_fifo_nonstall_allow(struct nvkm_event *event, int type, int index)
+{
+ struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), nonstall.event);
+ unsigned long flags;
+
+ spin_lock_irqsave(&fifo->lock, flags);
+ nvkm_mask(fifo->engine.subdev.device, 0x002140, 0x80000000, 0x80000000);
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+void
+gf100_fifo_nonstall_block(struct nvkm_event *event, int type, int index)
+{
+ struct nvkm_fifo *fifo = container_of(event, typeof(*fifo), nonstall.event);
+ unsigned long flags;
+
+ spin_lock_irqsave(&fifo->lock, flags);
+ nvkm_mask(fifo->engine.subdev.device, 0x002140, 0x80000000, 0x00000000);
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+const struct nvkm_event_func
+gf100_fifo_nonstall = {
+ .init = gf100_fifo_nonstall_allow,
+ .fini = gf100_fifo_nonstall_block,
+};
+
static const struct nvkm_enum
-gf100_fifo_fault_engine[] = {
+gf100_fifo_mmu_fault_engine[] = {
{ 0x00, "PGRAPH", NULL, NVKM_ENGINE_GR },
{ 0x03, "PEEPHOLE", NULL, NVKM_ENGINE_IFB },
{ 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
{ 0x05, "BAR3", NULL, NVKM_SUBDEV_INSTMEM },
- { 0x07, "PFIFO", NULL, NVKM_ENGINE_FIFO },
+ { 0x07, "PFIFO" },
{ 0x10, "PMSVLD", NULL, NVKM_ENGINE_MSVLD },
{ 0x11, "PMSPPP", NULL, NVKM_ENGINE_MSPPP },
{ 0x13, "PCOUNTER" },
@@ -212,7 +482,7 @@ gf100_fifo_fault_engine[] = {
};
static const struct nvkm_enum
-gf100_fifo_fault_reason[] = {
+gf100_fifo_mmu_fault_reason[] = {
{ 0x00, "PT_NOT_PRESENT" },
{ 0x01, "PT_TOO_SHORT" },
{ 0x02, "PAGE_NOT_PRESENT" },
@@ -226,7 +496,7 @@ gf100_fifo_fault_reason[] = {
};
static const struct nvkm_enum
-gf100_fifo_fault_hubclient[] = {
+gf100_fifo_mmu_fault_hubclient[] = {
{ 0x01, "PCOPY0" },
{ 0x02, "PCOPY1" },
{ 0x04, "DISPATCH" },
@@ -245,7 +515,7 @@ gf100_fifo_fault_hubclient[] = {
};
static const struct nvkm_enum
-gf100_fifo_fault_gpcclient[] = {
+gf100_fifo_mmu_fault_gpcclient[] = {
{ 0x01, "TEX" },
{ 0x0c, "ESETUP" },
{ 0x0e, "CTXCTL" },
@@ -253,29 +523,55 @@ gf100_fifo_fault_gpcclient[] = {
{}
};
-static void
-gf100_fifo_fault(struct nvkm_fifo *base, struct nvkm_fault_data *info)
-{
- struct gf100_fifo *fifo = gf100_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const struct nvkm_enum *er, *eu, *ec;
- struct nvkm_engine *engine = NULL;
- struct nvkm_fifo_chan *chan;
- unsigned long flags;
- char gpcid[8] = "";
+const struct nvkm_enum
+gf100_fifo_mmu_fault_access[] = {
+ { 0x00, "READ" },
+ { 0x01, "WRITE" },
+ {}
+};
- er = nvkm_enum_find(gf100_fifo_fault_reason, info->reason);
- eu = nvkm_enum_find(gf100_fifo_fault_engine, info->engine);
- if (info->hub) {
- ec = nvkm_enum_find(gf100_fifo_fault_hubclient, info->client);
- } else {
- ec = nvkm_enum_find(gf100_fifo_fault_gpcclient, info->client);
- snprintf(gpcid, sizeof(gpcid), "GPC%d/", info->gpc);
+void
+gf100_fifo_mmu_fault_recover(struct nvkm_fifo *fifo, struct nvkm_fault_data *info)
+{
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
+ struct nvkm_device *device = subdev->device;
+ const struct nvkm_enum *er, *ee, *ec, *ea;
+ struct nvkm_engine *engine = NULL;
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+ struct nvkm_chan *chan;
+ unsigned long flags;
+ char ct[8] = "HUB/";
+
+ /* Lookup engine by MMU fault ID. */
+ nvkm_runl_foreach(runl, fifo) {
+ engn = nvkm_runl_find_engn(engn, runl, engn->fault == info->engine);
+ if (engn) {
+ /* Fault triggered by CTXSW_TIMEOUT recovery procedure. */
+ if (engn->func->mmu_fault_triggered &&
+ engn->func->mmu_fault_triggered(engn)) {
+ nvkm_runl_rc_engn(runl, engn);
+ return;
+ }
+
+ engine = engn->engine;
+ break;
+ }
}
- if (eu && eu->data2) {
- switch (eu->data2) {
+ er = nvkm_enum_find(fifo->func->mmu_fault->reason, info->reason);
+ ee = nvkm_enum_find(fifo->func->mmu_fault->engine, info->engine);
+ if (info->hub) {
+ ec = nvkm_enum_find(fifo->func->mmu_fault->hubclient, info->client);
+ } else {
+ ec = nvkm_enum_find(fifo->func->mmu_fault->gpcclient, info->client);
+ snprintf(ct, sizeof(ct), "GPC%d/", info->gpc);
+ }
+ ea = nvkm_enum_find(fifo->func->mmu_fault->access, info->access);
+
+ /* Handle BAR faults. */
+ if (ee && ee->data2) {
+ switch (ee->data2) {
case NVKM_SUBDEV_BAR:
nvkm_bar_bar1_reset(device);
break;
@@ -286,77 +582,104 @@ gf100_fifo_fault(struct nvkm_fifo *base, struct nvkm_fault_data *info)
nvkm_mask(device, 0x001718, 0x00000000, 0x00000000);
break;
default:
- engine = nvkm_device_engine(device, eu->data2, eu->inst);
break;
}
}
- chan = nvkm_fifo_chan_inst(&fifo->base, info->inst, &flags);
+ chan = nvkm_chan_get_inst(&fifo->engine, info->inst, &flags);
nvkm_error(subdev,
- "%s fault at %010llx engine %02x [%s] client %02x [%s%s] "
- "reason %02x [%s] on channel %d [%010llx %s]\n",
- info->access ? "write" : "read", info->addr,
- info->engine, eu ? eu->name : "",
- info->client, gpcid, ec ? ec->name : "",
- info->reason, er ? er->name : "", chan ? chan->chid : -1,
- info->inst, chan ? chan->object.client->name : "unknown");
+ "fault %02x [%s] at %016llx engine %02x [%s] client %02x "
+ "[%s%s] reason %02x [%s] on channel %d [%010llx %s]\n",
+ info->access, ea ? ea->name : "", info->addr,
+ info->engine, ee ? ee->name : engine ? engine->subdev.name : "",
+ info->client, ct, ec ? ec->name : "",
+ info->reason, er ? er->name : "",
+ chan ? chan->id : -1, info->inst, chan ? chan->name : "unknown");
- if (engine && chan)
- gf100_fifo_recover(fifo, engine, (void *)chan);
- nvkm_fifo_chan_put(&fifo->base, flags, &chan);
+ /* Handle host/engine faults. */
+ if (chan)
+ nvkm_runl_rc_cgrp(chan->cgrp);
+
+ nvkm_chan_put(&chan, flags);
+}
+
+static const struct nvkm_fifo_func_mmu_fault
+gf100_fifo_mmu_fault = {
+ .recover = gf100_fifo_mmu_fault_recover,
+ .access = gf100_fifo_mmu_fault_access,
+ .engine = gf100_fifo_mmu_fault_engine,
+ .reason = gf100_fifo_mmu_fault_reason,
+ .hubclient = gf100_fifo_mmu_fault_hubclient,
+ .gpcclient = gf100_fifo_mmu_fault_gpcclient,
+};
+
+void
+gf100_fifo_intr_ctxsw_timeout(struct nvkm_fifo *fifo, u32 engm)
+{
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn, *engn2;
+ bool cgid, cgid2;
+ int id, id2;
+
+ nvkm_runl_foreach(runl, fifo) {
+ /* Stop the runlist, and go through all engines serving it. */
+ nvkm_runl_block(runl);
+ nvkm_runl_foreach_engn_cond(engn, runl, engm & BIT(engn->id)) {
+ /* Determine what channel (group) the engine is on. */
+ id = engn->func->cxid(engn, &cgid);
+ if (id >= 0) {
+ /* Trigger MMU fault on any engine(s) on that channel (group). */
+ nvkm_runl_foreach_engn_cond(engn2, runl, engn2->func->cxid) {
+ id2 = engn2->func->cxid(engn2, &cgid2);
+ if (cgid2 == cgid && id2 == id)
+ engn2->func->mmu_fault_trigger(engn2);
+ }
+ }
+ }
+ nvkm_runl_allow(runl); /* HW will keep runlist blocked via ERROR_SCHED_DISABLE. */
+ }
+}
+
+static void
+gf100_fifo_intr_sched_ctxsw(struct nvkm_fifo *fifo)
+{
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+ u32 engm = 0;
+
+ /* Look for any engines that are busy, and awaiting chsw ack. */
+ nvkm_runl_foreach(runl, fifo) {
+ nvkm_runl_foreach_engn_cond(engn, runl, engn->func->chsw) {
+ if (WARN_ON(engn->fault < 0) || !engn->func->chsw(engn))
+ continue;
+
+ engm |= BIT(engn->id);
+ }
+ }
+
+ if (!engm)
+ return;
+
+ fifo->func->intr_ctxsw_timeout(fifo, engm);
}
static const struct nvkm_enum
-gf100_fifo_sched_reason[] = {
+gf100_fifo_intr_sched_names[] = {
{ 0x0a, "CTXSW_TIMEOUT" },
{}
};
-static void
-gf100_fifo_intr_sched_ctxsw(struct gf100_fifo *fifo)
+void
+gf100_fifo_intr_sched(struct nvkm_fifo *fifo)
{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_engine *engine;
- struct gf100_fifo_chan *chan;
- unsigned long flags;
- u32 engn;
-
- spin_lock_irqsave(&fifo->base.lock, flags);
- for (engn = 0; engn < 6; engn++) {
- u32 stat = nvkm_rd32(device, 0x002640 + (engn * 0x04));
- u32 busy = (stat & 0x80000000);
- u32 save = (stat & 0x00100000); /* maybe? */
- u32 unk0 = (stat & 0x00040000);
- u32 unk1 = (stat & 0x00001000);
- u32 chid = (stat & 0x0000007f);
- (void)save;
-
- if (busy && unk0 && unk1) {
- list_for_each_entry(chan, &fifo->chan, head) {
- if (chan->base.chid == chid) {
- engine = gf100_fifo_id_engine(&fifo->base, engn);
- if (!engine)
- break;
- gf100_fifo_recover(fifo, engine, chan);
- break;
- }
- }
- }
- }
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-}
-
-static void
-gf100_fifo_intr_sched(struct gf100_fifo *fifo)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
u32 intr = nvkm_rd32(device, 0x00254c);
u32 code = intr & 0x000000ff;
const struct nvkm_enum *en;
- en = nvkm_enum_find(gf100_fifo_sched_reason, code);
+ en = nvkm_enum_find(gf100_fifo_intr_sched_names, code);
nvkm_error(subdev, "SCHED_ERROR %02x [%s]\n", code, en ? en->name : "");
@@ -370,7 +693,7 @@ gf100_fifo_intr_sched(struct gf100_fifo *fifo)
}
void
-gf100_fifo_intr_fault(struct nvkm_fifo *fifo, int unit)
+gf100_fifo_intr_mmu_fault_unit(struct nvkm_fifo *fifo, int unit)
{
struct nvkm_device *device = fifo->engine.subdev.device;
u32 inst = nvkm_rd32(device, 0x002800 + (unit * 0x10));
@@ -393,61 +716,45 @@ gf100_fifo_intr_fault(struct nvkm_fifo *fifo, int unit)
nvkm_fifo_fault(fifo, &info);
}
-static const struct nvkm_bitfield
-gf100_fifo_pbdma_intr[] = {
-/* { 0x00008000, "" } seen with null ib push */
- { 0x00200000, "ILLEGAL_MTHD" },
- { 0x00800000, "EMPTY_SUBC" },
- {}
-};
-
-static void
-gf100_fifo_intr_pbdma(struct gf100_fifo *fifo, int unit)
+void
+gf100_fifo_intr_mmu_fault(struct nvkm_fifo *fifo)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 stat = nvkm_rd32(device, 0x040108 + (unit * 0x2000));
- u32 addr = nvkm_rd32(device, 0x0400c0 + (unit * 0x2000));
- u32 data = nvkm_rd32(device, 0x0400c4 + (unit * 0x2000));
- u32 chid = nvkm_rd32(device, 0x040120 + (unit * 0x2000)) & 0x7f;
- u32 subc = (addr & 0x00070000) >> 16;
- u32 mthd = (addr & 0x00003ffc);
- struct nvkm_fifo_chan *chan;
- unsigned long flags;
- u32 show= stat;
- char msg[128];
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ unsigned long mask = nvkm_rd32(device, 0x00259c);
+ int unit;
- if (stat & 0x00800000) {
- if (device->sw) {
- if (nvkm_sw_mthd(device->sw, chid, subc, mthd, data))
- show &= ~0x00800000;
- }
+ for_each_set_bit(unit, &mask, 32) {
+ fifo->func->intr_mmu_fault_unit(fifo, unit);
+ nvkm_wr32(device, 0x00259c, BIT(unit));
+ }
+}
+
+bool
+gf100_fifo_intr_pbdma(struct nvkm_fifo *fifo)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_runq *runq;
+ u32 mask = nvkm_rd32(device, 0x0025a0);
+ bool handled = false;
+
+ nvkm_runq_foreach_cond(runq, fifo, mask & BIT(runq->id)) {
+ if (runq->func->intr(runq, NULL))
+ handled = true;
+
+ nvkm_wr32(device, 0x0025a0, BIT(runq->id));
}
- if (show) {
- nvkm_snprintbf(msg, sizeof(msg), gf100_fifo_pbdma_intr, show);
- chan = nvkm_fifo_chan_chid(&fifo->base, chid, &flags);
- nvkm_error(subdev, "PBDMA%d: %08x [%s] ch %d [%010llx %s] "
- "subc %d mthd %04x data %08x\n",
- unit, show, msg, chid, chan ? chan->inst->addr : 0,
- chan ? chan->object.client->name : "unknown",
- subc, mthd, data);
- nvkm_fifo_chan_put(&fifo->base, flags, &chan);
- }
-
- nvkm_wr32(device, 0x0400c0 + (unit * 0x2000), 0x80600008);
- nvkm_wr32(device, 0x040108 + (unit * 0x2000), stat);
+ return handled;
}
static void
-gf100_fifo_intr_runlist(struct gf100_fifo *fifo)
+gf100_fifo_intr_runlist(struct nvkm_fifo *fifo)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
u32 intr = nvkm_rd32(device, 0x002a00);
if (intr & 0x10000000) {
- wake_up(&fifo->runlist.wait);
nvkm_wr32(device, 0x002a00, 0x10000000);
intr &= ~0x10000000;
}
@@ -459,9 +766,9 @@ gf100_fifo_intr_runlist(struct gf100_fifo *fifo)
}
static void
-gf100_fifo_intr_engine_unit(struct gf100_fifo *fifo, int engn)
+gf100_fifo_intr_engine_unit(struct nvkm_fifo *fifo, int engn)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
u32 intr = nvkm_rd32(device, 0x0025a8 + (engn * 0x04));
u32 inte = nvkm_rd32(device, 0x002628);
@@ -472,22 +779,22 @@ gf100_fifo_intr_engine_unit(struct gf100_fifo *fifo, int engn)
for (unkn = 0; unkn < 8; unkn++) {
u32 ints = (intr >> (unkn * 0x04)) & inte;
if (ints & 0x1) {
- nvkm_fifo_uevent(&fifo->base);
+ nvkm_event_ntfy(&fifo->nonstall.event, 0, NVKM_FIFO_NONSTALL_EVENT);
ints &= ~1;
}
if (ints) {
- nvkm_error(subdev, "ENGINE %d %d %01x",
- engn, unkn, ints);
+ nvkm_error(subdev, "ENGINE %d %d %01x", engn, unkn, ints);
nvkm_mask(device, 0x002628, ints, 0);
}
}
}
-void
-gf100_fifo_intr_engine(struct gf100_fifo *fifo)
+static void
+gf100_fifo_intr_engine(struct nvkm_fifo *fifo)
{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
+ struct nvkm_device *device = fifo->engine.subdev.device;
u32 mask = nvkm_rd32(device, 0x0025a4);
+
while (mask) {
u32 unit = __ffs(mask);
gf100_fifo_intr_engine_unit(fifo, unit);
@@ -495,11 +802,11 @@ gf100_fifo_intr_engine(struct gf100_fifo *fifo)
}
}
-static void
-gf100_fifo_intr(struct nvkm_fifo *base)
+static irqreturn_t
+gf100_fifo_intr(struct nvkm_inth *inth)
{
- struct gf100_fifo *fifo = gf100_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_fifo *fifo = container_of(inth, typeof(*fifo), engine.subdev.inth);
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
u32 mask = nvkm_rd32(device, 0x002140);
u32 stat = nvkm_rd32(device, 0x002100) & mask;
@@ -532,25 +839,13 @@ gf100_fifo_intr(struct nvkm_fifo *base)
}
if (stat & 0x10000000) {
- u32 mask = nvkm_rd32(device, 0x00259c);
- while (mask) {
- u32 unit = __ffs(mask);
- gf100_fifo_intr_fault(&fifo->base, unit);
- nvkm_wr32(device, 0x00259c, (1 << unit));
- mask &= ~(1 << unit);
- }
+ gf100_fifo_intr_mmu_fault(fifo);
stat &= ~0x10000000;
}
if (stat & 0x20000000) {
- u32 mask = nvkm_rd32(device, 0x0025a0);
- while (mask) {
- u32 unit = __ffs(mask);
- gf100_fifo_intr_pbdma(fifo, unit);
- nvkm_wr32(device, 0x0025a0, (1 << unit));
- mask &= ~(1 << unit);
- }
- stat &= ~0x20000000;
+ if (gf100_fifo_intr_pbdma(fifo))
+ stat &= ~0x20000000;
}
if (stat & 0x40000000) {
@@ -565,71 +860,26 @@ gf100_fifo_intr(struct nvkm_fifo *base)
if (stat) {
nvkm_error(subdev, "INTR %08x\n", stat);
+ spin_lock(&fifo->lock);
nvkm_mask(device, 0x002140, stat, 0x00000000);
+ spin_unlock(&fifo->lock);
nvkm_wr32(device, 0x002100, stat);
}
-}
-static int
-gf100_fifo_oneinit(struct nvkm_fifo *base)
-{
- struct gf100_fifo *fifo = gf100_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- struct nvkm_vmm *bar = nvkm_bar_bar1_vmm(device);
- int ret;
-
- /* Determine number of PBDMAs by checking valid enable bits. */
- nvkm_wr32(device, 0x002204, 0xffffffff);
- fifo->pbdma_nr = hweight32(nvkm_rd32(device, 0x002204));
- nvkm_debug(subdev, "%d PBDMA(s)\n", fifo->pbdma_nr);
-
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 0x1000,
- false, &fifo->runlist.mem[0]);
- if (ret)
- return ret;
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 0x1000,
- false, &fifo->runlist.mem[1]);
- if (ret)
- return ret;
-
- init_waitqueue_head(&fifo->runlist.wait);
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 128 * 0x1000,
- 0x1000, false, &fifo->user.mem);
- if (ret)
- return ret;
-
- ret = nvkm_vmm_get(bar, 12, nvkm_memory_size(fifo->user.mem),
- &fifo->user.bar);
- if (ret)
- return ret;
-
- return nvkm_memory_map(fifo->user.mem, 0, bar, fifo->user.bar, NULL, 0);
+ return IRQ_HANDLED;
}
static void
-gf100_fifo_fini(struct nvkm_fifo *base)
+gf100_fifo_init_pbdmas(struct nvkm_fifo *fifo, u32 mask)
{
- struct gf100_fifo *fifo = gf100_fifo(base);
- flush_work(&fifo->recover.work);
-}
-
-static void
-gf100_fifo_init(struct nvkm_fifo *base)
-{
- struct gf100_fifo *fifo = gf100_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- int i;
+ struct nvkm_device *device = fifo->engine.subdev.device;
/* Enable PBDMAs. */
- nvkm_wr32(device, 0x000204, (1 << fifo->pbdma_nr) - 1);
- nvkm_wr32(device, 0x002204, (1 << fifo->pbdma_nr) - 1);
+ nvkm_wr32(device, 0x000204, mask);
+ nvkm_wr32(device, 0x002204, mask);
/* Assign engines to PBDMAs. */
- if (fifo->pbdma_nr >= 3) {
+ if ((mask & 7) == 7) {
nvkm_wr32(device, 0x002208, ~(1 << 0)); /* PGRAPH */
nvkm_wr32(device, 0x00220c, ~(1 << 1)); /* PVP */
nvkm_wr32(device, 0x002210, ~(1 << 1)); /* PMSPP */
@@ -638,62 +888,82 @@ gf100_fifo_init(struct nvkm_fifo *base)
nvkm_wr32(device, 0x00221c, ~(1 << 1)); /* PCE1 */
}
- /* PBDMA[n] */
- for (i = 0; i < fifo->pbdma_nr; i++) {
- nvkm_mask(device, 0x04013c + (i * 0x2000), 0x10000100, 0x00000000);
- nvkm_wr32(device, 0x040108 + (i * 0x2000), 0xffffffff); /* INTR */
- nvkm_wr32(device, 0x04010c + (i * 0x2000), 0xfffffeff); /* INTREN */
- }
+ nvkm_mask(device, 0x002a04, 0xbfffffff, 0xbfffffff);
+}
+
+static void
+gf100_fifo_init(struct nvkm_fifo *fifo)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
nvkm_mask(device, 0x002200, 0x00000001, 0x00000001);
- nvkm_wr32(device, 0x002254, 0x10000000 | fifo->user.bar->addr >> 12);
+ nvkm_wr32(device, 0x002254, 0x10000000 | fifo->userd.bar1->addr >> 12);
nvkm_wr32(device, 0x002100, 0xffffffff);
nvkm_wr32(device, 0x002140, 0x7fffffff);
nvkm_wr32(device, 0x002628, 0x00000001); /* ENGINE_INTR_EN */
}
-static void *
-gf100_fifo_dtor(struct nvkm_fifo *base)
+static int
+gf100_fifo_runl_ctor(struct nvkm_fifo *fifo)
{
- struct gf100_fifo *fifo = gf100_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- nvkm_vmm_put(nvkm_bar_bar1_vmm(device), &fifo->user.bar);
- nvkm_memory_unref(&fifo->user.mem);
- nvkm_memory_unref(&fifo->runlist.mem[0]);
- nvkm_memory_unref(&fifo->runlist.mem[1]);
- return fifo;
+ struct nvkm_runl *runl;
+
+ runl = nvkm_runl_new(fifo, 0, 0, 0);
+ if (IS_ERR(runl))
+ return PTR_ERR(runl);
+
+ nvkm_runl_add(runl, 0, fifo->func->engn, NVKM_ENGINE_GR, 0);
+ nvkm_runl_add(runl, 1, fifo->func->engn, NVKM_ENGINE_MSPDEC, 0);
+ nvkm_runl_add(runl, 2, fifo->func->engn, NVKM_ENGINE_MSPPP, 0);
+ nvkm_runl_add(runl, 3, fifo->func->engn, NVKM_ENGINE_MSVLD, 0);
+ nvkm_runl_add(runl, 4, fifo->func->engn, NVKM_ENGINE_CE, 0);
+ nvkm_runl_add(runl, 5, fifo->func->engn, NVKM_ENGINE_CE, 1);
+ nvkm_runl_add(runl, 15, &gf100_engn_sw, NVKM_ENGINE_SW, 0);
+ return 0;
+}
+
+int
+gf100_fifo_runq_nr(struct nvkm_fifo *fifo)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ u32 save;
+
+ /* Determine number of PBDMAs by checking valid enable bits. */
+ save = nvkm_mask(device, 0x000204, 0xffffffff, 0xffffffff);
+ save = nvkm_mask(device, 0x000204, 0xffffffff, save);
+ return hweight32(save);
+}
+
+int
+gf100_fifo_chid_ctor(struct nvkm_fifo *fifo, int nr)
+{
+ return nvkm_chid_new(&nvkm_chan_event, &fifo->engine.subdev, nr, 0, nr, &fifo->chid);
}
static const struct nvkm_fifo_func
gf100_fifo = {
- .dtor = gf100_fifo_dtor,
- .oneinit = gf100_fifo_oneinit,
+ .chid_nr = nv50_fifo_chid_nr,
+ .chid_ctor = gf100_fifo_chid_ctor,
+ .runq_nr = gf100_fifo_runq_nr,
+ .runl_ctor = gf100_fifo_runl_ctor,
.init = gf100_fifo_init,
- .fini = gf100_fifo_fini,
+ .init_pbdmas = gf100_fifo_init_pbdmas,
.intr = gf100_fifo_intr,
- .fault = gf100_fifo_fault,
- .engine_id = gf100_fifo_engine_id,
- .id_engine = gf100_fifo_id_engine,
- .uevent_init = gf100_fifo_uevent_init,
- .uevent_fini = gf100_fifo_uevent_fini,
- .chan = {
- &gf100_fifo_gpfifo_oclass,
- NULL
- },
+ .intr_mmu_fault_unit = gf100_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gf100_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gf100_runl,
+ .runq = &gf100_runq,
+ .engn = &gf100_engn,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, FERMI_CHANNEL_GPFIFO }, &gf100_chan },
};
int
gf100_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- struct gf100_fifo *fifo;
-
- if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
- return -ENOMEM;
- INIT_LIST_HEAD(&fifo->chan);
- INIT_WORK(&fifo->recover.work, gf100_fifo_recover_work);
- *pfifo = &fifo->base;
-
- return nvkm_fifo_ctor(&gf100_fifo, device, type, inst, 128, &fifo->base);
+ return nvkm_fifo_new_(&gf100_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.h
deleted file mode 100644
index b864249..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.h
+++ /dev/null
@@ -1,38 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __GF100_FIFO_H__
-#define __GF100_FIFO_H__
-#define gf100_fifo(p) container_of((p), struct gf100_fifo, base)
-#include "priv.h"
-
-#include <subdev/mmu.h>
-
-struct gf100_fifo_chan;
-struct gf100_fifo {
- struct nvkm_fifo base;
-
- struct list_head chan;
-
- struct {
- struct work_struct work;
- u64 mask;
- } recover;
-
- int pbdma_nr;
-
- struct {
- struct nvkm_memory *mem[2];
- int active;
- wait_queue_head_t wait;
- } runlist;
-
- struct {
- struct nvkm_memory *mem;
- struct nvkm_vma *bar;
- } user;
-};
-
-void gf100_fifo_intr_engine(struct gf100_fifo *);
-void gf100_fifo_runlist_insert(struct gf100_fifo *, struct gf100_fifo_chan *);
-void gf100_fifo_runlist_remove(struct gf100_fifo *, struct gf100_fifo_chan *);
-void gf100_fifo_runlist_commit(struct gf100_fifo *);
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c
index e771bd5..d8a4d77 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c
@@ -21,29 +21,189 @@
*
* Authors: Ben Skeggs
*/
-#include "gk104.h"
+#include "priv.h"
#include "cgrp.h"
-#include "changk104.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+#include "runq.h"
-#include <core/client.h>
#include <core/gpuobj.h>
-#include <subdev/bar.h>
-#include <subdev/fault.h>
-#include <subdev/timer.h>
+#include <subdev/mc.h>
+#include <subdev/mmu.h>
#include <subdev/top.h>
-#include <engine/sw.h>
#include <nvif/class.h>
-#include <nvif/cl0080.h>
+#include <nvif/if900d.h>
void
-gk104_fifo_engine_status(struct gk104_fifo *fifo, int engn,
- struct gk104_fifo_engine_status *status)
+gk104_chan_stop(struct nvkm_chan *chan)
{
- struct nvkm_engine *engine = fifo->engine[engn].engine;
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 stat = nvkm_rd32(device, 0x002640 + (engn * 0x08));
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_mask(device, 0x800004 + (chan->id * 8), 0x00000800, 0x00000800);
+}
+
+void
+gk104_chan_start(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_mask(device, 0x800004 + (chan->id * 8), 0x00000400, 0x00000400);
+}
+
+void
+gk104_chan_unbind(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x800000 + (chan->id * 8), 0x00000000);
+}
+
+void
+gk104_chan_bind_inst(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x800000 + (chan->id * 8), 0x80000000 | chan->inst->addr >> 12);
+}
+
+void
+gk104_chan_bind(struct nvkm_chan *chan)
+{
+ struct nvkm_runl *runl = chan->cgrp->runl;
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+
+ nvkm_mask(device, 0x800004 + (chan->id * 8), 0x000f0000, runl->id << 16);
+ gk104_chan_bind_inst(chan);
+}
+
+static int
+gk104_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ const u64 userd = nvkm_memory_addr(chan->userd.mem) + chan->userd.base;
+ const u32 limit2 = ilog2(length / 8);
+
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, 0x08, lower_32_bits(userd));
+ nvkm_wo32(chan->inst, 0x0c, upper_32_bits(userd));
+ nvkm_wo32(chan->inst, 0x10, 0x0000face);
+ nvkm_wo32(chan->inst, 0x30, 0xfffff902);
+ nvkm_wo32(chan->inst, 0x48, lower_32_bits(offset));
+ nvkm_wo32(chan->inst, 0x4c, upper_32_bits(offset) | (limit2 << 16));
+ nvkm_wo32(chan->inst, 0x84, 0x20400000);
+ nvkm_wo32(chan->inst, 0x94, 0x30000000 | devm);
+ nvkm_wo32(chan->inst, 0x9c, 0x00000100);
+ nvkm_wo32(chan->inst, 0xac, 0x0000001f);
+ nvkm_wo32(chan->inst, 0xe4, priv ? 0x00000020 : 0x00000000);
+ nvkm_wo32(chan->inst, 0xe8, chan->id);
+ nvkm_wo32(chan->inst, 0xb8, 0xf8000000);
+ nvkm_wo32(chan->inst, 0xf8, 0x10003080); /* 0x002310 */
+ nvkm_wo32(chan->inst, 0xfc, 0x10000010); /* 0x002350 */
+ nvkm_done(chan->inst);
+ return 0;
+}
+
+const struct nvkm_chan_func_ramfc
+gk104_chan_ramfc = {
+ .write = gk104_chan_ramfc_write,
+ .devm = 0xfff,
+ .priv = true,
+};
+
+const struct nvkm_chan_func_userd
+gk104_chan_userd = {
+ .bar = 1,
+ .size = 0x200,
+ .clear = gf100_chan_userd_clear,
+};
+
+static const struct nvkm_chan_func
+gk104_chan = {
+ .inst = &gf100_chan_inst,
+ .userd = &gk104_chan_userd,
+ .ramfc = &gk104_chan_ramfc,
+ .bind = gk104_chan_bind,
+ .unbind = gk104_chan_unbind,
+ .start = gk104_chan_start,
+ .stop = gk104_chan_stop,
+ .preempt = gf100_chan_preempt,
+};
+
+static void
+gk104_ectx_bind(struct nvkm_engn *engn, struct nvkm_cctx *cctx, struct nvkm_chan *chan)
+{
+ u32 ptr0, ptr1 = 0;
+ u64 addr = 0ULL;
+
+ switch (engn->engine->subdev.type) {
+ case NVKM_ENGINE_SW : return;
+ case NVKM_ENGINE_GR : ptr0 = 0x0210; break;
+ case NVKM_ENGINE_SEC : ptr0 = 0x0220; break;
+ case NVKM_ENGINE_MSPDEC: ptr0 = 0x0250; break;
+ case NVKM_ENGINE_MSPPP : ptr0 = 0x0260; break;
+ case NVKM_ENGINE_MSVLD : ptr0 = 0x0270; break;
+ case NVKM_ENGINE_VIC : ptr0 = 0x0280; break;
+ case NVKM_ENGINE_MSENC : ptr0 = 0x0290; break;
+ case NVKM_ENGINE_NVDEC :
+ ptr1 = 0x0270;
+ ptr0 = 0x0210;
+ break;
+ case NVKM_ENGINE_NVENC :
+ if (!engn->engine->subdev.inst)
+ ptr1 = 0x0290;
+ ptr0 = 0x0210;
+ break;
+ default:
+ WARN_ON(1);
+ return;
+ }
+
+ if (cctx) {
+ addr = cctx->vctx->vma->addr;
+ addr |= 4ULL;
+ }
+
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, ptr0 + 0, lower_32_bits(addr));
+ nvkm_wo32(chan->inst, ptr0 + 4, upper_32_bits(addr));
+ if (ptr1) {
+ nvkm_wo32(chan->inst, ptr1 + 0, lower_32_bits(addr));
+ nvkm_wo32(chan->inst, ptr1 + 4, upper_32_bits(addr));
+ }
+ nvkm_done(chan->inst);
+}
+
+int
+gk104_ectx_ctor(struct nvkm_engn *engn, struct nvkm_vctx *vctx)
+{
+ struct gf100_vmm_map_v0 args = { .priv = 1 };
+ int ret;
+
+ ret = nvkm_vmm_get(vctx->vmm, 12, vctx->inst->size, &vctx->vma);
+ if (ret)
+ return ret;
+
+ return nvkm_memory_map(vctx->inst, 0, vctx->vmm, vctx->vma, &args, sizeof(args));
+}
+
+/*TODO: clean this up */
+struct gk104_engn_status {
+ bool busy;
+ bool faulted;
+ bool chsw;
+ bool save;
+ bool load;
+ struct {
+ bool tsg;
+ u32 id;
+ } prev, next, *chan;
+};
+
+static void
+gk104_engn_status(struct nvkm_engn *engn, struct gk104_engn_status *status)
+{
+ u32 stat = nvkm_rd32(engn->runl->fifo->engine.subdev.device, 0x002640 + (engn->id * 0x08));
status->busy = !!(stat & 0x80000000);
status->faulted = !!(stat & 0x40000000);
@@ -58,7 +218,7 @@ gk104_fifo_engine_status(struct gk104_fifo *fifo, int engn,
if (status->busy && status->chsw) {
if (status->load && status->save) {
- if (engine && nvkm_engine_chsw_load(engine))
+ if (nvkm_engine_chsw_load(engn->engine))
status->chan = &status->next;
else
status->chan = &status->prev;
@@ -73,10 +233,8 @@ gk104_fifo_engine_status(struct gk104_fifo *fifo, int engn,
status->chan = &status->prev;
}
- nvkm_debug(subdev, "engine %02d: busy %d faulted %d chsw %d "
- "save %d load %d %sid %d%s-> %sid %d%s\n",
- engn, status->busy, status->faulted,
- status->chsw, status->save, status->load,
+ ENGN_DEBUG(engn, "%08x: busy %d faulted %d chsw %d save %d load %d %sid %d%s-> %sid %d%s",
+ stat, status->busy, status->faulted, status->chsw, status->save, status->load,
status->prev.tsg ? "tsg" : "ch", status->prev.id,
status->chan == &status->prev ? "*" : " ",
status->next.tsg ? "tsg" : "ch", status->next.id,
@@ -84,580 +242,97 @@ gk104_fifo_engine_status(struct gk104_fifo *fifo, int engn,
}
int
-gk104_fifo_class_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *argv, u32 argc, struct nvkm_object **pobject)
+gk104_engn_cxid(struct nvkm_engn *engn, bool *cgid)
{
- struct gk104_fifo *fifo = gk104_fifo(base);
- if (oclass->engn == &fifo->func->chan) {
- const struct gk104_fifo_chan_user *user = oclass->engn;
- return user->ctor(fifo, oclass, argv, argc, pobject);
- } else
- if (oclass->engn == &fifo->func->user) {
- const struct gk104_fifo_user_user *user = oclass->engn;
- return user->ctor(oclass, argv, argc, pobject);
- }
- WARN_ON(1);
- return -EINVAL;
-}
+ struct gk104_engn_status status;
-int
-gk104_fifo_class_get(struct nvkm_fifo *base, int index,
- struct nvkm_oclass *oclass)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- int c = 0;
-
- if (fifo->func->user.ctor && c++ == index) {
- oclass->base = fifo->func->user.user;
- oclass->engn = &fifo->func->user;
- return 0;
- }
-
- if (fifo->func->chan.ctor && c++ == index) {
- oclass->base = fifo->func->chan.user;
- oclass->engn = &fifo->func->chan;
- return 0;
- }
-
- return c;
-}
-
-void
-gk104_fifo_uevent_fini(struct nvkm_fifo *fifo)
-{
- struct nvkm_device *device = fifo->engine.subdev.device;
- nvkm_mask(device, 0x002140, 0x80000000, 0x00000000);
-}
-
-void
-gk104_fifo_uevent_init(struct nvkm_fifo *fifo)
-{
- struct nvkm_device *device = fifo->engine.subdev.device;
- nvkm_mask(device, 0x002140, 0x80000000, 0x80000000);
-}
-
-void
-gk104_fifo_runlist_commit(struct gk104_fifo *fifo, int runl,
- struct nvkm_memory *mem, int nr)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- int target;
-
- switch (nvkm_memory_target(mem)) {
- case NVKM_MEM_TARGET_VRAM: target = 0; break;
- case NVKM_MEM_TARGET_NCOH: target = 3; break;
- default:
- WARN_ON(1);
- return;
- }
-
- nvkm_wr32(device, 0x002270, (nvkm_memory_addr(mem) >> 12) |
- (target << 28));
- nvkm_wr32(device, 0x002274, (runl << 20) | nr);
-
- if (nvkm_msec(device, 2000,
- if (!(nvkm_rd32(device, 0x002284 + (runl * 0x08)) & 0x00100000))
- break;
- ) < 0)
- nvkm_error(subdev, "runlist %d update timeout\n", runl);
-}
-
-void
-gk104_fifo_runlist_update(struct gk104_fifo *fifo, int runl)
-{
- const struct gk104_fifo_runlist_func *func = fifo->func->runlist;
- struct gk104_fifo_chan *chan;
- struct nvkm_memory *mem;
- struct nvkm_fifo_cgrp *cgrp;
- int nr = 0;
-
- mutex_lock(&fifo->base.mutex);
- mem = fifo->runlist[runl].mem[fifo->runlist[runl].next];
- fifo->runlist[runl].next = !fifo->runlist[runl].next;
-
- nvkm_kmap(mem);
- list_for_each_entry(chan, &fifo->runlist[runl].chan, head) {
- func->chan(chan, mem, nr++ * func->size);
- }
-
- list_for_each_entry(cgrp, &fifo->runlist[runl].cgrp, head) {
- func->cgrp(cgrp, mem, nr++ * func->size);
- list_for_each_entry(chan, &cgrp->chan, head) {
- func->chan(chan, mem, nr++ * func->size);
- }
- }
- nvkm_done(mem);
-
- func->commit(fifo, runl, mem, nr);
- mutex_unlock(&fifo->base.mutex);
-}
-
-void
-gk104_fifo_runlist_remove(struct gk104_fifo *fifo, struct gk104_fifo_chan *chan)
-{
- struct nvkm_fifo_cgrp *cgrp = chan->cgrp;
- mutex_lock(&fifo->base.mutex);
- if (!list_empty(&chan->head)) {
- list_del_init(&chan->head);
- if (cgrp && !--cgrp->chan_nr)
- list_del_init(&cgrp->head);
- }
- mutex_unlock(&fifo->base.mutex);
-}
-
-void
-gk104_fifo_runlist_insert(struct gk104_fifo *fifo, struct gk104_fifo_chan *chan)
-{
- struct nvkm_fifo_cgrp *cgrp = chan->cgrp;
- mutex_lock(&fifo->base.mutex);
- if (cgrp) {
- if (!cgrp->chan_nr++)
- list_add_tail(&cgrp->head, &fifo->runlist[chan->runl].cgrp);
- list_add_tail(&chan->head, &cgrp->chan);
- } else {
- list_add_tail(&chan->head, &fifo->runlist[chan->runl].chan);
- }
- mutex_unlock(&fifo->base.mutex);
-}
-
-void
-gk104_fifo_runlist_chan(struct gk104_fifo_chan *chan,
- struct nvkm_memory *memory, u32 offset)
-{
- nvkm_wo32(memory, offset + 0, chan->base.chid);
- nvkm_wo32(memory, offset + 4, 0x00000000);
-}
-
-const struct gk104_fifo_runlist_func
-gk104_fifo_runlist = {
- .size = 8,
- .chan = gk104_fifo_runlist_chan,
- .commit = gk104_fifo_runlist_commit,
-};
-
-void
-gk104_fifo_pbdma_init(struct gk104_fifo *fifo)
-{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- nvkm_wr32(device, 0x000204, (1 << fifo->pbdma_nr) - 1);
-}
-
-int
-gk104_fifo_pbdma_nr(struct gk104_fifo *fifo)
-{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- /* Determine number of PBDMAs by checking valid enable bits. */
- nvkm_wr32(device, 0x000204, 0xffffffff);
- return hweight32(nvkm_rd32(device, 0x000204));
-}
-
-const struct gk104_fifo_pbdma_func
-gk104_fifo_pbdma = {
- .nr = gk104_fifo_pbdma_nr,
- .init = gk104_fifo_pbdma_init,
-};
-
-struct nvkm_engine *
-gk104_fifo_id_engine(struct nvkm_fifo *base, int engi)
-{
- if (engi == GK104_FIFO_ENGN_SW)
- return nvkm_device_engine(base->engine.subdev.device, NVKM_ENGINE_SW, 0);
-
- return gk104_fifo(base)->engine[engi].engine;
-}
-
-int
-gk104_fifo_engine_id(struct nvkm_fifo *base, struct nvkm_engine *engine)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- int engn;
-
- if (engine->subdev.type == NVKM_ENGINE_SW)
- return GK104_FIFO_ENGN_SW;
-
- for (engn = 0; engn < fifo->engine_nr && engine; engn++) {
- if (fifo->engine[engn].engine == engine)
- return engn;
- }
-
- WARN_ON(1);
- return -1;
-}
-
-static void
-gk104_fifo_recover_work(struct work_struct *w)
-{
- struct gk104_fifo *fifo = container_of(w, typeof(*fifo), recover.work);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_engine *engine;
- unsigned long flags;
- u32 engm, runm, todo;
- int engn, runl;
-
- spin_lock_irqsave(&fifo->base.lock, flags);
- runm = fifo->recover.runm;
- engm = fifo->recover.engm;
- fifo->recover.engm = 0;
- fifo->recover.runm = 0;
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-
- nvkm_mask(device, 0x002630, runm, runm);
-
- for (todo = engm; engn = __ffs(todo), todo; todo &= ~BIT(engn)) {
- if ((engine = fifo->engine[engn].engine)) {
- nvkm_subdev_fini(&engine->subdev, false);
- WARN_ON(nvkm_subdev_init(&engine->subdev));
- }
- }
-
- for (todo = runm; runl = __ffs(todo), todo; todo &= ~BIT(runl))
- gk104_fifo_runlist_update(fifo, runl);
-
- nvkm_wr32(device, 0x00262c, runm);
- nvkm_mask(device, 0x002630, runm, 0x00000000);
-}
-
-static void gk104_fifo_recover_engn(struct gk104_fifo *fifo, int engn);
-
-static void
-gk104_fifo_recover_runl(struct gk104_fifo *fifo, int runl)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const u32 runm = BIT(runl);
-
- assert_spin_locked(&fifo->base.lock);
- if (fifo->recover.runm & runm)
- return;
- fifo->recover.runm |= runm;
-
- /* Block runlist to prevent channel assignment(s) from changing. */
- nvkm_mask(device, 0x002630, runm, runm);
-
- /* Schedule recovery. */
- nvkm_warn(subdev, "runlist %d: scheduled for recovery\n", runl);
- schedule_work(&fifo->recover.work);
-}
-
-static struct gk104_fifo_chan *
-gk104_fifo_recover_chid(struct gk104_fifo *fifo, int runl, int chid)
-{
- struct gk104_fifo_chan *chan;
- struct nvkm_fifo_cgrp *cgrp;
-
- list_for_each_entry(chan, &fifo->runlist[runl].chan, head) {
- if (chan->base.chid == chid) {
- list_del_init(&chan->head);
- return chan;
- }
- }
-
- list_for_each_entry(cgrp, &fifo->runlist[runl].cgrp, head) {
- if (cgrp->id == chid) {
- chan = list_first_entry(&cgrp->chan, typeof(*chan), head);
- list_del_init(&chan->head);
- if (!--cgrp->chan_nr)
- list_del_init(&cgrp->head);
- return chan;
- }
- }
-
- return NULL;
-}
-
-static void
-gk104_fifo_recover_chan(struct nvkm_fifo *base, int chid)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const u32 stat = nvkm_rd32(device, 0x800004 + (chid * 0x08));
- const u32 runl = (stat & 0x000f0000) >> 16;
- const bool used = (stat & 0x00000001);
- unsigned long engn, engm = fifo->runlist[runl].engm;
- struct gk104_fifo_chan *chan;
-
- assert_spin_locked(&fifo->base.lock);
- if (!used)
- return;
-
- /* Lookup SW state for channel, and mark it as dead. */
- chan = gk104_fifo_recover_chid(fifo, runl, chid);
- if (chan) {
- chan->killed = true;
- nvkm_fifo_kevent(&fifo->base, chid);
- }
-
- /* Disable channel. */
- nvkm_wr32(device, 0x800004 + (chid * 0x08), stat | 0x00000800);
- nvkm_warn(subdev, "channel %d: killed\n", chid);
-
- /* Block channel assignments from changing during recovery. */
- gk104_fifo_recover_runl(fifo, runl);
-
- /* Schedule recovery for any engines the channel is on. */
- for_each_set_bit(engn, &engm, fifo->engine_nr) {
- struct gk104_fifo_engine_status status;
- gk104_fifo_engine_status(fifo, engn, &status);
- if (!status.chan || status.chan->id != chid)
- continue;
- gk104_fifo_recover_engn(fifo, engn);
- }
-}
-
-static void
-gk104_fifo_recover_engn(struct gk104_fifo *fifo, int engn)
-{
- struct nvkm_engine *engine = fifo->engine[engn].engine;
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const u32 runl = fifo->engine[engn].runl;
- const u32 engm = BIT(engn);
- struct gk104_fifo_engine_status status;
- int mmui = -1;
-
- assert_spin_locked(&fifo->base.lock);
- if (fifo->recover.engm & engm)
- return;
- fifo->recover.engm |= engm;
-
- /* Block channel assignments from changing during recovery. */
- gk104_fifo_recover_runl(fifo, runl);
-
- /* Determine which channel (if any) is currently on the engine. */
- gk104_fifo_engine_status(fifo, engn, &status);
+ gk104_engn_status(engn, &status);
if (status.chan) {
- /* The channel is not longer viable, kill it. */
- gk104_fifo_recover_chan(&fifo->base, status.chan->id);
+ *cgid = status.chan->tsg;
+ return status.chan->id;
}
- /* Determine MMU fault ID for the engine, if we're not being
- * called from the fault handler already.
- */
- if (!status.faulted && engine) {
- mmui = nvkm_top_fault_id(device, engine->subdev.type, engine->subdev.inst);
- if (mmui < 0) {
- const struct nvkm_enum *en = fifo->func->fault.engine;
- for (; en && en->name; en++) {
- if (en->data2 == engine->subdev.type &&
- en->inst == engine->subdev.inst) {
- mmui = en->value;
- break;
- }
- }
- }
- WARN_ON(mmui < 0);
- }
-
- /* Trigger a MMU fault for the engine.
- *
- * No good idea why this is needed, but nvgpu does something similar,
- * and it makes recovery from CTXSW_TIMEOUT a lot more reliable.
- */
- if (mmui >= 0) {
- nvkm_wr32(device, 0x002a30 + (engn * 0x04), 0x00000100 | mmui);
-
- /* Wait for fault to trigger. */
- nvkm_msec(device, 2000,
- gk104_fifo_engine_status(fifo, engn, &status);
- if (status.faulted)
- break;
- );
-
- /* Release MMU fault trigger, and ACK the fault. */
- nvkm_wr32(device, 0x002a30 + (engn * 0x04), 0x00000000);
- nvkm_wr32(device, 0x00259c, BIT(mmui));
- nvkm_wr32(device, 0x002100, 0x10000000);
- }
-
- /* Schedule recovery. */
- nvkm_warn(subdev, "engine %d: scheduled for recovery\n", engn);
- schedule_work(&fifo->recover.work);
+ return -ENODEV;
}
-static void
-gk104_fifo_fault(struct nvkm_fifo *base, struct nvkm_fault_data *info)
+bool
+gk104_engn_chsw(struct nvkm_engn *engn)
{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const struct nvkm_enum *er, *ee, *ec, *ea;
- struct nvkm_engine *engine = NULL;
- struct nvkm_fifo_chan *chan;
- unsigned long flags;
- const char *en = "";
- char ct[8] = "HUB/";
+ struct gk104_engn_status status;
- er = nvkm_enum_find(fifo->func->fault.reason, info->reason);
- ee = nvkm_enum_find(fifo->func->fault.engine, info->engine);
- if (info->hub) {
- ec = nvkm_enum_find(fifo->func->fault.hubclient, info->client);
- } else {
- ec = nvkm_enum_find(fifo->func->fault.gpcclient, info->client);
- snprintf(ct, sizeof(ct), "GPC%d/", info->gpc);
- }
- ea = nvkm_enum_find(fifo->func->fault.access, info->access);
+ gk104_engn_status(engn, &status);
+ if (status.busy && status.chsw)
+ return true;
- if (ee && ee->data2) {
- switch (ee->data2) {
- case NVKM_SUBDEV_BAR:
- nvkm_bar_bar1_reset(device);
- break;
- case NVKM_SUBDEV_INSTMEM:
- nvkm_bar_bar2_reset(device);
- break;
- case NVKM_ENGINE_IFB:
- nvkm_mask(device, 0x001718, 0x00000000, 0x00000000);
- break;
- default:
- engine = nvkm_device_engine(device, ee->data2, 0);
- break;
- }
- }
-
- if (ee == NULL) {
- struct nvkm_subdev *subdev = nvkm_top_fault(device, info->engine);
- if (subdev) {
- if (subdev->func == &nvkm_engine)
- engine = container_of(subdev, typeof(*engine), subdev);
- en = engine->subdev.name;
- }
- } else {
- en = ee->name;
- }
-
- spin_lock_irqsave(&fifo->base.lock, flags);
- chan = nvkm_fifo_chan_inst_locked(&fifo->base, info->inst);
-
- nvkm_error(subdev,
- "fault %02x [%s] at %016llx engine %02x [%s] client %02x "
- "[%s%s] reason %02x [%s] on channel %d [%010llx %s]\n",
- info->access, ea ? ea->name : "", info->addr,
- info->engine, ee ? ee->name : en,
- info->client, ct, ec ? ec->name : "",
- info->reason, er ? er->name : "", chan ? chan->chid : -1,
- info->inst, chan ? chan->object.client->name : "unknown");
-
- /* Kill the channel that caused the fault. */
- if (chan)
- gk104_fifo_recover_chan(&fifo->base, chan->chid);
-
- /* Channel recovery will probably have already done this for the
- * correct engine(s), but just in case we can't find the channel
- * information...
- */
- if (engine) {
- int engn = fifo->base.func->engine_id(&fifo->base, engine);
- if (engn >= 0 && engn != GK104_FIFO_ENGN_SW)
- gk104_fifo_recover_engn(fifo, engn);
- }
-
- spin_unlock_irqrestore(&fifo->base.lock, flags);
+ return false;
}
-static const struct nvkm_enum
-gk104_fifo_bind_reason[] = {
- { 0x01, "BIND_NOT_UNBOUND" },
- { 0x02, "SNOOP_WITHOUT_BAR1" },
- { 0x03, "UNBIND_WHILE_RUNNING" },
- { 0x05, "INVALID_RUNLIST" },
- { 0x06, "INVALID_CTX_TGT" },
- { 0x0b, "UNBIND_WHILE_PARKED" },
+const struct nvkm_engn_func
+gk104_engn = {
+ .chsw = gk104_engn_chsw,
+ .cxid = gk104_engn_cxid,
+ .mmu_fault_trigger = gf100_engn_mmu_fault_trigger,
+ .mmu_fault_triggered = gf100_engn_mmu_fault_triggered,
+ .ctor = gk104_ectx_ctor,
+ .bind = gk104_ectx_bind,
+};
+
+const struct nvkm_engn_func
+gk104_engn_ce = {
+ .chsw = gk104_engn_chsw,
+ .cxid = gk104_engn_cxid,
+ .mmu_fault_trigger = gf100_engn_mmu_fault_trigger,
+ .mmu_fault_triggered = gf100_engn_mmu_fault_triggered,
+};
+
+bool
+gk104_runq_idle(struct nvkm_runq *runq)
+{
+ struct nvkm_device *device = runq->fifo->engine.subdev.device;
+
+ return !(nvkm_rd32(device, 0x003080 + (runq->id * 4)) & 0x0000e000);
+}
+
+static const struct nvkm_bitfield
+gk104_runq_intr_1_names[] = {
+ { 0x00000001, "HCE_RE_ILLEGAL_OP" },
+ { 0x00000002, "HCE_RE_ALIGNB" },
+ { 0x00000004, "HCE_PRIV" },
+ { 0x00000008, "HCE_ILLEGAL_MTHD" },
+ { 0x00000010, "HCE_ILLEGAL_CLASS" },
{}
};
-void
-gk104_fifo_intr_bind(struct gk104_fifo *fifo)
+static bool
+gk104_runq_intr_1(struct nvkm_runq *runq)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_subdev *subdev = &runq->fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
- u32 intr = nvkm_rd32(device, 0x00252c);
- u32 code = intr & 0x000000ff;
- const struct nvkm_enum *en =
- nvkm_enum_find(gk104_fifo_bind_reason, code);
+ u32 mask = nvkm_rd32(device, 0x04014c + (runq->id * 0x2000));
+ u32 stat = nvkm_rd32(device, 0x040148 + (runq->id * 0x2000)) & mask;
+ u32 chid = nvkm_rd32(device, 0x040120 + (runq->id * 0x2000)) & 0xfff;
+ char msg[128];
- nvkm_error(subdev, "BIND_ERROR %02x [%s]\n", code, en ? en->name : "");
-}
-
-static const struct nvkm_enum
-gk104_fifo_sched_reason[] = {
- { 0x0a, "CTXSW_TIMEOUT" },
- {}
-};
-
-static void
-gk104_fifo_intr_sched_ctxsw(struct gk104_fifo *fifo)
-{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- unsigned long flags, engm = 0;
- u32 engn;
-
- /* We need to ACK the SCHED_ERROR here, and prevent it reasserting,
- * as MMU_FAULT cannot be triggered while it's pending.
- */
- spin_lock_irqsave(&fifo->base.lock, flags);
- nvkm_mask(device, 0x002140, 0x00000100, 0x00000000);
- nvkm_wr32(device, 0x002100, 0x00000100);
-
- for (engn = 0; engn < fifo->engine_nr; engn++) {
- struct gk104_fifo_engine_status status;
-
- gk104_fifo_engine_status(fifo, engn, &status);
- if (!status.busy || !status.chsw)
- continue;
-
- engm |= BIT(engn);
+ if (stat & 0x80000000) {
+ if (runq->func->intr_1_ctxnotvalid &&
+ runq->func->intr_1_ctxnotvalid(runq, chid))
+ stat &= ~0x80000000;
}
- for_each_set_bit(engn, &engm, fifo->engine_nr)
- gk104_fifo_recover_engn(fifo, engn);
-
- nvkm_mask(device, 0x002140, 0x00000100, 0x00000100);
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-}
-
-static void
-gk104_fifo_intr_sched(struct gk104_fifo *fifo)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 intr = nvkm_rd32(device, 0x00254c);
- u32 code = intr & 0x000000ff;
- const struct nvkm_enum *en =
- nvkm_enum_find(gk104_fifo_sched_reason, code);
-
- nvkm_error(subdev, "SCHED_ERROR %02x [%s]\n", code, en ? en->name : "");
-
- switch (code) {
- case 0x0a:
- gk104_fifo_intr_sched_ctxsw(fifo);
- break;
- default:
- break;
+ if (stat) {
+ nvkm_snprintbf(msg, sizeof(msg), gk104_runq_intr_1_names, stat);
+ nvkm_error(subdev, "PBDMA%d: %08x [%s] ch %d %08x %08x\n",
+ runq->id, stat, msg, chid,
+ nvkm_rd32(device, 0x040150 + (runq->id * 0x2000)),
+ nvkm_rd32(device, 0x040154 + (runq->id * 0x2000)));
}
+
+ nvkm_wr32(device, 0x040148 + (runq->id * 0x2000), stat);
+ return true;
}
-void
-gk104_fifo_intr_chsw(struct gk104_fifo *fifo)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 stat = nvkm_rd32(device, 0x00256c);
- nvkm_error(subdev, "CHSW_ERROR %08x\n", stat);
- nvkm_wr32(device, 0x00256c, stat);
-}
-
-void
-gk104_fifo_intr_dropped_fault(struct gk104_fifo *fifo)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 stat = nvkm_rd32(device, 0x00259c);
- nvkm_error(subdev, "DROPPED_MMU_FAULT %08x\n", stat);
-}
-
-static const struct nvkm_bitfield gk104_fifo_pbdma_intr_0[] = {
+const struct nvkm_bitfield
+gk104_runq_intr_0_names[] = {
{ 0x00000001, "MEMREQ" },
{ 0x00000002, "MEMACK_TIMEOUT" },
{ 0x00000004, "MEMACK_EXTRA" },
@@ -691,430 +366,111 @@ static const struct nvkm_bitfield gk104_fifo_pbdma_intr_0[] = {
{}
};
-void
-gk104_fifo_intr_pbdma_0(struct gk104_fifo *fifo, int unit)
+bool
+gk104_runq_intr(struct nvkm_runq *runq, struct nvkm_runl *null)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 mask = nvkm_rd32(device, 0x04010c + (unit * 0x2000));
- u32 stat = nvkm_rd32(device, 0x040108 + (unit * 0x2000)) & mask;
- u32 addr = nvkm_rd32(device, 0x0400c0 + (unit * 0x2000));
- u32 data = nvkm_rd32(device, 0x0400c4 + (unit * 0x2000));
- u32 chid = nvkm_rd32(device, 0x040120 + (unit * 0x2000)) & 0xfff;
- u32 subc = (addr & 0x00070000) >> 16;
- u32 mthd = (addr & 0x00003ffc);
- u32 show = stat;
- struct nvkm_fifo_chan *chan;
- unsigned long flags;
- char msg[128];
+ bool intr0 = gf100_runq_intr(runq, NULL);
+ bool intr1 = gk104_runq_intr_1(runq);
- if (stat & 0x00800000) {
- if (device->sw) {
- if (nvkm_sw_mthd(device->sw, chid, subc, mthd, data))
- show &= ~0x00800000;
- }
- }
-
- nvkm_wr32(device, 0x0400c0 + (unit * 0x2000), 0x80600008);
-
- if (show) {
- nvkm_snprintbf(msg, sizeof(msg), gk104_fifo_pbdma_intr_0, show);
- chan = nvkm_fifo_chan_chid(&fifo->base, chid, &flags);
- nvkm_error(subdev, "PBDMA%d: %08x [%s] ch %d [%010llx %s] "
- "subc %d mthd %04x data %08x\n",
- unit, show, msg, chid, chan ? chan->inst->addr : 0,
- chan ? chan->object.client->name : "unknown",
- subc, mthd, data);
- nvkm_fifo_chan_put(&fifo->base, flags, &chan);
- }
-
- nvkm_wr32(device, 0x040108 + (unit * 0x2000), stat);
+ return intr0 || intr1;
}
-static const struct nvkm_bitfield gk104_fifo_pbdma_intr_1[] = {
- { 0x00000001, "HCE_RE_ILLEGAL_OP" },
- { 0x00000002, "HCE_RE_ALIGNB" },
- { 0x00000004, "HCE_PRIV" },
- { 0x00000008, "HCE_ILLEGAL_MTHD" },
- { 0x00000010, "HCE_ILLEGAL_CLASS" },
- {}
+void
+gk104_runq_init(struct nvkm_runq *runq)
+{
+ struct nvkm_device *device = runq->fifo->engine.subdev.device;
+
+ gf100_runq_init(runq);
+
+ nvkm_wr32(device, 0x040148 + (runq->id * 0x2000), 0xffffffff); /* HCE.INTR */
+ nvkm_wr32(device, 0x04014c + (runq->id * 0x2000), 0xffffffff); /* HCE.INTREN */
+}
+
+static u32
+gk104_runq_runm(struct nvkm_runq *runq)
+{
+ return nvkm_rd32(runq->fifo->engine.subdev.device, 0x002390 + (runq->id * 0x04));
+}
+
+const struct nvkm_runq_func
+gk104_runq = {
+ .init = gk104_runq_init,
+ .intr = gk104_runq_intr,
+ .intr_0_names = gk104_runq_intr_0_names,
+ .idle = gk104_runq_idle,
};
void
-gk104_fifo_intr_pbdma_1(struct gk104_fifo *fifo, int unit)
+gk104_runl_fault_clear(struct nvkm_runl *runl)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 mask = nvkm_rd32(device, 0x04014c + (unit * 0x2000));
- u32 stat = nvkm_rd32(device, 0x040148 + (unit * 0x2000)) & mask;
- u32 chid = nvkm_rd32(device, 0x040120 + (unit * 0x2000)) & 0xfff;
- char msg[128];
-
- if (stat) {
- nvkm_snprintbf(msg, sizeof(msg), gk104_fifo_pbdma_intr_1, stat);
- nvkm_error(subdev, "PBDMA%d: %08x [%s] ch %d %08x %08x\n",
- unit, stat, msg, chid,
- nvkm_rd32(device, 0x040150 + (unit * 0x2000)),
- nvkm_rd32(device, 0x040154 + (unit * 0x2000)));
- }
-
- nvkm_wr32(device, 0x040148 + (unit * 0x2000), stat);
+ nvkm_wr32(runl->fifo->engine.subdev.device, 0x00262c, BIT(runl->id));
}
void
-gk104_fifo_intr_runlist(struct gk104_fifo *fifo)
+gk104_runl_allow(struct nvkm_runl *runl, u32 engm)
{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u32 mask = nvkm_rd32(device, 0x002a00);
- while (mask) {
- int runl = __ffs(mask);
- wake_up(&fifo->runlist[runl].wait);
- nvkm_wr32(device, 0x002a00, 1 << runl);
- mask &= ~(1 << runl);
- }
+ nvkm_mask(runl->fifo->engine.subdev.device, 0x002630, BIT(runl->id), 0x00000000);
}
void
-gk104_fifo_intr_engine(struct gk104_fifo *fifo)
+gk104_runl_block(struct nvkm_runl *runl, u32 engm)
{
- nvkm_fifo_uevent(&fifo->base);
+ nvkm_mask(runl->fifo->engine.subdev.device, 0x002630, BIT(runl->id), BIT(runl->id));
}
-static void
-gk104_fifo_intr(struct nvkm_fifo *base)
+bool
+gk104_runl_pending(struct nvkm_runl *runl)
{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 mask = nvkm_rd32(device, 0x002140);
- u32 stat = nvkm_rd32(device, 0x002100) & mask;
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
- if (stat & 0x00000001) {
- gk104_fifo_intr_bind(fifo);
- nvkm_wr32(device, 0x002100, 0x00000001);
- stat &= ~0x00000001;
- }
-
- if (stat & 0x00000010) {
- nvkm_error(subdev, "PIO_ERROR\n");
- nvkm_wr32(device, 0x002100, 0x00000010);
- stat &= ~0x00000010;
- }
-
- if (stat & 0x00000100) {
- gk104_fifo_intr_sched(fifo);
- nvkm_wr32(device, 0x002100, 0x00000100);
- stat &= ~0x00000100;
- }
-
- if (stat & 0x00010000) {
- gk104_fifo_intr_chsw(fifo);
- nvkm_wr32(device, 0x002100, 0x00010000);
- stat &= ~0x00010000;
- }
-
- if (stat & 0x00800000) {
- nvkm_error(subdev, "FB_FLUSH_TIMEOUT\n");
- nvkm_wr32(device, 0x002100, 0x00800000);
- stat &= ~0x00800000;
- }
-
- if (stat & 0x01000000) {
- nvkm_error(subdev, "LB_ERROR\n");
- nvkm_wr32(device, 0x002100, 0x01000000);
- stat &= ~0x01000000;
- }
-
- if (stat & 0x08000000) {
- gk104_fifo_intr_dropped_fault(fifo);
- nvkm_wr32(device, 0x002100, 0x08000000);
- stat &= ~0x08000000;
- }
-
- if (stat & 0x10000000) {
- u32 mask = nvkm_rd32(device, 0x00259c);
- while (mask) {
- u32 unit = __ffs(mask);
- fifo->func->intr.fault(&fifo->base, unit);
- nvkm_wr32(device, 0x00259c, (1 << unit));
- mask &= ~(1 << unit);
- }
- stat &= ~0x10000000;
- }
-
- if (stat & 0x20000000) {
- u32 mask = nvkm_rd32(device, 0x0025a0);
- while (mask) {
- u32 unit = __ffs(mask);
- gk104_fifo_intr_pbdma_0(fifo, unit);
- gk104_fifo_intr_pbdma_1(fifo, unit);
- nvkm_wr32(device, 0x0025a0, (1 << unit));
- mask &= ~(1 << unit);
- }
- stat &= ~0x20000000;
- }
-
- if (stat & 0x40000000) {
- gk104_fifo_intr_runlist(fifo);
- stat &= ~0x40000000;
- }
-
- if (stat & 0x80000000) {
- nvkm_wr32(device, 0x002100, 0x80000000);
- gk104_fifo_intr_engine(fifo);
- stat &= ~0x80000000;
- }
-
- if (stat) {
- nvkm_error(subdev, "INTR %08x\n", stat);
- nvkm_mask(device, 0x002140, stat, 0x00000000);
- nvkm_wr32(device, 0x002100, stat);
- }
+ return nvkm_rd32(device, 0x002284 + (runl->id * 0x08)) & 0x00100000;
}
void
-gk104_fifo_fini(struct nvkm_fifo *base)
+gk104_runl_commit(struct nvkm_runl *runl, struct nvkm_memory *memory, u32 start, int count)
{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- flush_work(&fifo->recover.work);
- /* allow mmu fault interrupts, even when we're not using fifo */
- nvkm_mask(device, 0x002140, 0x10000000, 0x10000000);
-}
+ struct nvkm_fifo *fifo = runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ u64 addr = nvkm_memory_addr(memory) + start;
+ int target;
-int
-gk104_fifo_info(struct nvkm_fifo *base, u64 mthd, u64 *data)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- switch (mthd) {
- case NV_DEVICE_HOST_RUNLISTS:
- *data = (1ULL << fifo->runlist_nr) - 1;
- return 0;
- case NV_DEVICE_HOST_RUNLIST_ENGINES: {
- if (*data < fifo->runlist_nr) {
- unsigned long engm = fifo->runlist[*data].engm;
- struct nvkm_engine *engine;
- int engn;
- *data = 0;
- for_each_set_bit(engn, &engm, fifo->engine_nr) {
- if ((engine = fifo->engine[engn].engine)) {
-#define CASE(n) case NVKM_ENGINE_##n: *data |= NV_DEVICE_HOST_RUNLIST_ENGINES_##n; break
- switch (engine->subdev.type) {
- CASE(SW );
- CASE(GR );
- CASE(MPEG );
- CASE(ME );
- CASE(CIPHER);
- CASE(BSP );
- CASE(VP );
- CASE(CE );
- CASE(SEC );
- CASE(MSVLD );
- CASE(MSPDEC);
- CASE(MSPPP );
- CASE(MSENC );
- CASE(VIC );
- CASE(SEC2 );
- CASE(NVDEC );
- CASE(NVENC );
- default:
- WARN_ON(1);
- break;
- }
- }
- }
- return 0;
- }
- }
- return -EINVAL;
+ switch (nvkm_memory_target(memory)) {
+ case NVKM_MEM_TARGET_VRAM: target = 0; break;
+ case NVKM_MEM_TARGET_NCOH: target = 3; break;
default:
- return -EINVAL;
- }
-}
-
-int
-gk104_fifo_oneinit(struct nvkm_fifo *base)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- struct nvkm_vmm *bar = nvkm_bar_bar1_vmm(device);
- struct nvkm_top_device *tdev;
- int pbid, ret, i, j;
- u32 *map;
-
- fifo->pbdma_nr = fifo->func->pbdma->nr(fifo);
- nvkm_debug(subdev, "%d PBDMA(s)\n", fifo->pbdma_nr);
-
- /* Read PBDMA->runlist(s) mapping from HW. */
- if (!(map = kcalloc(fifo->pbdma_nr, sizeof(*map), GFP_KERNEL)))
- return -ENOMEM;
-
- for (i = 0; i < fifo->pbdma_nr; i++)
- map[i] = nvkm_rd32(device, 0x002390 + (i * 0x04));
-
- /* Determine runlist configuration from topology device info. */
- list_for_each_entry(tdev, &device->top->device, head) {
- const int engn = tdev->engine;
- char _en[16], *en;
-
- if (engn < 0)
- continue;
-
- /* Determine which PBDMA handles requests for this engine. */
- for (j = 0, pbid = -1; j < fifo->pbdma_nr; j++) {
- if (map[j] & BIT(tdev->runlist)) {
- pbid = j;
- break;
- }
- }
-
- fifo->engine[engn].engine = nvkm_device_engine(device, tdev->type, tdev->inst);
- if (!fifo->engine[engn].engine) {
- snprintf(_en, sizeof(_en), "%s, %d",
- nvkm_subdev_type[tdev->type], tdev->inst);
- en = _en;
- } else {
- en = fifo->engine[engn].engine->subdev.name;
- }
-
- nvkm_debug(subdev, "engine %2d: runlist %2d pbdma %2d (%s)\n",
- tdev->engine, tdev->runlist, pbid, en);
-
- fifo->engine[engn].runl = tdev->runlist;
- fifo->engine[engn].pbid = pbid;
- fifo->engine_nr = max(fifo->engine_nr, engn + 1);
- fifo->runlist[tdev->runlist].engm |= BIT(engn);
- fifo->runlist[tdev->runlist].engm_sw |= BIT(engn);
- if (tdev->type == NVKM_ENGINE_GR)
- fifo->runlist[tdev->runlist].engm_sw |= BIT(GK104_FIFO_ENGN_SW);
- fifo->runlist_nr = max(fifo->runlist_nr, tdev->runlist + 1);
+ WARN_ON(1);
+ return;
}
- kfree(map);
-
- for (i = 0; i < fifo->runlist_nr; i++) {
- for (j = 0; j < ARRAY_SIZE(fifo->runlist[i].mem); j++) {
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST,
- fifo->base.nr * 2/* TSG+chan */ *
- fifo->func->runlist->size,
- 0x1000, false,
- &fifo->runlist[i].mem[j]);
- if (ret)
- return ret;
- }
-
- init_waitqueue_head(&fifo->runlist[i].wait);
- INIT_LIST_HEAD(&fifo->runlist[i].cgrp);
- INIT_LIST_HEAD(&fifo->runlist[i].chan);
- }
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST,
- fifo->base.nr * 0x200, 0x1000, true,
- &fifo->user.mem);
- if (ret)
- return ret;
-
- ret = nvkm_vmm_get(bar, 12, nvkm_memory_size(fifo->user.mem),
- &fifo->user.bar);
- if (ret)
- return ret;
-
- return nvkm_memory_map(fifo->user.mem, 0, bar, fifo->user.bar, NULL, 0);
+ spin_lock_irq(&fifo->lock);
+ nvkm_wr32(device, 0x002270, (target << 28) | (addr >> 12));
+ nvkm_wr32(device, 0x002274, (runl->id << 20) | count);
+ spin_unlock_irq(&fifo->lock);
}
void
-gk104_fifo_init(struct nvkm_fifo *base)
+gk104_runl_insert_chan(struct nvkm_chan *chan, struct nvkm_memory *memory, u64 offset)
{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- int i;
-
- /* Enable PBDMAs. */
- fifo->func->pbdma->init(fifo);
-
- /* PBDMA[n] */
- for (i = 0; i < fifo->pbdma_nr; i++) {
- nvkm_mask(device, 0x04013c + (i * 0x2000), 0x10000100, 0x00000000);
- nvkm_wr32(device, 0x040108 + (i * 0x2000), 0xffffffff); /* INTR */
- nvkm_wr32(device, 0x04010c + (i * 0x2000), 0xfffffeff); /* INTREN */
- }
-
- /* PBDMA[n].HCE */
- for (i = 0; i < fifo->pbdma_nr; i++) {
- nvkm_wr32(device, 0x040148 + (i * 0x2000), 0xffffffff); /* INTR */
- nvkm_wr32(device, 0x04014c + (i * 0x2000), 0xffffffff); /* INTREN */
- }
-
- nvkm_wr32(device, 0x002254, 0x10000000 | fifo->user.bar->addr >> 12);
-
- if (fifo->func->pbdma->init_timeout)
- fifo->func->pbdma->init_timeout(fifo);
-
- nvkm_wr32(device, 0x002100, 0xffffffff);
- nvkm_wr32(device, 0x002140, 0x7fffffff);
+ nvkm_wo32(memory, offset + 0, chan->id);
+ nvkm_wo32(memory, offset + 4, 0x00000000);
}
-void *
-gk104_fifo_dtor(struct nvkm_fifo *base)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- int i;
-
- nvkm_vmm_put(nvkm_bar_bar1_vmm(device), &fifo->user.bar);
- nvkm_memory_unref(&fifo->user.mem);
-
- for (i = 0; i < fifo->runlist_nr; i++) {
- nvkm_memory_unref(&fifo->runlist[i].mem[1]);
- nvkm_memory_unref(&fifo->runlist[i].mem[0]);
- }
-
- return fifo;
-}
-
-static const struct nvkm_fifo_func
-gk104_fifo_ = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .info = gk104_fifo_info,
- .init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
- .intr = gk104_fifo_intr,
- .fault = gk104_fifo_fault,
- .engine_id = gk104_fifo_engine_id,
- .id_engine = gk104_fifo_id_engine,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
- .recover_chan = gk104_fifo_recover_chan,
- .class_get = gk104_fifo_class_get,
- .class_new = gk104_fifo_class_new,
+static const struct nvkm_runl_func
+gk104_runl = {
+ .size = 8,
+ .update = nv50_runl_update,
+ .insert_chan = gk104_runl_insert_chan,
+ .commit = gk104_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = gk104_runl_pending,
+ .block = gk104_runl_block,
+ .allow = gk104_runl_allow,
+ .fault_clear = gk104_runl_fault_clear,
+ .preempt_pending = gf100_runl_preempt_pending,
};
-int
-gk104_fifo_new_(const struct gk104_fifo_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, int nr, struct nvkm_fifo **pfifo)
-{
- struct gk104_fifo *fifo;
-
- if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
- return -ENOMEM;
- fifo->func = func;
- INIT_WORK(&fifo->recover.work, gk104_fifo_recover_work);
- *pfifo = &fifo->base;
-
- return nvkm_fifo_ctor(&gk104_fifo_, device, type, inst, nr, &fifo->base);
-}
-
-const struct nvkm_enum
-gk104_fifo_fault_access[] = {
- { 0x0, "READ" },
- { 0x1, "WRITE" },
- {}
-};
-
-const struct nvkm_enum
-gk104_fifo_fault_engine[] = {
+static const struct nvkm_enum
+gk104_fifo_mmu_fault_engine[] = {
{ 0x00, "GR", NULL, NVKM_ENGINE_GR },
{ 0x01, "DISPLAY" },
{ 0x02, "CAPTURE" },
@@ -1122,14 +478,14 @@ gk104_fifo_fault_engine[] = {
{ 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
{ 0x05, "BAR2", NULL, NVKM_SUBDEV_INSTMEM },
{ 0x06, "SCHED" },
- { 0x07, "HOST0", NULL, NVKM_ENGINE_FIFO },
- { 0x08, "HOST1", NULL, NVKM_ENGINE_FIFO },
- { 0x09, "HOST2", NULL, NVKM_ENGINE_FIFO },
- { 0x0a, "HOST3", NULL, NVKM_ENGINE_FIFO },
- { 0x0b, "HOST4", NULL, NVKM_ENGINE_FIFO },
- { 0x0c, "HOST5", NULL, NVKM_ENGINE_FIFO },
- { 0x0d, "HOST6", NULL, NVKM_ENGINE_FIFO },
- { 0x0e, "HOST7", NULL, NVKM_ENGINE_FIFO },
+ { 0x07, "HOST0" },
+ { 0x08, "HOST1" },
+ { 0x09, "HOST2" },
+ { 0x0a, "HOST3" },
+ { 0x0b, "HOST4" },
+ { 0x0c, "HOST5" },
+ { 0x0d, "HOST6" },
+ { 0x0e, "HOST7" },
{ 0x0f, "HOSTSR" },
{ 0x10, "MSVLD", NULL, NVKM_ENGINE_MSVLD },
{ 0x11, "MSPPP", NULL, NVKM_ENGINE_MSPPP },
@@ -1145,7 +501,7 @@ gk104_fifo_fault_engine[] = {
};
const struct nvkm_enum
-gk104_fifo_fault_reason[] = {
+gk104_fifo_mmu_fault_reason[] = {
{ 0x00, "PDE" },
{ 0x01, "PDE_SIZE" },
{ 0x02, "PTE" },
@@ -1166,7 +522,7 @@ gk104_fifo_fault_reason[] = {
};
const struct nvkm_enum
-gk104_fifo_fault_hubclient[] = {
+gk104_fifo_mmu_fault_hubclient[] = {
{ 0x00, "VIP" },
{ 0x01, "CE0" },
{ 0x02, "CE1" },
@@ -1203,7 +559,7 @@ gk104_fifo_fault_hubclient[] = {
};
const struct nvkm_enum
-gk104_fifo_fault_gpcclient[] = {
+gk104_fifo_mmu_fault_gpcclient[] = {
{ 0x00, "L1_0" }, { 0x01, "T1_0" }, { 0x02, "PE_0" },
{ 0x03, "L1_1" }, { 0x04, "T1_1" }, { 0x05, "PE_1" },
{ 0x06, "L1_2" }, { 0x07, "T1_2" }, { 0x08, "PE_2" },
@@ -1228,22 +584,250 @@ gk104_fifo_fault_gpcclient[] = {
{}
};
-static const struct gk104_fifo_func
+const struct nvkm_fifo_func_mmu_fault
+gk104_fifo_mmu_fault = {
+ .recover = gf100_fifo_mmu_fault_recover,
+ .access = gf100_fifo_mmu_fault_access,
+ .engine = gk104_fifo_mmu_fault_engine,
+ .reason = gk104_fifo_mmu_fault_reason,
+ .hubclient = gk104_fifo_mmu_fault_hubclient,
+ .gpcclient = gk104_fifo_mmu_fault_gpcclient,
+};
+
+static const struct nvkm_enum
+gk104_fifo_intr_bind_reason[] = {
+ { 0x01, "BIND_NOT_UNBOUND" },
+ { 0x02, "SNOOP_WITHOUT_BAR1" },
+ { 0x03, "UNBIND_WHILE_RUNNING" },
+ { 0x05, "INVALID_RUNLIST" },
+ { 0x06, "INVALID_CTX_TGT" },
+ { 0x0b, "UNBIND_WHILE_PARKED" },
+ {}
+};
+
+void
+gk104_fifo_intr_bind(struct nvkm_fifo *fifo)
+{
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
+ u32 intr = nvkm_rd32(subdev->device, 0x00252c);
+ u32 code = intr & 0x000000ff;
+ const struct nvkm_enum *en = nvkm_enum_find(gk104_fifo_intr_bind_reason, code);
+
+ nvkm_error(subdev, "BIND_ERROR %02x [%s]\n", code, en ? en->name : "");
+}
+
+void
+gk104_fifo_intr_chsw(struct nvkm_fifo *fifo)
+{
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
+ struct nvkm_device *device = subdev->device;
+ u32 stat = nvkm_rd32(device, 0x00256c);
+
+ nvkm_error(subdev, "CHSW_ERROR %08x\n", stat);
+ nvkm_wr32(device, 0x00256c, stat);
+}
+
+static void
+gk104_fifo_intr_dropped_fault(struct nvkm_fifo *fifo)
+{
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
+ u32 stat = nvkm_rd32(subdev->device, 0x00259c);
+
+ nvkm_error(subdev, "DROPPED_MMU_FAULT %08x\n", stat);
+}
+
+void
+gk104_fifo_intr_runlist(struct nvkm_fifo *fifo)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_runl *runl;
+ u32 mask = nvkm_rd32(device, 0x002a00);
+
+ nvkm_runl_foreach_cond(runl, fifo, mask & BIT(runl->id)) {
+ nvkm_wr32(device, 0x002a00, BIT(runl->id));
+ }
+}
+
+irqreturn_t
+gk104_fifo_intr(struct nvkm_inth *inth)
+{
+ struct nvkm_fifo *fifo = container_of(inth, typeof(*fifo), engine.subdev.inth);
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
+ struct nvkm_device *device = subdev->device;
+ u32 mask = nvkm_rd32(device, 0x002140);
+ u32 stat = nvkm_rd32(device, 0x002100) & mask;
+
+ if (stat & 0x00000001) {
+ gk104_fifo_intr_bind(fifo);
+ nvkm_wr32(device, 0x002100, 0x00000001);
+ stat &= ~0x00000001;
+ }
+
+ if (stat & 0x00000010) {
+ nvkm_error(subdev, "PIO_ERROR\n");
+ nvkm_wr32(device, 0x002100, 0x00000010);
+ stat &= ~0x00000010;
+ }
+
+ if (stat & 0x00000100) {
+ gf100_fifo_intr_sched(fifo);
+ nvkm_wr32(device, 0x002100, 0x00000100);
+ stat &= ~0x00000100;
+ }
+
+ if (stat & 0x00010000) {
+ gk104_fifo_intr_chsw(fifo);
+ nvkm_wr32(device, 0x002100, 0x00010000);
+ stat &= ~0x00010000;
+ }
+
+ if (stat & 0x00800000) {
+ nvkm_error(subdev, "FB_FLUSH_TIMEOUT\n");
+ nvkm_wr32(device, 0x002100, 0x00800000);
+ stat &= ~0x00800000;
+ }
+
+ if (stat & 0x01000000) {
+ nvkm_error(subdev, "LB_ERROR\n");
+ nvkm_wr32(device, 0x002100, 0x01000000);
+ stat &= ~0x01000000;
+ }
+
+ if (stat & 0x08000000) {
+ gk104_fifo_intr_dropped_fault(fifo);
+ nvkm_wr32(device, 0x002100, 0x08000000);
+ stat &= ~0x08000000;
+ }
+
+ if (stat & 0x10000000) {
+ gf100_fifo_intr_mmu_fault(fifo);
+ stat &= ~0x10000000;
+ }
+
+ if (stat & 0x20000000) {
+ if (gf100_fifo_intr_pbdma(fifo))
+ stat &= ~0x20000000;
+ }
+
+ if (stat & 0x40000000) {
+ gk104_fifo_intr_runlist(fifo);
+ stat &= ~0x40000000;
+ }
+
+ if (stat & 0x80000000) {
+ nvkm_wr32(device, 0x002100, 0x80000000);
+ nvkm_event_ntfy(&fifo->nonstall.event, 0, NVKM_FIFO_NONSTALL_EVENT);
+ stat &= ~0x80000000;
+ }
+
+ if (stat) {
+ nvkm_error(subdev, "INTR %08x\n", stat);
+ spin_lock(&fifo->lock);
+ nvkm_mask(device, 0x002140, stat, 0x00000000);
+ spin_unlock(&fifo->lock);
+ nvkm_wr32(device, 0x002100, stat);
+ }
+
+ return IRQ_HANDLED;
+}
+
+void
+gk104_fifo_init_pbdmas(struct nvkm_fifo *fifo, u32 mask)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x000204, mask);
+ nvkm_mask(device, 0x002a04, 0xbfffffff, 0xbfffffff);
+}
+
+void
+gk104_fifo_init(struct nvkm_fifo *fifo)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+
+ if (fifo->func->chan.func->userd->bar == 1)
+ nvkm_wr32(device, 0x002254, 0x10000000 | fifo->userd.bar1->addr >> 12);
+
+ nvkm_wr32(device, 0x002100, 0xffffffff);
+ nvkm_wr32(device, 0x002140, 0x7fffffff);
+}
+
+int
+gk104_fifo_runl_ctor(struct nvkm_fifo *fifo)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_top_device *tdev;
+ struct nvkm_runl *runl;
+ struct nvkm_runq *runq;
+ const struct nvkm_engn_func *func;
+
+ nvkm_list_foreach(tdev, &device->top->device, head, tdev->runlist >= 0) {
+ runl = nvkm_runl_get(fifo, tdev->runlist, tdev->runlist);
+ if (!runl) {
+ runl = nvkm_runl_new(fifo, tdev->runlist, tdev->runlist, 0);
+ if (IS_ERR(runl))
+ return PTR_ERR(runl);
+
+ nvkm_runq_foreach_cond(runq, fifo, gk104_runq_runm(runq) & BIT(runl->id)) {
+ if (WARN_ON(runl->runq_nr == ARRAY_SIZE(runl->runq)))
+ return -ENOMEM;
+
+ runl->runq[runl->runq_nr++] = runq;
+ }
+
+ }
+
+ if (tdev->engine < 0)
+ continue;
+
+ switch (tdev->type) {
+ case NVKM_ENGINE_CE:
+ func = fifo->func->engn_ce;
+ break;
+ case NVKM_ENGINE_GR:
+ nvkm_runl_add(runl, 15, &gf100_engn_sw, NVKM_ENGINE_SW, 0);
+ fallthrough;
+ default:
+ func = fifo->func->engn;
+ break;
+ }
+
+ nvkm_runl_add(runl, tdev->engine, func, tdev->type, tdev->inst);
+ }
+
+ return 0;
+}
+
+int
+gk104_fifo_chid_nr(struct nvkm_fifo *fifo)
+{
+ return 4096;
+}
+
+static const struct nvkm_fifo_func
gk104_fifo = {
- .intr.fault = gf100_fifo_intr_fault,
- .pbdma = &gk104_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gk104_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gk104_fifo_runlist,
- .chan = {{0,0,KEPLER_CHANNEL_GPFIFO_A}, gk104_fifo_gpfifo_new },
+ .chid_nr = gk104_fifo_chid_nr,
+ .chid_ctor = gf100_fifo_chid_ctor,
+ .runq_nr = gf100_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_mmu_fault_unit = gf100_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gk104_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gk104_runl,
+ .runq = &gk104_runq,
+ .engn = &gk104_engn,
+ .engn_ce = &gk104_engn_ce,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, KEPLER_CHANNEL_GPFIFO_A }, &gk104_chan },
};
int
gk104_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gk104_fifo, device, type, inst, 4096, pfifo);
+ return nvkm_fifo_new_(&gk104_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.h
deleted file mode 100644
index f2d12ae..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.h
+++ /dev/null
@@ -1,168 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __GK104_FIFO_H__
-#define __GK104_FIFO_H__
-#define gk104_fifo(p) container_of((p), struct gk104_fifo, base)
-#include "priv.h"
-struct nvkm_fifo_cgrp;
-
-#include <core/enum.h>
-#include <subdev/mmu.h>
-
-struct gk104_fifo_chan;
-struct gk104_fifo {
- const struct gk104_fifo_func *func;
- struct nvkm_fifo base;
-
- struct {
- struct work_struct work;
- u32 engm;
- u32 runm;
- } recover;
-
- int pbdma_nr;
-
- struct {
- struct nvkm_engine *engine;
- int runl;
- int pbid;
- } engine[16];
- int engine_nr;
-
- struct {
- struct nvkm_memory *mem[2];
- int next;
- wait_queue_head_t wait;
- struct list_head cgrp;
- struct list_head chan;
- u32 engm;
- u32 engm_sw;
- } runlist[16];
- int runlist_nr;
-
- struct {
- struct nvkm_memory *mem;
- struct nvkm_vma *bar;
- } user;
-};
-
-struct gk104_fifo_func {
- struct {
- void (*fault)(struct nvkm_fifo *, int unit);
- } intr;
-
- const struct gk104_fifo_pbdma_func {
- int (*nr)(struct gk104_fifo *);
- void (*init)(struct gk104_fifo *);
- void (*init_timeout)(struct gk104_fifo *);
- } *pbdma;
-
- struct {
- const struct nvkm_enum *access;
- const struct nvkm_enum *engine;
- const struct nvkm_enum *reason;
- const struct nvkm_enum *hubclient;
- const struct nvkm_enum *gpcclient;
- } fault;
-
- const struct gk104_fifo_runlist_func {
- u8 size;
- void (*cgrp)(struct nvkm_fifo_cgrp *,
- struct nvkm_memory *, u32 offset);
- void (*chan)(struct gk104_fifo_chan *,
- struct nvkm_memory *, u32 offset);
- void (*commit)(struct gk104_fifo *, int runl,
- struct nvkm_memory *, int entries);
- } *runlist;
-
- struct gk104_fifo_user_user {
- struct nvkm_sclass user;
- int (*ctor)(const struct nvkm_oclass *, void *, u32,
- struct nvkm_object **);
- } user;
-
- struct gk104_fifo_chan_user {
- struct nvkm_sclass user;
- int (*ctor)(struct gk104_fifo *, const struct nvkm_oclass *,
- void *, u32, struct nvkm_object **);
- } chan;
- bool cgrp_force;
-};
-
-struct gk104_fifo_engine_status {
- bool busy;
- bool faulted;
- bool chsw;
- bool save;
- bool load;
- struct {
- bool tsg;
- u32 id;
- } prev, next, *chan;
-};
-
-int gk104_fifo_new_(const struct gk104_fifo_func *, struct nvkm_device *, enum nvkm_subdev_type,
- int index, int nr, struct nvkm_fifo **);
-void gk104_fifo_runlist_insert(struct gk104_fifo *, struct gk104_fifo_chan *);
-void gk104_fifo_runlist_remove(struct gk104_fifo *, struct gk104_fifo_chan *);
-void gk104_fifo_runlist_update(struct gk104_fifo *, int runl);
-void gk104_fifo_engine_status(struct gk104_fifo *fifo, int engn,
- struct gk104_fifo_engine_status *status);
-void gk104_fifo_intr_bind(struct gk104_fifo *fifo);
-void gk104_fifo_intr_chsw(struct gk104_fifo *fifo);
-void gk104_fifo_intr_dropped_fault(struct gk104_fifo *fifo);
-void gk104_fifo_intr_pbdma_0(struct gk104_fifo *fifo, int unit);
-void gk104_fifo_intr_pbdma_1(struct gk104_fifo *fifo, int unit);
-void gk104_fifo_intr_runlist(struct gk104_fifo *fifo);
-void gk104_fifo_intr_engine(struct gk104_fifo *fifo);
-void *gk104_fifo_dtor(struct nvkm_fifo *base);
-int gk104_fifo_oneinit(struct nvkm_fifo *base);
-int gk104_fifo_info(struct nvkm_fifo *base, u64 mthd, u64 *data);
-void gk104_fifo_init(struct nvkm_fifo *base);
-void gk104_fifo_fini(struct nvkm_fifo *base);
-int gk104_fifo_class_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *argv, u32 argc, struct nvkm_object **pobject);
-int gk104_fifo_class_get(struct nvkm_fifo *base, int index,
- struct nvkm_oclass *oclass);
-void gk104_fifo_uevent_fini(struct nvkm_fifo *fifo);
-void gk104_fifo_uevent_init(struct nvkm_fifo *fifo);
-
-extern const struct gk104_fifo_pbdma_func gk104_fifo_pbdma;
-int gk104_fifo_pbdma_nr(struct gk104_fifo *);
-void gk104_fifo_pbdma_init(struct gk104_fifo *);
-extern const struct nvkm_enum gk104_fifo_fault_access[];
-extern const struct nvkm_enum gk104_fifo_fault_engine[];
-extern const struct nvkm_enum gk104_fifo_fault_reason[];
-extern const struct nvkm_enum gk104_fifo_fault_hubclient[];
-extern const struct nvkm_enum gk104_fifo_fault_gpcclient[];
-extern const struct gk104_fifo_runlist_func gk104_fifo_runlist;
-void gk104_fifo_runlist_chan(struct gk104_fifo_chan *,
- struct nvkm_memory *, u32);
-void gk104_fifo_runlist_commit(struct gk104_fifo *, int runl,
- struct nvkm_memory *, int);
-
-extern const struct gk104_fifo_runlist_func gk110_fifo_runlist;
-void gk110_fifo_runlist_cgrp(struct nvkm_fifo_cgrp *,
- struct nvkm_memory *, u32);
-
-extern const struct gk104_fifo_pbdma_func gk208_fifo_pbdma;
-void gk208_fifo_pbdma_init_timeout(struct gk104_fifo *);
-
-void gm107_fifo_intr_fault(struct nvkm_fifo *, int);
-extern const struct nvkm_enum gm107_fifo_fault_engine[];
-extern const struct gk104_fifo_runlist_func gm107_fifo_runlist;
-
-extern const struct gk104_fifo_pbdma_func gm200_fifo_pbdma;
-int gm200_fifo_pbdma_nr(struct gk104_fifo *);
-
-void gp100_fifo_intr_fault(struct nvkm_fifo *, int);
-extern const struct nvkm_enum gp100_fifo_fault_engine[];
-
-extern const struct nvkm_enum gv100_fifo_fault_access[];
-extern const struct nvkm_enum gv100_fifo_fault_reason[];
-extern const struct nvkm_enum gv100_fifo_fault_hubclient[];
-extern const struct nvkm_enum gv100_fifo_fault_gpcclient[];
-void gv100_fifo_runlist_cgrp(struct nvkm_fifo_cgrp *,
- struct nvkm_memory *, u32);
-void gv100_fifo_runlist_chan(struct gk104_fifo_chan *,
- struct nvkm_memory *, u32);
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c
index 915278c..a8ff21c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c
@@ -21,47 +21,112 @@
*
* Authors: Ben Skeggs
*/
-#include "gk104.h"
+#include "priv.h"
#include "cgrp.h"
-#include "changk104.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
#include <core/memory.h>
+#include <subdev/timer.h>
#include <nvif/class.h>
void
-gk110_fifo_runlist_cgrp(struct nvkm_fifo_cgrp *cgrp,
- struct nvkm_memory *memory, u32 offset)
+gk110_chan_preempt(struct nvkm_chan *chan)
+{
+ struct nvkm_cgrp *cgrp = chan->cgrp;
+
+ if (cgrp->hw) {
+ cgrp->func->preempt(cgrp);
+ return;
+ }
+
+ gf100_chan_preempt(chan);
+}
+
+const struct nvkm_chan_func
+gk110_chan = {
+ .inst = &gf100_chan_inst,
+ .userd = &gk104_chan_userd,
+ .ramfc = &gk104_chan_ramfc,
+ .bind = gk104_chan_bind,
+ .unbind = gk104_chan_unbind,
+ .start = gk104_chan_start,
+ .stop = gk104_chan_stop,
+ .preempt = gk110_chan_preempt,
+};
+
+static void
+gk110_cgrp_preempt(struct nvkm_cgrp *cgrp)
+{
+ nvkm_wr32(cgrp->runl->fifo->engine.subdev.device, 0x002634, 0x01000000 | cgrp->id);
+}
+
+const struct nvkm_cgrp_func
+gk110_cgrp = {
+ .preempt = gk110_cgrp_preempt,
+};
+
+void
+gk110_runl_insert_cgrp(struct nvkm_cgrp *cgrp, struct nvkm_memory *memory, u64 offset)
{
nvkm_wo32(memory, offset + 0, (cgrp->chan_nr << 26) | (128 << 18) |
(3 << 14) | 0x00002000 | cgrp->id);
nvkm_wo32(memory, offset + 4, 0x00000000);
}
-const struct gk104_fifo_runlist_func
-gk110_fifo_runlist = {
+const struct nvkm_runl_func
+gk110_runl = {
.size = 8,
- .cgrp = gk110_fifo_runlist_cgrp,
- .chan = gk104_fifo_runlist_chan,
- .commit = gk104_fifo_runlist_commit,
+ .update = nv50_runl_update,
+ .insert_cgrp = gk110_runl_insert_cgrp,
+ .insert_chan = gk104_runl_insert_chan,
+ .commit = gk104_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = gk104_runl_pending,
+ .block = gk104_runl_block,
+ .allow = gk104_runl_allow,
+ .fault_clear = gk104_runl_fault_clear,
+ .preempt_pending = gf100_runl_preempt_pending,
};
-static const struct gk104_fifo_func
+int
+gk110_fifo_chid_ctor(struct nvkm_fifo *fifo, int nr)
+{
+ int ret;
+
+ ret = nvkm_chid_new(&nvkm_chan_event, &fifo->engine.subdev, nr, 0, nr, &fifo->cgid);
+ if (ret)
+ return ret;
+
+ return gf100_fifo_chid_ctor(fifo, nr);
+}
+
+static const struct nvkm_fifo_func
gk110_fifo = {
- .intr.fault = gf100_fifo_intr_fault,
- .pbdma = &gk104_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gk104_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gk110_fifo_runlist,
- .chan = {{0,0,KEPLER_CHANNEL_GPFIFO_B}, gk104_fifo_gpfifo_new },
+ .chid_nr = gk104_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gf100_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_mmu_fault_unit = gf100_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gk104_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gk110_runl,
+ .runq = &gk104_runq,
+ .engn = &gk104_engn,
+ .engn_ce = &gk104_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &gk110_cgrp },
+ .chan = {{ 0, 0, KEPLER_CHANNEL_GPFIFO_B }, &gk110_chan },
};
int
gk110_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gk110_fifo, device, type, inst, 4096, pfifo);
+ return nvkm_fifo_new_(&gk110_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c
index cb70369..8fa2b0b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c
@@ -21,44 +21,57 @@
*
* Authors: Ben Skeggs
*/
-#include "gk104.h"
-#include "changk104.h"
+#include "priv.h"
+#include "runq.h"
#include <nvif/class.h>
void
-gk208_fifo_pbdma_init_timeout(struct gk104_fifo *fifo)
+gk208_runq_init(struct nvkm_runq *runq)
{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- int i;
+ gk104_runq_init(runq);
- for (i = 0; i < fifo->pbdma_nr; i++)
- nvkm_wr32(device, 0x04012c + (i * 0x2000), 0x0000ffff);
+ nvkm_wr32(runq->fifo->engine.subdev.device, 0x04012c + (runq->id * 0x2000), 0x000f4240);
}
-const struct gk104_fifo_pbdma_func
-gk208_fifo_pbdma = {
- .nr = gk104_fifo_pbdma_nr,
- .init = gk104_fifo_pbdma_init,
- .init_timeout = gk208_fifo_pbdma_init_timeout,
+const struct nvkm_runq_func
+gk208_runq = {
+ .init = gk208_runq_init,
+ .intr = gk104_runq_intr,
+ .intr_0_names = gk104_runq_intr_0_names,
+ .idle = gk104_runq_idle,
};
-static const struct gk104_fifo_func
+static int
+gk208_fifo_chid_nr(struct nvkm_fifo *fifo)
+{
+ return 1024;
+}
+
+static const struct nvkm_fifo_func
gk208_fifo = {
- .intr.fault = gf100_fifo_intr_fault,
- .pbdma = &gk208_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gk104_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gk110_fifo_runlist,
- .chan = {{0,0,KEPLER_CHANNEL_GPFIFO_A}, gk104_fifo_gpfifo_new },
+ .chid_nr = gk208_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gf100_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_mmu_fault_unit = gf100_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gk104_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gk110_runl,
+ .runq = &gk208_runq,
+ .engn = &gk104_engn,
+ .engn_ce = &gk104_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &gk110_cgrp },
+ .chan = {{ 0, 0, KEPLER_CHANNEL_GPFIFO_A }, &gk110_chan },
};
int
gk208_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gk208_fifo, device, type, inst, 1024, pfifo);
+ return nvkm_fifo_new_(&gk208_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c
index 6e35cf4..b63ca83 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c
@@ -19,27 +19,34 @@
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*/
-#include "gk104.h"
-#include "changk104.h"
+#include "priv.h"
#include <nvif/class.h>
-static const struct gk104_fifo_func
+static const struct nvkm_fifo_func
gk20a_fifo = {
- .intr.fault = gf100_fifo_intr_fault,
- .pbdma = &gk208_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gk104_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gk110_fifo_runlist,
- .chan = {{0,0,KEPLER_CHANNEL_GPFIFO_A}, gk104_fifo_gpfifo_new },
+ .chid_nr = nv50_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gf100_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_mmu_fault_unit = gf100_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gk104_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gk110_runl,
+ .runq = &gk208_runq,
+ .engn = &gk104_engn,
+ .engn_ce = &gk104_engn_ce,
+ .cgrp = {{ }, &gk110_cgrp },
+ .chan = {{ 0, 0, KEPLER_CHANNEL_GPFIFO_A }, &gk110_chan },
};
int
gk20a_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gk20a_fifo, device, type, inst, 128, pfifo);
+ return nvkm_fifo_new_(&gk20a_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c
index 7af6e68..5ba6002 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c
@@ -21,46 +21,65 @@
*
* Authors: Ben Skeggs
*/
-#include "gk104.h"
-#include "changk104.h"
+#include "priv.h"
+#include "chan.h"
+#include "runl.h"
#include <core/gpuobj.h>
#include <subdev/fault.h>
#include <nvif/class.h>
-static void
-gm107_fifo_runlist_chan(struct gk104_fifo_chan *chan,
- struct nvkm_memory *memory, u32 offset)
-{
- nvkm_wo32(memory, offset + 0, chan->base.chid);
- nvkm_wo32(memory, offset + 4, chan->base.inst->addr >> 12);
-}
-
-const struct gk104_fifo_runlist_func
-gm107_fifo_runlist = {
- .size = 8,
- .cgrp = gk110_fifo_runlist_cgrp,
- .chan = gm107_fifo_runlist_chan,
- .commit = gk104_fifo_runlist_commit,
+const struct nvkm_chan_func
+gm107_chan = {
+ .inst = &gf100_chan_inst,
+ .userd = &gk104_chan_userd,
+ .ramfc = &gk104_chan_ramfc,
+ .bind = gk104_chan_bind_inst,
+ .unbind = gk104_chan_unbind,
+ .start = gk104_chan_start,
+ .stop = gk104_chan_stop,
+ .preempt = gk110_chan_preempt,
};
-const struct nvkm_enum
-gm107_fifo_fault_engine[] = {
+static void
+gm107_runl_insert_chan(struct nvkm_chan *chan, struct nvkm_memory *memory, u64 offset)
+{
+ nvkm_wo32(memory, offset + 0, chan->id);
+ nvkm_wo32(memory, offset + 4, chan->inst->addr >> 12);
+}
+
+const struct nvkm_runl_func
+gm107_runl = {
+ .size = 8,
+ .update = nv50_runl_update,
+ .insert_cgrp = gk110_runl_insert_cgrp,
+ .insert_chan = gm107_runl_insert_chan,
+ .commit = gk104_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = gk104_runl_pending,
+ .block = gk104_runl_block,
+ .allow = gk104_runl_allow,
+ .fault_clear = gk104_runl_fault_clear,
+ .preempt_pending = gf100_runl_preempt_pending,
+};
+
+static const struct nvkm_enum
+gm107_fifo_mmu_fault_engine[] = {
{ 0x01, "DISPLAY" },
{ 0x02, "CAPTURE" },
{ 0x03, "IFB", NULL, NVKM_ENGINE_IFB },
{ 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
{ 0x05, "BAR2", NULL, NVKM_SUBDEV_INSTMEM },
{ 0x06, "SCHED" },
- { 0x07, "HOST0", NULL, NVKM_ENGINE_FIFO },
- { 0x08, "HOST1", NULL, NVKM_ENGINE_FIFO },
- { 0x09, "HOST2", NULL, NVKM_ENGINE_FIFO },
- { 0x0a, "HOST3", NULL, NVKM_ENGINE_FIFO },
- { 0x0b, "HOST4", NULL, NVKM_ENGINE_FIFO },
- { 0x0c, "HOST5", NULL, NVKM_ENGINE_FIFO },
- { 0x0d, "HOST6", NULL, NVKM_ENGINE_FIFO },
- { 0x0e, "HOST7", NULL, NVKM_ENGINE_FIFO },
+ { 0x07, "HOST0" },
+ { 0x08, "HOST1" },
+ { 0x09, "HOST2" },
+ { 0x0a, "HOST3" },
+ { 0x0b, "HOST4" },
+ { 0x0c, "HOST5" },
+ { 0x0d, "HOST6" },
+ { 0x0e, "HOST7" },
{ 0x0f, "HOSTSR" },
{ 0x13, "PERF" },
{ 0x17, "PMU" },
@@ -68,8 +87,18 @@ gm107_fifo_fault_engine[] = {
{}
};
+const struct nvkm_fifo_func_mmu_fault
+gm107_fifo_mmu_fault = {
+ .recover = gf100_fifo_mmu_fault_recover,
+ .access = gf100_fifo_mmu_fault_access,
+ .engine = gm107_fifo_mmu_fault_engine,
+ .reason = gk104_fifo_mmu_fault_reason,
+ .hubclient = gk104_fifo_mmu_fault_hubclient,
+ .gpcclient = gk104_fifo_mmu_fault_gpcclient,
+};
+
void
-gm107_fifo_intr_fault(struct nvkm_fifo *fifo, int unit)
+gm107_fifo_intr_mmu_fault_unit(struct nvkm_fifo *fifo, int unit)
{
struct nvkm_device *device = fifo->engine.subdev.device;
u32 inst = nvkm_rd32(device, 0x002800 + (unit * 0x10));
@@ -92,22 +121,36 @@ gm107_fifo_intr_fault(struct nvkm_fifo *fifo, int unit)
nvkm_fifo_fault(fifo, &info);
}
-static const struct gk104_fifo_func
+static int
+gm107_fifo_chid_nr(struct nvkm_fifo *fifo)
+{
+ return 2048;
+}
+
+static const struct nvkm_fifo_func
gm107_fifo = {
- .intr.fault = gm107_fifo_intr_fault,
- .pbdma = &gk208_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gm107_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gm107_fifo_runlist,
- .chan = {{0,0,KEPLER_CHANNEL_GPFIFO_B}, gk104_fifo_gpfifo_new },
+ .chid_nr = gm107_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gf100_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_mmu_fault_unit = gm107_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gm107_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gm107_runl,
+ .runq = &gk208_runq,
+ .engn = &gk104_engn,
+ .engn_ce = &gk104_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &gk110_cgrp },
+ .chan = {{ 0, 0, KEPLER_CHANNEL_GPFIFO_B }, &gm107_chan },
};
int
gm107_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gm107_fifo, device, type, inst, 2048, pfifo);
+ return nvkm_fifo_new_(&gm107_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c
index 573658c..d92d1ac 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c
@@ -21,41 +21,46 @@
*
* Authors: Ben Skeggs
*/
-#include "gk104.h"
-#include "changk104.h"
+#include "priv.h"
#include <nvif/class.h>
int
-gm200_fifo_pbdma_nr(struct gk104_fifo *fifo)
+gm200_fifo_runq_nr(struct nvkm_fifo *fifo)
{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- return nvkm_rd32(device, 0x002004) & 0x000000ff;
+ return nvkm_rd32(fifo->engine.subdev.device, 0x002004) & 0x000000ff;
}
-const struct gk104_fifo_pbdma_func
-gm200_fifo_pbdma = {
- .nr = gm200_fifo_pbdma_nr,
- .init = gk104_fifo_pbdma_init,
- .init_timeout = gk208_fifo_pbdma_init_timeout,
-};
+int
+gm200_fifo_chid_nr(struct nvkm_fifo *fifo)
+{
+ return nvkm_rd32(fifo->engine.subdev.device, 0x002008);
+}
-static const struct gk104_fifo_func
+static const struct nvkm_fifo_func
gm200_fifo = {
- .intr.fault = gm107_fifo_intr_fault,
- .pbdma = &gm200_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gm107_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gm107_fifo_runlist,
- .chan = {{0,0,MAXWELL_CHANNEL_GPFIFO_A}, gk104_fifo_gpfifo_new },
+ .chid_nr = gm200_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gm200_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_mmu_fault_unit = gm107_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gm107_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gm107_runl,
+ .runq = &gk208_runq,
+ .engn = &gk104_engn,
+ .engn_ce = &gk104_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &gk110_cgrp },
+ .chan = {{ 0, 0, MAXWELL_CHANNEL_GPFIFO_A }, &gm107_chan },
};
int
gm200_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gm200_fifo, device, type, inst, 4096, pfifo);
+ return nvkm_fifo_new_(&gm200_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm20b.c
deleted file mode 100644
index 556c97e..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm20b.c
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Copyright (c) 2015, NVIDIA CORPORATION. All rights reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- * DEALINGS IN THE SOFTWARE.
- */
-#include "gk104.h"
-#include "changk104.h"
-
-#include <nvif/class.h>
-
-static const struct gk104_fifo_func
-gm20b_fifo = {
- .intr.fault = gm107_fifo_intr_fault,
- .pbdma = &gm200_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gm107_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gm107_fifo_runlist,
- .chan = {{0,0,MAXWELL_CHANNEL_GPFIFO_A}, gk104_fifo_gpfifo_new },
-};
-
-int
-gm20b_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
- struct nvkm_fifo **pfifo)
-{
- return gk104_fifo_new_(&gm20b_fifo, device, type, inst, 512, pfifo);
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp100.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp100.c
index 6b46b6b..65bdb6a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp100.c
@@ -21,30 +21,54 @@
*
* Authors: Ben Skeggs
*/
-#include "gk104.h"
-#include "changk104.h"
+#include "priv.h"
+#include "runl.h"
+#include <core/gpuobj.h>
#include <subdev/fault.h>
#include <nvif/class.h>
-const struct nvkm_enum
-gp100_fifo_fault_engine[] = {
+static void
+gp100_runl_insert_chan(struct nvkm_chan *chan, struct nvkm_memory *memory, u64 offset)
+{
+ nvkm_wo32(memory, offset + 0, chan->id | chan->runq << 14);
+ nvkm_wo32(memory, offset + 4, chan->inst->addr >> 12);
+}
+
+static const struct nvkm_runl_func
+gp100_runl = {
+ .runqs = 2,
+ .size = 8,
+ .update = nv50_runl_update,
+ .insert_cgrp = gk110_runl_insert_cgrp,
+ .insert_chan = gp100_runl_insert_chan,
+ .commit = gk104_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = gk104_runl_pending,
+ .block = gk104_runl_block,
+ .allow = gk104_runl_allow,
+ .fault_clear = gk104_runl_fault_clear,
+ .preempt_pending = gf100_runl_preempt_pending,
+};
+
+static const struct nvkm_enum
+gp100_fifo_mmu_fault_engine[] = {
{ 0x01, "DISPLAY" },
{ 0x03, "IFB", NULL, NVKM_ENGINE_IFB },
{ 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
{ 0x05, "BAR2", NULL, NVKM_SUBDEV_INSTMEM },
- { 0x06, "HOST0", NULL, NVKM_ENGINE_FIFO },
- { 0x07, "HOST1", NULL, NVKM_ENGINE_FIFO },
- { 0x08, "HOST2", NULL, NVKM_ENGINE_FIFO },
- { 0x09, "HOST3", NULL, NVKM_ENGINE_FIFO },
- { 0x0a, "HOST4", NULL, NVKM_ENGINE_FIFO },
- { 0x0b, "HOST5", NULL, NVKM_ENGINE_FIFO },
- { 0x0c, "HOST6", NULL, NVKM_ENGINE_FIFO },
- { 0x0d, "HOST7", NULL, NVKM_ENGINE_FIFO },
- { 0x0e, "HOST8", NULL, NVKM_ENGINE_FIFO },
- { 0x0f, "HOST9", NULL, NVKM_ENGINE_FIFO },
- { 0x10, "HOST10", NULL, NVKM_ENGINE_FIFO },
+ { 0x06, "HOST0" },
+ { 0x07, "HOST1" },
+ { 0x08, "HOST2" },
+ { 0x09, "HOST3" },
+ { 0x0a, "HOST4" },
+ { 0x0b, "HOST5" },
+ { 0x0c, "HOST6" },
+ { 0x0d, "HOST7" },
+ { 0x0e, "HOST8" },
+ { 0x0f, "HOST9" },
+ { 0x10, "HOST10" },
{ 0x13, "PERF" },
{ 0x17, "PMU" },
{ 0x18, "PTP" },
@@ -52,8 +76,18 @@ gp100_fifo_fault_engine[] = {
{}
};
-void
-gp100_fifo_intr_fault(struct nvkm_fifo *fifo, int unit)
+static const struct nvkm_fifo_func_mmu_fault
+gp100_fifo_mmu_fault = {
+ .recover = gf100_fifo_mmu_fault_recover,
+ .access = gf100_fifo_mmu_fault_access,
+ .engine = gp100_fifo_mmu_fault_engine,
+ .reason = gk104_fifo_mmu_fault_reason,
+ .hubclient = gk104_fifo_mmu_fault_hubclient,
+ .gpcclient = gk104_fifo_mmu_fault_gpcclient,
+};
+
+static void
+gp100_fifo_intr_mmu_fault_unit(struct nvkm_fifo *fifo, int unit)
{
struct nvkm_device *device = fifo->engine.subdev.device;
u32 inst = nvkm_rd32(device, 0x002800 + (unit * 0x10));
@@ -76,23 +110,30 @@ gp100_fifo_intr_fault(struct nvkm_fifo *fifo, int unit)
nvkm_fifo_fault(fifo, &info);
}
-static const struct gk104_fifo_func
+static const struct nvkm_fifo_func
gp100_fifo = {
- .intr.fault = gp100_fifo_intr_fault,
- .pbdma = &gm200_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gp100_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gm107_fifo_runlist,
- .chan = {{0,0,PASCAL_CHANNEL_GPFIFO_A}, gk104_fifo_gpfifo_new },
- .cgrp_force = true,
+ .chid_nr = gm200_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gm200_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_mmu_fault_unit = gp100_fifo_intr_mmu_fault_unit,
+ .intr_ctxsw_timeout = gf100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gp100_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gp100_runl,
+ .runq = &gk208_runq,
+ .engn = &gk104_engn,
+ .engn_ce = &gk104_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &gk110_cgrp, .force = true },
+ .chan = {{ 0, 0, PASCAL_CHANNEL_GPFIFO_A }, &gm107_chan },
};
int
gp100_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gp100_fifo, device, type, inst, 4096, pfifo);
+ return nvkm_fifo_new_(&gp100_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp10b.c
deleted file mode 100644
index 7a5929c..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp10b.c
+++ /dev/null
@@ -1,46 +0,0 @@
-/*
- * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- * DEALINGS IN THE SOFTWARE.
- */
-#include "gk104.h"
-#include "changk104.h"
-
-#include <nvif/class.h>
-
-static const struct gk104_fifo_func
-gp10b_fifo = {
- .intr.fault = gp100_fifo_intr_fault,
- .pbdma = &gm200_fifo_pbdma,
- .fault.access = gk104_fifo_fault_access,
- .fault.engine = gp100_fifo_fault_engine,
- .fault.reason = gk104_fifo_fault_reason,
- .fault.hubclient = gk104_fifo_fault_hubclient,
- .fault.gpcclient = gk104_fifo_fault_gpcclient,
- .runlist = &gm107_fifo_runlist,
- .chan = {{0,0,PASCAL_CHANNEL_GPFIFO_A}, gk104_fifo_gpfifo_new },
- .cgrp_force = true,
-};
-
-int
-gp10b_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
- struct nvkm_fifo **pfifo)
-{
- return gk104_fifo_new_(&gp10b_fifo, device, type, inst, 512, pfifo);
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifog84.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifog84.c
deleted file mode 100644
index 2121f51..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifog84.c
+++ /dev/null
@@ -1,95 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv50.h"
-
-#include <core/client.h>
-#include <core/ramht.h>
-
-#include <nvif/class.h>
-#include <nvif/cl826f.h>
-#include <nvif/unpack.h>
-
-static int
-g84_fifo_gpfifo_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct g82_channel_gpfifo_v0 v0;
- } *args = data;
- struct nv50_fifo *fifo = nv50_fifo(base);
- struct nv50_fifo_chan *chan;
- u64 ioffset, ilength;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel gpfifo size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel gpfifo vers %d vmm %llx "
- "pushbuf %llx ioffset %016llx "
- "ilength %08x\n",
- args->v0.version, args->v0.vmm, args->v0.pushbuf,
- args->v0.ioffset, args->v0.ilength);
- if (!args->v0.pushbuf)
- return -EINVAL;
- } else
- return ret;
-
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
-
- ret = g84_fifo_chan_ctor(fifo, args->v0.vmm, args->v0.pushbuf,
- oclass, chan);
- if (ret)
- return ret;
-
- args->v0.chid = chan->base.chid;
- ioffset = args->v0.ioffset;
- ilength = order_base_2(args->v0.ilength / 8);
-
- nvkm_kmap(chan->ramfc);
- nvkm_wo32(chan->ramfc, 0x3c, 0x403f6078);
- nvkm_wo32(chan->ramfc, 0x44, 0x01003fff);
- nvkm_wo32(chan->ramfc, 0x48, chan->base.push->node->offset >> 4);
- nvkm_wo32(chan->ramfc, 0x50, lower_32_bits(ioffset));
- nvkm_wo32(chan->ramfc, 0x54, upper_32_bits(ioffset) | (ilength << 16));
- nvkm_wo32(chan->ramfc, 0x60, 0x7fffffff);
- nvkm_wo32(chan->ramfc, 0x78, 0x00000000);
- nvkm_wo32(chan->ramfc, 0x7c, 0x30000001);
- nvkm_wo32(chan->ramfc, 0x80, ((chan->ramht->bits - 9) << 27) |
- (4 << 24) /* SEARCH_FULL */ |
- (chan->ramht->gpuobj->node->offset >> 4));
- nvkm_wo32(chan->ramfc, 0x88, chan->cache->addr >> 10);
- nvkm_wo32(chan->ramfc, 0x98, chan->base.inst->addr >> 12);
- nvkm_done(chan->ramfc);
- return 0;
-}
-
-const struct nvkm_fifo_chan_oclass
-g84_fifo_gpfifo_oclass = {
- .base.oclass = G82_CHANNEL_GPFIFO,
- .base.minver = 0,
- .base.maxver = 0,
- .ctor = g84_fifo_gpfifo_new,
-};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogf100.c
deleted file mode 100644
index 4e78bbe..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogf100.c
+++ /dev/null
@@ -1,308 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "changf100.h"
-
-#include <core/client.h>
-#include <core/gpuobj.h>
-#include <subdev/fb.h>
-#include <subdev/timer.h>
-
-#include <nvif/class.h>
-#include <nvif/cl906f.h>
-#include <nvif/unpack.h>
-
-int
-gf100_fifo_chan_ntfy(struct nvkm_fifo_chan *chan, u32 type,
- struct nvkm_event **pevent)
-{
- switch (type) {
- case NV906F_V0_NTFY_NON_STALL_INTERRUPT:
- *pevent = &chan->fifo->uevent;
- return 0;
- case NV906F_V0_NTFY_KILLED:
- *pevent = &chan->fifo->kevent;
- return 0;
- default:
- break;
- }
- return -EINVAL;
-}
-
-static u32
-gf100_fifo_gpfifo_engine_addr(struct nvkm_engine *engine)
-{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_SW : return 0;
- case NVKM_ENGINE_GR : return 0x0210;
- case NVKM_ENGINE_CE : return 0x0230 + (engine->subdev.inst * 0x10);
- case NVKM_ENGINE_MSPDEC: return 0x0250;
- case NVKM_ENGINE_MSPPP : return 0x0260;
- case NVKM_ENGINE_MSVLD : return 0x0270;
- default:
- WARN_ON(1);
- return 0;
- }
-}
-
-static struct gf100_fifo_engn *
-gf100_fifo_gpfifo_engine(struct gf100_fifo_chan *chan, struct nvkm_engine *engine)
-{
- int engi = chan->base.fifo->func->engine_id(chan->base.fifo, engine);
- if (engi >= 0)
- return &chan->engn[engi];
- return NULL;
-}
-
-static int
-gf100_fifo_gpfifo_engine_fini(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine, bool suspend)
-{
- const u32 offset = gf100_fifo_gpfifo_engine_addr(engine);
- struct gf100_fifo_chan *chan = gf100_fifo_chan(base);
- struct nvkm_subdev *subdev = &chan->fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- struct nvkm_gpuobj *inst = chan->base.inst;
- int ret = 0;
-
- mutex_lock(&chan->fifo->base.mutex);
- nvkm_wr32(device, 0x002634, chan->base.chid);
- if (nvkm_msec(device, 2000,
- if (nvkm_rd32(device, 0x002634) == chan->base.chid)
- break;
- ) < 0) {
- nvkm_error(subdev, "channel %d [%s] kick timeout\n",
- chan->base.chid, chan->base.object.client->name);
- ret = -ETIMEDOUT;
- }
- mutex_unlock(&chan->fifo->base.mutex);
-
- if (ret && suspend)
- return ret;
-
- if (offset) {
- nvkm_kmap(inst);
- nvkm_wo32(inst, offset + 0x00, 0x00000000);
- nvkm_wo32(inst, offset + 0x04, 0x00000000);
- nvkm_done(inst);
- }
-
- return ret;
-}
-
-static int
-gf100_fifo_gpfifo_engine_init(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- const u32 offset = gf100_fifo_gpfifo_engine_addr(engine);
- struct gf100_fifo_chan *chan = gf100_fifo_chan(base);
- struct gf100_fifo_engn *engn = gf100_fifo_gpfifo_engine(chan, engine);
- struct nvkm_gpuobj *inst = chan->base.inst;
-
- if (offset) {
- nvkm_kmap(inst);
- nvkm_wo32(inst, offset + 0x00, lower_32_bits(engn->vma->addr) | 4);
- nvkm_wo32(inst, offset + 0x04, upper_32_bits(engn->vma->addr));
- nvkm_done(inst);
- }
-
- return 0;
-}
-
-static void
-gf100_fifo_gpfifo_engine_dtor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct gf100_fifo_chan *chan = gf100_fifo_chan(base);
- struct gf100_fifo_engn *engn = gf100_fifo_gpfifo_engine(chan, engine);
- nvkm_vmm_put(chan->base.vmm, &engn->vma);
- nvkm_gpuobj_del(&engn->inst);
-}
-
-static int
-gf100_fifo_gpfifo_engine_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine,
- struct nvkm_object *object)
-{
- struct gf100_fifo_chan *chan = gf100_fifo_chan(base);
- struct gf100_fifo_engn *engn = gf100_fifo_gpfifo_engine(chan, engine);
- int ret;
-
- if (!gf100_fifo_gpfifo_engine_addr(engine))
- return 0;
-
- ret = nvkm_object_bind(object, NULL, 0, &engn->inst);
- if (ret)
- return ret;
-
- ret = nvkm_vmm_get(chan->base.vmm, 12, engn->inst->size, &engn->vma);
- if (ret)
- return ret;
-
- return nvkm_memory_map(engn->inst, 0, chan->base.vmm, engn->vma, NULL, 0);
-}
-
-static void
-gf100_fifo_gpfifo_fini(struct nvkm_fifo_chan *base)
-{
- struct gf100_fifo_chan *chan = gf100_fifo_chan(base);
- struct gf100_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u32 coff = chan->base.chid * 8;
-
- if (!list_empty(&chan->head) && !chan->killed) {
- gf100_fifo_runlist_remove(fifo, chan);
- nvkm_mask(device, 0x003004 + coff, 0x00000001, 0x00000000);
- gf100_fifo_runlist_commit(fifo);
- }
-
- gf100_fifo_intr_engine(fifo);
-
- nvkm_wr32(device, 0x003000 + coff, 0x00000000);
-}
-
-static void
-gf100_fifo_gpfifo_init(struct nvkm_fifo_chan *base)
-{
- struct gf100_fifo_chan *chan = gf100_fifo_chan(base);
- struct gf100_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u32 addr = chan->base.inst->addr >> 12;
- u32 coff = chan->base.chid * 8;
-
- nvkm_wr32(device, 0x003000 + coff, 0xc0000000 | addr);
-
- if (list_empty(&chan->head) && !chan->killed) {
- gf100_fifo_runlist_insert(fifo, chan);
- nvkm_wr32(device, 0x003004 + coff, 0x001f0001);
- gf100_fifo_runlist_commit(fifo);
- }
-}
-
-static void *
-gf100_fifo_gpfifo_dtor(struct nvkm_fifo_chan *base)
-{
- return gf100_fifo_chan(base);
-}
-
-static const struct nvkm_fifo_chan_func
-gf100_fifo_gpfifo_func = {
- .dtor = gf100_fifo_gpfifo_dtor,
- .init = gf100_fifo_gpfifo_init,
- .fini = gf100_fifo_gpfifo_fini,
- .ntfy = gf100_fifo_chan_ntfy,
- .engine_ctor = gf100_fifo_gpfifo_engine_ctor,
- .engine_dtor = gf100_fifo_gpfifo_engine_dtor,
- .engine_init = gf100_fifo_gpfifo_engine_init,
- .engine_fini = gf100_fifo_gpfifo_engine_fini,
-};
-
-static int
-gf100_fifo_gpfifo_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- union {
- struct fermi_channel_gpfifo_v0 v0;
- } *args = data;
- struct gf100_fifo *fifo = gf100_fifo(base);
- struct nvkm_object *parent = oclass->parent;
- struct gf100_fifo_chan *chan;
- u64 usermem, ioffset, ilength;
- int ret = -ENOSYS, i;
-
- nvif_ioctl(parent, "create channel gpfifo size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel gpfifo vers %d vmm %llx "
- "ioffset %016llx ilength %08x\n",
- args->v0.version, args->v0.vmm, args->v0.ioffset,
- args->v0.ilength);
- if (!args->v0.vmm)
- return -EINVAL;
- } else
- return ret;
-
- /* allocate channel */
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
- chan->fifo = fifo;
- INIT_LIST_HEAD(&chan->head);
-
- ret = nvkm_fifo_chan_ctor(&gf100_fifo_gpfifo_func, &fifo->base,
- 0x1000, 0x1000, true, args->v0.vmm, 0,
- BIT(GF100_FIFO_ENGN_GR) |
- BIT(GF100_FIFO_ENGN_MSPDEC) |
- BIT(GF100_FIFO_ENGN_MSPPP) |
- BIT(GF100_FIFO_ENGN_MSVLD) |
- BIT(GF100_FIFO_ENGN_CE0) |
- BIT(GF100_FIFO_ENGN_CE1) |
- BIT(GF100_FIFO_ENGN_SW),
- 1, fifo->user.bar->addr, 0x1000,
- oclass, &chan->base);
- if (ret)
- return ret;
-
- args->v0.chid = chan->base.chid;
-
- /* clear channel control registers */
-
- usermem = chan->base.chid * 0x1000;
- ioffset = args->v0.ioffset;
- ilength = order_base_2(args->v0.ilength / 8);
-
- nvkm_kmap(fifo->user.mem);
- for (i = 0; i < 0x1000; i += 4)
- nvkm_wo32(fifo->user.mem, usermem + i, 0x00000000);
- nvkm_done(fifo->user.mem);
- usermem = nvkm_memory_addr(fifo->user.mem) + usermem;
-
- /* RAMFC */
- nvkm_kmap(chan->base.inst);
- nvkm_wo32(chan->base.inst, 0x08, lower_32_bits(usermem));
- nvkm_wo32(chan->base.inst, 0x0c, upper_32_bits(usermem));
- nvkm_wo32(chan->base.inst, 0x10, 0x0000face);
- nvkm_wo32(chan->base.inst, 0x30, 0xfffff902);
- nvkm_wo32(chan->base.inst, 0x48, lower_32_bits(ioffset));
- nvkm_wo32(chan->base.inst, 0x4c, upper_32_bits(ioffset) |
- (ilength << 16));
- nvkm_wo32(chan->base.inst, 0x54, 0x00000002);
- nvkm_wo32(chan->base.inst, 0x84, 0x20400000);
- nvkm_wo32(chan->base.inst, 0x94, 0x30000001);
- nvkm_wo32(chan->base.inst, 0x9c, 0x00000100);
- nvkm_wo32(chan->base.inst, 0xa4, 0x1f1f1f1f);
- nvkm_wo32(chan->base.inst, 0xa8, 0x1f1f1f1f);
- nvkm_wo32(chan->base.inst, 0xac, 0x0000001f);
- nvkm_wo32(chan->base.inst, 0xb8, 0xf8000000);
- nvkm_wo32(chan->base.inst, 0xf8, 0x10003080); /* 0x002310 */
- nvkm_wo32(chan->base.inst, 0xfc, 0x10000010); /* 0x002350 */
- nvkm_done(chan->base.inst);
- return 0;
-}
-
-const struct nvkm_fifo_chan_oclass
-gf100_fifo_gpfifo_oclass = {
- .base.oclass = FERMI_CHANNEL_GPFIFO,
- .base.minver = 0,
- .base.maxver = 0,
- .ctor = gf100_fifo_gpfifo_new,
-};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogk104.c
deleted file mode 100644
index 80456ec..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogk104.c
+++ /dev/null
@@ -1,361 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "changk104.h"
-#include "cgrp.h"
-
-#include <core/client.h>
-#include <core/gpuobj.h>
-#include <subdev/fb.h>
-#include <subdev/mmu.h>
-#include <subdev/timer.h>
-
-#include <nvif/class.h>
-#include <nvif/cla06f.h>
-#include <nvif/unpack.h>
-
-int
-gk104_fifo_gpfifo_kick_locked(struct gk104_fifo_chan *chan)
-{
- struct gk104_fifo *fifo = chan->fifo;
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- struct nvkm_client *client = chan->base.object.client;
- struct nvkm_fifo_cgrp *cgrp = chan->cgrp;
- int ret = 0;
-
- if (cgrp)
- nvkm_wr32(device, 0x002634, cgrp->id | 0x01000000);
- else
- nvkm_wr32(device, 0x002634, chan->base.chid);
- if (nvkm_msec(device, 2000,
- if (!(nvkm_rd32(device, 0x002634) & 0x00100000))
- break;
- ) < 0) {
- nvkm_error(subdev, "%s %d [%s] kick timeout\n",
- cgrp ? "tsg" : "channel",
- cgrp ? cgrp->id : chan->base.chid, client->name);
- nvkm_fifo_recover_chan(&fifo->base, chan->base.chid);
- ret = -ETIMEDOUT;
- }
- return ret;
-}
-
-int
-gk104_fifo_gpfifo_kick(struct gk104_fifo_chan *chan)
-{
- int ret;
- mutex_lock(&chan->base.fifo->mutex);
- ret = gk104_fifo_gpfifo_kick_locked(chan);
- mutex_unlock(&chan->base.fifo->mutex);
- return ret;
-}
-
-static u32
-gk104_fifo_gpfifo_engine_addr(struct nvkm_engine *engine)
-{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_SW :
- case NVKM_ENGINE_CE : return 0;
- case NVKM_ENGINE_GR : return 0x0210;
- case NVKM_ENGINE_SEC : return 0x0220;
- case NVKM_ENGINE_MSPDEC: return 0x0250;
- case NVKM_ENGINE_MSPPP : return 0x0260;
- case NVKM_ENGINE_MSVLD : return 0x0270;
- case NVKM_ENGINE_VIC : return 0x0280;
- case NVKM_ENGINE_MSENC : return 0x0290;
- case NVKM_ENGINE_NVDEC : return 0x02100270;
- case NVKM_ENGINE_NVENC :
- if (engine->subdev.inst)
- return 0x0210;
- return 0x02100290;
- default:
- WARN_ON(1);
- return 0;
- }
-}
-
-struct gk104_fifo_engn *
-gk104_fifo_gpfifo_engine(struct gk104_fifo_chan *chan, struct nvkm_engine *engine)
-{
- int engi = chan->base.fifo->func->engine_id(chan->base.fifo, engine);
- if (engi >= 0)
- return &chan->engn[engi];
- return NULL;
-}
-
-static int
-gk104_fifo_gpfifo_engine_fini(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine, bool suspend)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct nvkm_gpuobj *inst = chan->base.inst;
- u32 offset = gk104_fifo_gpfifo_engine_addr(engine);
- int ret;
-
- ret = gk104_fifo_gpfifo_kick(chan);
- if (ret && suspend)
- return ret;
-
- if (offset) {
- nvkm_kmap(inst);
- nvkm_wo32(inst, (offset & 0xffff) + 0x00, 0x00000000);
- nvkm_wo32(inst, (offset & 0xffff) + 0x04, 0x00000000);
- if ((offset >>= 16)) {
- nvkm_wo32(inst, offset + 0x00, 0x00000000);
- nvkm_wo32(inst, offset + 0x04, 0x00000000);
- }
- nvkm_done(inst);
- }
-
- return ret;
-}
-
-static int
-gk104_fifo_gpfifo_engine_init(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct gk104_fifo_engn *engn = gk104_fifo_gpfifo_engine(chan, engine);
- struct nvkm_gpuobj *inst = chan->base.inst;
- u32 offset = gk104_fifo_gpfifo_engine_addr(engine);
-
- if (offset) {
- u32 datalo = lower_32_bits(engn->vma->addr) | 0x00000004;
- u32 datahi = upper_32_bits(engn->vma->addr);
- nvkm_kmap(inst);
- nvkm_wo32(inst, (offset & 0xffff) + 0x00, datalo);
- nvkm_wo32(inst, (offset & 0xffff) + 0x04, datahi);
- if ((offset >>= 16)) {
- nvkm_wo32(inst, offset + 0x00, datalo);
- nvkm_wo32(inst, offset + 0x04, datahi);
- }
- nvkm_done(inst);
- }
-
- return 0;
-}
-
-void
-gk104_fifo_gpfifo_engine_dtor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct gk104_fifo_engn *engn = gk104_fifo_gpfifo_engine(chan, engine);
- nvkm_vmm_put(chan->base.vmm, &engn->vma);
- nvkm_gpuobj_del(&engn->inst);
-}
-
-int
-gk104_fifo_gpfifo_engine_ctor(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine,
- struct nvkm_object *object)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct gk104_fifo_engn *engn = gk104_fifo_gpfifo_engine(chan, engine);
- int ret;
-
- if (!gk104_fifo_gpfifo_engine_addr(engine)) {
- if (engine->subdev.type != NVKM_ENGINE_CE ||
- engine->subdev.device->card_type < GV100)
- return 0;
- }
-
- ret = nvkm_object_bind(object, NULL, 0, &engn->inst);
- if (ret)
- return ret;
-
- if (!gk104_fifo_gpfifo_engine_addr(engine))
- return 0;
-
- ret = nvkm_vmm_get(chan->base.vmm, 12, engn->inst->size, &engn->vma);
- if (ret)
- return ret;
-
- return nvkm_memory_map(engn->inst, 0, chan->base.vmm, engn->vma, NULL, 0);
-}
-
-void
-gk104_fifo_gpfifo_fini(struct nvkm_fifo_chan *base)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct gk104_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u32 coff = chan->base.chid * 8;
-
- if (!list_empty(&chan->head)) {
- gk104_fifo_runlist_remove(fifo, chan);
- nvkm_mask(device, 0x800004 + coff, 0x00000800, 0x00000800);
- gk104_fifo_gpfifo_kick(chan);
- gk104_fifo_runlist_update(fifo, chan->runl);
- }
-
- nvkm_wr32(device, 0x800000 + coff, 0x00000000);
-}
-
-void
-gk104_fifo_gpfifo_init(struct nvkm_fifo_chan *base)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct gk104_fifo *fifo = chan->fifo;
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u32 addr = chan->base.inst->addr >> 12;
- u32 coff = chan->base.chid * 8;
-
- nvkm_mask(device, 0x800004 + coff, 0x000f0000, chan->runl << 16);
- nvkm_wr32(device, 0x800000 + coff, 0x80000000 | addr);
-
- if (list_empty(&chan->head) && !chan->killed) {
- gk104_fifo_runlist_insert(fifo, chan);
- nvkm_mask(device, 0x800004 + coff, 0x00000400, 0x00000400);
- gk104_fifo_runlist_update(fifo, chan->runl);
- nvkm_mask(device, 0x800004 + coff, 0x00000400, 0x00000400);
- }
-}
-
-void *
-gk104_fifo_gpfifo_dtor(struct nvkm_fifo_chan *base)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- kfree(chan->cgrp);
- return chan;
-}
-
-const struct nvkm_fifo_chan_func
-gk104_fifo_gpfifo_func = {
- .dtor = gk104_fifo_gpfifo_dtor,
- .init = gk104_fifo_gpfifo_init,
- .fini = gk104_fifo_gpfifo_fini,
- .ntfy = gf100_fifo_chan_ntfy,
- .engine_ctor = gk104_fifo_gpfifo_engine_ctor,
- .engine_dtor = gk104_fifo_gpfifo_engine_dtor,
- .engine_init = gk104_fifo_gpfifo_engine_init,
- .engine_fini = gk104_fifo_gpfifo_engine_fini,
-};
-
-static int
-gk104_fifo_gpfifo_new_(struct gk104_fifo *fifo, u64 *runlists, u16 *chid,
- u64 vmm, u64 ioffset, u64 ilength, u64 *inst, bool priv,
- const struct nvkm_oclass *oclass,
- struct nvkm_object **pobject)
-{
- struct gk104_fifo_chan *chan;
- int runlist = ffs(*runlists) -1, ret, i;
- u64 usermem;
-
- if (!vmm || runlist < 0 || runlist >= fifo->runlist_nr)
- return -EINVAL;
- *runlists = BIT_ULL(runlist);
-
- /* Allocate the channel. */
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
- chan->fifo = fifo;
- chan->runl = runlist;
- INIT_LIST_HEAD(&chan->head);
-
- ret = nvkm_fifo_chan_ctor(&gk104_fifo_gpfifo_func, &fifo->base,
- 0x1000, 0x1000, true, vmm, 0, fifo->runlist[runlist].engm_sw,
- 1, fifo->user.bar->addr, 0x200,
- oclass, &chan->base);
- if (ret)
- return ret;
-
- *chid = chan->base.chid;
- *inst = chan->base.inst->addr;
-
- /* Hack to support GPUs where even individual channels should be
- * part of a channel group.
- */
- if (fifo->func->cgrp_force) {
- if (!(chan->cgrp = kmalloc(sizeof(*chan->cgrp), GFP_KERNEL)))
- return -ENOMEM;
- chan->cgrp->id = chan->base.chid;
- INIT_LIST_HEAD(&chan->cgrp->head);
- INIT_LIST_HEAD(&chan->cgrp->chan);
- chan->cgrp->chan_nr = 0;
- }
-
- /* Clear channel control registers. */
- usermem = chan->base.chid * 0x200;
- ilength = order_base_2(ilength / 8);
-
- nvkm_kmap(fifo->user.mem);
- for (i = 0; i < 0x200; i += 4)
- nvkm_wo32(fifo->user.mem, usermem + i, 0x00000000);
- nvkm_done(fifo->user.mem);
- usermem = nvkm_memory_addr(fifo->user.mem) + usermem;
-
- /* RAMFC */
- nvkm_kmap(chan->base.inst);
- nvkm_wo32(chan->base.inst, 0x08, lower_32_bits(usermem));
- nvkm_wo32(chan->base.inst, 0x0c, upper_32_bits(usermem));
- nvkm_wo32(chan->base.inst, 0x10, 0x0000face);
- nvkm_wo32(chan->base.inst, 0x30, 0xfffff902);
- nvkm_wo32(chan->base.inst, 0x48, lower_32_bits(ioffset));
- nvkm_wo32(chan->base.inst, 0x4c, upper_32_bits(ioffset) |
- (ilength << 16));
- nvkm_wo32(chan->base.inst, 0x84, 0x20400000);
- nvkm_wo32(chan->base.inst, 0x94, 0x30000001);
- nvkm_wo32(chan->base.inst, 0x9c, 0x00000100);
- nvkm_wo32(chan->base.inst, 0xac, 0x0000001f);
- nvkm_wo32(chan->base.inst, 0xe4, priv ? 0x00000020 : 0x00000000);
- nvkm_wo32(chan->base.inst, 0xe8, chan->base.chid);
- nvkm_wo32(chan->base.inst, 0xb8, 0xf8000000);
- nvkm_wo32(chan->base.inst, 0xf8, 0x10003080); /* 0x002310 */
- nvkm_wo32(chan->base.inst, 0xfc, 0x10000010); /* 0x002350 */
- nvkm_done(chan->base.inst);
- return 0;
-}
-
-int
-gk104_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct kepler_channel_gpfifo_a_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel gpfifo size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel gpfifo vers %d vmm %llx "
- "ioffset %016llx ilength %08x "
- "runlist %016llx priv %d\n",
- args->v0.version, args->v0.vmm, args->v0.ioffset,
- args->v0.ilength, args->v0.runlist, args->v0.priv);
- return gk104_fifo_gpfifo_new_(fifo,
- &args->v0.runlist,
- &args->v0.chid,
- args->v0.vmm,
- args->v0.ioffset,
- args->v0.ilength,
- &args->v0.inst,
- args->v0.priv,
- oclass, pobject);
- }
-
- return ret;
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogv100.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogv100.c
deleted file mode 100644
index 428f9b4..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogv100.c
+++ /dev/null
@@ -1,241 +0,0 @@
-/*
- * Copyright 2018 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- */
-#include "changk104.h"
-#include "cgrp.h"
-
-#include <core/client.h>
-#include <core/gpuobj.h>
-
-#include <nvif/clc36f.h>
-#include <nvif/unpack.h>
-
-static u32
-gv100_fifo_gpfifo_submit_token(struct nvkm_fifo_chan *chan)
-{
- return chan->chid;
-}
-
-static int
-gv100_fifo_gpfifo_engine_valid(struct gk104_fifo_chan *chan, bool ce, bool valid)
-{
- struct nvkm_subdev *subdev = &chan->base.fifo->engine.subdev;
- struct nvkm_device *device = subdev->device;
- const u32 mask = ce ? 0x00020000 : 0x00010000;
- const u32 data = valid ? mask : 0x00000000;
- int ret;
-
- /* Block runlist to prevent the channel from being rescheduled. */
- mutex_lock(&chan->fifo->base.mutex);
- nvkm_mask(device, 0x002630, BIT(chan->runl), BIT(chan->runl));
-
- /* Preempt the channel. */
- ret = gk104_fifo_gpfifo_kick_locked(chan);
- if (ret == 0) {
- /* Update engine context validity. */
- nvkm_kmap(chan->base.inst);
- nvkm_mo32(chan->base.inst, 0x0ac, mask, data);
- nvkm_done(chan->base.inst);
- }
-
- /* Resume runlist. */
- nvkm_mask(device, 0x002630, BIT(chan->runl), 0);
- mutex_unlock(&chan->fifo->base.mutex);
- return ret;
-}
-
-int
-gv100_fifo_gpfifo_engine_fini(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine, bool suspend)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct nvkm_gpuobj *inst = chan->base.inst;
- int ret;
-
- if (engine->subdev.type == NVKM_ENGINE_CE) {
- ret = gv100_fifo_gpfifo_engine_valid(chan, true, false);
- if (ret && suspend)
- return ret;
-
- nvkm_kmap(inst);
- nvkm_wo32(chan->base.inst, 0x220, 0x00000000);
- nvkm_wo32(chan->base.inst, 0x224, 0x00000000);
- nvkm_done(inst);
- return ret;
- }
-
- ret = gv100_fifo_gpfifo_engine_valid(chan, false, false);
- if (ret && suspend)
- return ret;
-
- nvkm_kmap(inst);
- nvkm_wo32(inst, 0x0210, 0x00000000);
- nvkm_wo32(inst, 0x0214, 0x00000000);
- nvkm_done(inst);
- return ret;
-}
-
-int
-gv100_fifo_gpfifo_engine_init(struct nvkm_fifo_chan *base,
- struct nvkm_engine *engine)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- struct gk104_fifo_engn *engn = gk104_fifo_gpfifo_engine(chan, engine);
- struct nvkm_gpuobj *inst = chan->base.inst;
-
- if (engine->subdev.type == NVKM_ENGINE_CE) {
- const u64 bar2 = nvkm_memory_bar2(engn->inst->memory);
-
- nvkm_kmap(inst);
- nvkm_wo32(chan->base.inst, 0x220, lower_32_bits(bar2));
- nvkm_wo32(chan->base.inst, 0x224, upper_32_bits(bar2));
- nvkm_done(inst);
-
- return gv100_fifo_gpfifo_engine_valid(chan, true, true);
- }
-
- nvkm_kmap(inst);
- nvkm_wo32(inst, 0x210, lower_32_bits(engn->vma->addr) | 0x00000004);
- nvkm_wo32(inst, 0x214, upper_32_bits(engn->vma->addr));
- nvkm_done(inst);
-
- return gv100_fifo_gpfifo_engine_valid(chan, false, true);
-}
-
-static const struct nvkm_fifo_chan_func
-gv100_fifo_gpfifo = {
- .dtor = gk104_fifo_gpfifo_dtor,
- .init = gk104_fifo_gpfifo_init,
- .fini = gk104_fifo_gpfifo_fini,
- .ntfy = gf100_fifo_chan_ntfy,
- .engine_ctor = gk104_fifo_gpfifo_engine_ctor,
- .engine_dtor = gk104_fifo_gpfifo_engine_dtor,
- .engine_init = gv100_fifo_gpfifo_engine_init,
- .engine_fini = gv100_fifo_gpfifo_engine_fini,
- .submit_token = gv100_fifo_gpfifo_submit_token,
-};
-
-int
-gv100_fifo_gpfifo_new_(const struct nvkm_fifo_chan_func *func,
- struct gk104_fifo *fifo, u64 *runlists, u16 *chid,
- u64 vmm, u64 ioffset, u64 ilength, u64 *inst, bool priv,
- u32 *token, const struct nvkm_oclass *oclass,
- struct nvkm_object **pobject)
-{
- struct gk104_fifo_chan *chan;
- int runlist = ffs(*runlists) -1, ret, i;
- u64 usermem;
-
- if (!vmm || runlist < 0 || runlist >= fifo->runlist_nr)
- return -EINVAL;
- *runlists = BIT_ULL(runlist);
-
- /* Allocate the channel. */
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
- chan->fifo = fifo;
- chan->runl = runlist;
- INIT_LIST_HEAD(&chan->head);
-
- ret = nvkm_fifo_chan_ctor(func, &fifo->base, 0x1000, 0x1000, true, vmm,
- 0, fifo->runlist[runlist].engm, 1, fifo->user.bar->addr, 0x200,
- oclass, &chan->base);
- if (ret)
- return ret;
-
- *chid = chan->base.chid;
- *inst = chan->base.inst->addr;
- *token = chan->base.func->submit_token(&chan->base);
-
- /* Hack to support GPUs where even individual channels should be
- * part of a channel group.
- */
- if (fifo->func->cgrp_force) {
- if (!(chan->cgrp = kmalloc(sizeof(*chan->cgrp), GFP_KERNEL)))
- return -ENOMEM;
- chan->cgrp->id = chan->base.chid;
- INIT_LIST_HEAD(&chan->cgrp->head);
- INIT_LIST_HEAD(&chan->cgrp->chan);
- chan->cgrp->chan_nr = 0;
- }
-
- /* Clear channel control registers. */
- usermem = chan->base.chid * 0x200;
- ilength = order_base_2(ilength / 8);
-
- nvkm_kmap(fifo->user.mem);
- for (i = 0; i < 0x200; i += 4)
- nvkm_wo32(fifo->user.mem, usermem + i, 0x00000000);
- nvkm_done(fifo->user.mem);
- usermem = nvkm_memory_addr(fifo->user.mem) + usermem;
-
- /* RAMFC */
- nvkm_kmap(chan->base.inst);
- nvkm_wo32(chan->base.inst, 0x008, lower_32_bits(usermem));
- nvkm_wo32(chan->base.inst, 0x00c, upper_32_bits(usermem));
- nvkm_wo32(chan->base.inst, 0x010, 0x0000face);
- nvkm_wo32(chan->base.inst, 0x030, 0x7ffff902);
- nvkm_wo32(chan->base.inst, 0x048, lower_32_bits(ioffset));
- nvkm_wo32(chan->base.inst, 0x04c, upper_32_bits(ioffset) |
- (ilength << 16));
- nvkm_wo32(chan->base.inst, 0x084, 0x20400000);
- nvkm_wo32(chan->base.inst, 0x094, 0x30000001);
- nvkm_wo32(chan->base.inst, 0x0e4, priv ? 0x00000020 : 0x00000000);
- nvkm_wo32(chan->base.inst, 0x0e8, chan->base.chid);
- nvkm_wo32(chan->base.inst, 0x0f4, 0x00001000);
- nvkm_wo32(chan->base.inst, 0x0f8, 0x10003080);
- nvkm_mo32(chan->base.inst, 0x218, 0x00000000, 0x00000000);
- nvkm_done(chan->base.inst);
- return 0;
-}
-
-int
-gv100_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct volta_channel_gpfifo_a_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel gpfifo size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel gpfifo vers %d vmm %llx "
- "ioffset %016llx ilength %08x "
- "runlist %016llx priv %d\n",
- args->v0.version, args->v0.vmm, args->v0.ioffset,
- args->v0.ilength, args->v0.runlist, args->v0.priv);
- return gv100_fifo_gpfifo_new_(&gv100_fifo_gpfifo, fifo,
- &args->v0.runlist,
- &args->v0.chid,
- args->v0.vmm,
- args->v0.ioffset,
- args->v0.ilength,
- &args->v0.inst,
- args->v0.priv,
- &args->v0.token,
- oclass, pobject);
- }
-
- return ret;
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifonv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifonv50.c
deleted file mode 100644
index d8f28ec..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifonv50.c
+++ /dev/null
@@ -1,93 +0,0 @@
-/*
- * Copyright 2012 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- *
- * Authors: Ben Skeggs
- */
-#include "channv50.h"
-
-#include <core/client.h>
-#include <core/ramht.h>
-
-#include <nvif/class.h>
-#include <nvif/cl506f.h>
-#include <nvif/unpack.h>
-
-static int
-nv50_fifo_gpfifo_new(struct nvkm_fifo *base, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct nv50_channel_gpfifo_v0 v0;
- } *args = data;
- struct nv50_fifo *fifo = nv50_fifo(base);
- struct nv50_fifo_chan *chan;
- u64 ioffset, ilength;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel gpfifo size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel gpfifo vers %d vmm %llx "
- "pushbuf %llx ioffset %016llx "
- "ilength %08x\n",
- args->v0.version, args->v0.vmm, args->v0.pushbuf,
- args->v0.ioffset, args->v0.ilength);
- if (!args->v0.pushbuf)
- return -EINVAL;
- } else
- return ret;
-
- if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
- return -ENOMEM;
- *pobject = &chan->base.object;
-
- ret = nv50_fifo_chan_ctor(fifo, args->v0.vmm, args->v0.pushbuf,
- oclass, chan);
- if (ret)
- return ret;
-
- args->v0.chid = chan->base.chid;
- ioffset = args->v0.ioffset;
- ilength = order_base_2(args->v0.ilength / 8);
-
- nvkm_kmap(chan->ramfc);
- nvkm_wo32(chan->ramfc, 0x3c, 0x403f6078);
- nvkm_wo32(chan->ramfc, 0x44, 0x01003fff);
- nvkm_wo32(chan->ramfc, 0x48, chan->base.push->node->offset >> 4);
- nvkm_wo32(chan->ramfc, 0x50, lower_32_bits(ioffset));
- nvkm_wo32(chan->ramfc, 0x54, upper_32_bits(ioffset) | (ilength << 16));
- nvkm_wo32(chan->ramfc, 0x60, 0x7fffffff);
- nvkm_wo32(chan->ramfc, 0x78, 0x00000000);
- nvkm_wo32(chan->ramfc, 0x7c, 0x30000001);
- nvkm_wo32(chan->ramfc, 0x80, ((chan->ramht->bits - 9) << 27) |
- (4 << 24) /* SEARCH_FULL */ |
- (chan->ramht->gpuobj->node->offset >> 4));
- nvkm_done(chan->ramfc);
- return 0;
-}
-
-const struct nvkm_fifo_chan_oclass
-nv50_fifo_gpfifo_oclass = {
- .base.oclass = NV50_CHANNEL_GPFIFO,
- .base.minver = 0,
- .base.maxver = 0,
- .ctor = nv50_fifo_gpfifo_new,
-};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifotu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifotu102.c
deleted file mode 100644
index 99aafa1..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifotu102.c
+++ /dev/null
@@ -1,81 +0,0 @@
-/*
- * Copyright 2018 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- */
-#include "changk104.h"
-#include "cgrp.h"
-
-#include <core/client.h>
-#include <core/gpuobj.h>
-
-#include <nvif/clc36f.h>
-#include <nvif/unpack.h>
-
-static u32
-tu102_fifo_gpfifo_submit_token(struct nvkm_fifo_chan *base)
-{
- struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
- return (chan->runl << 16) | chan->base.chid;
-}
-
-static const struct nvkm_fifo_chan_func
-tu102_fifo_gpfifo = {
- .dtor = gk104_fifo_gpfifo_dtor,
- .init = gk104_fifo_gpfifo_init,
- .fini = gk104_fifo_gpfifo_fini,
- .ntfy = gf100_fifo_chan_ntfy,
- .engine_ctor = gk104_fifo_gpfifo_engine_ctor,
- .engine_dtor = gk104_fifo_gpfifo_engine_dtor,
- .engine_init = gv100_fifo_gpfifo_engine_init,
- .engine_fini = gv100_fifo_gpfifo_engine_fini,
- .submit_token = tu102_fifo_gpfifo_submit_token,
-};
-
-int
-tu102_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass,
- void *data, u32 size, struct nvkm_object **pobject)
-{
- struct nvkm_object *parent = oclass->parent;
- union {
- struct volta_channel_gpfifo_a_v0 v0;
- } *args = data;
- int ret = -ENOSYS;
-
- nvif_ioctl(parent, "create channel gpfifo size %d\n", size);
- if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false))) {
- nvif_ioctl(parent, "create channel gpfifo vers %d vmm %llx "
- "ioffset %016llx ilength %08x "
- "runlist %016llx priv %d\n",
- args->v0.version, args->v0.vmm, args->v0.ioffset,
- args->v0.ilength, args->v0.runlist, args->v0.priv);
- return gv100_fifo_gpfifo_new_(&tu102_fifo_gpfifo, fifo,
- &args->v0.runlist,
- &args->v0.chid,
- args->v0.vmm,
- args->v0.ioffset,
- args->v0.ilength,
- &args->v0.inst,
- args->v0.priv,
- &args->v0.token,
- oclass, pobject);
- }
-
- return ret;
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gv100.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gv100.c
index faf0fe9f7..33066c8 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gv100.c
@@ -19,32 +19,180 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include "gk104.h"
+#include "priv.h"
+#include "chan.h"
+#include "chid.h"
#include "cgrp.h"
-#include "changk104.h"
-#include "user.h"
+#include "runl.h"
+#include "runq.h"
#include <core/gpuobj.h>
+#include <subdev/mmu.h>
#include <nvif/class.h>
-void
-gv100_fifo_runlist_chan(struct gk104_fifo_chan *chan,
- struct nvkm_memory *memory, u32 offset)
+static u32
+gv100_chan_doorbell_handle(struct nvkm_chan *chan)
{
- struct nvkm_memory *usermem = chan->fifo->user.mem;
- const u64 user = nvkm_memory_addr(usermem) + (chan->base.chid * 0x200);
- const u64 inst = chan->base.inst->addr;
+ return chan->id;
+}
- nvkm_wo32(memory, offset + 0x0, lower_32_bits(user));
+static int
+gv100_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ const u64 userd = nvkm_memory_addr(chan->userd.mem) + chan->userd.base;
+ const u32 limit2 = ilog2(length / 8);
+
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, 0x008, lower_32_bits(userd));
+ nvkm_wo32(chan->inst, 0x00c, upper_32_bits(userd));
+ nvkm_wo32(chan->inst, 0x010, 0x0000face);
+ nvkm_wo32(chan->inst, 0x030, 0x7ffff902);
+ nvkm_wo32(chan->inst, 0x048, lower_32_bits(offset));
+ nvkm_wo32(chan->inst, 0x04c, upper_32_bits(offset) | (limit2 << 16));
+ nvkm_wo32(chan->inst, 0x084, 0x20400000);
+ nvkm_wo32(chan->inst, 0x094, 0x30000000 | devm);
+ nvkm_wo32(chan->inst, 0x0e4, priv ? 0x00000020 : 0x00000000);
+ nvkm_wo32(chan->inst, 0x0e8, chan->id);
+ nvkm_wo32(chan->inst, 0x0f4, 0x00001000 | (priv ? 0x00000100 : 0x00000000));
+ nvkm_wo32(chan->inst, 0x0f8, 0x10003080);
+ nvkm_mo32(chan->inst, 0x218, 0x00000000, 0x00000000);
+ nvkm_done(chan->inst);
+ return 0;
+}
+
+const struct nvkm_chan_func_ramfc
+gv100_chan_ramfc = {
+ .write = gv100_chan_ramfc_write,
+ .devm = 0xfff,
+ .priv = true,
+};
+
+const struct nvkm_chan_func_userd
+gv100_chan_userd = {
+ .bar = -1,
+ .size = 0x200,
+ .clear = gf100_chan_userd_clear,
+};
+
+static const struct nvkm_chan_func
+gv100_chan = {
+ .inst = &gf100_chan_inst,
+ .userd = &gv100_chan_userd,
+ .ramfc = &gv100_chan_ramfc,
+ .bind = gk104_chan_bind_inst,
+ .unbind = gk104_chan_unbind,
+ .start = gk104_chan_start,
+ .stop = gk104_chan_stop,
+ .preempt = gk110_chan_preempt,
+ .doorbell_handle = gv100_chan_doorbell_handle,
+};
+
+void
+gv100_ectx_bind(struct nvkm_engn *engn, struct nvkm_cctx *cctx, struct nvkm_chan *chan)
+{
+ u64 addr = 0ULL;
+
+ if (cctx) {
+ addr = cctx->vctx->vma->addr;
+ addr |= 4ULL;
+ }
+
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, 0x210, lower_32_bits(addr));
+ nvkm_wo32(chan->inst, 0x214, upper_32_bits(addr));
+ nvkm_mo32(chan->inst, 0x0ac, 0x00010000, cctx ? 0x00010000 : 0x00000000);
+ nvkm_done(chan->inst);
+}
+
+const struct nvkm_engn_func
+gv100_engn = {
+ .chsw = gk104_engn_chsw,
+ .cxid = gk104_engn_cxid,
+ .ctor = gk104_ectx_ctor,
+ .bind = gv100_ectx_bind,
+};
+
+void
+gv100_ectx_ce_bind(struct nvkm_engn *engn, struct nvkm_cctx *cctx, struct nvkm_chan *chan)
+{
+ const u64 bar2 = cctx ? nvkm_memory_bar2(cctx->vctx->inst->memory) : 0ULL;
+
+ nvkm_kmap(chan->inst);
+ nvkm_wo32(chan->inst, 0x220, lower_32_bits(bar2));
+ nvkm_wo32(chan->inst, 0x224, upper_32_bits(bar2));
+ nvkm_mo32(chan->inst, 0x0ac, 0x00020000, cctx ? 0x00020000 : 0x00000000);
+ nvkm_done(chan->inst);
+}
+
+int
+gv100_ectx_ce_ctor(struct nvkm_engn *engn, struct nvkm_vctx *vctx)
+{
+ if (nvkm_memory_bar2(vctx->inst->memory) == ~0ULL)
+ return -EFAULT;
+
+ return 0;
+}
+
+const struct nvkm_engn_func
+gv100_engn_ce = {
+ .chsw = gk104_engn_chsw,
+ .cxid = gk104_engn_cxid,
+ .ctor = gv100_ectx_ce_ctor,
+ .bind = gv100_ectx_ce_bind,
+};
+
+static bool
+gv100_runq_intr_1_ctxnotvalid(struct nvkm_runq *runq, int chid)
+{
+ struct nvkm_fifo *fifo = runq->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_chan *chan;
+ unsigned long flags;
+
+ RUNQ_ERROR(runq, "CTXNOTVALID chid:%d", chid);
+
+ chan = nvkm_chan_get_chid(&fifo->engine, chid, &flags);
+ if (WARN_ON_ONCE(!chan))
+ return false;
+
+ nvkm_chan_error(chan, true);
+ nvkm_chan_put(&chan, flags);
+
+ nvkm_mask(device, 0x0400ac + (runq->id * 0x2000), 0x00030000, 0x00030000);
+ nvkm_wr32(device, 0x040148 + (runq->id * 0x2000), 0x80000000);
+ return true;
+}
+
+const struct nvkm_runq_func
+gv100_runq = {
+ .init = gk208_runq_init,
+ .intr = gk104_runq_intr,
+ .intr_0_names = gk104_runq_intr_0_names,
+ .intr_1_ctxnotvalid = gv100_runq_intr_1_ctxnotvalid,
+ .idle = gk104_runq_idle,
+};
+
+void
+gv100_runl_preempt(struct nvkm_runl *runl)
+{
+ nvkm_wr32(runl->fifo->engine.subdev.device, 0x002638, BIT(runl->id));
+}
+
+void
+gv100_runl_insert_chan(struct nvkm_chan *chan, struct nvkm_memory *memory, u64 offset)
+{
+ const u64 user = nvkm_memory_addr(chan->userd.mem) + chan->userd.base;
+ const u64 inst = chan->inst->addr;
+
+ nvkm_wo32(memory, offset + 0x0, lower_32_bits(user) | chan->runq << 1);
nvkm_wo32(memory, offset + 0x4, upper_32_bits(user));
- nvkm_wo32(memory, offset + 0x8, lower_32_bits(inst) | chan->base.chid);
+ nvkm_wo32(memory, offset + 0x8, lower_32_bits(inst) | chan->id);
nvkm_wo32(memory, offset + 0xc, upper_32_bits(inst));
}
void
-gv100_fifo_runlist_cgrp(struct nvkm_fifo_cgrp *cgrp,
- struct nvkm_memory *memory, u32 offset)
+gv100_runl_insert_cgrp(struct nvkm_cgrp *cgrp, struct nvkm_memory *memory, u64 offset)
{
nvkm_wo32(memory, offset + 0x0, (128 << 24) | (3 << 16) | 0x00000001);
nvkm_wo32(memory, offset + 0x4, cgrp->chan_nr);
@@ -52,16 +200,24 @@ gv100_fifo_runlist_cgrp(struct nvkm_fifo_cgrp *cgrp,
nvkm_wo32(memory, offset + 0xc, 0x00000000);
}
-static const struct gk104_fifo_runlist_func
-gv100_fifo_runlist = {
+static const struct nvkm_runl_func
+gv100_runl = {
+ .runqs = 2,
.size = 16,
- .cgrp = gv100_fifo_runlist_cgrp,
- .chan = gv100_fifo_runlist_chan,
- .commit = gk104_fifo_runlist_commit,
+ .update = nv50_runl_update,
+ .insert_cgrp = gv100_runl_insert_cgrp,
+ .insert_chan = gv100_runl_insert_chan,
+ .commit = gk104_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = gk104_runl_pending,
+ .block = gk104_runl_block,
+ .allow = gk104_runl_allow,
+ .preempt = gv100_runl_preempt,
+ .preempt_pending = gf100_runl_preempt_pending,
};
const struct nvkm_enum
-gv100_fifo_fault_gpcclient[] = {
+gv100_fifo_mmu_fault_gpcclient[] = {
{ 0x00, "T1_0" },
{ 0x01, "T1_1" },
{ 0x02, "T1_2" },
@@ -163,7 +319,7 @@ gv100_fifo_fault_gpcclient[] = {
};
const struct nvkm_enum
-gv100_fifo_fault_hubclient[] = {
+gv100_fifo_mmu_fault_hubclient[] = {
{ 0x00, "VIP" },
{ 0x01, "CE0" },
{ 0x02, "CE1" },
@@ -225,7 +381,7 @@ gv100_fifo_fault_hubclient[] = {
};
const struct nvkm_enum
-gv100_fifo_fault_reason[] = {
+gv100_fifo_mmu_fault_reason[] = {
{ 0x00, "PDE" },
{ 0x01, "PDE_SIZE" },
{ 0x02, "PTE" },
@@ -246,7 +402,7 @@ gv100_fifo_fault_reason[] = {
};
static const struct nvkm_enum
-gv100_fifo_fault_engine[] = {
+gv100_fifo_mmu_fault_engine[] = {
{ 0x01, "DISPLAY" },
{ 0x03, "PTP" },
{ 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
@@ -273,7 +429,7 @@ gv100_fifo_fault_engine[] = {
};
const struct nvkm_enum
-gv100_fifo_fault_access[] = {
+gv100_fifo_mmu_fault_access[] = {
{ 0x0, "VIRT_READ" },
{ 0x1, "VIRT_WRITE" },
{ 0x2, "VIRT_ATOMIC" },
@@ -286,23 +442,51 @@ gv100_fifo_fault_access[] = {
{}
};
-static const struct gk104_fifo_func
+static const struct nvkm_fifo_func_mmu_fault
+gv100_fifo_mmu_fault = {
+ .recover = gf100_fifo_mmu_fault_recover,
+ .access = gv100_fifo_mmu_fault_access,
+ .engine = gv100_fifo_mmu_fault_engine,
+ .reason = gv100_fifo_mmu_fault_reason,
+ .hubclient = gv100_fifo_mmu_fault_hubclient,
+ .gpcclient = gv100_fifo_mmu_fault_gpcclient,
+};
+
+static void
+gv100_fifo_intr_ctxsw_timeout(struct nvkm_fifo *fifo, u32 engm)
+{
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+
+ nvkm_runl_foreach(runl, fifo) {
+ nvkm_runl_foreach_engn_cond(engn, runl, engm & BIT(engn->id))
+ nvkm_runl_rc_engn(runl, engn);
+ }
+}
+
+static const struct nvkm_fifo_func
gv100_fifo = {
- .pbdma = &gm200_fifo_pbdma,
- .fault.access = gv100_fifo_fault_access,
- .fault.engine = gv100_fifo_fault_engine,
- .fault.reason = gv100_fifo_fault_reason,
- .fault.hubclient = gv100_fifo_fault_hubclient,
- .fault.gpcclient = gv100_fifo_fault_gpcclient,
- .runlist = &gv100_fifo_runlist,
- .user = {{-1,-1,VOLTA_USERMODE_A }, gv100_fifo_user_new },
- .chan = {{ 0, 0,VOLTA_CHANNEL_GPFIFO_A}, gv100_fifo_gpfifo_new },
- .cgrp_force = true,
+ .chid_nr = gm200_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gm200_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
+ .init = gk104_fifo_init,
+ .init_pbdmas = gk104_fifo_init_pbdmas,
+ .intr = gk104_fifo_intr,
+ .intr_ctxsw_timeout = gv100_fifo_intr_ctxsw_timeout,
+ .mmu_fault = &gv100_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &gv100_runl,
+ .runq = &gv100_runq,
+ .engn = &gv100_engn,
+ .engn_ce = &gv100_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &gk110_cgrp, .force = true },
+ .chan = {{ 0, 0, VOLTA_CHANNEL_GPFIFO_A }, &gv100_chan },
};
int
gv100_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return gk104_fifo_new_(&gv100_fifo, device, type, inst, 4096, pfifo);
+ return nvkm_fifo_new_(&gv100_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.c
index c6730c1..674faf0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.c
@@ -21,38 +21,201 @@
*
* Authors: Ben Skeggs
*/
-#include "nv04.h"
-#include "channv04.h"
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+
#include "regsnv04.h"
-#include <core/client.h>
#include <core/ramht.h>
#include <subdev/instmem.h>
+#include <subdev/mc.h>
#include <subdev/timer.h>
#include <engine/sw.h>
-static const struct nv04_fifo_ramfc
-nv04_fifo_ramfc[] = {
- { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
- { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
- { 16, 0, 0x08, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
- { 16, 16, 0x08, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
- { 32, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_STATE },
- { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
- { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_ENGINE },
- { 32, 0, 0x18, 0, NV04_PFIFO_CACHE1_PULL1 },
- {}
+#include <nvif/class.h>
+
+void
+nv04_chan_stop(struct nvkm_chan *chan)
+{
+ struct nvkm_fifo *fifo = chan->cgrp->runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_memory *fctx = device->imem->ramfc;
+ const struct nvkm_ramfc_layout *c;
+ unsigned long flags;
+ u32 data = chan->ramfc_offset;
+ u32 chid;
+
+ /* prevent fifo context switches */
+ spin_lock_irqsave(&fifo->lock, flags);
+ nvkm_wr32(device, NV03_PFIFO_CACHES, 0);
+
+ /* if this channel is active, replace it with a null context */
+ chid = nvkm_rd32(device, NV03_PFIFO_CACHE1_PUSH1) & fifo->chid->mask;
+ if (chid == chan->id) {
+ nvkm_mask(device, NV04_PFIFO_CACHE1_DMA_PUSH, 0x00000001, 0);
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH0, 0);
+ nvkm_mask(device, NV04_PFIFO_CACHE1_PULL0, 0x00000001, 0);
+
+ c = chan->func->ramfc->layout;
+ nvkm_kmap(fctx);
+ do {
+ u32 rm = ((1ULL << c->bits) - 1) << c->regs;
+ u32 cm = ((1ULL << c->bits) - 1) << c->ctxs;
+ u32 rv = (nvkm_rd32(device, c->regp) & rm) >> c->regs;
+ u32 cv = (nvkm_ro32(fctx, c->ctxp + data) & ~cm);
+ nvkm_wo32(fctx, c->ctxp + data, cv | (rv << c->ctxs));
+ } while ((++c)->bits);
+ nvkm_done(fctx);
+
+ c = chan->func->ramfc->layout;
+ do {
+ nvkm_wr32(device, c->regp, 0x00000000);
+ } while ((++c)->bits);
+
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_GET, 0);
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_PUT, 0);
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, fifo->chid->mask);
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH0, 1);
+ nvkm_wr32(device, NV04_PFIFO_CACHE1_PULL0, 1);
+ }
+
+ /* restore normal operation, after disabling dma mode */
+ nvkm_mask(device, NV04_PFIFO_MODE, BIT(chan->id), 0);
+ nvkm_wr32(device, NV03_PFIFO_CACHES, 1);
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+void
+nv04_chan_start(struct nvkm_chan *chan)
+{
+ struct nvkm_fifo *fifo = chan->cgrp->runl->fifo;
+ unsigned long flags;
+
+ spin_lock_irqsave(&fifo->lock, flags);
+ nvkm_mask(fifo->engine.subdev.device, NV04_PFIFO_MODE, BIT(chan->id), BIT(chan->id));
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+void
+nv04_chan_ramfc_clear(struct nvkm_chan *chan)
+{
+ struct nvkm_memory *ramfc = chan->cgrp->runl->fifo->engine.subdev.device->imem->ramfc;
+ const struct nvkm_ramfc_layout *c = chan->func->ramfc->layout;
+
+ nvkm_kmap(ramfc);
+ do {
+ nvkm_wo32(ramfc, chan->ramfc_offset + c->ctxp, 0x00000000);
+ } while ((++c)->bits);
+ nvkm_done(ramfc);
+}
+
+static int
+nv04_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ struct nvkm_memory *ramfc = chan->cgrp->runl->fifo->engine.subdev.device->imem->ramfc;
+ const u32 base = chan->id * 32;
+
+ chan->ramfc_offset = base;
+
+ nvkm_kmap(ramfc);
+ nvkm_wo32(ramfc, base + 0x00, offset);
+ nvkm_wo32(ramfc, base + 0x04, offset);
+ nvkm_wo32(ramfc, base + 0x08, chan->push->addr >> 4);
+ nvkm_wo32(ramfc, base + 0x10, NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
+ NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
+#ifdef __BIG_ENDIAN
+ NV_PFIFO_CACHE1_BIG_ENDIAN |
+#endif
+ NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
+ nvkm_done(ramfc);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+nv04_chan_ramfc = {
+ .layout = (const struct nvkm_ramfc_layout[]) {
+ { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
+ { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
+ { 16, 0, 0x08, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
+ { 16, 16, 0x08, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
+ { 32, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_STATE },
+ { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
+ { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_ENGINE },
+ { 32, 0, 0x18, 0, NV04_PFIFO_CACHE1_PULL1 },
+ {}
+ },
+ .write = nv04_chan_ramfc_write,
+ .clear = nv04_chan_ramfc_clear,
+ .ctxdma = true,
+};
+
+const struct nvkm_chan_func_userd
+nv04_chan_userd = {
+ .bar = 0,
+ .base = 0x800000,
+ .size = 0x010000,
+};
+
+const struct nvkm_chan_func_inst
+nv04_chan_inst = {
+ .size = 0x1000,
+};
+
+static const struct nvkm_chan_func
+nv04_chan = {
+ .inst = &nv04_chan_inst,
+ .userd = &nv04_chan_userd,
+ .ramfc = &nv04_chan_ramfc,
+ .start = nv04_chan_start,
+ .stop = nv04_chan_stop,
+};
+
+const struct nvkm_cgrp_func
+nv04_cgrp = {
};
void
-nv04_fifo_pause(struct nvkm_fifo *base, unsigned long *pflags)
-__acquires(fifo->base.lock)
+nv04_eobj_ramht_del(struct nvkm_chan *chan, int hash)
{
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
+ struct nvkm_fifo *fifo = chan->cgrp->runl->fifo;
+ struct nvkm_instmem *imem = fifo->engine.subdev.device->imem;
+
+ mutex_lock(&fifo->mutex);
+ nvkm_ramht_remove(imem->ramht, hash);
+ mutex_unlock(&fifo->mutex);
+}
+
+static int
+nv04_eobj_ramht_add(struct nvkm_engn *engn, struct nvkm_object *eobj, struct nvkm_chan *chan)
+{
+ struct nvkm_fifo *fifo = chan->cgrp->runl->fifo;
+ struct nvkm_instmem *imem = fifo->engine.subdev.device->imem;
+ u32 context = 0x80000000 | chan->id << 24 | engn->id << 16;
+ int hash;
+
+ mutex_lock(&fifo->mutex);
+ hash = nvkm_ramht_insert(imem->ramht, eobj, chan->id, 4, eobj->handle, context);
+ mutex_unlock(&fifo->mutex);
+ return hash;
+}
+
+const struct nvkm_engn_func
+nv04_engn = {
+ .ramht_add = nv04_eobj_ramht_add,
+ .ramht_del = nv04_eobj_ramht_del,
+};
+
+void
+nv04_fifo_pause(struct nvkm_fifo *fifo, unsigned long *pflags)
+__acquires(fifo->lock)
+{
+ struct nvkm_device *device = fifo->engine.subdev.device;
unsigned long flags;
- spin_lock_irqsave(&fifo->base.lock, flags);
+ spin_lock_irqsave(&fifo->lock, flags);
*pflags = flags;
nvkm_wr32(device, NV03_PFIFO_CACHES, 0x00000000);
@@ -81,50 +244,21 @@ __acquires(fifo->base.lock)
}
void
-nv04_fifo_start(struct nvkm_fifo *base, unsigned long *pflags)
-__releases(fifo->base.lock)
+nv04_fifo_start(struct nvkm_fifo *fifo, unsigned long *pflags)
+__releases(fifo->lock)
{
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
+ struct nvkm_device *device = fifo->engine.subdev.device;
unsigned long flags = *pflags;
nvkm_mask(device, NV04_PFIFO_CACHE1_PULL0, 0x00000001, 0x00000001);
nvkm_wr32(device, NV03_PFIFO_CACHES, 0x00000001);
- spin_unlock_irqrestore(&fifo->base.lock, flags);
+ spin_unlock_irqrestore(&fifo->lock, flags);
}
-struct nvkm_engine *
-nv04_fifo_id_engine(struct nvkm_fifo *fifo, int engi)
-{
- enum nvkm_subdev_type type;
-
- switch (engi) {
- case NV04_FIFO_ENGN_SW : type = NVKM_ENGINE_SW; break;
- case NV04_FIFO_ENGN_GR : type = NVKM_ENGINE_GR; break;
- case NV04_FIFO_ENGN_MPEG: type = NVKM_ENGINE_MPEG; break;
- case NV04_FIFO_ENGN_DMA : type = NVKM_ENGINE_DMAOBJ; break;
- default:
- WARN_ON(1);
- return NULL;
- }
-
- return nvkm_device_engine(fifo->engine.subdev.device, type, 0);
-}
-
-int
-nv04_fifo_engine_id(struct nvkm_fifo *base, struct nvkm_engine *engine)
-{
- switch (engine->subdev.type) {
- case NVKM_ENGINE_SW : return NV04_FIFO_ENGN_SW;
- case NVKM_ENGINE_GR : return NV04_FIFO_ENGN_GR;
- case NVKM_ENGINE_MPEG : return NV04_FIFO_ENGN_MPEG;
- case NVKM_ENGINE_DMAOBJ: return NV04_FIFO_ENGN_DMA;
- default:
- WARN_ON(1);
- return 0;
- }
-}
+const struct nvkm_runl_func
+nv04_runl = {
+};
static const char *
nv_dma_state_err(u32 state)
@@ -166,11 +300,11 @@ nv04_fifo_swmthd(struct nvkm_device *device, u32 chid, u32 addr, u32 data)
}
static void
-nv04_fifo_cache_error(struct nv04_fifo *fifo, u32 chid, u32 get)
+nv04_fifo_intr_cache_error(struct nvkm_fifo *fifo, u32 chid, u32 get)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
unsigned long flags;
u32 pull0 = nvkm_rd32(device, 0x003250);
u32 mthd, data;
@@ -193,12 +327,12 @@ nv04_fifo_cache_error(struct nv04_fifo *fifo, u32 chid, u32 get)
if (!(pull0 & 0x00000100) ||
!nv04_fifo_swmthd(device, chid, mthd, data)) {
- chan = nvkm_fifo_chan_chid(&fifo->base, chid, &flags);
+ chan = nvkm_chan_get_chid(&fifo->engine, chid, &flags);
nvkm_error(subdev, "CACHE_ERROR - "
"ch %d [%s] subc %d mthd %04x data %08x\n",
- chid, chan ? chan->object.client->name : "unknown",
+ chid, chan ? chan->name : "unknown",
(mthd >> 13) & 7, mthd & 0x1ffc, data);
- nvkm_fifo_chan_put(&fifo->base, flags, &chan);
+ nvkm_chan_put(&chan, flags);
}
nvkm_wr32(device, NV04_PFIFO_CACHE1_DMA_PUSH, 0);
@@ -217,20 +351,20 @@ nv04_fifo_cache_error(struct nv04_fifo *fifo, u32 chid, u32 get)
}
static void
-nv04_fifo_dma_pusher(struct nv04_fifo *fifo, u32 chid)
+nv04_fifo_intr_dma_pusher(struct nvkm_fifo *fifo, u32 chid)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
u32 dma_get = nvkm_rd32(device, 0x003244);
u32 dma_put = nvkm_rd32(device, 0x003240);
u32 push = nvkm_rd32(device, 0x003220);
u32 state = nvkm_rd32(device, 0x003228);
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
unsigned long flags;
const char *name;
- chan = nvkm_fifo_chan_chid(&fifo->base, chid, &flags);
- name = chan ? chan->object.client->name : "unknown";
+ chan = nvkm_chan_get_chid(&fifo->engine, chid, &flags);
+ name = chan ? chan->name : "unknown";
if (device->card_type == NV_50) {
u32 ho_get = nvkm_rd32(device, 0x003328);
u32 ho_put = nvkm_rd32(device, 0x003320);
@@ -261,18 +395,18 @@ nv04_fifo_dma_pusher(struct nv04_fifo *fifo, u32 chid)
if (dma_get != dma_put)
nvkm_wr32(device, 0x003244, dma_put);
}
- nvkm_fifo_chan_put(&fifo->base, flags, &chan);
+ nvkm_chan_put(&chan, flags);
nvkm_wr32(device, 0x003228, 0x00000000);
nvkm_wr32(device, 0x003220, 0x00000001);
nvkm_wr32(device, 0x002100, NV_PFIFO_INTR_DMA_PUSHER);
}
-void
-nv04_fifo_intr(struct nvkm_fifo *base)
+irqreturn_t
+nv04_fifo_intr(struct nvkm_inth *inth)
{
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_fifo *fifo = container_of(inth, typeof(*fifo), engine.subdev.inth);
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
u32 mask = nvkm_rd32(device, NV03_PFIFO_INTR_EN_0);
u32 stat = nvkm_rd32(device, NV03_PFIFO_INTR_0) & mask;
@@ -281,16 +415,16 @@ nv04_fifo_intr(struct nvkm_fifo *base)
reassign = nvkm_rd32(device, NV03_PFIFO_CACHES) & 1;
nvkm_wr32(device, NV03_PFIFO_CACHES, 0);
- chid = nvkm_rd32(device, NV03_PFIFO_CACHE1_PUSH1) & (fifo->base.nr - 1);
+ chid = nvkm_rd32(device, NV03_PFIFO_CACHE1_PUSH1) & fifo->chid->mask;
get = nvkm_rd32(device, NV03_PFIFO_CACHE1_GET);
if (stat & NV_PFIFO_INTR_CACHE_ERROR) {
- nv04_fifo_cache_error(fifo, chid, get);
+ nv04_fifo_intr_cache_error(fifo, chid, get);
stat &= ~NV_PFIFO_INTR_CACHE_ERROR;
}
if (stat & NV_PFIFO_INTR_DMA_PUSHER) {
- nv04_fifo_dma_pusher(fifo, chid);
+ nv04_fifo_intr_dma_pusher(fifo, chid);
stat &= ~NV_PFIFO_INTR_DMA_PUSHER;
}
@@ -313,7 +447,7 @@ nv04_fifo_intr(struct nvkm_fifo *base)
if (stat & 0x40000000) {
nvkm_wr32(device, 0x002100, 0x40000000);
- nvkm_fifo_uevent(&fifo->base);
+ nvkm_event_ntfy(&fifo->nonstall.event, 0, NVKM_FIFO_NONSTALL_EVENT);
stat &= ~0x40000000;
}
}
@@ -325,13 +459,13 @@ nv04_fifo_intr(struct nvkm_fifo *base)
}
nvkm_wr32(device, NV03_PFIFO_CACHES, reassign);
+ return IRQ_HANDLED;
}
void
-nv04_fifo_init(struct nvkm_fifo *base)
+nv04_fifo_init(struct nvkm_fifo *fifo)
{
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
+ struct nvkm_device *device = fifo->engine.subdev.device;
struct nvkm_instmem *imem = device->imem;
struct nvkm_ramht *ramht = imem->ramht;
struct nvkm_memory *ramro = imem->ramro;
@@ -346,7 +480,7 @@ nv04_fifo_init(struct nvkm_fifo *base)
nvkm_wr32(device, NV03_PFIFO_RAMRO, nvkm_memory_addr(ramro) >> 8);
nvkm_wr32(device, NV03_PFIFO_RAMFC, nvkm_memory_addr(ramfc) >> 8);
- nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, fifo->base.nr - 1);
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, fifo->chid->mask);
nvkm_wr32(device, NV03_PFIFO_INTR_0, 0xffffffff);
nvkm_wr32(device, NV03_PFIFO_INTR_EN_0, 0xffffffff);
@@ -357,43 +491,53 @@ nv04_fifo_init(struct nvkm_fifo *base)
}
int
-nv04_fifo_new_(const struct nvkm_fifo_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, int nr, const struct nv04_fifo_ramfc *ramfc,
- struct nvkm_fifo **pfifo)
+nv04_fifo_runl_ctor(struct nvkm_fifo *fifo)
{
- struct nv04_fifo *fifo;
- int ret;
+ struct nvkm_runl *runl;
- if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
- return -ENOMEM;
- fifo->ramfc = ramfc;
- *pfifo = &fifo->base;
+ runl = nvkm_runl_new(fifo, 0, 0, 0);
+ if (IS_ERR(runl))
+ return PTR_ERR(runl);
- ret = nvkm_fifo_ctor(func, device, type, inst, nr, &fifo->base);
- if (ret)
- return ret;
-
- set_bit(nr - 1, fifo->base.mask); /* inactive channel */
+ nvkm_runl_add(runl, 0, fifo->func->engn_sw, NVKM_ENGINE_SW, 0);
+ nvkm_runl_add(runl, 0, fifo->func->engn_sw, NVKM_ENGINE_DMAOBJ, 0);
+ nvkm_runl_add(runl, 1, fifo->func->engn , NVKM_ENGINE_GR, 0);
+ nvkm_runl_add(runl, 2, fifo->func->engn , NVKM_ENGINE_MPEG, 0); /* NV31- */
return 0;
}
+int
+nv04_fifo_chid_ctor(struct nvkm_fifo *fifo, int nr)
+{
+ /* The last CHID is reserved by HW as a "channel invalid" marker. */
+ return nvkm_chid_new(&nvkm_chan_event, &fifo->engine.subdev, nr, 0, nr - 1, &fifo->chid);
+}
+
+static int
+nv04_fifo_chid_nr(struct nvkm_fifo *fifo)
+{
+ return 16;
+}
+
static const struct nvkm_fifo_func
nv04_fifo = {
+ .chid_nr = nv04_fifo_chid_nr,
+ .chid_ctor = nv04_fifo_chid_ctor,
+ .runl_ctor = nv04_fifo_runl_ctor,
.init = nv04_fifo_init,
.intr = nv04_fifo_intr,
- .engine_id = nv04_fifo_engine_id,
- .id_engine = nv04_fifo_id_engine,
.pause = nv04_fifo_pause,
.start = nv04_fifo_start,
- .chan = {
- &nv04_fifo_dma_oclass,
- NULL
- },
+ .runl = &nv04_runl,
+ .engn = &nv04_engn,
+ .engn_sw = &nv04_engn,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, NV03_CHANNEL_DMA }, &nv04_chan },
};
int
nv04_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return nv04_fifo_new_(&nv04_fifo, device, type, inst, 16, nv04_fifo_ramfc, pfifo);
+ return nvkm_fifo_new_(&nv04_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.h
deleted file mode 100644
index 3f23bcd..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.h
+++ /dev/null
@@ -1,23 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NV04_FIFO_H__
-#define __NV04_FIFO_H__
-#define nv04_fifo(p) container_of((p), struct nv04_fifo, base)
-#include "priv.h"
-
-struct nv04_fifo_ramfc {
- unsigned bits:6;
- unsigned ctxs:5;
- unsigned ctxp:8;
- unsigned regs:5;
- unsigned regp;
-};
-
-struct nv04_fifo {
- struct nvkm_fifo base;
- const struct nv04_fifo_ramfc *ramfc;
-};
-
-int nv04_fifo_new_(const struct nvkm_fifo_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
- int nr, const struct nv04_fifo_ramfc *, struct nvkm_fifo **);
-void nv04_fifo_init(struct nvkm_fifo *);
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv10.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv10.c
index f8887f0..a4bcf6b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv10.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv10.c
@@ -21,41 +21,93 @@
*
* Authors: Ben Skeggs
*/
-#include "nv04.h"
-#include "channv04.h"
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "runl.h"
+
+#include <core/gpuobj.h>
+#include <subdev/instmem.h>
+
#include "regsnv04.h"
-static const struct nv04_fifo_ramfc
-nv10_fifo_ramfc[] = {
- { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
- { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
- { 32, 0, 0x08, 0, NV10_PFIFO_CACHE1_REF_CNT },
- { 16, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
- { 16, 16, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
- { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_STATE },
- { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
- { 32, 0, 0x18, 0, NV04_PFIFO_CACHE1_ENGINE },
- { 32, 0, 0x1c, 0, NV04_PFIFO_CACHE1_PULL1 },
- {}
+#include <nvif/class.h>
+
+static int
+nv10_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ struct nvkm_memory *ramfc = chan->cgrp->runl->fifo->engine.subdev.device->imem->ramfc;
+ const u32 base = chan->id * 32;
+
+ chan->ramfc_offset = base;
+
+ nvkm_kmap(ramfc);
+ nvkm_wo32(ramfc, base + 0x00, offset);
+ nvkm_wo32(ramfc, base + 0x04, offset);
+ nvkm_wo32(ramfc, base + 0x0c, chan->push->addr >> 4);
+ nvkm_wo32(ramfc, base + 0x14, NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
+ NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
+#ifdef __BIG_ENDIAN
+ NV_PFIFO_CACHE1_BIG_ENDIAN |
+#endif
+ NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
+ nvkm_done(ramfc);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+nv10_chan_ramfc = {
+ .layout = (const struct nvkm_ramfc_layout[]) {
+ { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
+ { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
+ { 32, 0, 0x08, 0, NV10_PFIFO_CACHE1_REF_CNT },
+ { 16, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
+ { 16, 16, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
+ { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_STATE },
+ { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
+ { 32, 0, 0x18, 0, NV04_PFIFO_CACHE1_ENGINE },
+ { 32, 0, 0x1c, 0, NV04_PFIFO_CACHE1_PULL1 },
+ {}
+ },
+ .write = nv10_chan_ramfc_write,
+ .clear = nv04_chan_ramfc_clear,
+ .ctxdma = true,
};
+static const struct nvkm_chan_func
+nv10_chan = {
+ .inst = &nv04_chan_inst,
+ .userd = &nv04_chan_userd,
+ .ramfc = &nv10_chan_ramfc,
+ .start = nv04_chan_start,
+ .stop = nv04_chan_stop,
+};
+
+int
+nv10_fifo_chid_nr(struct nvkm_fifo *fifo)
+{
+ return 32;
+}
+
static const struct nvkm_fifo_func
nv10_fifo = {
+ .chid_nr = nv10_fifo_chid_nr,
+ .chid_ctor = nv04_fifo_chid_ctor,
+ .runl_ctor = nv04_fifo_runl_ctor,
.init = nv04_fifo_init,
.intr = nv04_fifo_intr,
- .engine_id = nv04_fifo_engine_id,
- .id_engine = nv04_fifo_id_engine,
.pause = nv04_fifo_pause,
.start = nv04_fifo_start,
- .chan = {
- &nv10_fifo_dma_oclass,
- NULL
- },
+ .runl = &nv04_runl,
+ .engn = &nv04_engn,
+ .engn_sw = &nv04_engn,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, NV10_CHANNEL_DMA }, &nv10_chan },
};
int
nv10_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return nv04_fifo_new_(&nv10_fifo, device, type, inst, 32, nv10_fifo_ramfc, pfifo);
+ return nvkm_fifo_new_(&nv10_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv17.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv17.c
index 3f94c7b..c70f44f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv17.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv17.c
@@ -21,37 +21,78 @@
*
* Authors: Ben Skeggs
*/
-#include "nv04.h"
-#include "channv04.h"
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+
#include "regsnv04.h"
#include <core/ramht.h>
#include <subdev/instmem.h>
-static const struct nv04_fifo_ramfc
-nv17_fifo_ramfc[] = {
- { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
- { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
- { 32, 0, 0x08, 0, NV10_PFIFO_CACHE1_REF_CNT },
- { 16, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
- { 16, 16, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
- { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_STATE },
- { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
- { 32, 0, 0x18, 0, NV04_PFIFO_CACHE1_ENGINE },
- { 32, 0, 0x1c, 0, NV04_PFIFO_CACHE1_PULL1 },
- { 32, 0, 0x20, 0, NV10_PFIFO_CACHE1_ACQUIRE_VALUE },
- { 32, 0, 0x24, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMESTAMP },
- { 32, 0, 0x28, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMEOUT },
- { 32, 0, 0x2c, 0, NV10_PFIFO_CACHE1_SEMAPHORE },
- { 32, 0, 0x30, 0, NV10_PFIFO_CACHE1_DMA_SUBROUTINE },
- {}
+#include <nvif/class.h>
+
+static int
+nv17_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ struct nvkm_memory *ramfc = chan->cgrp->runl->fifo->engine.subdev.device->imem->ramfc;
+ const u32 base = chan->id * 64;
+
+ chan->ramfc_offset = base;
+
+ nvkm_kmap(ramfc);
+ nvkm_wo32(ramfc, base + 0x00, offset);
+ nvkm_wo32(ramfc, base + 0x04, offset);
+ nvkm_wo32(ramfc, base + 0x0c, chan->push->addr >> 4);
+ nvkm_wo32(ramfc, base + 0x14, NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
+ NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
+#ifdef __BIG_ENDIAN
+ NV_PFIFO_CACHE1_BIG_ENDIAN |
+#endif
+ NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
+ nvkm_done(ramfc);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+nv17_chan_ramfc = {
+ .layout = (const struct nvkm_ramfc_layout[]) {
+ { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
+ { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
+ { 32, 0, 0x08, 0, NV10_PFIFO_CACHE1_REF_CNT },
+ { 16, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
+ { 16, 16, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
+ { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_STATE },
+ { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
+ { 32, 0, 0x18, 0, NV04_PFIFO_CACHE1_ENGINE },
+ { 32, 0, 0x1c, 0, NV04_PFIFO_CACHE1_PULL1 },
+ { 32, 0, 0x20, 0, NV10_PFIFO_CACHE1_ACQUIRE_VALUE },
+ { 32, 0, 0x24, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMESTAMP },
+ { 32, 0, 0x28, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMEOUT },
+ { 32, 0, 0x2c, 0, NV10_PFIFO_CACHE1_SEMAPHORE },
+ { 32, 0, 0x30, 0, NV10_PFIFO_CACHE1_DMA_SUBROUTINE },
+ {}
+ },
+ .write = nv17_chan_ramfc_write,
+ .clear = nv04_chan_ramfc_clear,
+ .ctxdma = true,
+};
+
+static const struct nvkm_chan_func
+nv17_chan = {
+ .inst = &nv04_chan_inst,
+ .userd = &nv04_chan_userd,
+ .ramfc = &nv17_chan_ramfc,
+ .start = nv04_chan_start,
+ .stop = nv04_chan_stop,
};
static void
-nv17_fifo_init(struct nvkm_fifo *base)
+nv17_fifo_init(struct nvkm_fifo *fifo)
{
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
+ struct nvkm_device *device = fifo->engine.subdev.device;
struct nvkm_instmem *imem = device->imem;
struct nvkm_ramht *ramht = imem->ramht;
struct nvkm_memory *ramro = imem->ramro;
@@ -67,7 +108,7 @@ nv17_fifo_init(struct nvkm_fifo *base)
nvkm_wr32(device, NV03_PFIFO_RAMFC, nvkm_memory_addr(ramfc) >> 8 |
0x00010000);
- nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, fifo->base.nr - 1);
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, fifo->chid->mask);
nvkm_wr32(device, NV03_PFIFO_INTR_0, 0xffffffff);
nvkm_wr32(device, NV03_PFIFO_INTR_EN_0, 0xffffffff);
@@ -79,21 +120,23 @@ nv17_fifo_init(struct nvkm_fifo *base)
static const struct nvkm_fifo_func
nv17_fifo = {
+ .chid_nr = nv10_fifo_chid_nr,
+ .chid_ctor = nv04_fifo_chid_ctor,
+ .runl_ctor = nv04_fifo_runl_ctor,
.init = nv17_fifo_init,
.intr = nv04_fifo_intr,
- .engine_id = nv04_fifo_engine_id,
- .id_engine = nv04_fifo_id_engine,
.pause = nv04_fifo_pause,
.start = nv04_fifo_start,
- .chan = {
- &nv17_fifo_dma_oclass,
- NULL
- },
+ .runl = &nv04_runl,
+ .engn = &nv04_engn,
+ .engn_sw = &nv04_engn,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, NV17_CHANNEL_DMA }, &nv17_chan },
};
int
nv17_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return nv04_fifo_new_(&nv17_fifo, device, type, inst, 32, nv17_fifo_ramfc, pfifo);
+ return nvkm_fifo_new_(&nv17_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv40.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv40.c
index f9ea468..e50a94b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv40.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv40.c
@@ -21,46 +21,166 @@
*
* Authors: Ben Skeggs
*/
-#include "nv04.h"
-#include "channv04.h"
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+
#include "regsnv04.h"
#include <core/ramht.h>
#include <subdev/fb.h>
#include <subdev/instmem.h>
-static const struct nv04_fifo_ramfc
-nv40_fifo_ramfc[] = {
- { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
- { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
- { 32, 0, 0x08, 0, NV10_PFIFO_CACHE1_REF_CNT },
- { 32, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
- { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
- { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_DMA_STATE },
- { 28, 0, 0x18, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
- { 2, 28, 0x18, 28, 0x002058 },
- { 32, 0, 0x1c, 0, NV04_PFIFO_CACHE1_ENGINE },
- { 32, 0, 0x20, 0, NV04_PFIFO_CACHE1_PULL1 },
- { 32, 0, 0x24, 0, NV10_PFIFO_CACHE1_ACQUIRE_VALUE },
- { 32, 0, 0x28, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMESTAMP },
- { 32, 0, 0x2c, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMEOUT },
- { 32, 0, 0x30, 0, NV10_PFIFO_CACHE1_SEMAPHORE },
- { 32, 0, 0x34, 0, NV10_PFIFO_CACHE1_DMA_SUBROUTINE },
- { 32, 0, 0x38, 0, NV40_PFIFO_GRCTX_INSTANCE },
- { 17, 0, 0x3c, 0, NV04_PFIFO_DMA_TIMESLICE },
- { 32, 0, 0x40, 0, 0x0032e4 },
- { 32, 0, 0x44, 0, 0x0032e8 },
- { 32, 0, 0x4c, 0, 0x002088 },
- { 32, 0, 0x50, 0, 0x003300 },
- { 32, 0, 0x54, 0, 0x00330c },
- {}
+#include <nvif/class.h>
+
+static int
+nv40_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ struct nvkm_memory *ramfc = chan->cgrp->runl->fifo->engine.subdev.device->imem->ramfc;
+ const u32 base = chan->id * 128;
+
+ chan->ramfc_offset = base;
+
+ nvkm_kmap(ramfc);
+ nvkm_wo32(ramfc, base + 0x00, offset);
+ nvkm_wo32(ramfc, base + 0x04, offset);
+ nvkm_wo32(ramfc, base + 0x0c, chan->push->addr >> 4);
+ nvkm_wo32(ramfc, base + 0x18, 0x30000000 |
+ NV_PFIFO_CACHE1_DMA_FETCH_TRIG_128_BYTES |
+ NV_PFIFO_CACHE1_DMA_FETCH_SIZE_128_BYTES |
+#ifdef __BIG_ENDIAN
+ NV_PFIFO_CACHE1_BIG_ENDIAN |
+#endif
+ NV_PFIFO_CACHE1_DMA_FETCH_MAX_REQS_8);
+ nvkm_wo32(ramfc, base + 0x3c, 0x0001ffff);
+ nvkm_done(ramfc);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+nv40_chan_ramfc = {
+ .layout = (const struct nvkm_ramfc_layout[]) {
+ { 32, 0, 0x00, 0, NV04_PFIFO_CACHE1_DMA_PUT },
+ { 32, 0, 0x04, 0, NV04_PFIFO_CACHE1_DMA_GET },
+ { 32, 0, 0x08, 0, NV10_PFIFO_CACHE1_REF_CNT },
+ { 32, 0, 0x0c, 0, NV04_PFIFO_CACHE1_DMA_INSTANCE },
+ { 32, 0, 0x10, 0, NV04_PFIFO_CACHE1_DMA_DCOUNT },
+ { 32, 0, 0x14, 0, NV04_PFIFO_CACHE1_DMA_STATE },
+ { 28, 0, 0x18, 0, NV04_PFIFO_CACHE1_DMA_FETCH },
+ { 2, 28, 0x18, 28, 0x002058 },
+ { 32, 0, 0x1c, 0, NV04_PFIFO_CACHE1_ENGINE },
+ { 32, 0, 0x20, 0, NV04_PFIFO_CACHE1_PULL1 },
+ { 32, 0, 0x24, 0, NV10_PFIFO_CACHE1_ACQUIRE_VALUE },
+ { 32, 0, 0x28, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMESTAMP },
+ { 32, 0, 0x2c, 0, NV10_PFIFO_CACHE1_ACQUIRE_TIMEOUT },
+ { 32, 0, 0x30, 0, NV10_PFIFO_CACHE1_SEMAPHORE },
+ { 32, 0, 0x34, 0, NV10_PFIFO_CACHE1_DMA_SUBROUTINE },
+ { 32, 0, 0x38, 0, NV40_PFIFO_GRCTX_INSTANCE },
+ { 17, 0, 0x3c, 0, NV04_PFIFO_DMA_TIMESLICE },
+ { 32, 0, 0x40, 0, 0x0032e4 },
+ { 32, 0, 0x44, 0, 0x0032e8 },
+ { 32, 0, 0x4c, 0, 0x002088 },
+ { 32, 0, 0x50, 0, 0x003300 },
+ { 32, 0, 0x54, 0, 0x00330c },
+ {}
+ },
+ .write = nv40_chan_ramfc_write,
+ .clear = nv04_chan_ramfc_clear,
+ .ctxdma = true,
+};
+
+static const struct nvkm_chan_func_userd
+nv40_chan_userd = {
+ .bar = 0,
+ .base = 0xc00000,
+ .size = 0x001000,
+};
+
+static const struct nvkm_chan_func
+nv40_chan = {
+ .inst = &nv04_chan_inst,
+ .userd = &nv40_chan_userd,
+ .ramfc = &nv40_chan_ramfc,
+ .start = nv04_chan_start,
+ .stop = nv04_chan_stop,
+};
+
+static int
+nv40_eobj_ramht_add(struct nvkm_engn *engn, struct nvkm_object *eobj, struct nvkm_chan *chan)
+{
+ struct nvkm_fifo *fifo = chan->cgrp->runl->fifo;
+ struct nvkm_instmem *imem = fifo->engine.subdev.device->imem;
+ u32 context = chan->id << 23 | engn->id << 20;
+ int hash;
+
+ mutex_lock(&fifo->mutex);
+ hash = nvkm_ramht_insert(imem->ramht, eobj, chan->id, 4, eobj->handle, context);
+ mutex_unlock(&fifo->mutex);
+ return hash;
+}
+
+static void
+nv40_ectx_bind(struct nvkm_engn *engn, struct nvkm_cctx *cctx, struct nvkm_chan *chan)
+{
+ struct nvkm_fifo *fifo = chan->cgrp->runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_memory *ramfc = device->imem->ramfc;
+ u32 inst = 0x00000000, reg, ctx;
+ int chid;
+
+ switch (engn->engine->subdev.type) {
+ case NVKM_ENGINE_GR:
+ reg = 0x0032e0;
+ ctx = 0x38;
+ break;
+ case NVKM_ENGINE_MPEG:
+ if (WARN_ON(device->chipset < 0x44))
+ return;
+ reg = 0x00330c;
+ ctx = 0x54;
+ break;
+ default:
+ WARN_ON(1);
+ return;
+ }
+
+ if (cctx)
+ inst = cctx->vctx->inst->addr >> 4;
+
+ spin_lock_irq(&fifo->lock);
+ nvkm_mask(device, 0x002500, 0x00000001, 0x00000000);
+
+ chid = nvkm_rd32(device, 0x003204) & (fifo->chid->nr - 1);
+ if (chid == chan->id)
+ nvkm_wr32(device, reg, inst);
+
+ nvkm_kmap(ramfc);
+ nvkm_wo32(ramfc, chan->ramfc_offset + ctx, inst);
+ nvkm_done(ramfc);
+
+ nvkm_mask(device, 0x002500, 0x00000001, 0x00000001);
+ spin_unlock_irq(&fifo->lock);
+}
+
+static const struct nvkm_engn_func
+nv40_engn = {
+ .bind = nv40_ectx_bind,
+ .ramht_add = nv40_eobj_ramht_add,
+ .ramht_del = nv04_eobj_ramht_del,
+};
+
+static const struct nvkm_engn_func
+nv40_engn_sw = {
+ .ramht_add = nv40_eobj_ramht_add,
+ .ramht_del = nv04_eobj_ramht_del,
};
static void
-nv40_fifo_init(struct nvkm_fifo *base)
+nv40_fifo_init(struct nvkm_fifo *fifo)
{
- struct nv04_fifo *fifo = nv04_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
+ struct nvkm_device *device = fifo->engine.subdev.device;
struct nvkm_fb *fb = device->fb;
struct nvkm_instmem *imem = device->imem;
struct nvkm_ramht *ramht = imem->ramht;
@@ -98,7 +218,7 @@ nv40_fifo_init(struct nvkm_fifo *base)
break;
}
- nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, fifo->base.nr - 1);
+ nvkm_wr32(device, NV03_PFIFO_CACHE1_PUSH1, fifo->chid->mask);
nvkm_wr32(device, NV03_PFIFO_INTR_0, 0xffffffff);
nvkm_wr32(device, NV03_PFIFO_INTR_EN_0, 0xffffffff);
@@ -110,21 +230,23 @@ nv40_fifo_init(struct nvkm_fifo *base)
static const struct nvkm_fifo_func
nv40_fifo = {
+ .chid_nr = nv10_fifo_chid_nr,
+ .chid_ctor = nv04_fifo_chid_ctor,
+ .runl_ctor = nv04_fifo_runl_ctor,
.init = nv40_fifo_init,
.intr = nv04_fifo_intr,
- .engine_id = nv04_fifo_engine_id,
- .id_engine = nv04_fifo_id_engine,
.pause = nv04_fifo_pause,
.start = nv04_fifo_start,
- .chan = {
- &nv40_fifo_dma_oclass,
- NULL
- },
+ .runl = &nv04_runl,
+ .engn = &nv40_engn,
+ .engn_sw = &nv40_engn_sw,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, NV40_CHANNEL_DMA }, &nv40_chan },
};
int
nv40_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return nv04_fifo_new_(&nv40_fifo, device, type, inst, 32, nv40_fifo_ramfc, pfifo);
+ return nvkm_fifo_new_(&nv40_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.c
index a08742c..954b5f3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.c
@@ -21,62 +21,325 @@
*
* Authors: Ben Skeggs
*/
-#include "nv50.h"
-#include "channv50.h"
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
-#include <core/gpuobj.h>
+#include <core/ramht.h>
+#include <subdev/timer.h>
-static void
-nv50_fifo_runlist_update_locked(struct nv50_fifo *fifo)
-{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_memory *cur;
- int i, p;
-
- cur = fifo->runlist[fifo->cur_runlist];
- fifo->cur_runlist = !fifo->cur_runlist;
-
- nvkm_kmap(cur);
- for (i = 0, p = 0; i < fifo->base.nr; i++) {
- if (nvkm_rd32(device, 0x002600 + (i * 4)) & 0x80000000)
- nvkm_wo32(cur, p++ * 4, i);
- }
- nvkm_done(cur);
-
- nvkm_wr32(device, 0x0032f4, nvkm_memory_addr(cur) >> 12);
- nvkm_wr32(device, 0x0032ec, p);
- nvkm_wr32(device, 0x002500, 0x00000101);
-}
+#include <nvif/class.h>
void
-nv50_fifo_runlist_update(struct nv50_fifo *fifo)
+nv50_eobj_ramht_del(struct nvkm_chan *chan, int hash)
{
- mutex_lock(&fifo->base.mutex);
- nv50_fifo_runlist_update_locked(fifo);
- mutex_unlock(&fifo->base.mutex);
+ nvkm_ramht_remove(chan->ramht, hash);
}
int
-nv50_fifo_oneinit(struct nvkm_fifo *base)
+nv50_eobj_ramht_add(struct nvkm_engn *engn, struct nvkm_object *eobj, struct nvkm_chan *chan)
{
- struct nv50_fifo *fifo = nv50_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- int ret;
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 128 * 4, 0x1000,
- false, &fifo->runlist[0]);
- if (ret)
- return ret;
-
- return nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 128 * 4, 0x1000,
- false, &fifo->runlist[1]);
+ return nvkm_ramht_insert(chan->ramht, eobj, 0, 4, eobj->handle, engn->id << 20);
}
void
-nv50_fifo_init(struct nvkm_fifo *base)
+nv50_chan_stop(struct nvkm_chan *chan)
{
- struct nv50_fifo *fifo = nv50_fifo(base);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_mask(device, 0x002600 + (chan->id * 4), 0x80000000, 0x00000000);
+}
+
+void
+nv50_chan_start(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_mask(device, 0x002600 + (chan->id * 4), 0x80000000, 0x80000000);
+}
+
+void
+nv50_chan_unbind(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x002600 + (chan->id * 4), 0x00000000);
+}
+
+static void
+nv50_chan_bind(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ nvkm_wr32(device, 0x002600 + (chan->id * 4), chan->ramfc->addr >> 12);
+}
+
+static int
+nv50_chan_ramfc_write(struct nvkm_chan *chan, u64 offset, u64 length, u32 devm, bool priv)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+ const u32 limit2 = ilog2(length / 8);
+ int ret;
+
+ ret = nvkm_gpuobj_new(device, 0x0200, 0x1000, true, chan->inst, &chan->ramfc);
+ if (ret)
+ return ret;
+
+ ret = nvkm_gpuobj_new(device, 0x1200, 0, true, chan->inst, &chan->eng);
+ if (ret)
+ return ret;
+
+ ret = nvkm_gpuobj_new(device, 0x4000, 0, false, chan->inst, &chan->pgd);
+ if (ret)
+ return ret;
+
+ ret = nvkm_ramht_new(device, 0x8000, 16, chan->inst, &chan->ramht);
+ if (ret)
+ return ret;
+
+ nvkm_kmap(chan->ramfc);
+ nvkm_wo32(chan->ramfc, 0x3c, 0x403f6078);
+ nvkm_wo32(chan->ramfc, 0x44, 0x01003fff);
+ nvkm_wo32(chan->ramfc, 0x48, chan->push->node->offset >> 4);
+ nvkm_wo32(chan->ramfc, 0x50, lower_32_bits(offset));
+ nvkm_wo32(chan->ramfc, 0x54, upper_32_bits(offset) | (limit2 << 16));
+ nvkm_wo32(chan->ramfc, 0x60, 0x7fffffff);
+ nvkm_wo32(chan->ramfc, 0x78, 0x00000000);
+ nvkm_wo32(chan->ramfc, 0x7c, 0x30000000 | devm);
+ nvkm_wo32(chan->ramfc, 0x80, ((chan->ramht->bits - 9) << 27) |
+ (4 << 24) /* SEARCH_FULL */ |
+ (chan->ramht->gpuobj->node->offset >> 4));
+ nvkm_done(chan->ramfc);
+ return 0;
+}
+
+static const struct nvkm_chan_func_ramfc
+nv50_chan_ramfc = {
+ .write = nv50_chan_ramfc_write,
+ .ctxdma = true,
+ .devm = 0xfff,
+};
+
+const struct nvkm_chan_func_userd
+nv50_chan_userd = {
+ .bar = 0,
+ .base = 0xc00000,
+ .size = 0x002000,
+};
+
+const struct nvkm_chan_func_inst
+nv50_chan_inst = {
+ .size = 0x10000,
+ .vmm = true,
+};
+
+static const struct nvkm_chan_func
+nv50_chan = {
+ .inst = &nv50_chan_inst,
+ .userd = &nv50_chan_userd,
+ .ramfc = &nv50_chan_ramfc,
+ .bind = nv50_chan_bind,
+ .unbind = nv50_chan_unbind,
+ .start = nv50_chan_start,
+ .stop = nv50_chan_stop,
+};
+
+static void
+nv50_ectx_bind(struct nvkm_engn *engn, struct nvkm_cctx *cctx, struct nvkm_chan *chan)
+{
+ struct nvkm_subdev *subdev = &chan->cgrp->runl->fifo->engine.subdev;
+ struct nvkm_device *device = subdev->device;
+ u64 start = 0, limit = 0;
+ u32 flags = 0, ptr0, save;
+
+ switch (engn->engine->subdev.type) {
+ case NVKM_ENGINE_GR : ptr0 = 0x0000; break;
+ case NVKM_ENGINE_MPEG : ptr0 = 0x0060; break;
+ default:
+ WARN_ON(1);
+ return;
+ }
+
+ if (!cctx) {
+ /* HW bug workaround:
+ *
+ * PFIFO will hang forever if the connected engines don't report
+ * that they've processed the context switch request.
+ *
+ * In order for the kickoff to work, we need to ensure all the
+ * connected engines are in a state where they can answer.
+ *
+ * Newer chipsets don't seem to suffer from this issue, and well,
+ * there's also a "ignore these engines" bitmask reg we can use
+ * if we hit the issue there..
+ */
+ save = nvkm_mask(device, 0x00b860, 0x00000001, 0x00000001);
+
+ /* Tell engines to save out contexts. */
+ nvkm_wr32(device, 0x0032fc, chan->inst->addr >> 12);
+ nvkm_msec(device, 2000,
+ if (nvkm_rd32(device, 0x0032fc) != 0xffffffff)
+ break;
+ );
+ nvkm_wr32(device, 0x00b860, save);
+ } else {
+ flags = 0x00190000;
+ start = cctx->vctx->inst->addr;
+ limit = start + cctx->vctx->inst->size - 1;
+ }
+
+ nvkm_kmap(chan->eng);
+ nvkm_wo32(chan->eng, ptr0 + 0x00, flags);
+ nvkm_wo32(chan->eng, ptr0 + 0x04, lower_32_bits(limit));
+ nvkm_wo32(chan->eng, ptr0 + 0x08, lower_32_bits(start));
+ nvkm_wo32(chan->eng, ptr0 + 0x0c, upper_32_bits(limit) << 24 |
+ lower_32_bits(start));
+ nvkm_wo32(chan->eng, ptr0 + 0x10, 0x00000000);
+ nvkm_wo32(chan->eng, ptr0 + 0x14, 0x00000000);
+ nvkm_done(chan->eng);
+}
+
+static const struct nvkm_engn_func
+nv50_engn = {
+ .bind = nv50_ectx_bind,
+ .ramht_add = nv50_eobj_ramht_add,
+ .ramht_del = nv50_eobj_ramht_del,
+};
+
+const struct nvkm_engn_func
+nv50_engn_sw = {
+ .ramht_add = nv50_eobj_ramht_add,
+ .ramht_del = nv50_eobj_ramht_del,
+};
+
+static bool
+nv50_runl_pending(struct nvkm_runl *runl)
+{
+ return nvkm_rd32(runl->fifo->engine.subdev.device, 0x0032ec) & 0x00000100;
+}
+
+int
+nv50_runl_wait(struct nvkm_runl *runl)
+{
+ struct nvkm_fifo *fifo = runl->fifo;
+
+ nvkm_msec(fifo->engine.subdev.device, fifo->timeout.chan_msec,
+ if (!nvkm_runl_update_pending(runl))
+ return 0;
+ usleep_range(1, 2);
+ );
+
+ return -ETIMEDOUT;
+}
+
+static void
+nv50_runl_commit(struct nvkm_runl *runl, struct nvkm_memory *memory, u32 start, int count)
+{
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+ u64 addr = nvkm_memory_addr(memory) + start;
+
+ nvkm_wr32(device, 0x0032f4, addr >> 12);
+ nvkm_wr32(device, 0x0032ec, count);
+}
+
+static void
+nv50_runl_insert_chan(struct nvkm_chan *chan, struct nvkm_memory *memory, u64 offset)
+{
+ nvkm_wo32(memory, offset, chan->id);
+}
+
+static struct nvkm_memory *
+nv50_runl_alloc(struct nvkm_runl *runl, u32 *offset)
+{
+ const u32 segment = ALIGN((runl->cgrp_nr + runl->chan_nr) * runl->func->size, 0x1000);
+ const u32 maxsize = (runl->cgid ? runl->cgid->nr : 0) + runl->chid->nr;
+ int ret;
+
+ if (unlikely(!runl->mem)) {
+ ret = nvkm_memory_new(runl->fifo->engine.subdev.device, NVKM_MEM_TARGET_INST,
+ maxsize * 2 * runl->func->size, 0, false, &runl->mem);
+ if (ret) {
+ RUNL_ERROR(runl, "alloc %d\n", ret);
+ return ERR_PTR(ret);
+ }
+ } else {
+ if (runl->offset + segment >= nvkm_memory_size(runl->mem)) {
+ ret = runl->func->wait(runl);
+ if (ret) {
+ RUNL_DEBUG(runl, "rewind timeout");
+ return ERR_PTR(ret);
+ }
+
+ runl->offset = 0;
+ }
+ }
+
+ *offset = runl->offset;
+ runl->offset += segment;
+ return runl->mem;
+}
+
+int
+nv50_runl_update(struct nvkm_runl *runl)
+{
+ struct nvkm_memory *memory;
+ struct nvkm_cgrp *cgrp;
+ struct nvkm_chan *chan;
+ u32 start, offset, count;
+
+ /*TODO: prio, interleaving. */
+
+ RUNL_TRACE(runl, "RAMRL: update cgrps:%d chans:%d", runl->cgrp_nr, runl->chan_nr);
+ memory = nv50_runl_alloc(runl, &start);
+ if (IS_ERR(memory))
+ return PTR_ERR(memory);
+
+ RUNL_TRACE(runl, "RAMRL: update start:%08x", start);
+ offset = start;
+
+ nvkm_kmap(memory);
+ nvkm_runl_foreach_cgrp(cgrp, runl) {
+ if (cgrp->hw) {
+ CGRP_TRACE(cgrp, " RAMRL+%08x: chans:%d", offset, cgrp->chan_nr);
+ runl->func->insert_cgrp(cgrp, memory, offset);
+ offset += runl->func->size;
+ }
+
+ nvkm_cgrp_foreach_chan(chan, cgrp) {
+ CHAN_TRACE(chan, "RAMRL+%08x: [%s]", offset, chan->name);
+ runl->func->insert_chan(chan, memory, offset);
+ offset += runl->func->size;
+ }
+ }
+ nvkm_done(memory);
+
+ /*TODO: look into using features on newer HW to guarantee forward progress. */
+ list_rotate_left(&runl->cgrps);
+
+ count = (offset - start) / runl->func->size;
+ RUNL_TRACE(runl, "RAMRL: commit start:%08x count:%d", start, count);
+
+ runl->func->commit(runl, memory, start, count);
+ return 0;
+}
+
+const struct nvkm_runl_func
+nv50_runl = {
+ .size = 4,
+ .update = nv50_runl_update,
+ .insert_chan = nv50_runl_insert_chan,
+ .commit = nv50_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = nv50_runl_pending,
+};
+
+void
+nv50_fifo_init(struct nvkm_fifo *fifo)
+{
+ struct nvkm_runl *runl = nvkm_runl_first(fifo);
+ struct nvkm_device *device = fifo->engine.subdev.device;
int i;
nvkm_mask(device, 0x000200, 0x00000100, 0x00000000);
@@ -89,61 +352,47 @@ nv50_fifo_init(struct nvkm_fifo *base)
for (i = 0; i < 128; i++)
nvkm_wr32(device, 0x002600 + (i * 4), 0x00000000);
- nv50_fifo_runlist_update_locked(fifo);
+
+ atomic_set(&runl->changed, 1);
+ runl->func->update(runl);
nvkm_wr32(device, 0x003200, 0x00000001);
nvkm_wr32(device, 0x003250, 0x00000001);
nvkm_wr32(device, 0x002500, 0x00000001);
}
-void *
-nv50_fifo_dtor(struct nvkm_fifo *base)
+int
+nv50_fifo_chid_ctor(struct nvkm_fifo *fifo, int nr)
{
- struct nv50_fifo *fifo = nv50_fifo(base);
- nvkm_memory_unref(&fifo->runlist[1]);
- nvkm_memory_unref(&fifo->runlist[0]);
- return fifo;
+ /* CHID 0 is unusable (some kind of PIO channel?), 127 is "channel invalid". */
+ return nvkm_chid_new(&nvkm_chan_event, &fifo->engine.subdev, nr, 1, nr - 2, &fifo->chid);
}
int
-nv50_fifo_new_(const struct nvkm_fifo_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_fifo **pfifo)
+nv50_fifo_chid_nr(struct nvkm_fifo *fifo)
{
- struct nv50_fifo *fifo;
- int ret;
-
- if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
- return -ENOMEM;
- *pfifo = &fifo->base;
-
- ret = nvkm_fifo_ctor(func, device, type, inst, 128, &fifo->base);
- if (ret)
- return ret;
-
- set_bit(0, fifo->base.mask); /* PIO channel */
- set_bit(127, fifo->base.mask); /* inactive channel */
- return 0;
+ return 128;
}
static const struct nvkm_fifo_func
nv50_fifo = {
- .dtor = nv50_fifo_dtor,
- .oneinit = nv50_fifo_oneinit,
+ .chid_nr = nv50_fifo_chid_nr,
+ .chid_ctor = nv50_fifo_chid_ctor,
+ .runl_ctor = nv04_fifo_runl_ctor,
.init = nv50_fifo_init,
.intr = nv04_fifo_intr,
- .engine_id = nv04_fifo_engine_id,
- .id_engine = nv04_fifo_id_engine,
.pause = nv04_fifo_pause,
.start = nv04_fifo_start,
- .chan = {
- &nv50_fifo_gpfifo_oclass,
- NULL
- },
+ .runl = &nv50_runl,
+ .engn = &nv50_engn,
+ .engn_sw = &nv50_engn_sw,
+ .cgrp = {{ }, &nv04_cgrp },
+ .chan = {{ 0, 0, NV50_CHANNEL_GPFIFO }, &nv50_chan },
};
int
nv50_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- return nv50_fifo_new_(&nv50_fifo, device, type, inst, pfifo);
+ return nvkm_fifo_new_(&nv50_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.h
deleted file mode 100644
index 0111e7e..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.h
+++ /dev/null
@@ -1,20 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __NV50_FIFO_H__
-#define __NV50_FIFO_H__
-#define nv50_fifo(p) container_of((p), struct nv50_fifo, base)
-#include "priv.h"
-
-struct nv50_fifo {
- struct nvkm_fifo base;
- struct nvkm_memory *runlist[2];
- int cur_runlist;
-};
-
-int nv50_fifo_new_(const struct nvkm_fifo_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
- struct nvkm_fifo **);
-
-void *nv50_fifo_dtor(struct nvkm_fifo *);
-int nv50_fifo_oneinit(struct nvkm_fifo *);
-void nv50_fifo_init(struct nvkm_fifo *);
-void nv50_fifo_runlist_update(struct nv50_fifo *);
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h
index 79cec57..4d448be 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h
@@ -3,46 +3,207 @@
#define __NVKM_FIFO_PRIV_H__
#define nvkm_fifo(p) container_of((p), struct nvkm_fifo, engine)
#include <engine/fifo.h>
+#include <core/enum.h>
+struct nvkm_cctx;
+struct nvkm_cgrp;
+struct nvkm_engn;
+struct nvkm_memory;
+struct nvkm_runl;
+struct nvkm_runq;
+struct nvkm_vctx;
-int nvkm_fifo_ctor(const struct nvkm_fifo_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
- int nr, struct nvkm_fifo *);
-void nvkm_fifo_uevent(struct nvkm_fifo *);
-void nvkm_fifo_kevent(struct nvkm_fifo *, int chid);
-void nvkm_fifo_recover_chan(struct nvkm_fifo *, int chid);
-
-struct nvkm_fifo_chan *
-nvkm_fifo_chan_inst_locked(struct nvkm_fifo *, u64 inst);
-
-struct nvkm_fifo_chan_oclass;
struct nvkm_fifo_func {
- void *(*dtor)(struct nvkm_fifo *);
- int (*oneinit)(struct nvkm_fifo *);
- int (*info)(struct nvkm_fifo *, u64 mthd, u64 *data);
+ int (*chid_nr)(struct nvkm_fifo *);
+ int (*chid_ctor)(struct nvkm_fifo *, int nr);
+ int (*runq_nr)(struct nvkm_fifo *);
+ int (*runl_ctor)(struct nvkm_fifo *);
+
void (*init)(struct nvkm_fifo *);
- void (*fini)(struct nvkm_fifo *);
- void (*intr)(struct nvkm_fifo *);
- void (*fault)(struct nvkm_fifo *, struct nvkm_fault_data *);
- int (*engine_id)(struct nvkm_fifo *, struct nvkm_engine *);
- struct nvkm_engine *(*id_engine)(struct nvkm_fifo *, int engi);
+ void (*init_pbdmas)(struct nvkm_fifo *, u32 mask);
+
+ irqreturn_t (*intr)(struct nvkm_inth *);
+ void (*intr_mmu_fault_unit)(struct nvkm_fifo *, int unit);
+ void (*intr_ctxsw_timeout)(struct nvkm_fifo *, u32 engm);
+
+ const struct nvkm_fifo_func_mmu_fault {
+ void (*recover)(struct nvkm_fifo *, struct nvkm_fault_data *);
+ const struct nvkm_enum *access;
+ const struct nvkm_enum *engine;
+ const struct nvkm_enum *reason;
+ const struct nvkm_enum *hubclient;
+ const struct nvkm_enum *gpcclient;
+ } *mmu_fault;
+
void (*pause)(struct nvkm_fifo *, unsigned long *);
void (*start)(struct nvkm_fifo *, unsigned long *);
- void (*uevent_init)(struct nvkm_fifo *);
- void (*uevent_fini)(struct nvkm_fifo *);
- void (*recover_chan)(struct nvkm_fifo *, int chid);
- int (*class_get)(struct nvkm_fifo *, int index, struct nvkm_oclass *);
- int (*class_new)(struct nvkm_fifo *, const struct nvkm_oclass *,
- void *, u32, struct nvkm_object **);
- const struct nvkm_fifo_chan_oclass *chan[];
+
+ int (*nonstall_ctor)(struct nvkm_fifo *);
+ const struct nvkm_event_func *nonstall;
+
+ const struct nvkm_runl_func *runl;
+ const struct nvkm_runq_func *runq;
+ const struct nvkm_engn_func *engn;
+ const struct nvkm_engn_func *engn_sw;
+ const struct nvkm_engn_func *engn_ce;
+
+ struct nvkm_fifo_func_cgrp {
+ struct nvkm_sclass user;
+ const struct nvkm_cgrp_func *func;
+ bool force;
+ } cgrp;
+
+ struct nvkm_fifo_func_chan {
+ struct nvkm_sclass user;
+ const struct nvkm_chan_func *func;
+ } chan;
};
-void nv04_fifo_intr(struct nvkm_fifo *);
-int nv04_fifo_engine_id(struct nvkm_fifo *, struct nvkm_engine *);
-struct nvkm_engine *nv04_fifo_id_engine(struct nvkm_fifo *, int);
+int nvkm_fifo_new_(const struct nvkm_fifo_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
+ struct nvkm_fifo **);
+
+int nv04_fifo_chid_ctor(struct nvkm_fifo *, int);
+int nv04_fifo_runl_ctor(struct nvkm_fifo *);
+void nv04_fifo_init(struct nvkm_fifo *);
+irqreturn_t nv04_fifo_intr(struct nvkm_inth *);
void nv04_fifo_pause(struct nvkm_fifo *, unsigned long *);
void nv04_fifo_start(struct nvkm_fifo *, unsigned long *);
+extern const struct nvkm_runl_func nv04_runl;
+extern const struct nvkm_engn_func nv04_engn;
+extern const struct nvkm_cgrp_func nv04_cgrp;
+extern const struct nvkm_chan_func_inst nv04_chan_inst;
+extern const struct nvkm_chan_func_userd nv04_chan_userd;
+void nv04_chan_ramfc_clear(struct nvkm_chan *);
+void nv04_chan_start(struct nvkm_chan *);
+void nv04_chan_stop(struct nvkm_chan *);
+void nv04_eobj_ramht_del(struct nvkm_chan *, int);
-void gf100_fifo_intr_fault(struct nvkm_fifo *, int);
+int nv10_fifo_chid_nr(struct nvkm_fifo *);
-int gk104_fifo_engine_id(struct nvkm_fifo *, struct nvkm_engine *);
-struct nvkm_engine *gk104_fifo_id_engine(struct nvkm_fifo *, int);
+int nv50_fifo_chid_nr(struct nvkm_fifo *);
+int nv50_fifo_chid_ctor(struct nvkm_fifo *, int);
+void nv50_fifo_init(struct nvkm_fifo *);
+extern const struct nvkm_runl_func nv50_runl;
+int nv50_runl_update(struct nvkm_runl *);
+int nv50_runl_wait(struct nvkm_runl *);
+extern const struct nvkm_engn_func nv50_engn_sw;
+extern const struct nvkm_chan_func_inst nv50_chan_inst;
+extern const struct nvkm_chan_func_userd nv50_chan_userd;
+void nv50_chan_unbind(struct nvkm_chan *);
+void nv50_chan_start(struct nvkm_chan *);
+void nv50_chan_stop(struct nvkm_chan *);
+void nv50_chan_preempt(struct nvkm_chan *);
+int nv50_eobj_ramht_add(struct nvkm_engn *, struct nvkm_object *, struct nvkm_chan *);
+void nv50_eobj_ramht_del(struct nvkm_chan *, int);
+
+extern const struct nvkm_event_func g84_fifo_nonstall;
+extern const struct nvkm_engn_func g84_engn;
+extern const struct nvkm_chan_func g84_chan;
+
+int gf100_fifo_chid_ctor(struct nvkm_fifo *, int);
+int gf100_fifo_runq_nr(struct nvkm_fifo *);
+bool gf100_fifo_intr_pbdma(struct nvkm_fifo *);
+void gf100_fifo_intr_mmu_fault(struct nvkm_fifo *);
+void gf100_fifo_intr_mmu_fault_unit(struct nvkm_fifo *, int);
+void gf100_fifo_intr_sched(struct nvkm_fifo *);
+void gf100_fifo_intr_ctxsw_timeout(struct nvkm_fifo *, u32);
+void gf100_fifo_mmu_fault_recover(struct nvkm_fifo *, struct nvkm_fault_data *);
+extern const struct nvkm_enum gf100_fifo_mmu_fault_access[];
+extern const struct nvkm_event_func gf100_fifo_nonstall;
+bool gf100_runl_preempt_pending(struct nvkm_runl *);
+void gf100_runq_init(struct nvkm_runq *);
+bool gf100_runq_intr(struct nvkm_runq *, struct nvkm_runl *);
+void gf100_engn_mmu_fault_trigger(struct nvkm_engn *);
+bool gf100_engn_mmu_fault_triggered(struct nvkm_engn *);
+extern const struct nvkm_engn_func gf100_engn_sw;
+extern const struct nvkm_chan_func_inst gf100_chan_inst;
+void gf100_chan_userd_clear(struct nvkm_chan *);
+void gf100_chan_preempt(struct nvkm_chan *);
+
+int gk104_fifo_chid_nr(struct nvkm_fifo *);
+int gk104_fifo_runl_ctor(struct nvkm_fifo *);
+void gk104_fifo_init(struct nvkm_fifo *);
+void gk104_fifo_init_pbdmas(struct nvkm_fifo *, u32);
+irqreturn_t gk104_fifo_intr(struct nvkm_inth *);
+void gk104_fifo_intr_runlist(struct nvkm_fifo *);
+void gk104_fifo_intr_chsw(struct nvkm_fifo *);
+void gk104_fifo_intr_bind(struct nvkm_fifo *);
+extern const struct nvkm_fifo_func_mmu_fault gk104_fifo_mmu_fault;
+extern const struct nvkm_enum gk104_fifo_mmu_fault_reason[];
+extern const struct nvkm_enum gk104_fifo_mmu_fault_hubclient[];
+extern const struct nvkm_enum gk104_fifo_mmu_fault_gpcclient[];
+void gk104_runl_insert_chan(struct nvkm_chan *, struct nvkm_memory *, u64);
+void gk104_runl_commit(struct nvkm_runl *, struct nvkm_memory *, u32, int);
+bool gk104_runl_pending(struct nvkm_runl *);
+void gk104_runl_block(struct nvkm_runl *, u32);
+void gk104_runl_allow(struct nvkm_runl *, u32);
+void gk104_runl_fault_clear(struct nvkm_runl *);
+extern const struct nvkm_runq_func gk104_runq;
+void gk104_runq_init(struct nvkm_runq *);
+bool gk104_runq_intr(struct nvkm_runq *, struct nvkm_runl *);
+extern const struct nvkm_bitfield gk104_runq_intr_0_names[];
+bool gk104_runq_idle(struct nvkm_runq *);
+extern const struct nvkm_engn_func gk104_engn;
+bool gk104_engn_chsw(struct nvkm_engn *);
+int gk104_engn_cxid(struct nvkm_engn *, bool *cgid);
+int gk104_ectx_ctor(struct nvkm_engn *, struct nvkm_vctx *);
+extern const struct nvkm_engn_func gk104_engn_ce;
+extern const struct nvkm_chan_func_userd gk104_chan_userd;
+extern const struct nvkm_chan_func_ramfc gk104_chan_ramfc;
+void gk104_chan_bind(struct nvkm_chan *);
+void gk104_chan_bind_inst(struct nvkm_chan *);
+void gk104_chan_unbind(struct nvkm_chan *);
+void gk104_chan_start(struct nvkm_chan *);
+void gk104_chan_stop(struct nvkm_chan *);
+
+int gk110_fifo_chid_ctor(struct nvkm_fifo *, int);
+extern const struct nvkm_runl_func gk110_runl;
+extern const struct nvkm_cgrp_func gk110_cgrp;
+void gk110_runl_insert_cgrp(struct nvkm_cgrp *, struct nvkm_memory *, u64);
+extern const struct nvkm_chan_func gk110_chan;
+void gk110_chan_preempt(struct nvkm_chan *);
+
+extern const struct nvkm_runq_func gk208_runq;
+void gk208_runq_init(struct nvkm_runq *);
+
+void gm107_fifo_intr_mmu_fault_unit(struct nvkm_fifo *, int);
+extern const struct nvkm_fifo_func_mmu_fault gm107_fifo_mmu_fault;
+extern const struct nvkm_runl_func gm107_runl;
+extern const struct nvkm_chan_func gm107_chan;
+
+int gm200_fifo_chid_nr(struct nvkm_fifo *);
+int gm200_fifo_runq_nr(struct nvkm_fifo *);
+
+extern const struct nvkm_enum gv100_fifo_mmu_fault_access[];
+extern const struct nvkm_enum gv100_fifo_mmu_fault_reason[];
+extern const struct nvkm_enum gv100_fifo_mmu_fault_hubclient[];
+extern const struct nvkm_enum gv100_fifo_mmu_fault_gpcclient[];
+void gv100_runl_insert_cgrp(struct nvkm_cgrp *, struct nvkm_memory *, u64);
+void gv100_runl_insert_chan(struct nvkm_chan *, struct nvkm_memory *, u64);
+void gv100_runl_preempt(struct nvkm_runl *);
+extern const struct nvkm_runq_func gv100_runq;
+extern const struct nvkm_engn_func gv100_engn;
+void gv100_ectx_bind(struct nvkm_engn *, struct nvkm_cctx *, struct nvkm_chan *);
+extern const struct nvkm_engn_func gv100_engn_ce;
+int gv100_ectx_ce_ctor(struct nvkm_engn *, struct nvkm_vctx *);
+void gv100_ectx_ce_bind(struct nvkm_engn *, struct nvkm_cctx *, struct nvkm_chan *);
+extern const struct nvkm_chan_func_userd gv100_chan_userd;
+extern const struct nvkm_chan_func_ramfc gv100_chan_ramfc;
+
+void tu102_fifo_intr_ctxsw_timeout_info(struct nvkm_engn *, u32 info);
+extern const struct nvkm_fifo_func_mmu_fault tu102_fifo_mmu_fault;
+
+int ga100_fifo_runl_ctor(struct nvkm_fifo *);
+int ga100_fifo_nonstall_ctor(struct nvkm_fifo *);
+extern const struct nvkm_event_func ga100_fifo_nonstall;
+extern const struct nvkm_runl_func ga100_runl;
+extern const struct nvkm_runq_func ga100_runq;
+extern const struct nvkm_engn_func ga100_engn;
+extern const struct nvkm_engn_func ga100_engn_ce;
+extern const struct nvkm_cgrp_func ga100_cgrp;
+extern const struct nvkm_chan_func ga100_chan;
+
+int nvkm_uchan_new(struct nvkm_fifo *, struct nvkm_cgrp *, const struct nvkm_oclass *,
+ void *argv, u32 argc, struct nvkm_object **);
+int nvkm_ucgrp_new(struct nvkm_fifo *, const struct nvkm_oclass *, void *argv, u32 argc,
+ struct nvkm_object **);
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runl.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runl.c
new file mode 100644
index 0000000..b5836cb
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runl.c
@@ -0,0 +1,430 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "runl.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "priv.h"
+#include "runq.h"
+
+#include <core/gpuobj.h>
+#include <subdev/timer.h>
+#include <subdev/top.h>
+
+struct nvkm_cgrp *
+nvkm_engn_cgrp_get(struct nvkm_engn *engn, unsigned long *pirqflags)
+{
+ struct nvkm_cgrp *cgrp = NULL;
+ struct nvkm_chan *chan;
+ bool cgid;
+ int id;
+
+ id = engn->func->cxid(engn, &cgid);
+ if (id < 0)
+ return NULL;
+
+ if (!cgid) {
+ chan = nvkm_runl_chan_get_chid(engn->runl, id, pirqflags);
+ if (chan)
+ cgrp = chan->cgrp;
+ } else {
+ cgrp = nvkm_runl_cgrp_get_cgid(engn->runl, id, pirqflags);
+ }
+
+ WARN_ON(!cgrp);
+ return cgrp;
+}
+
+static void
+nvkm_runl_rc(struct nvkm_runl *runl)
+{
+ struct nvkm_fifo *fifo = runl->fifo;
+ struct nvkm_cgrp *cgrp, *gtmp;
+ struct nvkm_chan *chan, *ctmp;
+ struct nvkm_engn *engn;
+ unsigned long flags;
+ int rc, state, i;
+ bool reset;
+
+ /* Runlist is blocked before scheduling recovery - fetch count. */
+ BUG_ON(!mutex_is_locked(&runl->mutex));
+ rc = atomic_xchg(&runl->rc_pending, 0);
+ if (!rc)
+ return;
+
+ /* Look for channel groups flagged for RC. */
+ nvkm_runl_foreach_cgrp_safe(cgrp, gtmp, runl) {
+ state = atomic_cmpxchg(&cgrp->rc, NVKM_CGRP_RC_PENDING, NVKM_CGRP_RC_RUNNING);
+ if (state == NVKM_CGRP_RC_PENDING) {
+ /* Disable all channels in them, and remove from runlist. */
+ nvkm_cgrp_foreach_chan_safe(chan, ctmp, cgrp) {
+ nvkm_chan_error(chan, false);
+ nvkm_chan_remove_locked(chan);
+ }
+ }
+ }
+
+ /* On GPUs with runlist preempt, wait for PBDMA(s) servicing runlist to go idle. */
+ if (runl->func->preempt) {
+ for (i = 0; i < runl->runq_nr; i++) {
+ struct nvkm_runq *runq = runl->runq[i];
+
+ if (runq) {
+ nvkm_msec(fifo->engine.subdev.device, 2000,
+ if (runq->func->idle(runq))
+ break;
+ );
+ }
+ }
+ }
+
+ /* Look for engines that are still on flagged channel groups - reset them. */
+ nvkm_runl_foreach_engn_cond(engn, runl, engn->func->cxid) {
+ cgrp = nvkm_engn_cgrp_get(engn, &flags);
+ if (!cgrp) {
+ ENGN_DEBUG(engn, "cxid not valid");
+ continue;
+ }
+
+ reset = atomic_read(&cgrp->rc) == NVKM_CGRP_RC_RUNNING;
+ nvkm_cgrp_put(&cgrp, flags);
+ if (!reset) {
+ ENGN_DEBUG(engn, "cxid not in recovery");
+ continue;
+ }
+
+ ENGN_DEBUG(engn, "resetting...");
+ /*TODO: can we do something less of a potential catastrophe on failure? */
+ WARN_ON(nvkm_engine_reset(engn->engine));
+ }
+
+ /* Submit runlist update, and clear any remaining exception state. */
+ runl->func->update(runl);
+ if (runl->func->fault_clear)
+ runl->func->fault_clear(runl);
+
+ /* Unblock runlist processing. */
+ while (rc--)
+ nvkm_runl_allow(runl);
+ runl->func->wait(runl);
+}
+
+static void
+nvkm_runl_rc_runl(struct nvkm_runl *runl)
+{
+ RUNL_ERROR(runl, "rc scheduled");
+
+ nvkm_runl_block(runl);
+ if (runl->func->preempt)
+ runl->func->preempt(runl);
+
+ atomic_inc(&runl->rc_pending);
+ schedule_work(&runl->work);
+}
+
+void
+nvkm_runl_rc_cgrp(struct nvkm_cgrp *cgrp)
+{
+ if (atomic_cmpxchg(&cgrp->rc, NVKM_CGRP_RC_NONE, NVKM_CGRP_RC_PENDING) != NVKM_CGRP_RC_NONE)
+ return;
+
+ CGRP_ERROR(cgrp, "rc scheduled");
+ nvkm_runl_rc_runl(cgrp->runl);
+}
+
+void
+nvkm_runl_rc_engn(struct nvkm_runl *runl, struct nvkm_engn *engn)
+{
+ struct nvkm_cgrp *cgrp;
+ unsigned long flags;
+
+ /* Lookup channel group currently on engine. */
+ cgrp = nvkm_engn_cgrp_get(engn, &flags);
+ if (!cgrp) {
+ ENGN_DEBUG(engn, "rc skipped, not on channel");
+ return;
+ }
+
+ nvkm_runl_rc_cgrp(cgrp);
+ nvkm_cgrp_put(&cgrp, flags);
+}
+
+static void
+nvkm_runl_work(struct work_struct *work)
+{
+ struct nvkm_runl *runl = container_of(work, typeof(*runl), work);
+
+ mutex_lock(&runl->mutex);
+ nvkm_runl_rc(runl);
+ mutex_unlock(&runl->mutex);
+
+}
+
+struct nvkm_chan *
+nvkm_runl_chan_get_inst(struct nvkm_runl *runl, u64 inst, unsigned long *pirqflags)
+{
+ struct nvkm_chid *chid = runl->chid;
+ struct nvkm_chan *chan;
+ unsigned long flags;
+ int id;
+
+ spin_lock_irqsave(&chid->lock, flags);
+ for_each_set_bit(id, chid->used, chid->nr) {
+ chan = chid->data[id];
+ if (likely(chan)) {
+ if (chan->inst->addr == inst) {
+ spin_lock(&chan->cgrp->lock);
+ *pirqflags = flags;
+ spin_unlock(&chid->lock);
+ return chan;
+ }
+ }
+ }
+ spin_unlock_irqrestore(&chid->lock, flags);
+ return NULL;
+}
+
+struct nvkm_chan *
+nvkm_runl_chan_get_chid(struct nvkm_runl *runl, int id, unsigned long *pirqflags)
+{
+ struct nvkm_chid *chid = runl->chid;
+ struct nvkm_chan *chan;
+ unsigned long flags;
+
+ spin_lock_irqsave(&chid->lock, flags);
+ if (!WARN_ON(id >= chid->nr)) {
+ chan = chid->data[id];
+ if (likely(chan)) {
+ spin_lock(&chan->cgrp->lock);
+ *pirqflags = flags;
+ spin_unlock(&chid->lock);
+ return chan;
+ }
+ }
+ spin_unlock_irqrestore(&chid->lock, flags);
+ return NULL;
+}
+
+struct nvkm_cgrp *
+nvkm_runl_cgrp_get_cgid(struct nvkm_runl *runl, int id, unsigned long *pirqflags)
+{
+ struct nvkm_chid *cgid = runl->cgid;
+ struct nvkm_cgrp *cgrp;
+ unsigned long flags;
+
+ spin_lock_irqsave(&cgid->lock, flags);
+ if (!WARN_ON(id >= cgid->nr)) {
+ cgrp = cgid->data[id];
+ if (likely(cgrp)) {
+ spin_lock(&cgrp->lock);
+ *pirqflags = flags;
+ spin_unlock(&cgid->lock);
+ return cgrp;
+ }
+ }
+ spin_unlock_irqrestore(&cgid->lock, flags);
+ return NULL;
+}
+
+int
+nvkm_runl_preempt_wait(struct nvkm_runl *runl)
+{
+ return nvkm_msec(runl->fifo->engine.subdev.device, runl->fifo->timeout.chan_msec,
+ if (!runl->func->preempt_pending(runl))
+ break;
+
+ nvkm_runl_rc(runl);
+ usleep_range(1, 2);
+ ) < 0 ? -ETIMEDOUT : 0;
+}
+
+bool
+nvkm_runl_update_pending(struct nvkm_runl *runl)
+{
+ if (!runl->func->pending(runl))
+ return false;
+
+ nvkm_runl_rc(runl);
+ return true;
+}
+
+void
+nvkm_runl_update_locked(struct nvkm_runl *runl, bool wait)
+{
+ if (atomic_xchg(&runl->changed, 0) && runl->func->update) {
+ runl->func->update(runl);
+ if (wait)
+ runl->func->wait(runl);
+ }
+}
+
+void
+nvkm_runl_allow(struct nvkm_runl *runl)
+{
+ struct nvkm_fifo *fifo = runl->fifo;
+ unsigned long flags;
+
+ spin_lock_irqsave(&fifo->lock, flags);
+ if (!--runl->blocked) {
+ RUNL_TRACE(runl, "running");
+ runl->func->allow(runl, ~0);
+ }
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+void
+nvkm_runl_block(struct nvkm_runl *runl)
+{
+ struct nvkm_fifo *fifo = runl->fifo;
+ unsigned long flags;
+
+ spin_lock_irqsave(&fifo->lock, flags);
+ if (!runl->blocked++) {
+ RUNL_TRACE(runl, "stopped");
+ runl->func->block(runl, ~0);
+ }
+ spin_unlock_irqrestore(&fifo->lock, flags);
+}
+
+void
+nvkm_runl_fini(struct nvkm_runl *runl)
+{
+ if (runl->func->fini)
+ runl->func->fini(runl);
+
+ flush_work(&runl->work);
+}
+
+void
+nvkm_runl_del(struct nvkm_runl *runl)
+{
+ struct nvkm_engn *engn, *engt;
+
+ nvkm_memory_unref(&runl->mem);
+
+ list_for_each_entry_safe(engn, engt, &runl->engns, head) {
+ list_del(&engn->head);
+ kfree(engn);
+ }
+
+ nvkm_chid_unref(&runl->chid);
+ nvkm_chid_unref(&runl->cgid);
+
+ list_del(&runl->head);
+ mutex_destroy(&runl->mutex);
+ kfree(runl);
+}
+
+struct nvkm_engn *
+nvkm_runl_add(struct nvkm_runl *runl, int engi, const struct nvkm_engn_func *func,
+ enum nvkm_subdev_type type, int inst)
+{
+ struct nvkm_fifo *fifo = runl->fifo;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_engine *engine;
+ struct nvkm_engn *engn;
+
+ engine = nvkm_device_engine(device, type, inst);
+ if (!engine) {
+ RUNL_DEBUG(runl, "engn %d.%d[%s] not found", engi, inst, nvkm_subdev_type[type]);
+ return NULL;
+ }
+
+ if (!(engn = kzalloc(sizeof(*engn), GFP_KERNEL)))
+ return NULL;
+
+ engn->func = func;
+ engn->runl = runl;
+ engn->id = engi;
+ engn->engine = engine;
+ engn->fault = -1;
+ list_add_tail(&engn->head, &runl->engns);
+
+ /* Lookup MMU engine ID for fault handling. */
+ if (device->top)
+ engn->fault = nvkm_top_fault_id(device, engine->subdev.type, engine->subdev.inst);
+
+ if (engn->fault < 0 && fifo->func->mmu_fault) {
+ const struct nvkm_enum *map = fifo->func->mmu_fault->engine;
+
+ while (map->name) {
+ if (map->data2 == engine->subdev.type && map->inst == engine->subdev.inst) {
+ engn->fault = map->value;
+ break;
+ }
+ map++;
+ }
+ }
+
+ return engn;
+}
+
+struct nvkm_runl *
+nvkm_runl_get(struct nvkm_fifo *fifo, int runi, u32 addr)
+{
+ struct nvkm_runl *runl;
+
+ nvkm_runl_foreach(runl, fifo) {
+ if ((runi >= 0 && runl->id == runi) || (runi < 0 && runl->addr == addr))
+ return runl;
+ }
+
+ return NULL;
+}
+
+struct nvkm_runl *
+nvkm_runl_new(struct nvkm_fifo *fifo, int runi, u32 addr, int id_nr)
+{
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
+ struct nvkm_runl *runl;
+ int ret;
+
+ if (!(runl = kzalloc(sizeof(*runl), GFP_KERNEL)))
+ return NULL;
+
+ runl->func = fifo->func->runl;
+ runl->fifo = fifo;
+ runl->id = runi;
+ runl->addr = addr;
+ INIT_LIST_HEAD(&runl->engns);
+ INIT_LIST_HEAD(&runl->cgrps);
+ atomic_set(&runl->changed, 0);
+ mutex_init(&runl->mutex);
+ INIT_WORK(&runl->work, nvkm_runl_work);
+ atomic_set(&runl->rc_triggered, 0);
+ atomic_set(&runl->rc_pending, 0);
+ list_add_tail(&runl->head, &fifo->runls);
+
+ if (!fifo->chid) {
+ if ((ret = nvkm_chid_new(&nvkm_chan_event, subdev, id_nr, 0, id_nr, &runl->cgid)) ||
+ (ret = nvkm_chid_new(&nvkm_chan_event, subdev, id_nr, 0, id_nr, &runl->chid))) {
+ RUNL_ERROR(runl, "cgid/chid: %d", ret);
+ nvkm_runl_del(runl);
+ return NULL;
+ }
+ } else {
+ runl->cgid = nvkm_chid_ref(fifo->cgid);
+ runl->chid = nvkm_chid_ref(fifo->chid);
+ }
+
+ return runl;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runl.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runl.h
new file mode 100644
index 0000000..c93d21b
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runl.h
@@ -0,0 +1,125 @@
+#ifndef __NVKM_RUNL_H__
+#define __NVKM_RUNL_H__
+#include <core/intr.h>
+struct nvkm_cctx;
+struct nvkm_cgrp;
+struct nvkm_chan;
+struct nvkm_memory;
+struct nvkm_object;
+struct nvkm_vctx;
+enum nvkm_subdev_type;
+
+struct nvkm_engn {
+ const struct nvkm_engn_func {
+ bool (*chsw)(struct nvkm_engn *);
+ int (*cxid)(struct nvkm_engn *, bool *cgid);
+ void (*mmu_fault_trigger)(struct nvkm_engn *);
+ bool (*mmu_fault_triggered)(struct nvkm_engn *);
+ int (*ctor)(struct nvkm_engn *, struct nvkm_vctx *);
+ void (*bind)(struct nvkm_engn *, struct nvkm_cctx *, struct nvkm_chan *);
+ int (*ramht_add)(struct nvkm_engn *, struct nvkm_object *, struct nvkm_chan *);
+ void (*ramht_del)(struct nvkm_chan *, int hash);
+ } *func;
+ struct nvkm_runl *runl;
+ int id;
+
+ struct nvkm_engine *engine;
+
+ int fault;
+
+ struct list_head head;
+};
+
+#define ENGN_PRINT(e,l,p,f,a...) \
+ RUNL_PRINT((e)->runl, l, p, "%02d[%8s]:"f, (e)->id, (e)->engine->subdev.name, ##a)
+#define ENGN_DEBUG(e,f,a...) ENGN_PRINT((e), DEBUG, info, " "f"\n", ##a)
+
+struct nvkm_runl {
+ const struct nvkm_runl_func {
+ void (*init)(struct nvkm_runl *);
+ void (*fini)(struct nvkm_runl *);
+ int runqs;
+ u8 size;
+ int (*update)(struct nvkm_runl *);
+ void (*insert_cgrp)(struct nvkm_cgrp *, struct nvkm_memory *, u64 offset);
+ void (*insert_chan)(struct nvkm_chan *, struct nvkm_memory *, u64 offset);
+ void (*commit)(struct nvkm_runl *, struct nvkm_memory *, u32 start, int count);
+ int (*wait)(struct nvkm_runl *);
+ bool (*pending)(struct nvkm_runl *);
+ void (*block)(struct nvkm_runl *, u32 engm);
+ void (*allow)(struct nvkm_runl *, u32 engm);
+ void (*fault_clear)(struct nvkm_runl *);
+ void (*preempt)(struct nvkm_runl *);
+ bool (*preempt_pending)(struct nvkm_runl *);
+ } *func;
+ struct nvkm_fifo *fifo;
+ int id;
+ u32 addr;
+ u32 chan;
+ u16 doorbell;
+
+ struct nvkm_chid *cgid;
+#define NVKM_CHAN_EVENT_ERRORED BIT(0)
+ struct nvkm_chid *chid;
+
+ struct list_head engns;
+
+ struct nvkm_runq *runq[2];
+ int runq_nr;
+
+ struct nvkm_inth inth;
+
+ struct list_head cgrps;
+ int cgrp_nr;
+ int chan_nr;
+ atomic_t changed;
+ struct nvkm_memory *mem;
+ u32 offset;
+ struct mutex mutex;
+
+ int blocked;
+
+ struct work_struct work;
+ atomic_t rc_triggered;
+ atomic_t rc_pending;
+
+ struct list_head head;
+};
+
+struct nvkm_runl *nvkm_runl_new(struct nvkm_fifo *, int runi, u32 addr, int id_nr);
+struct nvkm_runl *nvkm_runl_get(struct nvkm_fifo *, int runi, u32 addr);
+struct nvkm_engn *nvkm_runl_add(struct nvkm_runl *, int engi, const struct nvkm_engn_func *,
+ enum nvkm_subdev_type, int inst);
+void nvkm_runl_del(struct nvkm_runl *);
+void nvkm_runl_fini(struct nvkm_runl *);
+void nvkm_runl_block(struct nvkm_runl *);
+void nvkm_runl_allow(struct nvkm_runl *);
+void nvkm_runl_update_locked(struct nvkm_runl *, bool wait);
+bool nvkm_runl_update_pending(struct nvkm_runl *);
+int nvkm_runl_preempt_wait(struct nvkm_runl *);
+
+void nvkm_runl_rc_engn(struct nvkm_runl *, struct nvkm_engn *);
+void nvkm_runl_rc_cgrp(struct nvkm_cgrp *);
+
+struct nvkm_cgrp *nvkm_runl_cgrp_get_cgid(struct nvkm_runl *, int cgid, unsigned long *irqflags);
+struct nvkm_chan *nvkm_runl_chan_get_chid(struct nvkm_runl *, int chid, unsigned long *irqflags);
+struct nvkm_chan *nvkm_runl_chan_get_inst(struct nvkm_runl *, u64 inst, unsigned long *irqflags);
+
+#define nvkm_runl_find_engn(engn,runl,cond) nvkm_list_find(engn, &(runl)->engns, head, (cond))
+
+#define nvkm_runl_first(fifo) list_first_entry(&(fifo)->runls, struct nvkm_runl, head)
+#define nvkm_runl_foreach(runl,fifo) list_for_each_entry((runl), &(fifo)->runls, head)
+#define nvkm_runl_foreach_cond(runl,fifo,cond) nvkm_list_foreach(runl, &(fifo)->runls, head, (cond))
+#define nvkm_runl_foreach_engn(engn,runl) list_for_each_entry((engn), &(runl)->engns, head)
+#define nvkm_runl_foreach_engn_cond(engn,runl,cond) \
+ nvkm_list_foreach(engn, &(runl)->engns, head, (cond))
+#define nvkm_runl_foreach_cgrp(cgrp,runl) list_for_each_entry((cgrp), &(runl)->cgrps, head)
+#define nvkm_runl_foreach_cgrp_safe(cgrp,gtmp,runl) \
+ list_for_each_entry_safe((cgrp), (gtmp), &(runl)->cgrps, head)
+
+#define RUNL_PRINT(r,l,p,f,a...) \
+ nvkm_printk__(&(r)->fifo->engine.subdev, NV_DBG_##l, p, "%06x:"f, (r)->addr, ##a)
+#define RUNL_ERROR(r,f,a...) RUNL_PRINT((r), ERROR, err, " "f"\n", ##a)
+#define RUNL_DEBUG(r,f,a...) RUNL_PRINT((r), DEBUG, info, " "f"\n", ##a)
+#define RUNL_TRACE(r,f,a...) RUNL_PRINT((r), TRACE, info, " "f"\n", ##a)
+#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runq.c
similarity index 62%
copy from drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
copy to drivers/gpu/drm/nouveau/nvkm/engine/fifo/runq.c
index 217268f..33bcf5f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runq.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2018 Red Hat Inc.
+ * Copyright 2021 Red Hat Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -19,27 +19,27 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include "user.h"
+#include "runq.h"
+#include "priv.h"
-static int
-tu102_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc,
- enum nvkm_object_map *type, u64 *addr, u64 *size)
+void
+nvkm_runq_del(struct nvkm_runq *runq)
{
- struct nvkm_device *device = object->engine->subdev.device;
- *addr = 0xbb0000 + device->func->resource_addr(device, 0);
- *size = 0x010000;
- *type = NVKM_OBJECT_MAP_IO;
- return 0;
+ list_del(&runq->head);
+ kfree(runq);
}
-static const struct nvkm_object_func
-tu102_fifo_user = {
- .map = tu102_fifo_user_map,
-};
-
-int
-tu102_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
- struct nvkm_object **pobject)
+struct nvkm_runq *
+nvkm_runq_new(struct nvkm_fifo *fifo, int pbid)
{
- return nvkm_object_new_(&tu102_fifo_user, oclass, argv, argc, pobject);
+ struct nvkm_runq *runq;
+
+ if (!(runq = kzalloc(sizeof(*runq), GFP_KERNEL)))
+ return NULL;
+
+ runq->func = fifo->func->runq;
+ runq->fifo = fifo;
+ runq->id = pbid;
+ list_add_tail(&runq->head, &fifo->runqs);
+ return runq;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runq.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runq.h
new file mode 100644
index 0000000..2cb4836
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/runq.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVKM_RUNQ_H__
+#define __NVKM_RUNQ_H__
+#include <core/os.h>
+struct nvkm_runl;
+
+struct nvkm_runq {
+ const struct nvkm_runq_func {
+ void (*init)(struct nvkm_runq *);
+ bool (*intr)(struct nvkm_runq *, struct nvkm_runl *);
+ const struct nvkm_bitfield *intr_0_names;
+ bool (*intr_1_ctxnotvalid)(struct nvkm_runq *, int chid);
+ bool (*idle)(struct nvkm_runq *);
+ } *func;
+ struct nvkm_fifo *fifo;
+ int id;
+
+ struct list_head head;
+};
+
+struct nvkm_runq *nvkm_runq_new(struct nvkm_fifo *, int pbid);
+void nvkm_runq_del(struct nvkm_runq *);
+
+#define nvkm_runq_foreach(runq,fifo) list_for_each_entry((runq), &(fifo)->runqs, head)
+#define nvkm_runq_foreach_cond(runq,fifo,cond) nvkm_list_foreach(runq, &(fifo)->runqs, head, (cond))
+
+#define RUNQ_PRINT(r,l,p,f,a...) \
+ nvkm_printk__(&(r)->fifo->engine.subdev, NV_DBG_##l, p, "PBDMA%d:"f, (r)->id, ##a)
+#define RUNQ_ERROR(r,f,a...) RUNQ_PRINT((r), ERROR, err, " "f"\n", ##a)
+#define RUNQ_DEBUG(r,f,a...) RUNQ_PRINT((r), DEBUG, info, " "f"\n", ##a)
+#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c
index 260b197..ea9e151 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.c
@@ -19,46 +19,83 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include "gk104.h"
+#include "priv.h"
#include "cgrp.h"
-#include "changk104.h"
-#include "user.h"
+#include "chan.h"
+#include "runl.h"
-#include <core/client.h>
-#include <core/gpuobj.h>
-#include <subdev/bar.h>
-#include <subdev/fault.h>
-#include <subdev/top.h>
-#include <subdev/timer.h>
-#include <engine/sw.h>
+#include <core/memory.h>
+#include <subdev/mc.h>
+#include <subdev/vfn.h>
#include <nvif/class.h>
-static void
-tu102_fifo_runlist_commit(struct gk104_fifo *fifo, int runl,
- struct nvkm_memory *mem, int nr)
+static u32
+tu102_chan_doorbell_handle(struct nvkm_chan *chan)
{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- u64 addr = nvkm_memory_addr(mem);
- /*XXX: target? */
-
- nvkm_wr32(device, 0x002b00 + (runl * 0x10), lower_32_bits(addr));
- nvkm_wr32(device, 0x002b04 + (runl * 0x10), upper_32_bits(addr));
- nvkm_wr32(device, 0x002b08 + (runl * 0x10), nr);
-
- /*XXX: how to wait? can you even wait? */
+ return (chan->cgrp->runl->id << 16) | chan->id;
}
-static const struct gk104_fifo_runlist_func
-tu102_fifo_runlist = {
+static void
+tu102_chan_start(struct nvkm_chan *chan)
+{
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ gk104_chan_start(chan);
+ nvkm_wr32(device, device->vfn->addr.user + 0x0090, chan->func->doorbell_handle(chan));
+}
+
+static const struct nvkm_chan_func
+tu102_chan = {
+ .inst = &gf100_chan_inst,
+ .userd = &gv100_chan_userd,
+ .ramfc = &gv100_chan_ramfc,
+ .bind = gk104_chan_bind_inst,
+ .unbind = gk104_chan_unbind,
+ .start = tu102_chan_start,
+ .stop = gk104_chan_stop,
+ .preempt = gk110_chan_preempt,
+ .doorbell_handle = tu102_chan_doorbell_handle,
+};
+
+static bool
+tu102_runl_pending(struct nvkm_runl *runl)
+{
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+
+ return nvkm_rd32(device, 0x002b0c + (runl->id * 0x10)) & 0x00008000;
+}
+
+static void
+tu102_runl_commit(struct nvkm_runl *runl, struct nvkm_memory *memory, u32 start, int count)
+{
+ struct nvkm_device *device = runl->fifo->engine.subdev.device;
+ u64 addr = nvkm_memory_addr(memory) + start;
+ /*XXX: target? */
+
+ nvkm_wr32(device, 0x002b00 + (runl->id * 0x10), lower_32_bits(addr));
+ nvkm_wr32(device, 0x002b04 + (runl->id * 0x10), upper_32_bits(addr));
+ nvkm_wr32(device, 0x002b08 + (runl->id * 0x10), count);
+}
+
+static const struct nvkm_runl_func
+tu102_runl = {
+ .runqs = 2,
.size = 16,
- .cgrp = gv100_fifo_runlist_cgrp,
- .chan = gv100_fifo_runlist_chan,
- .commit = tu102_fifo_runlist_commit,
+ .update = nv50_runl_update,
+ .insert_cgrp = gv100_runl_insert_cgrp,
+ .insert_chan = gv100_runl_insert_chan,
+ .commit = tu102_runl_commit,
+ .wait = nv50_runl_wait,
+ .pending = tu102_runl_pending,
+ .block = gk104_runl_block,
+ .allow = gk104_runl_allow,
+ .preempt = gv100_runl_preempt,
+ .preempt_pending = gf100_runl_preempt_pending,
};
static const struct nvkm_enum
-tu102_fifo_fault_engine[] = {
+tu102_fifo_mmu_fault_engine[] = {
{ 0x01, "DISPLAY" },
{ 0x03, "PTP" },
{ 0x06, "PWR_PMU" },
@@ -85,305 +122,82 @@ tu102_fifo_fault_engine[] = {
{}
};
-static void
-tu102_fifo_pbdma_init(struct gk104_fifo *fifo)
-{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- const u32 mask = (1 << fifo->pbdma_nr) - 1;
- /*XXX: this is a bit of a guess at this point in time. */
- nvkm_mask(device, 0xb65000, 0x80000fff, 0x80000000 | mask);
-}
-
-static const struct gk104_fifo_pbdma_func
-tu102_fifo_pbdma = {
- .nr = gm200_fifo_pbdma_nr,
- .init = tu102_fifo_pbdma_init,
- .init_timeout = gk208_fifo_pbdma_init_timeout,
+const struct nvkm_fifo_func_mmu_fault
+tu102_fifo_mmu_fault = {
+ .recover = gf100_fifo_mmu_fault_recover,
+ .access = gv100_fifo_mmu_fault_access,
+ .engine = tu102_fifo_mmu_fault_engine,
+ .reason = gv100_fifo_mmu_fault_reason,
+ .hubclient = gv100_fifo_mmu_fault_hubclient,
+ .gpcclient = gv100_fifo_mmu_fault_gpcclient,
};
-static const struct gk104_fifo_func
-tu102_fifo = {
- .pbdma = &tu102_fifo_pbdma,
- .fault.access = gv100_fifo_fault_access,
- .fault.engine = tu102_fifo_fault_engine,
- .fault.reason = gv100_fifo_fault_reason,
- .fault.hubclient = gv100_fifo_fault_hubclient,
- .fault.gpcclient = gv100_fifo_fault_gpcclient,
- .runlist = &tu102_fifo_runlist,
- .user = {{-1,-1,VOLTA_USERMODE_A }, tu102_fifo_user_new },
- .chan = {{ 0, 0,TURING_CHANNEL_GPFIFO_A}, tu102_fifo_gpfifo_new },
- .cgrp_force = true,
-};
-
-static void
-tu102_fifo_recover_work(struct work_struct *w)
+void
+tu102_fifo_intr_ctxsw_timeout_info(struct nvkm_engn *engn, u32 info)
{
- struct gk104_fifo *fifo = container_of(w, typeof(*fifo), recover.work);
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- struct nvkm_engine *engine;
+ struct nvkm_runl *runl = engn->runl;
+ struct nvkm_cgrp *cgrp;
unsigned long flags;
- u32 engm, runm, todo;
- int engn, runl;
- spin_lock_irqsave(&fifo->base.lock, flags);
- runm = fifo->recover.runm;
- engm = fifo->recover.engm;
- fifo->recover.engm = 0;
- fifo->recover.runm = 0;
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-
- nvkm_mask(device, 0x002630, runm, runm);
-
- for (todo = engm; engn = __ffs(todo), todo; todo &= ~BIT(engn)) {
- if ((engine = fifo->engine[engn].engine)) {
- nvkm_subdev_fini(&engine->subdev, false);
- WARN_ON(nvkm_subdev_init(&engine->subdev));
- }
- }
-
- for (todo = runm; runl = __ffs(todo), todo; todo &= ~BIT(runl))
- gk104_fifo_runlist_update(fifo, runl);
-
- nvkm_mask(device, 0x002630, runm, 0x00000000);
-}
-
-static void tu102_fifo_recover_engn(struct gk104_fifo *fifo, int engn);
-
-static void
-tu102_fifo_recover_runl(struct gk104_fifo *fifo, int runl)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const u32 runm = BIT(runl);
-
- assert_spin_locked(&fifo->base.lock);
- if (fifo->recover.runm & runm)
- return;
- fifo->recover.runm |= runm;
-
- /* Block runlist to prevent channel assignment(s) from changing. */
- nvkm_mask(device, 0x002630, runm, runm);
-
- /* Schedule recovery. */
- nvkm_warn(subdev, "runlist %d: scheduled for recovery\n", runl);
- schedule_work(&fifo->recover.work);
-}
-
-static struct gk104_fifo_chan *
-tu102_fifo_recover_chid(struct gk104_fifo *fifo, int runl, int chid)
-{
- struct gk104_fifo_chan *chan;
- struct nvkm_fifo_cgrp *cgrp;
-
- list_for_each_entry(chan, &fifo->runlist[runl].chan, head) {
- if (chan->base.chid == chid) {
- list_del_init(&chan->head);
- return chan;
- }
- }
-
- list_for_each_entry(cgrp, &fifo->runlist[runl].cgrp, head) {
- if (cgrp->id == chid) {
- chan = list_first_entry(&cgrp->chan, typeof(*chan), head);
- list_del_init(&chan->head);
- if (!--cgrp->chan_nr)
- list_del_init(&cgrp->head);
- return chan;
- }
- }
-
- return NULL;
-}
-
-static void
-tu102_fifo_recover_chan(struct nvkm_fifo *base, int chid)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const u32 stat = nvkm_rd32(device, 0x800004 + (chid * 0x08));
- const u32 runl = (stat & 0x000f0000) >> 16;
- const bool used = (stat & 0x00000001);
- unsigned long engn, engm = fifo->runlist[runl].engm;
- struct gk104_fifo_chan *chan;
-
- assert_spin_locked(&fifo->base.lock);
- if (!used)
+ /* Check that engine hasn't become unstuck since timeout raised. */
+ ENGN_DEBUG(engn, "CTXSW_TIMEOUT %08x", info);
+ if (info & 0xc0000000)
return;
- /* Lookup SW state for channel, and mark it as dead. */
- chan = tu102_fifo_recover_chid(fifo, runl, chid);
- if (chan) {
- chan->killed = true;
- nvkm_fifo_kevent(&fifo->base, chid);
+ /* Determine channel group the engine is stuck on, and schedule recovery. */
+ switch (info & 0x0000c000) {
+ case 0x00004000: /* LOAD */
+ cgrp = nvkm_runl_cgrp_get_cgid(runl, info & 0x3fff0000, &flags);
+ break;
+ case 0x00008000: /* SAVE */
+ case 0x0000c000: /* SWITCH */
+ cgrp = nvkm_runl_cgrp_get_cgid(runl, info & 0x00003fff, &flags);
+ break;
+ default:
+ cgrp = NULL;
+ break;
}
- /* Disable channel. */
- nvkm_wr32(device, 0x800004 + (chid * 0x08), stat | 0x00000800);
- nvkm_warn(subdev, "channel %d: killed\n", chid);
-
- /* Block channel assignments from changing during recovery. */
- tu102_fifo_recover_runl(fifo, runl);
-
- /* Schedule recovery for any engines the channel is on. */
- for_each_set_bit(engn, &engm, fifo->engine_nr) {
- struct gk104_fifo_engine_status status;
-
- gk104_fifo_engine_status(fifo, engn, &status);
- if (!status.chan || status.chan->id != chid)
- continue;
- tu102_fifo_recover_engn(fifo, engn);
+ if (!WARN_ON(!cgrp)) {
+ nvkm_runl_rc_cgrp(cgrp);
+ nvkm_cgrp_put(&cgrp, flags);
}
}
static void
-tu102_fifo_recover_engn(struct gk104_fifo *fifo, int engn)
+tu102_fifo_intr_ctxsw_timeout(struct nvkm_fifo *fifo)
{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const u32 runl = fifo->engine[engn].runl;
- const u32 engm = BIT(engn);
- struct gk104_fifo_engine_status status;
+ struct nvkm_device *device = fifo->engine.subdev.device;
+ struct nvkm_runl *runl;
+ struct nvkm_engn *engn;
+ u32 engm = nvkm_rd32(device, 0x002a30);
+ u32 info;
- assert_spin_locked(&fifo->base.lock);
- if (fifo->recover.engm & engm)
- return;
- fifo->recover.engm |= engm;
-
- /* Block channel assignments from changing during recovery. */
- tu102_fifo_recover_runl(fifo, runl);
-
- /* Determine which channel (if any) is currently on the engine. */
- gk104_fifo_engine_status(fifo, engn, &status);
- if (status.chan) {
- /* The channel is not longer viable, kill it. */
- tu102_fifo_recover_chan(&fifo->base, status.chan->id);
- }
-
- /* Preempt the runlist */
- nvkm_wr32(device, 0x2638, BIT(runl));
-
- /* Schedule recovery. */
- nvkm_warn(subdev, "engine %d: scheduled for recovery\n", engn);
- schedule_work(&fifo->recover.work);
-}
-
-static void
-tu102_fifo_fault(struct nvkm_fifo *base, struct nvkm_fault_data *info)
-{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- const struct nvkm_enum *er, *ee, *ec, *ea;
- struct nvkm_engine *engine = NULL;
- struct nvkm_fifo_chan *chan;
- unsigned long flags;
- const char *en = "";
- char ct[8] = "HUB/";
- int engn;
-
- er = nvkm_enum_find(fifo->func->fault.reason, info->reason);
- ee = nvkm_enum_find(fifo->func->fault.engine, info->engine);
- if (info->hub) {
- ec = nvkm_enum_find(fifo->func->fault.hubclient, info->client);
- } else {
- ec = nvkm_enum_find(fifo->func->fault.gpcclient, info->client);
- snprintf(ct, sizeof(ct), "GPC%d/", info->gpc);
- }
- ea = nvkm_enum_find(fifo->func->fault.access, info->access);
-
- if (ee && ee->data2) {
- switch (ee->data2) {
- case NVKM_SUBDEV_BAR:
- nvkm_bar_bar1_reset(device);
- break;
- case NVKM_SUBDEV_INSTMEM:
- nvkm_bar_bar2_reset(device);
- break;
- case NVKM_ENGINE_IFB:
- nvkm_mask(device, 0x001718, 0x00000000, 0x00000000);
- break;
- default:
- engine = nvkm_device_engine(device, ee->data2, 0);
- break;
+ nvkm_runl_foreach(runl, fifo) {
+ nvkm_runl_foreach_engn_cond(engn, runl, engm & BIT(engn->id)) {
+ info = nvkm_rd32(device, 0x003200 + (engn->id * 4));
+ tu102_fifo_intr_ctxsw_timeout_info(engn, info);
}
}
- if (ee == NULL) {
- struct nvkm_subdev *subdev = nvkm_top_fault(device, info->engine);
- if (subdev) {
- if (subdev->func == &nvkm_engine)
- engine = container_of(subdev, typeof(*engine), subdev);
- en = engine->subdev.name;
- }
- } else {
- en = ee->name;
- }
-
- spin_lock_irqsave(&fifo->base.lock, flags);
- chan = nvkm_fifo_chan_inst_locked(&fifo->base, info->inst);
-
- nvkm_error(subdev,
- "fault %02x [%s] at %016llx engine %02x [%s] client %02x "
- "[%s%s] reason %02x [%s] on channel %d [%010llx %s]\n",
- info->access, ea ? ea->name : "", info->addr,
- info->engine, ee ? ee->name : en,
- info->client, ct, ec ? ec->name : "",
- info->reason, er ? er->name : "", chan ? chan->chid : -1,
- info->inst, chan ? chan->object.client->name : "unknown");
-
- /* Kill the channel that caused the fault. */
- if (chan)
- tu102_fifo_recover_chan(&fifo->base, chan->chid);
-
- /* Channel recovery will probably have already done this for the
- * correct engine(s), but just in case we can't find the channel
- * information...
- */
- for (engn = 0; engn < fifo->engine_nr && engine; engn++) {
- if (fifo->engine[engn].engine == engine) {
- tu102_fifo_recover_engn(fifo, engn);
- break;
- }
- }
-
- spin_unlock_irqrestore(&fifo->base.lock, flags);
+ nvkm_wr32(device, 0x002a30, engm);
}
static void
-tu102_fifo_intr_ctxsw_timeout(struct gk104_fifo *fifo)
+tu102_fifo_intr_sched(struct nvkm_fifo *fifo)
{
- struct nvkm_device *device = fifo->base.engine.subdev.device;
- unsigned long flags, engm;
- u32 engn;
-
- spin_lock_irqsave(&fifo->base.lock, flags);
-
- engm = nvkm_rd32(device, 0x2a30);
- nvkm_wr32(device, 0x2a30, engm);
-
- for_each_set_bit(engn, &engm, 32)
- tu102_fifo_recover_engn(fifo, engn);
-
- spin_unlock_irqrestore(&fifo->base.lock, flags);
-}
-
-static void
-tu102_fifo_intr_sched(struct gk104_fifo *fifo)
-{
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
- struct nvkm_device *device = subdev->device;
- u32 intr = nvkm_rd32(device, 0x00254c);
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
+ u32 intr = nvkm_rd32(subdev->device, 0x00254c);
u32 code = intr & 0x000000ff;
nvkm_error(subdev, "SCHED_ERROR %02x\n", code);
}
-static void
-tu102_fifo_intr(struct nvkm_fifo *base)
+static irqreturn_t
+tu102_fifo_intr(struct nvkm_inth *inth)
{
- struct gk104_fifo *fifo = gk104_fifo(base);
- struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
+ struct nvkm_fifo *fifo = container_of(inth, typeof(*fifo), engine.subdev.inth);
+ struct nvkm_subdev *subdev = &fifo->engine.subdev;
struct nvkm_device *device = subdev->device;
u32 mask = nvkm_rd32(device, 0x002140);
u32 stat = nvkm_rd32(device, 0x002100) & mask;
@@ -412,17 +226,8 @@ tu102_fifo_intr(struct nvkm_fifo *base)
}
if (stat & 0x20000000) {
- u32 mask = nvkm_rd32(device, 0x0025a0);
-
- while (mask) {
- u32 unit = __ffs(mask);
-
- gk104_fifo_intr_pbdma_0(fifo, unit);
- gk104_fifo_intr_pbdma_1(fifo, unit);
- nvkm_wr32(device, 0x0025a0, (1 << unit));
- mask &= ~(1 << unit);
- }
- stat &= ~0x20000000;
+ if (gf100_fifo_intr_pbdma(fifo))
+ stat &= ~0x20000000;
}
if (stat & 0x40000000) {
@@ -432,46 +237,50 @@ tu102_fifo_intr(struct nvkm_fifo *base)
if (stat & 0x80000000) {
nvkm_wr32(device, 0x002100, 0x80000000);
- gk104_fifo_intr_engine(fifo);
+ nvkm_event_ntfy(&fifo->nonstall.event, 0, NVKM_FIFO_NONSTALL_EVENT);
stat &= ~0x80000000;
}
if (stat) {
nvkm_error(subdev, "INTR %08x\n", stat);
+ spin_lock(&fifo->lock);
nvkm_mask(device, 0x002140, stat, 0x00000000);
+ spin_unlock(&fifo->lock);
nvkm_wr32(device, 0x002100, stat);
}
+
+ return IRQ_HANDLED;
+}
+
+static void
+tu102_fifo_init_pbdmas(struct nvkm_fifo *fifo, u32 mask)
+{
+ /* Not directly related to PBDMAs, but, enables doorbell to function. */
+ nvkm_mask(fifo->engine.subdev.device, 0xb65000, 0x80000000, 0x80000000);
}
static const struct nvkm_fifo_func
-tu102_fifo_ = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .info = gk104_fifo_info,
+tu102_fifo = {
+ .chid_nr = gm200_fifo_chid_nr,
+ .chid_ctor = gk110_fifo_chid_ctor,
+ .runq_nr = gm200_fifo_runq_nr,
+ .runl_ctor = gk104_fifo_runl_ctor,
.init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
+ .init_pbdmas = tu102_fifo_init_pbdmas,
.intr = tu102_fifo_intr,
- .fault = tu102_fifo_fault,
- .engine_id = gk104_fifo_engine_id,
- .id_engine = gk104_fifo_id_engine,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
- .recover_chan = tu102_fifo_recover_chan,
- .class_get = gk104_fifo_class_get,
- .class_new = gk104_fifo_class_new,
+ .mmu_fault = &tu102_fifo_mmu_fault,
+ .nonstall = &gf100_fifo_nonstall,
+ .runl = &tu102_runl,
+ .runq = &gv100_runq,
+ .engn = &gv100_engn,
+ .engn_ce = &gv100_engn_ce,
+ .cgrp = {{ 0, 0, KEPLER_CHANNEL_GROUP_A }, &gk110_cgrp, .force = true },
+ .chan = {{ 0, 0, TURING_CHANNEL_GPFIFO_A }, &tu102_chan },
};
int
tu102_fifo_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fifo **pfifo)
{
- struct gk104_fifo *fifo;
-
- if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
- return -ENOMEM;
- fifo->func = &tu102_fifo;
- INIT_WORK(&fifo->recover.work, tu102_fifo_recover_work);
- *pfifo = &fifo->base;
-
- return nvkm_fifo_ctor(&tu102_fifo_, device, type, inst, 4096, &fifo->base);
+ return nvkm_fifo_new_(&tu102_fifo, device, type, inst, pfifo);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ucgrp.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ucgrp.c
new file mode 100644
index 0000000..52c594d
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/ucgrp.c
@@ -0,0 +1,125 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#define nvkm_ucgrp(p) container_of((p), struct nvkm_ucgrp, object)
+#include "priv.h"
+#include "cgrp.h"
+#include "runl.h"
+
+#include <subdev/mmu.h>
+
+#include <nvif/if0021.h>
+
+struct nvkm_ucgrp {
+ struct nvkm_object object;
+ struct nvkm_cgrp *cgrp;
+};
+
+static int
+nvkm_ucgrp_chan_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
+ struct nvkm_object **pobject)
+{
+ struct nvkm_cgrp *cgrp = nvkm_ucgrp(oclass->parent)->cgrp;
+
+ return nvkm_uchan_new(cgrp->runl->fifo, cgrp, oclass, argv, argc, pobject);
+}
+
+static int
+nvkm_ucgrp_sclass(struct nvkm_object *object, int index, struct nvkm_oclass *oclass)
+{
+ struct nvkm_cgrp *cgrp = nvkm_ucgrp(object)->cgrp;
+ struct nvkm_fifo *fifo = cgrp->runl->fifo;
+ const struct nvkm_fifo_func_chan *chan = &fifo->func->chan;
+ int c = 0;
+
+ /* *_CHANNEL_GPFIFO_* */
+ if (chan->user.oclass) {
+ if (c++ == index) {
+ oclass->base = chan->user;
+ oclass->ctor = nvkm_ucgrp_chan_new;
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+
+static void *
+nvkm_ucgrp_dtor(struct nvkm_object *object)
+{
+ struct nvkm_ucgrp *ucgrp = nvkm_ucgrp(object);
+
+ nvkm_cgrp_unref(&ucgrp->cgrp);
+ return ucgrp;
+}
+
+static const struct nvkm_object_func
+nvkm_ucgrp = {
+ .dtor = nvkm_ucgrp_dtor,
+ .sclass = nvkm_ucgrp_sclass,
+};
+
+int
+nvkm_ucgrp_new(struct nvkm_fifo *fifo, const struct nvkm_oclass *oclass, void *argv, u32 argc,
+ struct nvkm_object **pobject)
+{
+ union nvif_cgrp_args *args = argv;
+ struct nvkm_runl *runl;
+ struct nvkm_vmm *vmm;
+ struct nvkm_ucgrp *ucgrp;
+ int ret;
+
+ if (argc < sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+ argc -= sizeof(args->v0);
+
+ if (args->v0.namelen != argc)
+ return -EINVAL;
+
+ /* Lookup objects referenced in args. */
+ runl = nvkm_runl_get(fifo, args->v0.runlist, 0);
+ if (!runl)
+ return -EINVAL;
+
+ vmm = nvkm_uvmm_search(oclass->client, args->v0.vmm);
+ if (IS_ERR(vmm))
+ return PTR_ERR(vmm);
+
+ /* Allocate channel group. */
+ if (!(ucgrp = kzalloc(sizeof(*ucgrp), GFP_KERNEL))) {
+ ret = -ENOMEM;
+ goto done;
+ }
+
+ nvkm_object_ctor(&nvkm_ucgrp, oclass, &ucgrp->object);
+ *pobject = &ucgrp->object;
+
+ ret = nvkm_cgrp_new(runl, args->v0.name, vmm, true, &ucgrp->cgrp);
+ if (ret)
+ goto done;
+
+ /* Return channel group info to caller. */
+ args->v0.cgid = ucgrp->cgrp->id;
+
+done:
+ nvkm_vmm_unref(&vmm);
+ return ret;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/uchan.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/uchan.c
new file mode 100644
index 0000000..1dac95a
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/uchan.c
@@ -0,0 +1,409 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#define nvkm_uchan(p) container_of((p), struct nvkm_uchan, object)
+#include "priv.h"
+#include "cgrp.h"
+#include "chan.h"
+#include "chid.h"
+#include "runl.h"
+
+#include <core/gpuobj.h>
+#include <core/oproxy.h>
+#include <subdev/mmu.h>
+#include <engine/dma.h>
+
+#include <nvif/if0020.h>
+
+struct nvkm_uchan {
+ struct nvkm_object object;
+ struct nvkm_chan *chan;
+};
+
+static int
+nvkm_uchan_uevent(struct nvkm_object *object, void *argv, u32 argc, struct nvkm_uevent *uevent)
+{
+ struct nvkm_chan *chan = nvkm_uchan(object)->chan;
+ struct nvkm_runl *runl = chan->cgrp->runl;
+ union nvif_chan_event_args *args = argv;
+
+ if (!uevent)
+ return 0;
+ if (argc != sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+
+ switch (args->v0.type) {
+ case NVIF_CHAN_EVENT_V0_NON_STALL_INTR:
+ return nvkm_uevent_add(uevent, &runl->fifo->nonstall.event, 0,
+ NVKM_FIFO_NONSTALL_EVENT, NULL);
+ case NVIF_CHAN_EVENT_V0_KILLED:
+ return nvkm_uevent_add(uevent, &runl->chid->event, chan->id,
+ NVKM_CHAN_EVENT_ERRORED, NULL);
+ default:
+ break;
+ }
+
+ return -ENOSYS;
+}
+
+struct nvkm_uobj {
+ struct nvkm_oproxy oproxy;
+ struct nvkm_chan *chan;
+ struct nvkm_cctx *cctx;
+ int hash;
+};
+
+static int
+nvkm_uchan_object_fini_1(struct nvkm_oproxy *oproxy, bool suspend)
+{
+ struct nvkm_uobj *uobj = container_of(oproxy, typeof(*uobj), oproxy);
+ struct nvkm_chan *chan = uobj->chan;
+ struct nvkm_cctx *cctx = uobj->cctx;
+ struct nvkm_ectx *ectx = cctx->vctx->ectx;
+
+ if (!ectx->object)
+ return 0;
+
+ /* Unbind engine context from channel, if no longer required. */
+ if (refcount_dec_and_mutex_lock(&cctx->uses, &chan->cgrp->mutex)) {
+ nvkm_chan_cctx_bind(chan, ectx->engn, NULL);
+
+ if (refcount_dec_and_test(&ectx->uses))
+ nvkm_object_fini(ectx->object, false);
+ mutex_unlock(&chan->cgrp->mutex);
+ }
+
+ return 0;
+}
+
+static int
+nvkm_uchan_object_init_0(struct nvkm_oproxy *oproxy)
+{
+ struct nvkm_uobj *uobj = container_of(oproxy, typeof(*uobj), oproxy);
+ struct nvkm_chan *chan = uobj->chan;
+ struct nvkm_cctx *cctx = uobj->cctx;
+ struct nvkm_ectx *ectx = cctx->vctx->ectx;
+ int ret = 0;
+
+ if (!ectx->object)
+ return 0;
+
+ /* Bind engine context to channel, if it hasn't been already. */
+ if (!refcount_inc_not_zero(&cctx->uses)) {
+ mutex_lock(&chan->cgrp->mutex);
+ if (!refcount_inc_not_zero(&cctx->uses)) {
+ if (!refcount_inc_not_zero(&ectx->uses)) {
+ ret = nvkm_object_init(ectx->object);
+ if (ret == 0)
+ refcount_set(&ectx->uses, 1);
+ }
+
+ if (ret == 0) {
+ nvkm_chan_cctx_bind(chan, ectx->engn, cctx);
+ refcount_set(&cctx->uses, 1);
+ }
+ }
+ mutex_unlock(&chan->cgrp->mutex);
+ }
+
+ return ret;
+}
+
+static void
+nvkm_uchan_object_dtor(struct nvkm_oproxy *oproxy)
+{
+ struct nvkm_uobj *uobj = container_of(oproxy, typeof(*uobj), oproxy);
+ struct nvkm_engn *engn;
+
+ if (!uobj->cctx)
+ return;
+
+ engn = uobj->cctx->vctx->ectx->engn;
+ if (engn->func->ramht_del)
+ engn->func->ramht_del(uobj->chan, uobj->hash);
+
+ nvkm_chan_cctx_put(uobj->chan, &uobj->cctx);
+}
+
+static const struct nvkm_oproxy_func
+nvkm_uchan_object = {
+ .dtor[1] = nvkm_uchan_object_dtor,
+ .init[0] = nvkm_uchan_object_init_0,
+ .fini[1] = nvkm_uchan_object_fini_1,
+};
+
+static int
+nvkm_uchan_object_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
+ struct nvkm_object **pobject)
+{
+ struct nvkm_chan *chan = nvkm_uchan(oclass->parent)->chan;
+ struct nvkm_cgrp *cgrp = chan->cgrp;
+ struct nvkm_engn *engn;
+ struct nvkm_uobj *uobj;
+ int ret;
+
+ /* Lookup host engine state for target engine. */
+ engn = nvkm_runl_find_engn(engn, cgrp->runl, engn->engine == oclass->engine);
+ if (WARN_ON(!engn))
+ return -EINVAL;
+
+ /* Allocate SW object. */
+ if (!(uobj = kzalloc(sizeof(*uobj), GFP_KERNEL)))
+ return -ENOMEM;
+
+ nvkm_oproxy_ctor(&nvkm_uchan_object, oclass, &uobj->oproxy);
+ uobj->chan = chan;
+ *pobject = &uobj->oproxy.base;
+
+ /* Ref. channel context for target engine.*/
+ ret = nvkm_chan_cctx_get(chan, engn, &uobj->cctx, oclass->client);
+ if (ret)
+ return ret;
+
+ /* Allocate HW object. */
+ ret = oclass->base.ctor(&(const struct nvkm_oclass) {
+ .base = oclass->base,
+ .engn = oclass->engn,
+ .handle = oclass->handle,
+ .object = oclass->object,
+ .client = oclass->client,
+ .parent = uobj->cctx->vctx->ectx->object ?: oclass->parent,
+ .engine = engn->engine,
+ }, argv, argc, &uobj->oproxy.object);
+ if (ret)
+ return ret;
+
+ if (engn->func->ramht_add) {
+ uobj->hash = engn->func->ramht_add(engn, uobj->oproxy.object, uobj->chan);
+ if (uobj->hash < 0)
+ return uobj->hash;
+ }
+
+ return 0;
+}
+
+static int
+nvkm_uchan_sclass(struct nvkm_object *object, int index, struct nvkm_oclass *oclass)
+{
+ struct nvkm_chan *chan = nvkm_uchan(object)->chan;
+ struct nvkm_engn *engn;
+ int ret, runq = 0;
+
+ nvkm_runl_foreach_engn(engn, chan->cgrp->runl) {
+ struct nvkm_engine *engine = engn->engine;
+ int c = 0;
+
+ /* Each runqueue, on runlists with multiple, has its own LCE. */
+ if (engn->runl->func->runqs) {
+ if (engine->subdev.type == NVKM_ENGINE_CE) {
+ if (chan->runq != runq++)
+ continue;
+ }
+ }
+
+ oclass->engine = engine;
+ oclass->base.oclass = 0;
+
+ if (engine->func->fifo.sclass) {
+ ret = engine->func->fifo.sclass(oclass, index);
+ if (oclass->base.oclass) {
+ if (!oclass->base.ctor)
+ oclass->base.ctor = nvkm_object_new;
+ oclass->ctor = nvkm_uchan_object_new;
+ return 0;
+ }
+
+ index -= ret;
+ continue;
+ }
+
+ while (engine->func->sclass[c].oclass) {
+ if (c++ == index) {
+ oclass->base = engine->func->sclass[index];
+ if (!oclass->base.ctor)
+ oclass->base.ctor = nvkm_object_new;
+ oclass->ctor = nvkm_uchan_object_new;
+ return 0;
+ }
+ }
+
+ index -= c;
+ }
+
+ return -EINVAL;
+}
+
+static int
+nvkm_uchan_map(struct nvkm_object *object, void *argv, u32 argc,
+ enum nvkm_object_map *type, u64 *addr, u64 *size)
+{
+ struct nvkm_chan *chan = nvkm_uchan(object)->chan;
+ struct nvkm_device *device = chan->cgrp->runl->fifo->engine.subdev.device;
+
+ if (chan->func->userd->bar < 0)
+ return -ENOSYS;
+
+ *type = NVKM_OBJECT_MAP_IO;
+ *addr = device->func->resource_addr(device, chan->func->userd->bar) +
+ chan->func->userd->base + chan->userd.base;
+ *size = chan->func->userd->size;
+ return 0;
+}
+
+static int
+nvkm_uchan_fini(struct nvkm_object *object, bool suspend)
+{
+ struct nvkm_chan *chan = nvkm_uchan(object)->chan;
+
+ nvkm_chan_block(chan);
+ nvkm_chan_remove(chan, true);
+
+ if (chan->func->unbind)
+ chan->func->unbind(chan);
+
+ return 0;
+}
+
+static int
+nvkm_uchan_init(struct nvkm_object *object)
+{
+ struct nvkm_chan *chan = nvkm_uchan(object)->chan;
+
+ if (atomic_read(&chan->errored))
+ return 0;
+
+ if (chan->func->bind)
+ chan->func->bind(chan);
+
+ nvkm_chan_allow(chan);
+ nvkm_chan_insert(chan);
+ return 0;
+}
+
+static void *
+nvkm_uchan_dtor(struct nvkm_object *object)
+{
+ struct nvkm_uchan *uchan = nvkm_uchan(object);
+
+ nvkm_chan_del(&uchan->chan);
+ return uchan;
+}
+
+static const struct nvkm_object_func
+nvkm_uchan = {
+ .dtor = nvkm_uchan_dtor,
+ .init = nvkm_uchan_init,
+ .fini = nvkm_uchan_fini,
+ .map = nvkm_uchan_map,
+ .sclass = nvkm_uchan_sclass,
+ .uevent = nvkm_uchan_uevent,
+};
+
+int
+nvkm_uchan_new(struct nvkm_fifo *fifo, struct nvkm_cgrp *cgrp, const struct nvkm_oclass *oclass,
+ void *argv, u32 argc, struct nvkm_object **pobject)
+{
+ union nvif_chan_args *args = argv;
+ struct nvkm_runl *runl;
+ struct nvkm_vmm *vmm = NULL;
+ struct nvkm_dmaobj *ctxdma = NULL;
+ struct nvkm_memory *userd = NULL;
+ struct nvkm_uchan *uchan;
+ struct nvkm_chan *chan;
+ int ret;
+
+ if (argc < sizeof(args->v0) || args->v0.version != 0)
+ return -ENOSYS;
+ argc -= sizeof(args->v0);
+
+ if (args->v0.namelen != argc)
+ return -EINVAL;
+
+ /* Lookup objects referenced in args. */
+ runl = nvkm_runl_get(fifo, args->v0.runlist, 0);
+ if (!runl)
+ return -EINVAL;
+
+ if (args->v0.vmm) {
+ vmm = nvkm_uvmm_search(oclass->client, args->v0.vmm);
+ if (IS_ERR(vmm))
+ return PTR_ERR(vmm);
+ }
+
+ if (args->v0.ctxdma) {
+ ctxdma = nvkm_dmaobj_search(oclass->client, args->v0.ctxdma);
+ if (IS_ERR(ctxdma)) {
+ ret = PTR_ERR(ctxdma);
+ goto done;
+ }
+ }
+
+ if (args->v0.huserd) {
+ userd = nvkm_umem_search(oclass->client, args->v0.huserd);
+ if (IS_ERR(userd)) {
+ ret = PTR_ERR(userd);
+ userd = NULL;
+ goto done;
+ }
+ }
+
+ /* Allocate channel. */
+ if (!(uchan = kzalloc(sizeof(*uchan), GFP_KERNEL))) {
+ ret = -ENOMEM;
+ goto done;
+ }
+
+ nvkm_object_ctor(&nvkm_uchan, oclass, &uchan->object);
+ *pobject = &uchan->object;
+
+ ret = nvkm_chan_new_(fifo->func->chan.func, runl, args->v0.runq, cgrp, args->v0.name,
+ args->v0.priv != 0, args->v0.devm, vmm, ctxdma, args->v0.offset,
+ args->v0.length, userd, args->v0.ouserd, &uchan->chan);
+ if (ret)
+ goto done;
+
+ chan = uchan->chan;
+
+ /* Return channel info to caller. */
+ if (chan->func->doorbell_handle)
+ args->v0.token = chan->func->doorbell_handle(chan);
+ else
+ args->v0.token = ~0;
+
+ args->v0.chid = chan->id;
+
+ switch (nvkm_memory_target(chan->inst->memory)) {
+ case NVKM_MEM_TARGET_INST: args->v0.aper = NVIF_CHAN_V0_INST_APER_INST; break;
+ case NVKM_MEM_TARGET_VRAM: args->v0.aper = NVIF_CHAN_V0_INST_APER_VRAM; break;
+ case NVKM_MEM_TARGET_HOST: args->v0.aper = NVIF_CHAN_V0_INST_APER_HOST; break;
+ case NVKM_MEM_TARGET_NCOH: args->v0.aper = NVIF_CHAN_V0_INST_APER_NCOH; break;
+ default:
+ WARN_ON(1);
+ ret = -EFAULT;
+ break;
+ }
+
+ args->v0.inst = nvkm_memory_addr(chan->inst->memory);
+done:
+ nvkm_memory_unref(&userd);
+ nvkm_vmm_unref(&vmm);
+ return ret;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/user.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/user.h
deleted file mode 100644
index 54a3a30..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/user.h
+++ /dev/null
@@ -1,8 +0,0 @@
-#ifndef __NVKM_FIFO_USER_H__
-#define __NVKM_FIFO_USER_H__
-#include "priv.h"
-int gv100_fifo_user_new(const struct nvkm_oclass *, void *, u32,
- struct nvkm_object **);
-int tu102_fifo_user_new(const struct nvkm_oclass *, void *, u32,
- struct nvkm_object **);
-#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/Kbuild b/drivers/gpu/drm/nouveau/nvkm/engine/gr/Kbuild
index 558c86f..b5418f0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/Kbuild
@@ -40,6 +40,7 @@
nvkm-y += nvkm/engine/gr/gp10b.o
nvkm-y += nvkm/engine/gr/gv100.o
nvkm-y += nvkm/engine/gr/tu102.o
+nvkm-y += nvkm/engine/gr/ga102.o
nvkm-y += nvkm/engine/gr/ctxnv40.o
nvkm-y += nvkm/engine/gr/ctxnv50.o
@@ -63,3 +64,4 @@
nvkm-y += nvkm/engine/gr/ctxgp107.o
nvkm-y += nvkm/engine/gr/ctxgv100.o
nvkm-y += nvkm/engine/gr/ctxtu102.o
+nvkm-y += nvkm/engine/gr/ctxga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c
index 61759f5..71b824e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c
@@ -136,6 +136,17 @@ nvkm_gr_oneinit(struct nvkm_engine *engine)
}
static int
+nvkm_gr_reset(struct nvkm_engine *engine)
+{
+ struct nvkm_gr *gr = nvkm_gr(engine);
+
+ if (gr->func->reset)
+ return gr->func->reset(gr);
+
+ return -ENOSYS;
+}
+
+static int
nvkm_gr_init(struct nvkm_engine *engine)
{
struct nvkm_gr *gr = nvkm_gr(engine);
@@ -166,6 +177,7 @@ nvkm_gr = {
.oneinit = nvkm_gr_oneinit,
.init = nvkm_gr_init,
.fini = nvkm_gr_fini,
+ .reset = nvkm_gr_reset,
.intr = nvkm_gr_intr,
.tile = nvkm_gr_tile,
.chsw_load = nvkm_gr_chsw_load,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxga102.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxga102.c
new file mode 100644
index 0000000..11461adf
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxga102.c
@@ -0,0 +1,77 @@
+/*
+ * Copyright 2019 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "ctxgf100.h"
+
+static void
+ga102_grctx_generate_sm_id(struct gf100_gr *gr, int gpc, int tpc, int sm)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ tpc = gv100_gr_nonpes_aware_tpc(gr, gpc, tpc);
+
+ nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x608), sm);
+}
+
+static void
+ga102_grctx_generate_unkn(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_mask(device, 0x41980c, 0x00000010, 0x00000010);
+ nvkm_mask(device, 0x41be08, 0x00000004, 0x00000004);
+}
+
+static void
+ga102_grctx_generate_r419ea8(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x419ea8, nvkm_rd32(device, 0x504728) | 0x08000000);
+}
+
+const struct gf100_grctx_func
+ga102_grctx = {
+ .main = gf100_grctx_generate_main,
+ .unkn = ga102_grctx_generate_unkn,
+ .bundle = gm107_grctx_generate_bundle,
+ .bundle_size = 0x3000,
+ .bundle_min_gpm_fifo_depth = 0x180,
+ .bundle_token_limit = 0x1140,
+ .pagepool = gp100_grctx_generate_pagepool,
+ .pagepool_size = 0x20000,
+ .attrib_cb_size = gp102_grctx_generate_attrib_cb_size,
+ .attrib_cb = gv100_grctx_generate_attrib_cb,
+ .attrib = gv100_grctx_generate_attrib,
+ .attrib_nr_max = 0x800,
+ .attrib_nr = 0x4a1,
+ .alpha_nr_max = 0xc00,
+ .alpha_nr = 0x800,
+ .unknown_size = 0x80000,
+ .unknown = tu102_grctx_generate_unknown,
+ .gfxp_nr = 0xd28,
+ .sm_id = ga102_grctx_generate_sm_id,
+ .skip_pd_num_tpc_per_gpc = true,
+ .rop_mapping = gv100_grctx_generate_rop_mapping,
+ .r406500 = gm200_grctx_generate_r406500,
+ .r400088 = gv100_grctx_generate_r400088,
+ .r419ea8 = ga102_grctx_generate_r419ea8,
+};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c
index 2979157..cb390e0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c
@@ -26,6 +26,7 @@
#include <subdev/fb.h>
#include <subdev/mc.h>
#include <subdev/timer.h>
+#include <engine/fifo.h>
/*******************************************************************************
* PGRAPH context register lists
@@ -990,43 +991,16 @@ gf100_grctx_pack_tpc[] = {
* PGRAPH context implementation
******************************************************************************/
-int
-gf100_grctx_mmio_data(struct gf100_grctx *info, u32 size, u32 align, bool priv)
-{
- if (info->data) {
- info->buffer[info->buffer_nr] = round_up(info->addr, align);
- info->addr = info->buffer[info->buffer_nr] + size;
- info->data->size = size;
- info->data->align = align;
- info->data->priv = priv;
- info->data++;
- return info->buffer_nr++;
- }
- return -1;
-}
-
void
-gf100_grctx_mmio_item(struct gf100_grctx *info, u32 addr, u32 data,
- int shift, int buffer)
+gf100_grctx_patch_wr32(struct gf100_gr_chan *chan, u32 addr, u32 data)
{
- struct nvkm_device *device = info->gr->base.engine.subdev.device;
- if (info->data) {
- if (shift >= 0) {
- info->mmio->addr = addr;
- info->mmio->data = data;
- info->mmio->shift = shift;
- info->mmio->buffer = buffer;
- if (buffer >= 0)
- data |= info->buffer[buffer] >> shift;
- info->mmio++;
- } else
- return;
- } else {
- if (buffer >= 0)
- return;
+ if (unlikely(!chan->mmio)) {
+ nvkm_wr32(chan->gr->base.engine.subdev.device, addr, data);
+ return;
}
- nvkm_wr32(device, addr, data);
+ nvkm_wo32(chan->mmio, chan->mmio_nr++ * 4, addr);
+ nvkm_wo32(chan->mmio, chan->mmio_nr++ * 4, data);
}
void
@@ -1037,56 +1011,60 @@ gf100_grctx_generate_r419cb8(struct gf100_gr *gr)
}
void
-gf100_grctx_generate_bundle(struct gf100_grctx *info)
+gf100_grctx_generate_bundle(struct gf100_gr_chan *chan, u64 addr, u32 size)
{
- const struct gf100_grctx_func *grctx = info->gr->func->grctx;
- const int s = 8;
- const int b = mmio_vram(info, grctx->bundle_size, (1 << s), true);
- mmio_refn(info, 0x408004, 0x00000000, s, b);
- mmio_wr32(info, 0x408008, 0x80000000 | (grctx->bundle_size >> s));
- mmio_refn(info, 0x418808, 0x00000000, s, b);
- mmio_wr32(info, 0x41880c, 0x80000000 | (grctx->bundle_size >> s));
+ gf100_grctx_patch_wr32(chan, 0x408004, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x408008, 0x80000000 | (size >> 8));
+ gf100_grctx_patch_wr32(chan, 0x418808, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x41880c, 0x80000000 | (size >> 8));
}
void
-gf100_grctx_generate_pagepool(struct gf100_grctx *info)
+gf100_grctx_generate_pagepool(struct gf100_gr_chan *chan, u64 addr)
{
- const struct gf100_grctx_func *grctx = info->gr->func->grctx;
- const int s = 8;
- const int b = mmio_vram(info, grctx->pagepool_size, (1 << s), true);
- mmio_refn(info, 0x40800c, 0x00000000, s, b);
- mmio_wr32(info, 0x408010, 0x80000000);
- mmio_refn(info, 0x419004, 0x00000000, s, b);
- mmio_wr32(info, 0x419008, 0x00000000);
+ gf100_grctx_patch_wr32(chan, 0x40800c, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x408010, 0x80000000);
+ gf100_grctx_patch_wr32(chan, 0x419004, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x419008, 0x00000000);
}
void
-gf100_grctx_generate_attrib(struct gf100_grctx *info)
+gf100_grctx_generate_attrib(struct gf100_gr_chan *chan)
{
- struct gf100_gr *gr = info->gr;
+ struct gf100_gr *gr = chan->gr;
const struct gf100_grctx_func *grctx = gr->func->grctx;
const u32 attrib = grctx->attrib_nr;
- const u32 size = 0x20 * (grctx->attrib_nr_max + grctx->alpha_nr_max);
- const int s = 12;
- const int b = mmio_vram(info, size * gr->tpc_total, (1 << s), false);
int gpc, tpc;
u32 bo = 0;
- mmio_refn(info, 0x418810, 0x80000000, s, b);
- mmio_refn(info, 0x419848, 0x10000000, s, b);
- mmio_wr32(info, 0x405830, (attrib << 16));
+ gf100_grctx_patch_wr32(chan, 0x405830, (attrib << 16));
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
for (tpc = 0; tpc < gr->tpc_nr[gpc]; tpc++) {
const u32 o = TPC_UNIT(gpc, tpc, 0x0520);
- mmio_skip(info, o, (attrib << 16) | ++bo);
- mmio_wr32(info, o, (attrib << 16) | --bo);
+
+ gf100_grctx_patch_wr32(chan, o, (attrib << 16) | bo);
bo += grctx->attrib_nr_max;
}
}
}
void
+gf100_grctx_generate_attrib_cb(struct gf100_gr_chan *chan, u64 addr, u32 size)
+{
+ gf100_grctx_patch_wr32(chan, 0x418810, 0x80000000 | addr >> 12);
+ gf100_grctx_patch_wr32(chan, 0x419848, 0x10000000 | addr >> 12);
+}
+
+u32
+gf100_grctx_generate_attrib_cb_size(struct gf100_gr *gr)
+{
+ const struct gf100_grctx_func *grctx = gr->func->grctx;
+
+ return 0x20 * (grctx->attrib_nr_max + grctx->alpha_nr_max) * gr->tpc_total;
+}
+
+void
gf100_grctx_generate_unkn(struct gf100_gr *gr)
{
}
@@ -1361,8 +1339,9 @@ gf100_grctx_generate_floorsweep(struct gf100_gr *gr)
}
void
-gf100_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
+gf100_grctx_generate_main(struct gf100_gr_chan *chan)
{
+ struct gf100_gr *gr = chan->gr;
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
u32 idle_timeout;
@@ -1380,15 +1359,23 @@ gf100_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
gf100_gr_mmio(gr, gr->sw_ctx);
}
+ if (gr->func->init_419bd8)
+ gr->func->init_419bd8(gr);
+ if (grctx->r419ea8)
+ grctx->r419ea8(gr);
+
gf100_gr_wait_idle(gr);
idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
- grctx->pagepool(info);
- grctx->bundle(info);
- grctx->attrib(info);
+ grctx->pagepool(chan, chan->pagepool->addr);
+ grctx->bundle(chan, chan->bundle_cb->addr, grctx->bundle_size);
+ grctx->attrib_cb(chan, chan->attrib_cb->addr, grctx->attrib_cb_size(gr));
+ grctx->attrib(chan);
if (grctx->patch_ltc)
- grctx->patch_ltc(info);
+ grctx->patch_ltc(chan);
+ if (grctx->unknown_size)
+ grctx->unknown(chan, chan->unknown->addr, grctx->unknown_size);
grctx->unkn(gr);
gf100_grctx_generate_floorsweep(gr);
@@ -1396,12 +1383,23 @@ gf100_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
gf100_gr_wait_idle(gr);
if (grctx->r400088) grctx->r400088(gr, false);
+
if (gr->bundle)
gf100_gr_icmd(gr, gr->bundle);
else
gf100_gr_icmd(gr, grctx->icmd);
- if (grctx->sw_veid_bundle_init)
+
+ if (gr->bundle_veid)
+ gf100_gr_icmd(gr, gr->bundle_veid);
+ else
gf100_gr_icmd(gr, grctx->sw_veid_bundle_init);
+
+ if (gr->bundle64)
+ gf100_gr_icmd(gr, gr->bundle64);
+ else
+ if (grctx->sw_bundle64_init)
+ gf100_gr_icmd(gr, grctx->sw_bundle64_init);
+
if (grctx->r400088) grctx->r400088(gr, true);
nvkm_wr32(device, 0x404154, idle_timeout);
@@ -1428,21 +1426,20 @@ gf100_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
grctx->r408840(gr);
if (grctx->r419c0c)
grctx->r419c0c(gr);
+
+ gf100_gr_wait_idle(gr);
}
#define CB_RESERVED 0x80000
int
-gf100_grctx_generate(struct gf100_gr *gr)
+gf100_grctx_generate(struct gf100_gr *gr, struct gf100_gr_chan *chan, struct nvkm_gpuobj *inst)
{
const struct gf100_grctx_func *grctx = gr->func->grctx;
struct nvkm_subdev *subdev = &gr->base.engine.subdev;
struct nvkm_device *device = subdev->device;
- struct nvkm_memory *inst = NULL;
struct nvkm_memory *data = NULL;
- struct nvkm_vmm *vmm = NULL;
struct nvkm_vma *ctx = NULL;
- struct gf100_grctx info;
int ret, i;
u64 addr;
@@ -1457,72 +1454,47 @@ gf100_grctx_generate(struct gf100_gr *gr)
grctx->unkn88c(gr, true);
/* Reset FECS. */
- nvkm_wr32(device, 0x409614, 0x00000070);
- nvkm_usec(device, 10, NVKM_DELAY);
- nvkm_mask(device, 0x409614, 0x00000700, 0x00000700);
- nvkm_usec(device, 10, NVKM_DELAY);
- nvkm_rd32(device, 0x409614);
+ gr->func->fecs.reset(gr);
if (grctx->unkn88c)
grctx->unkn88c(gr, false);
/* NV_PGRAPH_FE_PWR_MODE_AUTO. */
nvkm_wr32(device, 0x404170, 0x00000010);
+ nvkm_msec(device, 2000,
+ if (!(nvkm_rd32(device, 0x404170) & 0x00000010))
+ break;
+ );
/* Init SCC RAM. */
nvkm_wr32(device, 0x40802c, 0x00000001);
- /* Allocate memory to for a "channel", which we'll use to generate
- * the default context values.
- */
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST,
- 0x1000, 0x1000, true, &inst);
- if (ret)
- goto done;
-
- ret = nvkm_vmm_new(device, 0, 0, NULL, 0, NULL, "grctx", &vmm);
- if (ret)
- goto done;
-
- vmm->debug = subdev->debug;
-
- ret = nvkm_vmm_join(vmm, inst);
- if (ret)
- goto done;
-
+ /* Allocate memory to store context, and dummy global context buffers. */
ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST,
CB_RESERVED + gr->size, 0, true, &data);
if (ret)
goto done;
- ret = nvkm_vmm_get(vmm, 0, nvkm_memory_size(data), &ctx);
+ ret = nvkm_vmm_get(chan->vmm, 0, nvkm_memory_size(data), &ctx);
if (ret)
goto done;
- ret = nvkm_memory_map(data, 0, vmm, ctx, NULL, 0);
+ ret = nvkm_memory_map(data, 0, chan->vmm, ctx, NULL, 0);
if (ret)
goto done;
-
/* Setup context pointer. */
nvkm_kmap(inst);
nvkm_wo32(inst, 0x0210, lower_32_bits(ctx->addr + CB_RESERVED) | 4);
nvkm_wo32(inst, 0x0214, upper_32_bits(ctx->addr + CB_RESERVED));
nvkm_done(inst);
- /* Setup default state for mmio list construction. */
- info.gr = gr;
- info.data = gr->mmio_data;
- info.mmio = gr->mmio_list;
- info.addr = ctx->addr;
- info.buffer_nr = 0;
-
/* Make channel current. */
- addr = nvkm_memory_addr(inst) >> 12;
+ addr = inst->addr >> 12;
if (gr->firmware) {
ret = gf100_gr_fecs_bind_pointer(gr, 0x80000000 | addr);
if (ret)
- goto done;
+ goto done_inst;
nvkm_kmap(data);
nvkm_wo32(data, 0x1c, 1);
@@ -1540,19 +1512,27 @@ gf100_grctx_generate(struct gf100_gr *gr)
);
}
- grctx->main(gr, &info);
+ grctx->main(chan);
- /* Trigger a context unload by unsetting the "next channel valid" bit
- * and faking a context switch interrupt.
- */
- nvkm_mask(device, 0x409b04, 0x80000000, 0x00000000);
- nvkm_wr32(device, 0x409000, 0x00000100);
- if (nvkm_msec(device, 2000,
- if (!(nvkm_rd32(device, 0x409b00) & 0x80000000))
- break;
- ) < 0) {
- ret = -EBUSY;
- goto done;
+ if (!gr->firmware) {
+ /* Trigger a context unload by unsetting the "next channel valid" bit
+ * and faking a context switch interrupt.
+ */
+ nvkm_mask(device, 0x409b04, 0x80000000, 0x00000000);
+ nvkm_wr32(device, 0x409000, 0x00000100);
+ if (nvkm_msec(device, 2000,
+ if (!(nvkm_rd32(device, 0x409b00) & 0x80000000))
+ break;
+ ) < 0) {
+ ret = -EBUSY;
+ goto done_inst;
+ }
+ } else {
+ ret = gf100_gr_fecs_wfi_golden_save(gr, 0x80000000 | addr);
+ if (ret)
+ goto done_inst;
+
+ nvkm_mask(device, 0x409b00, 0x80000000, 0x00000000);
}
gr->data = kmalloc(gr->size, GFP_KERNEL);
@@ -1566,12 +1546,14 @@ gf100_grctx_generate(struct gf100_gr *gr)
ret = -ENOMEM;
}
+done_inst:
+ nvkm_kmap(inst);
+ nvkm_wo32(inst, 0x0210, 0);
+ nvkm_wo32(inst, 0x0214, 0);
+ nvkm_done(inst);
done:
- nvkm_vmm_put(vmm, &ctx);
+ nvkm_vmm_put(chan->vmm, &ctx);
nvkm_memory_unref(&data);
- nvkm_vmm_part(vmm, inst);
- nvkm_vmm_unref(&vmm);
- nvkm_memory_unref(&inst);
return ret;
}
@@ -1590,6 +1572,8 @@ gf100_grctx = {
.bundle_size = 0x1800,
.pagepool = gf100_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf100_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h
index 32bbddc..00dbeda 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h
@@ -3,27 +3,12 @@
#define __NVKM_GRCTX_NVC0_H__
#include "gf100.h"
-struct gf100_grctx {
- struct gf100_gr *gr;
- struct gf100_gr_data *data;
- struct gf100_gr_mmio *mmio;
- int buffer_nr;
- u64 buffer[4];
- u64 addr;
-};
-
-int gf100_grctx_mmio_data(struct gf100_grctx *, u32 size, u32 align, bool priv);
-void gf100_grctx_mmio_item(struct gf100_grctx *, u32 addr, u32 data, int s, int);
-
-#define mmio_vram(a,b,c,d) gf100_grctx_mmio_data((a), (b), (c), (d))
-#define mmio_refn(a,b,c,d,e) gf100_grctx_mmio_item((a), (b), (c), (d), (e))
-#define mmio_skip(a,b,c) mmio_refn((a), (b), (c), -1, -1)
-#define mmio_wr32(a,b,c) mmio_refn((a), (b), (c), 0, -1)
+void gf100_grctx_patch_wr32(struct gf100_gr_chan *, u32 addr, u32 data);
struct gf100_grctx_func {
void (*unkn88c)(struct gf100_gr *, bool on);
/* main context generation function */
- void (*main)(struct gf100_gr *, struct gf100_grctx *);
+ void (*main)(struct gf100_gr_chan *);
/* context-specific modify-on-first-load list generation function */
void (*unkn)(struct gf100_gr *);
/* mmio context data */
@@ -37,23 +22,29 @@ struct gf100_grctx_func {
const struct gf100_gr_pack *icmd;
const struct gf100_gr_pack *mthd;
const struct gf100_gr_pack *sw_veid_bundle_init;
+ const struct gf100_gr_pack *sw_bundle64_init;
/* bundle circular buffer */
- void (*bundle)(struct gf100_grctx *);
+ void (*bundle)(struct gf100_gr_chan *, u64 addr, u32 size);
u32 bundle_size;
u32 bundle_min_gpm_fifo_depth;
u32 bundle_token_limit;
/* pagepool */
- void (*pagepool)(struct gf100_grctx *);
+ void (*pagepool)(struct gf100_gr_chan *, u64 addr);
u32 pagepool_size;
/* attribute(/alpha) circular buffer */
- void (*attrib)(struct gf100_grctx *);
+ u32 (*attrib_cb_size)(struct gf100_gr *);
+ void (*attrib_cb)(struct gf100_gr_chan *, u64 addr, u32 size);
+ void (*attrib)(struct gf100_gr_chan *);
u32 attrib_nr_max;
u32 attrib_nr;
u32 alpha_nr_max;
u32 alpha_nr;
u32 gfxp_nr;
+ /* some other context buffer */
+ void (*unknown)(struct gf100_gr_chan *, u64 addr, u32 size);
+ u32 unknown_size;
/* other patch buffer stuff */
- void (*patch_ltc)(struct gf100_grctx *);
+ void (*patch_ltc)(struct gf100_gr_chan *);
/* floorsweeping */
void (*sm_id)(struct gf100_gr *, int gpc, int tpc, int sm);
void (*tpc_nr)(struct gf100_gr *, int gpc);
@@ -78,14 +69,17 @@ struct gf100_grctx_func {
void (*r419a3c)(struct gf100_gr *);
void (*r408840)(struct gf100_gr *);
void (*r419c0c)(struct gf100_gr *);
+ void (*r419ea8)(struct gf100_gr *);
};
extern const struct gf100_grctx_func gf100_grctx;
-int gf100_grctx_generate(struct gf100_gr *);
-void gf100_grctx_generate_main(struct gf100_gr *, struct gf100_grctx *);
-void gf100_grctx_generate_bundle(struct gf100_grctx *);
-void gf100_grctx_generate_pagepool(struct gf100_grctx *);
-void gf100_grctx_generate_attrib(struct gf100_grctx *);
+int gf100_grctx_generate(struct gf100_gr *, struct gf100_gr_chan *, struct nvkm_gpuobj *inst);
+void gf100_grctx_generate_main(struct gf100_gr_chan *);
+void gf100_grctx_generate_pagepool(struct gf100_gr_chan *, u64);
+void gf100_grctx_generate_bundle(struct gf100_gr_chan *, u64, u32);
+u32 gf100_grctx_generate_attrib_cb_size(struct gf100_gr *);
+void gf100_grctx_generate_attrib_cb(struct gf100_gr_chan *, u64, u32);
+void gf100_grctx_generate_attrib(struct gf100_gr_chan *);
void gf100_grctx_generate_unkn(struct gf100_gr *);
void gf100_grctx_generate_floorsweep(struct gf100_gr *);
void gf100_grctx_generate_sm_id(struct gf100_gr *, int, int, int);
@@ -97,14 +91,14 @@ void gf100_grctx_generate_max_ways_evict(struct gf100_gr *);
void gf100_grctx_generate_r419cb8(struct gf100_gr *);
extern const struct gf100_grctx_func gf108_grctx;
-void gf108_grctx_generate_attrib(struct gf100_grctx *);
+void gf108_grctx_generate_attrib(struct gf100_gr_chan *);
void gf108_grctx_generate_unkn(struct gf100_gr *);
extern const struct gf100_grctx_func gf104_grctx;
extern const struct gf100_grctx_func gf110_grctx;
extern const struct gf100_grctx_func gf117_grctx;
-void gf117_grctx_generate_attrib(struct gf100_grctx *);
+void gf117_grctx_generate_attrib(struct gf100_gr_chan *);
void gf117_grctx_generate_rop_mapping(struct gf100_gr *);
void gf117_grctx_generate_dist_skip_table(struct gf100_gr *);
@@ -115,9 +109,9 @@ void gk104_grctx_generate_alpha_beta_tables(struct gf100_gr *);
void gk104_grctx_generate_gpc_tpc_nr(struct gf100_gr *);
extern const struct gf100_grctx_func gk20a_grctx;
-void gk104_grctx_generate_bundle(struct gf100_grctx *);
-void gk104_grctx_generate_pagepool(struct gf100_grctx *);
-void gk104_grctx_generate_patch_ltc(struct gf100_grctx *);
+void gk104_grctx_generate_pagepool(struct gf100_gr_chan *, u64);
+void gk104_grctx_generate_bundle(struct gf100_gr_chan *, u64, u32);
+void gk104_grctx_generate_patch_ltc(struct gf100_gr_chan *);
void gk104_grctx_generate_unkn(struct gf100_gr *);
void gk104_grctx_generate_r418800(struct gf100_gr *);
@@ -128,9 +122,10 @@ extern const struct gf100_grctx_func gk110b_grctx;
extern const struct gf100_grctx_func gk208_grctx;
extern const struct gf100_grctx_func gm107_grctx;
-void gm107_grctx_generate_bundle(struct gf100_grctx *);
-void gm107_grctx_generate_pagepool(struct gf100_grctx *);
-void gm107_grctx_generate_attrib(struct gf100_grctx *);
+void gm107_grctx_generate_pagepool(struct gf100_gr_chan *, u64);
+void gm107_grctx_generate_bundle(struct gf100_gr_chan *, u64, u32);
+void gm107_grctx_generate_attrib_cb(struct gf100_gr_chan *, u64, u32);
+void gm107_grctx_generate_attrib(struct gf100_gr_chan *);
void gm107_grctx_generate_sm_id(struct gf100_gr *, int, int, int);
extern const struct gf100_grctx_func gm200_grctx;
@@ -143,11 +138,13 @@ void gm200_grctx_generate_r419a3c(struct gf100_gr *);
extern const struct gf100_grctx_func gm20b_grctx;
extern const struct gf100_grctx_func gp100_grctx;
-void gp100_grctx_generate_pagepool(struct gf100_grctx *);
+void gp100_grctx_generate_pagepool(struct gf100_gr_chan *, u64);
+void gp100_grctx_generate_attrib_cb(struct gf100_gr_chan *, u64, u32);
void gp100_grctx_generate_smid_config(struct gf100_gr *);
extern const struct gf100_grctx_func gp102_grctx;
-void gp102_grctx_generate_attrib(struct gf100_grctx *);
+u32 gp102_grctx_generate_attrib_cb_size(struct gf100_gr *);
+void gp102_grctx_generate_attrib(struct gf100_gr_chan *);
extern const struct gf100_grctx_func gp104_grctx;
@@ -158,11 +155,15 @@ extern const struct gf100_grctx_func gv100_grctx;
extern const struct gf100_grctx_func tu102_grctx;
void gv100_grctx_unkn88c(struct gf100_gr *, bool);
void gv100_grctx_generate_unkn(struct gf100_gr *);
-extern const struct gf100_gr_init gv100_grctx_init_sw_veid_bundle_init_0[];
-void gv100_grctx_generate_attrib(struct gf100_grctx *);
+void gv100_grctx_generate_attrib_cb(struct gf100_gr_chan *, u64, u32);
+void gv100_grctx_generate_attrib(struct gf100_gr_chan *);
void gv100_grctx_generate_rop_mapping(struct gf100_gr *);
void gv100_grctx_generate_r400088(struct gf100_gr *, bool);
+void tu102_grctx_generate_unknown(struct gf100_gr_chan *, u64, u32);
+
+extern const struct gf100_grctx_func ga102_grctx;
+
/* context init value lists */
extern const struct gf100_gr_pack gf100_grctx_pack_icmd[];
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf104.c
index 7a0564b..ba63a3b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf104.c
@@ -94,6 +94,8 @@ gf104_grctx = {
.bundle_size = 0x1800,
.pagepool = gf100_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf100_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf108.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf108.c
index dda2c32..0bc2eab6a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf108.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf108.c
@@ -733,25 +733,20 @@ gf108_grctx_pack_tpc[] = {
******************************************************************************/
void
-gf108_grctx_generate_attrib(struct gf100_grctx *info)
+gf108_grctx_generate_attrib(struct gf100_gr_chan *chan)
{
- struct gf100_gr *gr = info->gr;
+ struct gf100_gr *gr = chan->gr;
const struct gf100_grctx_func *grctx = gr->func->grctx;
const u32 alpha = grctx->alpha_nr;
const u32 beta = grctx->attrib_nr;
- const u32 size = 0x20 * (grctx->attrib_nr_max + grctx->alpha_nr_max);
- const int s = 12;
- const int b = mmio_vram(info, size * gr->tpc_total, (1 << s), false);
const int timeslice_mode = 1;
const int max_batches = 0xffff;
u32 bo = 0;
u32 ao = bo + grctx->attrib_nr_max * gr->tpc_total;
int gpc, tpc;
- mmio_refn(info, 0x418810, 0x80000000, s, b);
- mmio_refn(info, 0x419848, 0x10000000, s, b);
- mmio_wr32(info, 0x405830, (beta << 16) | alpha);
- mmio_wr32(info, 0x4064c4, ((alpha / 4) << 16) | max_batches);
+ gf100_grctx_patch_wr32(chan, 0x405830, (beta << 16) | alpha);
+ gf100_grctx_patch_wr32(chan, 0x4064c4, ((alpha / 4) << 16) | max_batches);
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
for (tpc = 0; tpc < gr->tpc_nr[gpc]; tpc++) {
@@ -759,10 +754,10 @@ gf108_grctx_generate_attrib(struct gf100_grctx *info)
const u32 b = beta;
const u32 t = timeslice_mode;
const u32 o = TPC_UNIT(gpc, tpc, 0x500);
- mmio_skip(info, o + 0x20, (t << 28) | (b << 16) | ++bo);
- mmio_wr32(info, o + 0x20, (t << 28) | (b << 16) | --bo);
+
+ gf100_grctx_patch_wr32(chan, o + 0x20, (t << 28) | (b << 16) | bo);
bo += grctx->attrib_nr_max;
- mmio_wr32(info, o + 0x44, (a << 16) | ao);
+ gf100_grctx_patch_wr32(chan, o + 0x44, (a << 16) | ao);
ao += grctx->alpha_nr_max;
}
}
@@ -795,6 +790,8 @@ gf108_grctx = {
.bundle_size = 0x1800,
.pagepool = gf100_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf108_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf110.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf110.c
index f5cca5e..64b723b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf110.c
@@ -342,6 +342,8 @@ gf110_grctx = {
.bundle_size = 0x1800,
.pagepool = gf100_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf100_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c
index 276c282..e34c5da 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c
@@ -241,38 +241,34 @@ gf117_grctx_generate_rop_mapping(struct gf100_gr *gr)
}
void
-gf117_grctx_generate_attrib(struct gf100_grctx *info)
+gf117_grctx_generate_attrib(struct gf100_gr_chan *chan)
{
- struct gf100_gr *gr = info->gr;
+ struct gf100_gr *gr = chan->gr;
const struct gf100_grctx_func *grctx = gr->func->grctx;
const u32 alpha = grctx->alpha_nr;
const u32 beta = grctx->attrib_nr;
- const u32 size = 0x20 * (grctx->attrib_nr_max + grctx->alpha_nr_max);
- const int s = 12;
- const int b = mmio_vram(info, size * gr->tpc_total, (1 << s), false);
const int timeslice_mode = 1;
const int max_batches = 0xffff;
u32 bo = 0;
u32 ao = bo + grctx->attrib_nr_max * gr->tpc_total;
int gpc, ppc;
- mmio_refn(info, 0x418810, 0x80000000, s, b);
- mmio_refn(info, 0x419848, 0x10000000, s, b);
- mmio_wr32(info, 0x405830, (beta << 16) | alpha);
- mmio_wr32(info, 0x4064c4, ((alpha / 4) << 16) | max_batches);
+ gf100_grctx_patch_wr32(chan, 0x405830, (beta << 16) | alpha);
+ gf100_grctx_patch_wr32(chan, 0x4064c4, ((alpha / 4) << 16) | max_batches);
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++) {
+ for (ppc = 0; ppc < gr->func->ppc_nr; ppc++) {
const u32 a = alpha * gr->ppc_tpc_nr[gpc][ppc];
const u32 b = beta * gr->ppc_tpc_nr[gpc][ppc];
const u32 t = timeslice_mode;
const u32 o = PPC_UNIT(gpc, ppc, 0);
+
if (!(gr->ppc_mask[gpc] & (1 << ppc)))
continue;
- mmio_skip(info, o + 0xc0, (t << 28) | (b << 16) | ++bo);
- mmio_wr32(info, o + 0xc0, (t << 28) | (b << 16) | --bo);
+
+ gf100_grctx_patch_wr32(chan, o + 0xc0, (t << 28) | (b << 16) | bo);
bo += grctx->attrib_nr_max * gr->ppc_tpc_nr[gpc][ppc];
- mmio_wr32(info, o + 0xe4, (a << 16) | ao);
+ gf100_grctx_patch_wr32(chan, o + 0xe4, (a << 16) | ao);
ao += grctx->alpha_nr_max * gr->ppc_tpc_nr[gpc][ppc];
}
}
@@ -294,6 +290,8 @@ gf117_grctx = {
.bundle_size = 0x1800,
.pagepool = gf100_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf117_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf119.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf119.c
index 0cfe463..426ad1b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf119.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf119.c
@@ -510,6 +510,8 @@ gf119_grctx = {
.bundle_size = 0x1800,
.pagepool = gf100_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf108_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c
index 304e9d2..94233d0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c
@@ -861,43 +861,33 @@ gk104_grctx_generate_r418800(struct gf100_gr *gr)
}
void
-gk104_grctx_generate_patch_ltc(struct gf100_grctx *info)
+gk104_grctx_generate_patch_ltc(struct gf100_gr_chan *chan)
{
- struct nvkm_device *device = info->gr->base.engine.subdev.device;
+ struct nvkm_device *device = chan->gr->base.engine.subdev.device;
u32 data0 = nvkm_rd32(device, 0x17e91c);
u32 data1 = nvkm_rd32(device, 0x17e920);
+
/*XXX: Figure out how to modify this correctly! */
- mmio_wr32(info, 0x17e91c, data0);
- mmio_wr32(info, 0x17e920, data1);
+ gf100_grctx_patch_wr32(chan, 0x17e91c, data0);
+ gf100_grctx_patch_wr32(chan, 0x17e920, data1);
}
void
-gk104_grctx_generate_bundle(struct gf100_grctx *info)
+gk104_grctx_generate_bundle(struct gf100_gr_chan *chan, u64 addr, u32 size)
{
- const struct gf100_grctx_func *grctx = info->gr->func->grctx;
- const u32 state_limit = min(grctx->bundle_min_gpm_fifo_depth,
- grctx->bundle_size / 0x20);
+ const struct gf100_grctx_func *grctx = chan->gr->func->grctx;
+ const u32 state_limit = min(grctx->bundle_min_gpm_fifo_depth, size / 0x20);
const u32 token_limit = grctx->bundle_token_limit;
- const int s = 8;
- const int b = mmio_vram(info, grctx->bundle_size, (1 << s), true);
- mmio_refn(info, 0x408004, 0x00000000, s, b);
- mmio_wr32(info, 0x408008, 0x80000000 | (grctx->bundle_size >> s));
- mmio_refn(info, 0x418808, 0x00000000, s, b);
- mmio_wr32(info, 0x41880c, 0x80000000 | (grctx->bundle_size >> s));
- mmio_wr32(info, 0x4064c8, (state_limit << 16) | token_limit);
+
+ gf100_grctx_generate_bundle(chan, addr, size);
+ gf100_grctx_patch_wr32(chan, 0x4064c8, (state_limit << 16) | token_limit);
}
void
-gk104_grctx_generate_pagepool(struct gf100_grctx *info)
+gk104_grctx_generate_pagepool(struct gf100_gr_chan *chan, u64 addr)
{
- const struct gf100_grctx_func *grctx = info->gr->func->grctx;
- const int s = 8;
- const int b = mmio_vram(info, grctx->pagepool_size, (1 << s), true);
- mmio_refn(info, 0x40800c, 0x00000000, s, b);
- mmio_wr32(info, 0x408010, 0x80000000);
- mmio_refn(info, 0x419004, 0x00000000, s, b);
- mmio_wr32(info, 0x419008, 0x00000000);
- mmio_wr32(info, 0x4064cc, 0x80000000);
+ gf100_grctx_generate_pagepool(chan, addr);
+ gf100_grctx_patch_wr32(chan, 0x4064cc, 0x80000000);
}
void
@@ -991,6 +981,8 @@ gk104_grctx = {
.bundle_token_limit = 0x600,
.pagepool = gk104_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf117_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110.c
index 86547cfc..4391458 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110.c
@@ -838,6 +838,8 @@ gk110_grctx = {
.bundle_token_limit = 0x7c0,
.pagepool = gk104_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf117_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110b.c
index ebb947b..7b9a34f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110b.c
@@ -87,6 +87,8 @@ gk110b_grctx = {
.bundle_token_limit = 0x600,
.pagepool = gk104_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf117_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk208.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk208.c
index 4d40512..c78d07a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk208.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk208.c
@@ -553,6 +553,8 @@ gk208_grctx = {
.bundle_token_limit = 0x200,
.pagepool = gk104_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf117_grctx_generate_attrib,
.attrib_nr_max = 0x324,
.attrib_nr = 0x218,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c
index c0d36bc..ac5fdcb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c
@@ -25,8 +25,9 @@
#include <subdev/mc.h>
static void
-gk20a_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
+gk20a_grctx_generate_main(struct gf100_gr_chan *chan)
{
+ struct gf100_gr *gr = chan->gr;
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
u32 idle_timeout;
@@ -38,7 +39,8 @@ gk20a_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
- grctx->attrib(info);
+ grctx->attrib_cb(chan, chan->attrib_cb->addr, grctx->attrib_cb_size(gr));
+ grctx->attrib(chan);
grctx->unkn(gr);
@@ -60,8 +62,8 @@ gk20a_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
gf100_gr_wait_idle(gr);
gf100_gr_icmd(gr, gr->bundle);
- grctx->pagepool(info);
- grctx->bundle(info);
+ grctx->pagepool(chan, chan->pagepool->addr);
+ grctx->bundle(chan, chan->bundle_cb->addr, grctx->bundle_size);
}
const struct gf100_grctx_func
@@ -74,6 +76,8 @@ gk20a_grctx = {
.bundle_token_limit = 0x100,
.pagepool = gk104_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gf100_grctx_generate_attrib_cb,
.attrib = gf117_grctx_generate_attrib,
.attrib_nr_max = 0x240,
.attrib_nr = 0x240,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c
index 0b3964e6..beac66e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c
@@ -876,75 +876,70 @@ gm107_grctx_generate_r419e00(struct gf100_gr *gr)
}
void
-gm107_grctx_generate_bundle(struct gf100_grctx *info)
+gm107_grctx_generate_bundle(struct gf100_gr_chan *chan, u64 addr, u32 size)
{
- const struct gf100_grctx_func *grctx = info->gr->func->grctx;
- const u32 state_limit = min(grctx->bundle_min_gpm_fifo_depth,
- grctx->bundle_size / 0x20);
+ const struct gf100_grctx_func *grctx = chan->gr->func->grctx;
+ const u32 state_limit = min(grctx->bundle_min_gpm_fifo_depth, size / 0x20);
const u32 token_limit = grctx->bundle_token_limit;
- const int s = 8;
- const int b = mmio_vram(info, grctx->bundle_size, (1 << s), true);
- mmio_refn(info, 0x408004, 0x00000000, s, b);
- mmio_wr32(info, 0x408008, 0x80000000 | (grctx->bundle_size >> s));
- mmio_refn(info, 0x418e24, 0x00000000, s, b);
- mmio_wr32(info, 0x418e28, 0x80000000 | (grctx->bundle_size >> s));
- mmio_wr32(info, 0x4064c8, (state_limit << 16) | token_limit);
+
+ gf100_grctx_patch_wr32(chan, 0x408004, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x408008, 0x80000000 | (size >> 8));
+ gf100_grctx_patch_wr32(chan, 0x418e24, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x418e28, 0x80000000 | (size >> 8));
+ gf100_grctx_patch_wr32(chan, 0x4064c8, (state_limit << 16) | token_limit);
}
void
-gm107_grctx_generate_pagepool(struct gf100_grctx *info)
+gm107_grctx_generate_pagepool(struct gf100_gr_chan *chan, u64 addr)
{
- const struct gf100_grctx_func *grctx = info->gr->func->grctx;
- const int s = 8;
- const int b = mmio_vram(info, grctx->pagepool_size, (1 << s), true);
- mmio_refn(info, 0x40800c, 0x00000000, s, b);
- mmio_wr32(info, 0x408010, 0x80000000);
- mmio_refn(info, 0x419004, 0x00000000, s, b);
- mmio_wr32(info, 0x419008, 0x00000000);
- mmio_wr32(info, 0x4064cc, 0x80000000);
- mmio_wr32(info, 0x418e30, 0x80000000); /* guess at it being related */
+ gk104_grctx_generate_pagepool(chan, addr);
+ gf100_grctx_patch_wr32(chan, 0x418e30, 0x80000000);
}
void
-gm107_grctx_generate_attrib(struct gf100_grctx *info)
+gm107_grctx_generate_attrib(struct gf100_gr_chan *chan)
{
- struct gf100_gr *gr = info->gr;
+ struct gf100_gr *gr = chan->gr;
const struct gf100_grctx_func *grctx = gr->func->grctx;
const u32 alpha = grctx->alpha_nr;
const u32 attrib = grctx->attrib_nr;
- const u32 size = 0x20 * (grctx->attrib_nr_max + grctx->alpha_nr_max);
- const int s = 12;
- const int b = mmio_vram(info, size * gr->tpc_total, (1 << s), false);
const int max_batches = 0xffff;
u32 bo = 0;
u32 ao = bo + grctx->attrib_nr_max * gr->tpc_total;
int gpc, ppc, n = 0;
- mmio_refn(info, 0x418810, 0x80000000, s, b);
- mmio_refn(info, 0x419848, 0x10000000, s, b);
- mmio_refn(info, 0x419c2c, 0x10000000, s, b);
- mmio_wr32(info, 0x405830, (attrib << 16) | alpha);
- mmio_wr32(info, 0x4064c4, ((alpha / 4) << 16) | max_batches);
+ gf100_grctx_patch_wr32(chan, 0x405830, (attrib << 16) | alpha);
+ gf100_grctx_patch_wr32(chan, 0x4064c4, ((alpha / 4) << 16) | max_batches);
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++, n++) {
+ for (ppc = 0; ppc < gr->func->ppc_nr; ppc++, n++) {
const u32 as = alpha * gr->ppc_tpc_nr[gpc][ppc];
const u32 bs = attrib * gr->ppc_tpc_nr[gpc][ppc];
const u32 u = 0x418ea0 + (n * 0x04);
const u32 o = PPC_UNIT(gpc, ppc, 0);
+
if (!(gr->ppc_mask[gpc] & (1 << ppc)))
continue;
- mmio_wr32(info, o + 0xc0, bs);
- mmio_wr32(info, o + 0xf4, bo);
+
+ gf100_grctx_patch_wr32(chan, o + 0xc0, bs);
+ gf100_grctx_patch_wr32(chan, o + 0xf4, bo);
bo += grctx->attrib_nr_max * gr->ppc_tpc_nr[gpc][ppc];
- mmio_wr32(info, o + 0xe4, as);
- mmio_wr32(info, o + 0xf8, ao);
+ gf100_grctx_patch_wr32(chan, o + 0xe4, as);
+ gf100_grctx_patch_wr32(chan, o + 0xf8, ao);
ao += grctx->alpha_nr_max * gr->ppc_tpc_nr[gpc][ppc];
- mmio_wr32(info, u, ((bs / 3) << 16) | bs);
+ gf100_grctx_patch_wr32(chan, u, ((bs / 3) << 16) | bs);
}
}
}
+void
+gm107_grctx_generate_attrib_cb(struct gf100_gr_chan *chan, u64 addr, u32 size)
+{
+ gf100_grctx_generate_attrib_cb(chan, addr, size);
+
+ gf100_grctx_patch_wr32(chan, 0x419c2c, 0x10000000 | addr >> 12);
+}
+
static void
gm107_grctx_generate_r406500(struct gf100_gr *gr)
{
@@ -978,6 +973,8 @@ gm107_grctx = {
.bundle_token_limit = 0x2c0,
.pagepool = gm107_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gm107_grctx_generate_attrib_cb,
.attrib = gm107_grctx_generate_attrib,
.attrib_nr_max = 0xff0,
.attrib_nr = 0xaa0,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c
index 013d05a..175da8a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c
@@ -87,7 +87,7 @@ gm200_grctx_generate_dist_skip_table(struct gf100_gr *gr)
int gpc, ppc, i;
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++) {
+ for (ppc = 0; ppc < gr->func->ppc_nr; ppc++) {
u8 ppc_tpcs = gr->ppc_tpc_nr[gpc][ppc];
u8 ppc_tpcm = gr->ppc_tpc_mask[gpc][ppc];
while (ppc_tpcs-- > gr->ppc_tpc_min)
@@ -111,6 +111,8 @@ gm200_grctx = {
.bundle_token_limit = 0x780,
.pagepool = gm107_grctx_generate_pagepool,
.pagepool_size = 0x20000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gm107_grctx_generate_attrib_cb,
.attrib = gm107_grctx_generate_attrib,
.attrib_nr_max = 0x600,
.attrib_nr = 0x400,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c
index 6b92f8a..b8edccf 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c
@@ -22,8 +22,9 @@
#include "ctxgf100.h"
static void
-gm20b_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
+gm20b_grctx_generate_main(struct gf100_gr_chan *chan)
{
+ struct gf100_gr *gr = chan->gr;
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
u32 idle_timeout;
@@ -35,7 +36,8 @@ gm20b_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
- grctx->attrib(info);
+ grctx->attrib_cb(chan, chan->attrib_cb->addr, grctx->attrib_cb_size(gr));
+ grctx->attrib(chan);
grctx->unkn(gr);
@@ -63,8 +65,8 @@ gm20b_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
gf100_gr_wait_idle(gr);
gf100_gr_icmd(gr, gr->bundle);
- grctx->pagepool(info);
- grctx->bundle(info);
+ grctx->pagepool(chan, chan->pagepool->addr);
+ grctx->bundle(chan, chan->bundle_cb->addr, grctx->bundle_size);
}
const struct gf100_grctx_func
@@ -77,6 +79,8 @@ gm20b_grctx = {
.bundle_token_limit = 0x1c0,
.pagepool = gm107_grctx_generate_pagepool,
.pagepool_size = 0x8000,
+ .attrib_cb_size = gf100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gm107_grctx_generate_attrib_cb,
.attrib = gm107_grctx_generate_attrib,
.attrib_nr_max = 0x600,
.attrib_nr = 0x400,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp100.c
index 0b33262..8485aae 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp100.c
@@ -30,66 +30,76 @@
******************************************************************************/
void
-gp100_grctx_generate_pagepool(struct gf100_grctx *info)
+gp100_grctx_generate_pagepool(struct gf100_gr_chan *chan, u64 addr)
{
- const struct gf100_grctx_func *grctx = info->gr->func->grctx;
- const int s = 8;
- const int b = mmio_vram(info, grctx->pagepool_size, (1 << s), true);
- mmio_refn(info, 0x40800c, 0x00000000, s, b);
- mmio_wr32(info, 0x408010, 0x8007d800);
- mmio_refn(info, 0x419004, 0x00000000, s, b);
- mmio_wr32(info, 0x419008, 0x00000000);
+ gf100_grctx_patch_wr32(chan, 0x40800c, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x408010, 0x8007d800);
+ gf100_grctx_patch_wr32(chan, 0x419004, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x419008, 0x00000000);
}
static void
-gp100_grctx_generate_attrib(struct gf100_grctx *info)
+gp100_grctx_generate_attrib(struct gf100_gr_chan *chan)
{
- struct gf100_gr *gr = info->gr;
+ struct gf100_gr *gr = chan->gr;
const struct gf100_grctx_func *grctx = gr->func->grctx;
const u32 alpha = grctx->alpha_nr;
const u32 attrib = grctx->attrib_nr;
- const int s = 12;
const int max_batches = 0xffff;
u32 size = grctx->alpha_nr_max * gr->tpc_total;
u32 ao = 0;
u32 bo = ao + size;
- int gpc, ppc, b, n = 0;
+ int gpc, ppc, n = 0;
- for (gpc = 0; gpc < gr->gpc_nr; gpc++)
- size += grctx->attrib_nr_max * gr->ppc_nr[gpc] * gr->ppc_tpc_max;
- size = ((size * 0x20) + 128) & ~127;
- b = mmio_vram(info, size, (1 << s), false);
-
- mmio_refn(info, 0x418810, 0x80000000, s, b);
- mmio_refn(info, 0x419848, 0x10000000, s, b);
- mmio_refn(info, 0x419c2c, 0x10000000, s, b);
- mmio_refn(info, 0x419b00, 0x00000000, s, b);
- mmio_wr32(info, 0x419b04, 0x80000000 | size >> 7);
- mmio_wr32(info, 0x405830, attrib);
- mmio_wr32(info, 0x40585c, alpha);
- mmio_wr32(info, 0x4064c4, ((alpha / 4) << 16) | max_batches);
+ gf100_grctx_patch_wr32(chan, 0x405830, attrib);
+ gf100_grctx_patch_wr32(chan, 0x40585c, alpha);
+ gf100_grctx_patch_wr32(chan, 0x4064c4, ((alpha / 4) << 16) | max_batches);
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++, n++) {
+ for (ppc = 0; ppc < gr->func->ppc_nr; ppc++, n++) {
const u32 as = alpha * gr->ppc_tpc_nr[gpc][ppc];
const u32 bs = attrib * gr->ppc_tpc_max;
const u32 u = 0x418ea0 + (n * 0x04);
const u32 o = PPC_UNIT(gpc, ppc, 0);
+
if (!(gr->ppc_mask[gpc] & (1 << ppc)))
continue;
- mmio_wr32(info, o + 0xc0, bs);
- mmio_wr32(info, o + 0xf4, bo);
- mmio_wr32(info, o + 0xf0, bs);
+
+ gf100_grctx_patch_wr32(chan, o + 0xc0, bs);
+ gf100_grctx_patch_wr32(chan, o + 0xf4, bo);
+ gf100_grctx_patch_wr32(chan, o + 0xf0, bs);
bo += grctx->attrib_nr_max * gr->ppc_tpc_max;
- mmio_wr32(info, o + 0xe4, as);
- mmio_wr32(info, o + 0xf8, ao);
+ gf100_grctx_patch_wr32(chan, o + 0xe4, as);
+ gf100_grctx_patch_wr32(chan, o + 0xf8, ao);
ao += grctx->alpha_nr_max * gr->ppc_tpc_nr[gpc][ppc];
- mmio_wr32(info, u, bs);
+ gf100_grctx_patch_wr32(chan, u, bs);
}
}
- mmio_wr32(info, 0x418eec, 0x00000000);
- mmio_wr32(info, 0x41befc, 0x00000000);
+ gf100_grctx_patch_wr32(chan, 0x418eec, 0x00000000);
+ gf100_grctx_patch_wr32(chan, 0x41befc, 0x00000000);
+}
+
+void
+gp100_grctx_generate_attrib_cb(struct gf100_gr_chan *chan, u64 addr, u32 size)
+{
+ gm107_grctx_generate_attrib_cb(chan, addr, size);
+
+ gf100_grctx_patch_wr32(chan, 0x419b00, 0x00000000 | addr >> 12);
+ gf100_grctx_patch_wr32(chan, 0x419b04, 0x80000000 | size >> 7);
+}
+
+static u32
+gp100_grctx_generate_attrib_cb_size(struct gf100_gr *gr)
+{
+ const struct gf100_grctx_func *grctx = gr->func->grctx;
+ u32 size = grctx->alpha_nr_max * gr->tpc_total;
+ int gpc;
+
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++)
+ size += grctx->attrib_nr_max * gr->func->ppc_nr * gr->ppc_tpc_max;
+
+ return ((size * 0x20) + 128) & ~127;
}
void
@@ -123,6 +133,8 @@ gp100_grctx = {
.bundle_token_limit = 0x1080,
.pagepool = gp100_grctx_generate_pagepool,
.pagepool_size = 0x20000,
+ .attrib_cb_size = gp100_grctx_generate_attrib_cb_size,
+ .attrib_cb = gp100_grctx_generate_attrib_cb,
.attrib = gp100_grctx_generate_attrib,
.attrib_nr_max = 0x660,
.attrib_nr = 0x440,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp102.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp102.c
index daee17b..7537979 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp102.c
@@ -37,58 +37,62 @@ gp102_grctx_generate_r408840(struct gf100_gr *gr)
}
void
-gp102_grctx_generate_attrib(struct gf100_grctx *info)
+gp102_grctx_generate_attrib(struct gf100_gr_chan *chan)
{
- struct gf100_gr *gr = info->gr;
+ struct gf100_gr *gr = chan->gr;
const struct gf100_grctx_func *grctx = gr->func->grctx;
const u32 alpha = grctx->alpha_nr;
const u32 attrib = grctx->attrib_nr;
const u32 gfxp = grctx->gfxp_nr;
- const int s = 12;
const int max_batches = 0xffff;
u32 size = grctx->alpha_nr_max * gr->tpc_total;
u32 ao = 0;
u32 bo = ao + size;
- int gpc, ppc, b, n = 0;
+ int gpc, ppc, n = 0;
- for (gpc = 0; gpc < gr->gpc_nr; gpc++)
- size += grctx->gfxp_nr * gr->ppc_nr[gpc] * gr->ppc_tpc_max;
- size = ((size * 0x20) + 128) & ~127;
- b = mmio_vram(info, size, (1 << s), false);
-
- mmio_refn(info, 0x418810, 0x80000000, s, b);
- mmio_refn(info, 0x419848, 0x10000000, s, b);
- mmio_refn(info, 0x419c2c, 0x10000000, s, b);
- mmio_refn(info, 0x419b00, 0x00000000, s, b);
- mmio_wr32(info, 0x419b04, 0x80000000 | size >> 7);
- mmio_wr32(info, 0x405830, attrib);
- mmio_wr32(info, 0x40585c, alpha);
- mmio_wr32(info, 0x4064c4, ((alpha / 4) << 16) | max_batches);
+ gf100_grctx_patch_wr32(chan, 0x405830, attrib);
+ gf100_grctx_patch_wr32(chan, 0x40585c, alpha);
+ gf100_grctx_patch_wr32(chan, 0x4064c4, ((alpha / 4) << 16) | max_batches);
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++, n++) {
+ for (ppc = 0; ppc < gr->func->ppc_nr; ppc++, n++) {
const u32 as = alpha * gr->ppc_tpc_nr[gpc][ppc];
const u32 bs = attrib * gr->ppc_tpc_max;
const u32 gs = gfxp * gr->ppc_tpc_max;
const u32 u = 0x418ea0 + (n * 0x04);
const u32 o = PPC_UNIT(gpc, ppc, 0);
const u32 p = GPC_UNIT(gpc, 0xc44 + (ppc * 4));
+
if (!(gr->ppc_mask[gpc] & (1 << ppc)))
continue;
- mmio_wr32(info, o + 0xc0, gs);
- mmio_wr32(info, p, bs);
- mmio_wr32(info, o + 0xf4, bo);
- mmio_wr32(info, o + 0xf0, bs);
+
+ gf100_grctx_patch_wr32(chan, o + 0xc0, gs);
+ gf100_grctx_patch_wr32(chan, p, bs);
+ gf100_grctx_patch_wr32(chan, o + 0xf4, bo);
+ gf100_grctx_patch_wr32(chan, o + 0xf0, bs);
bo += gs;
- mmio_wr32(info, o + 0xe4, as);
- mmio_wr32(info, o + 0xf8, ao);
+ gf100_grctx_patch_wr32(chan, o + 0xe4, as);
+ gf100_grctx_patch_wr32(chan, o + 0xf8, ao);
ao += grctx->alpha_nr_max * gr->ppc_tpc_nr[gpc][ppc];
- mmio_wr32(info, u, bs);
+ gf100_grctx_patch_wr32(chan, u, bs);
}
}
- mmio_wr32(info, 0x4181e4, 0x00000100);
- mmio_wr32(info, 0x41befc, 0x00000100);
+ gf100_grctx_patch_wr32(chan, 0x4181e4, 0x00000100);
+ gf100_grctx_patch_wr32(chan, 0x41befc, 0x00000100);
+}
+
+u32
+gp102_grctx_generate_attrib_cb_size(struct gf100_gr *gr)
+{
+ const struct gf100_grctx_func *grctx = gr->func->grctx;
+ u32 size = grctx->alpha_nr_max * gr->tpc_total;
+ int gpc;
+
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++)
+ size += grctx->gfxp_nr * gr->func->ppc_nr * gr->ppc_tpc_max;
+
+ return ((size * 0x20) + 127) & ~127;
}
const struct gf100_grctx_func
@@ -101,6 +105,8 @@ gp102_grctx = {
.bundle_token_limit = 0x900,
.pagepool = gp100_grctx_generate_pagepool,
.pagepool_size = 0x20000,
+ .attrib_cb_size = gp102_grctx_generate_attrib_cb_size,
+ .attrib_cb = gp100_grctx_generate_attrib_cb,
.attrib = gp102_grctx_generate_attrib,
.attrib_nr_max = 0x4b0,
.attrib_nr = 0x320,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp104.c
index 3b85e3d..90b5f79 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp104.c
@@ -31,6 +31,8 @@ gp104_grctx = {
.bundle_token_limit = 0x900,
.pagepool = gp100_grctx_generate_pagepool,
.pagepool_size = 0x20000,
+ .attrib_cb_size = gp102_grctx_generate_attrib_cb_size,
+ .attrib_cb = gp100_grctx_generate_attrib_cb,
.attrib = gp102_grctx_generate_attrib,
.attrib_nr_max = 0x4b0,
.attrib_nr = 0x320,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp107.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp107.c
index 5060c5e..d191761 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp107.c
@@ -39,6 +39,8 @@ gp107_grctx = {
.bundle_token_limit = 0x300,
.pagepool = gp100_grctx_generate_pagepool,
.pagepool_size = 0x20000,
+ .attrib_cb_size = gp102_grctx_generate_attrib_cb_size,
+ .attrib_cb = gp100_grctx_generate_attrib_cb,
.attrib = gp102_grctx_generate_attrib,
.attrib_nr_max = 0x15de,
.attrib_nr = 0x540,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgv100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgv100.c
index 39553d5..957ea9d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgv100.c
@@ -25,7 +25,7 @@
* PGRAPH context implementation
******************************************************************************/
-const struct gf100_gr_init
+static const struct gf100_gr_init
gv100_grctx_init_sw_veid_bundle_init_0[] = {
{ 0x00001000, 64, 0x00100000, 0x00000008 },
{ 0x00000941, 64, 0x00100000, 0x00000000 },
@@ -59,67 +59,70 @@ gv100_grctx_pack_sw_veid_bundle_init[] = {
};
void
-gv100_grctx_generate_attrib(struct gf100_grctx *info)
+gv100_grctx_generate_attrib(struct gf100_gr_chan *chan)
{
- struct gf100_gr *gr = info->gr;
+ struct gf100_gr *gr = chan->gr;
const struct gf100_grctx_func *grctx = gr->func->grctx;
const u32 alpha = grctx->alpha_nr;
const u32 attrib = grctx->attrib_nr;
const u32 gfxp = grctx->gfxp_nr;
- const int s = 12;
+ const int max_batches = 0xffff;
u32 size = grctx->alpha_nr_max * gr->tpc_total;
u32 ao = 0;
u32 bo = ao + size;
- int gpc, ppc, b, n = 0;
+ int gpc, ppc, n = 0;
- for (gpc = 0; gpc < gr->gpc_nr; gpc++)
- size += grctx->gfxp_nr * gr->ppc_nr[gpc] * gr->ppc_tpc_max;
- size = ((size * 0x20) + 127) & ~127;
- b = mmio_vram(info, size, (1 << s), false);
-
- mmio_refn(info, 0x418810, 0x80000000, s, b);
- mmio_refn(info, 0x419848, 0x10000000, s, b);
- mmio_refn(info, 0x419c2c, 0x10000000, s, b);
- mmio_refn(info, 0x419e00, 0x00000000, s, b);
- mmio_wr32(info, 0x419e04, 0x80000000 | size >> 7);
- mmio_wr32(info, 0x405830, attrib);
- mmio_wr32(info, 0x40585c, alpha);
+ gf100_grctx_patch_wr32(chan, 0x405830, attrib);
+ gf100_grctx_patch_wr32(chan, 0x40585c, alpha);
+ gf100_grctx_patch_wr32(chan, 0x4064c4, ((alpha / 4) << 16) | max_batches);
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++, n++) {
+ for (ppc = 0; ppc < gr->func->ppc_nr; ppc++, n++) {
const u32 as = alpha * gr->ppc_tpc_nr[gpc][ppc];
const u32 bs = attrib * gr->ppc_tpc_max;
const u32 gs = gfxp * gr->ppc_tpc_max;
const u32 u = 0x418ea0 + (n * 0x04);
const u32 o = PPC_UNIT(gpc, ppc, 0);
+
if (!(gr->ppc_mask[gpc] & (1 << ppc)))
continue;
- mmio_wr32(info, o + 0xc0, gs);
- mmio_wr32(info, o + 0xf4, bo);
- mmio_wr32(info, o + 0xf0, bs);
+
+ gf100_grctx_patch_wr32(chan, o + 0xc0, gs);
+ gf100_grctx_patch_wr32(chan, o + 0xf4, bo);
+ gf100_grctx_patch_wr32(chan, o + 0xf0, bs);
bo += gs;
- mmio_wr32(info, o + 0xe4, as);
- mmio_wr32(info, o + 0xf8, ao);
+ gf100_grctx_patch_wr32(chan, o + 0xe4, as);
+ gf100_grctx_patch_wr32(chan, o + 0xf8, ao);
ao += grctx->alpha_nr_max * gr->ppc_tpc_nr[gpc][ppc];
- mmio_wr32(info, u, bs);
+ gf100_grctx_patch_wr32(chan, u, bs);
}
}
- mmio_wr32(info, 0x4181e4, 0x00000100);
- mmio_wr32(info, 0x41befc, 0x00000100);
+ gf100_grctx_patch_wr32(chan, 0x4181e4, 0x00000100);
+ gf100_grctx_patch_wr32(chan, 0x41befc, 0x00000100);
+}
+
+void
+gv100_grctx_generate_attrib_cb(struct gf100_gr_chan *chan, u64 addr, u32 size)
+{
+ gm107_grctx_generate_attrib_cb(chan, addr, size);
+
+ gf100_grctx_patch_wr32(chan, 0x419e00, 0x00000000 | addr >> 12);
+ gf100_grctx_patch_wr32(chan, 0x419e04, 0x80000000 | size >> 7);
}
void
gv100_grctx_generate_rop_mapping(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
+ const u32 mapregs = DIV_ROUND_UP(gr->func->gpc_nr * gr->func->tpc_nr, 6);
u32 data;
int i, j;
/* Pack tile map into register format. */
nvkm_wr32(device, 0x418bb8, (gr->tpc_total << 8) |
gr->screen_tile_row_offset);
- for (i = 0; i < 11; i++) {
+ for (i = 0; i < mapregs; i++) {
for (data = 0, j = 0; j < 6; j++)
data |= (gr->tile[i * 6 + j] & 0x1f) << (j * 5);
nvkm_wr32(device, 0x418b08 + (i * 4), data);
@@ -157,6 +160,9 @@ static void
gv100_grctx_generate_sm_id(struct gf100_gr *gr, int gpc, int tpc, int sm)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ tpc = gv100_gr_nonpes_aware_tpc(gr, gpc, tpc);
+
nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x608), sm);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0c10 + tpc * 4), sm);
nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x088), sm);
@@ -198,6 +204,8 @@ gv100_grctx = {
.bundle_token_limit = 0x1680,
.pagepool = gp100_grctx_generate_pagepool,
.pagepool_size = 0x20000,
+ .attrib_cb_size = gp102_grctx_generate_attrib_cb_size,
+ .attrib_cb = gv100_grctx_generate_attrib_cb,
.attrib = gv100_grctx_generate_attrib,
.attrib_nr_max = 0x6c0,
.attrib_nr = 0x480,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxtu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxtu102.c
index 2299ca0..542ab0c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxtu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxtu102.c
@@ -34,6 +34,9 @@ static void
tu102_grctx_generate_sm_id(struct gf100_gr *gr, int gpc, int tpc, int sm)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ tpc = gv100_gr_nonpes_aware_tpc(gr, gpc, tpc);
+
nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x608), sm);
nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x088), sm);
}
@@ -47,42 +50,38 @@ tu102_grctx_init_unknown_bundle_init_0[] = {
};
static const struct gf100_gr_pack
-tu102_grctx_pack_sw_veid_bundle_init[] = {
- { gv100_grctx_init_sw_veid_bundle_init_0 },
- { tu102_grctx_init_unknown_bundle_init_0 },
+tu102_grctx_pack_sw_bundle64_init[] = {
+ { tu102_grctx_init_unknown_bundle_init_0, .type = 64 },
{}
};
-static void
-tu102_grctx_generate_attrib(struct gf100_grctx *info)
+void
+tu102_grctx_generate_unknown(struct gf100_gr_chan *chan, u64 addr, u32 size)
{
- const u64 size = 0x80000; /*XXX: educated guess */
- const int s = 8;
- const int b = mmio_vram(info, size, (1 << s), true);
-
- gv100_grctx_generate_attrib(info);
-
- mmio_refn(info, 0x408070, 0x00000000, s, b);
- mmio_wr32(info, 0x408074, size >> s); /*XXX: guess */
- mmio_refn(info, 0x419034, 0x00000000, s, b);
- mmio_wr32(info, 0x408078, 0x00000000);
+ gf100_grctx_patch_wr32(chan, 0x408070, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x408074, size >> 8); /*XXX: guess */
+ gf100_grctx_patch_wr32(chan, 0x419034, addr >> 8);
+ gf100_grctx_patch_wr32(chan, 0x408078, 0x00000000);
}
const struct gf100_grctx_func
tu102_grctx = {
- .unkn88c = gv100_grctx_unkn88c,
.main = gf100_grctx_generate_main,
.unkn = gv100_grctx_generate_unkn,
- .sw_veid_bundle_init = tu102_grctx_pack_sw_veid_bundle_init,
+ .sw_bundle64_init = tu102_grctx_pack_sw_bundle64_init,
.bundle = gm107_grctx_generate_bundle,
.bundle_size = 0x3000,
.bundle_min_gpm_fifo_depth = 0x180,
.bundle_token_limit = 0xa80,
.pagepool = gp100_grctx_generate_pagepool,
.pagepool_size = 0x20000,
- .attrib = tu102_grctx_generate_attrib,
+ .attrib_cb_size = gp102_grctx_generate_attrib_cb_size,
+ .attrib_cb = gv100_grctx_generate_attrib_cb,
+ .attrib = gv100_grctx_generate_attrib,
.attrib_nr_max = 0x800,
.attrib_nr = 0x700,
+ .unknown_size = 0x80000,
+ .unknown = tu102_grctx_generate_unknown,
.alpha_nr_max = 0xc00,
.alpha_nr = 0x800,
.gfxp_nr = 0xfa8,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ga102.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ga102.c
new file mode 100644
index 0000000..a5b5ac2
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ga102.c
@@ -0,0 +1,347 @@
+/*
+ * Copyright 2019 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "gf100.h"
+#include "ctxgf100.h"
+
+#include <core/firmware.h>
+#include <subdev/acr.h>
+#include <subdev/timer.h>
+#include <subdev/vfn.h>
+
+#include <nvfw/flcn.h>
+
+#include <nvif/class.h>
+
+static void
+ga102_gr_zbc_clear_color(struct gf100_gr *gr, int zbc)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+ u32 invalid[] = { 0, 0, 0, 0 }, *color;
+
+ if (gr->zbc_color[zbc].format)
+ color = gr->zbc_color[zbc].l2;
+ else
+ color = invalid;
+
+ nvkm_mask(device, 0x41bcb4, 0x0000001f, zbc);
+ nvkm_wr32(device, 0x41bcec, color[0]);
+ nvkm_wr32(device, 0x41bcf0, color[1]);
+ nvkm_wr32(device, 0x41bcf4, color[2]);
+ nvkm_wr32(device, 0x41bcf8, color[3]);
+}
+
+static const struct gf100_gr_func_zbc
+ga102_gr_zbc = {
+ .clear_color = ga102_gr_zbc_clear_color,
+ .clear_depth = gp100_gr_zbc_clear_depth,
+ .stencil_get = gp102_gr_zbc_stencil_get,
+ .clear_stencil = gp102_gr_zbc_clear_stencil,
+};
+
+static void
+ga102_gr_gpccs_reset(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x41a610, 0x00000000);
+ nvkm_msec(device, 1, NVKM_DELAY);
+ nvkm_wr32(device, 0x41a610, 0x00000001);
+}
+
+static const struct nvkm_acr_lsf_func
+ga102_gr_gpccs_acr = {
+ .flags = NVKM_ACR_LSF_FORCE_PRIV_LOAD,
+ .bl_entry = 0x3400,
+ .bld_size = sizeof(struct flcn_bl_dmem_desc_v2),
+ .bld_write = gp108_gr_acr_bld_write,
+ .bld_patch = gp108_gr_acr_bld_patch,
+};
+
+static void
+ga102_gr_fecs_reset(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x409614, 0x00000010);
+ nvkm_wr32(device, 0x41a614, 0x00000020);
+ nvkm_usec(device, 10, NVKM_DELAY);
+ nvkm_wr32(device, 0x409614, 0x00000110);
+ nvkm_wr32(device, 0x41a614, 0x00000a20);
+ nvkm_usec(device, 10, NVKM_DELAY);
+ nvkm_rd32(device, 0x409614);
+ nvkm_rd32(device, 0x41a614);
+}
+
+static const struct nvkm_acr_lsf_func
+ga102_gr_fecs_acr = {
+ .bl_entry = 0x7e00,
+ .bld_size = sizeof(struct flcn_bl_dmem_desc_v2),
+ .bld_write = gp108_gr_acr_bld_write,
+ .bld_patch = gp108_gr_acr_bld_patch,
+};
+
+static void
+ga102_gr_init_rop_exceptions(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x41bcbc, 0x40000000);
+ nvkm_wr32(device, 0x41bc38, 0x40000000);
+ nvkm_wr32(device, 0x41ac94, nvkm_rd32(device, 0x502c94));
+}
+
+static void
+ga102_gr_init_40a790(struct gf100_gr *gr)
+{
+ nvkm_wr32(gr->base.engine.subdev.device, 0x40a790, 0xc0000000);
+}
+
+static void
+ga102_gr_init_gpc_mmu(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x418880, nvkm_rd32(device, 0x100c80) & 0xf8001fff);
+ nvkm_wr32(device, 0x418894, 0x00000000);
+
+ nvkm_wr32(device, 0x4188b4, nvkm_rd32(device, 0x100cc8));
+ nvkm_wr32(device, 0x4188b8, nvkm_rd32(device, 0x100ccc));
+ nvkm_wr32(device, 0x4188b0, nvkm_rd32(device, 0x100cc4));
+}
+
+static struct nvkm_intr *
+ga102_gr_oneinit_intr(struct gf100_gr *gr, enum nvkm_intr_type *pvector)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ *pvector = nvkm_rd32(device, 0x400154) & 0x00000fff;
+ return &device->vfn->intr;
+}
+
+static const struct gf100_gr_func
+ga102_gr = {
+ .oneinit_intr = ga102_gr_oneinit_intr,
+ .oneinit_tiles = gm200_gr_oneinit_tiles,
+ .oneinit_sm_id = gv100_gr_oneinit_sm_id,
+ .init = gf100_gr_init,
+ .init_419bd8 = gv100_gr_init_419bd8,
+ .init_gpc_mmu = ga102_gr_init_gpc_mmu,
+ .init_vsc_stream_master = gk104_gr_init_vsc_stream_master,
+ .init_zcull = tu102_gr_init_zcull,
+ .init_num_active_ltcs = gf100_gr_init_num_active_ltcs,
+ .init_swdx_pes_mask = gp102_gr_init_swdx_pes_mask,
+ .init_fs = tu102_gr_init_fs,
+ .init_fecs_exceptions = tu102_gr_init_fecs_exceptions,
+ .init_40a790 = ga102_gr_init_40a790,
+ .init_ds_hww_esr_2 = gm200_gr_init_ds_hww_esr_2,
+ .init_sked_hww_esr = gk104_gr_init_sked_hww_esr,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
+ .init_504430 = gv100_gr_init_504430,
+ .init_shader_exceptions = gv100_gr_init_shader_exceptions,
+ .init_rop_exceptions = ga102_gr_init_rop_exceptions,
+ .init_4188a4 = gv100_gr_init_4188a4,
+ .trap_mp = gv100_gr_trap_mp,
+ .fecs.reset = ga102_gr_fecs_reset,
+ .gpccs.reset = ga102_gr_gpccs_reset,
+ .rops = gm200_gr_rops,
+ .gpc_nr = 7,
+ .tpc_nr = 6,
+ .ppc_nr = 3,
+ .grctx = &ga102_grctx,
+ .zbc = &ga102_gr_zbc,
+ .sclass = {
+ { -1, -1, FERMI_TWOD_A },
+ { -1, -1, KEPLER_INLINE_TO_MEMORY_B },
+ { -1, -1, AMPERE_B, &gf100_fermi },
+ { -1, -1, AMPERE_COMPUTE_B },
+ {}
+ }
+};
+
+MODULE_FIRMWARE("nvidia/ga102/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga102/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga102/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga102/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga102/gr/NET_img.bin");
+
+MODULE_FIRMWARE("nvidia/ga103/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga103/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga103/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga103/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga103/gr/NET_img.bin");
+
+MODULE_FIRMWARE("nvidia/ga104/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga104/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga104/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga104/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga104/gr/NET_img.bin");
+
+MODULE_FIRMWARE("nvidia/ga106/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga106/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga106/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga106/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga106/gr/NET_img.bin");
+
+MODULE_FIRMWARE("nvidia/ga107/gr/fecs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga107/gr/fecs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga107/gr/gpccs_bl.bin");
+MODULE_FIRMWARE("nvidia/ga107/gr/gpccs_sig.bin");
+MODULE_FIRMWARE("nvidia/ga107/gr/NET_img.bin");
+
+struct netlist_region {
+ u32 region_id;
+ u32 data_size;
+ u32 data_offset;
+};
+
+struct netlist_image_header {
+ u32 version;
+ u32 regions;
+};
+
+struct netlist_image {
+ struct netlist_image_header header;
+ struct netlist_region regions[];
+};
+
+struct netlist_av64 {
+ u32 addr;
+ u32 data_hi;
+ u32 data_lo;
+};
+
+static int
+ga102_gr_av64_to_init(struct nvkm_blob *blob, struct gf100_gr_pack **ppack)
+{
+ struct gf100_gr_init *init;
+ struct gf100_gr_pack *pack;
+ int nent;
+ int i;
+
+ nent = (blob->size / sizeof(struct netlist_av64));
+ pack = vzalloc((sizeof(*pack) * 2) + (sizeof(*init) * (nent + 1)));
+ if (!pack)
+ return -ENOMEM;
+
+ init = (void *)(pack + 2);
+ pack[0].init = init;
+ pack[0].type = 64;
+
+ for (i = 0; i < nent; i++) {
+ struct gf100_gr_init *ent = &init[i];
+ struct netlist_av64 *av = &((struct netlist_av64 *)blob->data)[i];
+
+ ent->addr = av->addr;
+ ent->data = ((u64)av->data_hi << 32) | av->data_lo;
+ ent->count = 1;
+ ent->pitch = 1;
+ }
+
+ *ppack = pack;
+ return 0;
+}
+
+static int
+ga102_gr_load(struct gf100_gr *gr, int ver, const struct gf100_gr_fwif *fwif)
+{
+ struct nvkm_subdev *subdev = &gr->base.engine.subdev;
+ const struct firmware *fw;
+ const struct netlist_image *net;
+ const struct netlist_region *fecs_inst = NULL;
+ const struct netlist_region *fecs_data = NULL;
+ const struct netlist_region *gpccs_inst = NULL;
+ const struct netlist_region *gpccs_data = NULL;
+ int ret, i;
+
+ ret = nvkm_firmware_get(subdev, "gr/NET_img", 0, &fw);
+ if (ret)
+ return ret;
+
+ net = (const void *)fw->data;
+ nvkm_debug(subdev, "netlist version %d, %d regions\n",
+ net->header.version, net->header.regions);
+
+ for (i = 0; i < net->header.regions; i++) {
+ const struct netlist_region *reg = &net->regions[i];
+ struct nvkm_blob blob = {
+ .data = (void *)fw->data + reg->data_offset,
+ .size = reg->data_size,
+ };
+
+ nvkm_debug(subdev, "\t%2d: %08x %08x\n",
+ reg->region_id, reg->data_offset, reg->data_size);
+
+ switch (reg->region_id) {
+ case 0: fecs_data = reg; break;
+ case 1: fecs_inst = reg; break;
+ case 2: gpccs_data = reg; break;
+ case 3: gpccs_inst = reg; break;
+ case 4: gk20a_gr_av_to_init(&blob, &gr->bundle); break;
+ case 5: gk20a_gr_aiv_to_init(&blob, &gr->sw_ctx); break;
+ case 7: gk20a_gr_av_to_method(&blob, &gr->method); break;
+ case 28: tu102_gr_av_to_init_veid(&blob, &gr->bundle_veid); break;
+ case 34: ga102_gr_av64_to_init(&blob, &gr->bundle64); break;
+ case 48: gk20a_gr_av_to_init(&blob, &gr->sw_nonctx1); break;
+ case 49: gk20a_gr_av_to_init(&blob, &gr->sw_nonctx2); break;
+ case 50: gk20a_gr_av_to_init(&blob, &gr->sw_nonctx3); break;
+ case 51: gk20a_gr_av_to_init(&blob, &gr->sw_nonctx4); break;
+ default:
+ break;
+ }
+ }
+
+ ret = nvkm_acr_lsfw_load_bl_sig_net(subdev, &gr->fecs.falcon, NVKM_ACR_LSF_FECS,
+ "gr/fecs_", ver, fwif->fecs,
+ fw->data + fecs_inst->data_offset,
+ fecs_inst->data_size,
+ fw->data + fecs_data->data_offset,
+ fecs_data->data_size);
+ if (ret)
+ return ret;
+
+ ret = nvkm_acr_lsfw_load_bl_sig_net(subdev, &gr->gpccs.falcon, NVKM_ACR_LSF_GPCCS,
+ "gr/gpccs_", ver, fwif->gpccs,
+ fw->data + gpccs_inst->data_offset,
+ gpccs_inst->data_size,
+ fw->data + gpccs_data->data_offset,
+ gpccs_data->data_size);
+ if (ret)
+ return ret;
+
+ gr->firmware = true;
+
+ nvkm_firmware_put(fw);
+ return 0;
+}
+
+static const struct gf100_gr_fwif
+ga102_gr_fwif[] = {
+ { 0, ga102_gr_load, &ga102_gr, &ga102_gr_fecs_acr, &ga102_gr_gpccs_acr },
+ { -1, gm200_gr_nofw },
+ {}
+};
+
+int
+ga102_gr_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, struct nvkm_gr **pgr)
+{
+ return gf100_gr_new_(ga102_gr_fwif, device, type, inst, pgr);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c
index f16eabf..5f20079 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c
@@ -67,7 +67,7 @@ gf100_gr_zbc_color_get(struct gf100_gr *gr, int format,
struct nvkm_ltc *ltc = gr->base.engine.subdev.device->ltc;
int zbc = -ENOSPC, i;
- for (i = ltc->zbc_min; i <= ltc->zbc_max; i++) {
+ for (i = ltc->zbc_color_min; i <= ltc->zbc_color_max; i++) {
if (gr->zbc_color[i].format) {
if (gr->zbc_color[i].format != format)
continue;
@@ -114,7 +114,7 @@ gf100_gr_zbc_depth_get(struct gf100_gr *gr, int format,
struct nvkm_ltc *ltc = gr->base.engine.subdev.device->ltc;
int zbc = -ENOSPC, i;
- for (i = ltc->zbc_min; i <= ltc->zbc_max; i++) {
+ for (i = ltc->zbc_depth_min; i <= ltc->zbc_depth_max; i++) {
if (gr->zbc_depth[i].format) {
if (gr->zbc_depth[i].format != format)
continue;
@@ -355,15 +355,14 @@ static void *
gf100_gr_chan_dtor(struct nvkm_object *object)
{
struct gf100_gr_chan *chan = gf100_gr_chan(object);
- int i;
-
- for (i = 0; i < ARRAY_SIZE(chan->data); i++) {
- nvkm_vmm_put(chan->vmm, &chan->data[i].vma);
- nvkm_memory_unref(&chan->data[i].mem);
- }
nvkm_vmm_put(chan->vmm, &chan->mmio_vma);
nvkm_memory_unref(&chan->mmio);
+
+ nvkm_vmm_put(chan->vmm, &chan->attrib_cb);
+ nvkm_vmm_put(chan->vmm, &chan->unknown);
+ nvkm_vmm_put(chan->vmm, &chan->bundle_cb);
+ nvkm_vmm_put(chan->vmm, &chan->pagepool);
nvkm_vmm_unref(&chan->vmm);
return chan;
}
@@ -380,12 +379,10 @@ gf100_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
struct nvkm_object **pobject)
{
struct gf100_gr *gr = gf100_gr(base);
- struct gf100_gr_data *data = gr->mmio_data;
- struct gf100_gr_mmio *mmio = gr->mmio_list;
struct gf100_gr_chan *chan;
struct gf100_vmm_map_v0 args = { .priv = 1 };
struct nvkm_device *device = gr->base.engine.subdev.device;
- int ret, i;
+ int ret;
if (!(chan = kzalloc(sizeof(*chan), GFP_KERNEL)))
return -ENOMEM;
@@ -394,6 +391,63 @@ gf100_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
chan->vmm = nvkm_vmm_ref(fifoch->vmm);
*pobject = &chan->object;
+ /* Map pagepool. */
+ ret = nvkm_vmm_get(chan->vmm, 12, nvkm_memory_size(gr->pagepool), &chan->pagepool);
+ if (ret)
+ return ret;
+
+ ret = nvkm_memory_map(gr->pagepool, 0, chan->vmm, chan->pagepool, &args, sizeof(args));
+ if (ret)
+ return ret;
+
+ /* Map bundle circular buffer. */
+ ret = nvkm_vmm_get(chan->vmm, 12, nvkm_memory_size(gr->bundle_cb), &chan->bundle_cb);
+ if (ret)
+ return ret;
+
+ ret = nvkm_memory_map(gr->bundle_cb, 0, chan->vmm, chan->bundle_cb, &args, sizeof(args));
+ if (ret)
+ return ret;
+
+ /* Map attribute circular buffer. */
+ ret = nvkm_vmm_get(chan->vmm, 12, nvkm_memory_size(gr->attrib_cb), &chan->attrib_cb);
+ if (ret)
+ return ret;
+
+ if (device->card_type < GP100) {
+ ret = nvkm_memory_map(gr->attrib_cb, 0, chan->vmm, chan->attrib_cb, NULL, 0);
+ if (ret)
+ return ret;
+ } else {
+ ret = nvkm_memory_map(gr->attrib_cb, 0, chan->vmm, chan->attrib_cb,
+ &args, sizeof(args));;
+ if (ret)
+ return ret;
+ }
+
+ /* Map some context buffer of unknown purpose. */
+ if (gr->func->grctx->unknown_size) {
+ ret = nvkm_vmm_get(chan->vmm, 12, nvkm_memory_size(gr->unknown), &chan->unknown);
+ if (ret)
+ return ret;
+
+ ret = nvkm_memory_map(gr->unknown, 0, chan->vmm, chan->unknown,
+ &args, sizeof(args));
+ if (ret)
+ return ret;
+ }
+
+ /* Generate golden context image. */
+ mutex_lock(&gr->fecs.mutex);
+ if (gr->data == NULL) {
+ ret = gf100_grctx_generate(gr, chan, fifoch->inst);
+ if (ret) {
+ nvkm_error(&base->engine.subdev, "failed to construct context\n");
+ return ret;
+ }
+ }
+ mutex_unlock(&gr->fecs.mutex);
+
/* allocate memory for a "mmio list" buffer that's used by the HUB
* fuc to modify some per-context register settings on first load
* of the context.
@@ -412,45 +466,16 @@ gf100_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
if (ret)
return ret;
- /* allocate buffers referenced by mmio list */
- for (i = 0; data->size && i < ARRAY_SIZE(gr->mmio_data); i++) {
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST,
- data->size, data->align, false,
- &chan->data[i].mem);
- if (ret)
- return ret;
-
- ret = nvkm_vmm_get(fifoch->vmm, 12,
- nvkm_memory_size(chan->data[i].mem),
- &chan->data[i].vma);
- if (ret)
- return ret;
-
- args.priv = data->priv;
-
- ret = nvkm_memory_map(chan->data[i].mem, 0, chan->vmm,
- chan->data[i].vma, &args, sizeof(args));
- if (ret)
- return ret;
-
- data++;
- }
-
/* finally, fill in the mmio list and point the context at it */
nvkm_kmap(chan->mmio);
- for (i = 0; mmio->addr && i < ARRAY_SIZE(gr->mmio_list); i++) {
- u32 addr = mmio->addr;
- u32 data = mmio->data;
-
- if (mmio->buffer >= 0) {
- u64 info = chan->data[mmio->buffer].vma->addr;
- data |= info >> mmio->shift;
- }
-
- nvkm_wo32(chan->mmio, chan->mmio_nr++ * 4, addr);
- nvkm_wo32(chan->mmio, chan->mmio_nr++ * 4, data);
- mmio++;
- }
+ gr->func->grctx->pagepool(chan, chan->pagepool->addr);
+ gr->func->grctx->bundle(chan, chan->bundle_cb->addr, gr->func->grctx->bundle_size);
+ gr->func->grctx->attrib_cb(chan, chan->attrib_cb->addr, gr->func->grctx->attrib_cb_size(gr));
+ gr->func->grctx->attrib(chan);
+ if (gr->func->grctx->patch_ltc)
+ gr->func->grctx->patch_ltc(chan);
+ if (gr->func->grctx->unknown_size)
+ gr->func->grctx->unknown(chan, chan->unknown->addr, gr->func->grctx->unknown_size);
nvkm_done(chan->mmio);
return 0;
}
@@ -727,7 +752,7 @@ gf100_gr_fecs_ctrl_ctxsw(struct gf100_gr *gr, u32 mthd)
struct nvkm_device *device = gr->base.engine.subdev.device;
nvkm_wr32(device, 0x409804, 0xffffffff);
- nvkm_wr32(device, 0x409840, 0xffffffff);
+ nvkm_wr32(device, 0x409800, 0x00000000);
nvkm_wr32(device, 0x409500, 0xffffffff);
nvkm_wr32(device, 0x409504, mthd);
nvkm_msec(device, 2000,
@@ -771,12 +796,45 @@ gf100_gr_fecs_stop_ctxsw(struct nvkm_gr *base)
return ret;
}
+static int
+gf100_gr_fecs_halt_pipeline(struct gf100_gr *gr)
+{
+ int ret = 0;
+
+ if (gr->firmware) {
+ mutex_lock(&gr->fecs.mutex);
+ ret = gf100_gr_fecs_ctrl_ctxsw(gr, 0x04);
+ mutex_unlock(&gr->fecs.mutex);
+ }
+
+ return ret;
+}
+
+int
+gf100_gr_fecs_wfi_golden_save(struct gf100_gr *gr, u32 inst)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_mask(device, 0x409800, 0x00000003, 0x00000000);
+ nvkm_wr32(device, 0x409500, inst);
+ nvkm_wr32(device, 0x409504, 0x00000009);
+ nvkm_msec(device, 2000,
+ u32 stat = nvkm_rd32(device, 0x409800);
+ if (stat & 0x00000002)
+ return -EIO;
+ if (stat & 0x00000001)
+ return 0;
+ );
+
+ return -ETIMEDOUT;
+}
+
int
gf100_gr_fecs_bind_pointer(struct gf100_gr *gr, u32 inst)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
- nvkm_wr32(device, 0x409840, 0x00000030);
+ nvkm_mask(device, 0x409800, 0x00000030, 0x00000000);
nvkm_wr32(device, 0x409500, inst);
nvkm_wr32(device, 0x409504, 0x00000003);
nvkm_msec(device, 2000,
@@ -867,7 +925,7 @@ gf100_gr_fecs_discover_pm_image_size(struct gf100_gr *gr, u32 *psize)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
- nvkm_wr32(device, 0x409840, 0xffffffff);
+ nvkm_wr32(device, 0x409800, 0x00000000);
nvkm_wr32(device, 0x409500, 0x00000000);
nvkm_wr32(device, 0x409504, 0x00000025);
nvkm_msec(device, 2000,
@@ -883,7 +941,7 @@ gf100_gr_fecs_discover_zcull_image_size(struct gf100_gr *gr, u32 *psize)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
- nvkm_wr32(device, 0x409840, 0xffffffff);
+ nvkm_wr32(device, 0x409800, 0x00000000);
nvkm_wr32(device, 0x409500, 0x00000000);
nvkm_wr32(device, 0x409504, 0x00000016);
nvkm_msec(device, 2000,
@@ -899,7 +957,7 @@ gf100_gr_fecs_discover_image_size(struct gf100_gr *gr, u32 *psize)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
- nvkm_wr32(device, 0x409840, 0xffffffff);
+ nvkm_wr32(device, 0x409800, 0x00000000);
nvkm_wr32(device, 0x409500, 0x00000000);
nvkm_wr32(device, 0x409504, 0x00000010);
nvkm_msec(device, 2000,
@@ -915,7 +973,7 @@ gf100_gr_fecs_set_watchdog_timeout(struct gf100_gr *gr, u32 timeout)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
- nvkm_wr32(device, 0x409840, 0xffffffff);
+ nvkm_wr32(device, 0x409800, 0x00000000);
nvkm_wr32(device, 0x409500, timeout);
nvkm_wr32(device, 0x409504, 0x00000021);
}
@@ -955,7 +1013,7 @@ gf100_gr_zbc_init(struct gf100_gr *gr)
const u32 f32_1[] = { 0x3f800000, 0x3f800000, 0x3f800000, 0x3f800000,
0x3f800000, 0x3f800000, 0x3f800000, 0x3f800000 };
struct nvkm_ltc *ltc = gr->base.engine.subdev.device->ltc;
- int index, c = ltc->zbc_min, d = ltc->zbc_min, s = ltc->zbc_min;
+ int index, c = ltc->zbc_color_min, d = ltc->zbc_depth_min, s = ltc->zbc_depth_min;
if (!gr->zbc_color[0].format) {
gf100_gr_zbc_color_get(gr, 1, & zero[0], &zero[4]); c++;
@@ -971,13 +1029,13 @@ gf100_gr_zbc_init(struct gf100_gr *gr)
}
}
- for (index = c; index <= ltc->zbc_max; index++)
+ for (index = c; index <= ltc->zbc_color_max; index++)
gr->func->zbc->clear_color(gr, index);
- for (index = d; index <= ltc->zbc_max; index++)
+ for (index = d; index <= ltc->zbc_depth_max; index++)
gr->func->zbc->clear_depth(gr, index);
if (gr->func->zbc->clear_stencil) {
- for (index = s; index <= ltc->zbc_max; index++)
+ for (index = s; index <= ltc->zbc_depth_max; index++)
gr->func->zbc->clear_stencil(gr, index);
}
}
@@ -1003,7 +1061,7 @@ gf100_gr_wait_idle(struct gf100_gr *gr)
nvkm_rd32(device, 0x400700);
gr_enabled = nvkm_rd32(device, 0x200) & 0x1000;
- ctxsw_active = nvkm_rd32(device, 0x2640) & 0x8000;
+ ctxsw_active = nvkm_fifo_ctxsw_in_progress(&gr->base.engine);
gr_busy = nvkm_rd32(device, 0x40060c) & 0x1;
if (!gr_enabled || (!gr_busy && !ctxsw_active))
@@ -1039,7 +1097,7 @@ gf100_gr_icmd(struct gf100_gr *gr, const struct gf100_gr_pack *p)
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_gr_pack *pack;
const struct gf100_gr_init *init;
- u32 data = 0;
+ u64 data = 0;
nvkm_wr32(device, 0x400208, 0x80000000);
@@ -1049,6 +1107,8 @@ gf100_gr_icmd(struct gf100_gr *gr, const struct gf100_gr_pack *p)
if ((pack == p && init == p->init) || data != init->data) {
nvkm_wr32(device, 0x400204, init->data);
+ if (pack->type == 64)
+ nvkm_wr32(device, 0x40020c, upper_32_bits(init->data));
data = init->data;
}
@@ -1542,13 +1602,13 @@ gf100_gr_ctxctl_isr(struct gf100_gr *gr)
}
}
-static void
-gf100_gr_intr(struct nvkm_gr *base)
+static irqreturn_t
+gf100_gr_intr(struct nvkm_inth *inth)
{
- struct gf100_gr *gr = gf100_gr(base);
+ struct gf100_gr *gr = container_of(inth, typeof(*gr), base.engine.subdev.inth);
struct nvkm_subdev *subdev = &gr->base.engine.subdev;
struct nvkm_device *device = subdev->device;
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
unsigned long flags;
u64 inst = nvkm_rd32(device, 0x409b00) & 0x0fffffff;
u32 stat = nvkm_rd32(device, 0x400100);
@@ -1561,10 +1621,10 @@ gf100_gr_intr(struct nvkm_gr *base)
const char *name = "unknown";
int chid = -1;
- chan = nvkm_fifo_chan_inst(device->fifo, (u64)inst << 12, &flags);
+ chan = nvkm_chan_get_inst(&gr->base.engine, (u64)inst << 12, &flags);
if (chan) {
- name = chan->object.client->name;
- chid = chan->chid;
+ name = chan->name;
+ chid = chan->id;
}
if (device->card_type < NV_E0 || subc < 4)
@@ -1631,7 +1691,8 @@ gf100_gr_intr(struct nvkm_gr *base)
}
nvkm_wr32(device, 0x400500, 0x00010001);
- nvkm_fifo_chan_put(device->fifo, flags, &chan);
+ nvkm_chan_put(&chan, flags);
+ return IRQ_HANDLED;
}
static void
@@ -1721,7 +1782,7 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr)
nvkm_mc_unk260(device, 1);
/* start both of them running */
- nvkm_wr32(device, 0x409840, 0xffffffff);
+ nvkm_wr32(device, 0x409800, 0x00000000);
nvkm_wr32(device, 0x41a10c, 0x00000000);
nvkm_wr32(device, 0x40910c, 0x00000000);
@@ -1763,15 +1824,6 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr)
return ret;
}
- /* Generate golden context image. */
- if (gr->data == NULL) {
- int ret = gf100_grctx_generate(gr);
- if (ret) {
- nvkm_error(subdev, "failed to construct context\n");
- return ret;
- }
- }
-
return 0;
}
@@ -1823,14 +1875,6 @@ gf100_gr_init_ctxctl_int(struct gf100_gr *gr)
}
gr->size = nvkm_rd32(device, 0x409804);
- if (gr->data == NULL) {
- int ret = gf100_grctx_generate(gr);
- if (ret) {
- nvkm_error(subdev, "failed to construct context\n");
- return ret;
- }
- }
-
return 0;
}
@@ -1847,10 +1891,11 @@ gf100_gr_init_ctxctl(struct gf100_gr *gr)
return ret;
}
-void
+int
gf100_gr_oneinit_sm_id(struct gf100_gr *gr)
{
int tpc, gpc;
+
for (tpc = 0; tpc < gr->tpc_max; tpc++) {
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
if (tpc < gr->tpc_nr[gpc]) {
@@ -1860,6 +1905,8 @@ gf100_gr_oneinit_sm_id(struct gf100_gr *gr)
}
}
}
+
+ return 0;
}
void
@@ -1944,7 +1991,17 @@ gf100_gr_oneinit(struct nvkm_gr *base)
struct gf100_gr *gr = gf100_gr(base);
struct nvkm_subdev *subdev = &gr->base.engine.subdev;
struct nvkm_device *device = subdev->device;
- int i, j;
+ struct nvkm_intr *intr = &device->mc->intr;
+ enum nvkm_intr_type intr_type = NVKM_INTR_SUBDEV;
+ int ret, i, j;
+
+ if (gr->func->oneinit_intr)
+ intr = gr->func->oneinit_intr(gr, &intr_type);
+
+ ret = nvkm_inth_add(intr, intr_type, NVKM_INTR_PRIO_NORMAL, &gr->base.engine.subdev,
+ gf100_gr_intr, &gr->base.engine.subdev.inth);
+ if (ret)
+ return ret;
nvkm_pmu_pgob(device->pmu, false);
@@ -1954,12 +2011,14 @@ gf100_gr_oneinit(struct nvkm_gr *base)
gr->tpc_nr[i] = nvkm_rd32(device, GPC_UNIT(i, 0x2608));
gr->tpc_max = max(gr->tpc_max, gr->tpc_nr[i]);
gr->tpc_total += gr->tpc_nr[i];
- gr->ppc_nr[i] = gr->func->ppc_nr;
- for (j = 0; j < gr->ppc_nr[i]; j++) {
+ for (j = 0; j < gr->func->ppc_nr; j++) {
gr->ppc_tpc_mask[i][j] =
nvkm_rd32(device, GPC_UNIT(i, 0x0c30 + (j * 4)));
if (gr->ppc_tpc_mask[i][j] == 0)
continue;
+
+ gr->ppc_nr[i]++;
+
gr->ppc_mask[i] |= (1 << j);
gr->ppc_tpc_nr[i][j] = hweight8(gr->ppc_tpc_mask[i][j]);
if (gr->ppc_tpc_min == 0 ||
@@ -1968,12 +2027,37 @@ gf100_gr_oneinit(struct nvkm_gr *base)
if (gr->ppc_tpc_max < gr->ppc_tpc_nr[i][j])
gr->ppc_tpc_max = gr->ppc_tpc_nr[i][j];
}
+
+ gr->ppc_total += gr->ppc_nr[i];
+ }
+
+ /* Allocate global context buffers. */
+ ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, gr->func->grctx->pagepool_size,
+ 0x100, false, &gr->pagepool);
+ if (ret)
+ return ret;
+
+ ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, gr->func->grctx->bundle_size,
+ 0x100, false, &gr->bundle_cb);
+ if (ret)
+ return ret;
+
+ ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, gr->func->grctx->attrib_cb_size(gr),
+ 0x1000, false, &gr->attrib_cb);
+ if (ret)
+ return ret;
+
+ if (gr->func->grctx->unknown_size) {
+ ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, gr->func->grctx->unknown_size,
+ 0x100, false, &gr->unknown);
+ if (ret)
+ return ret;
}
memset(gr->tile, 0xff, sizeof(gr->tile));
gr->func->oneinit_tiles(gr);
- gr->func->oneinit_sm_id(gr);
- return 0;
+
+ return gr->func->oneinit_sm_id(gr);
}
static int
@@ -1983,7 +2067,7 @@ gf100_gr_init_(struct nvkm_gr *base)
struct nvkm_subdev *subdev = &base->engine.subdev;
struct nvkm_device *device = subdev->device;
bool reset = device->chipset == 0x137 || device->chipset == 0x138;
- u32 ret;
+ int ret;
/* On certain GP107/GP108 boards, we trigger a weird issue where
* GR will stop responding to PRI accesses after we've asked the
@@ -2019,7 +2103,12 @@ gf100_gr_init_(struct nvkm_gr *base)
if (ret)
return ret;
- return gr->func->init(gr);
+ ret = gr->func->init(gr);
+ if (ret)
+ return ret;
+
+ nvkm_inth_allow(&subdev->inth);
+ return 0;
}
static int
@@ -2027,6 +2116,9 @@ gf100_gr_fini(struct nvkm_gr *base, bool suspend)
{
struct gf100_gr *gr = gf100_gr(base);
struct nvkm_subdev *subdev = &gr->base.engine.subdev;
+
+ nvkm_inth_block(&subdev->inth);
+
nvkm_falcon_put(&gr->gpccs.falcon, subdev);
nvkm_falcon_put(&gr->fecs.falcon, subdev);
return 0;
@@ -2039,6 +2131,11 @@ gf100_gr_dtor(struct nvkm_gr *base)
kfree(gr->data);
+ nvkm_memory_unref(&gr->unknown);
+ nvkm_memory_unref(&gr->attrib_cb);
+ nvkm_memory_unref(&gr->bundle_cb);
+ nvkm_memory_unref(&gr->pagepool);
+
nvkm_falcon_dtor(&gr->gpccs.falcon);
nvkm_falcon_dtor(&gr->fecs.falcon);
@@ -2047,81 +2144,27 @@ gf100_gr_dtor(struct nvkm_gr *base)
nvkm_blob_dtor(&gr->gpccs.inst);
nvkm_blob_dtor(&gr->gpccs.data);
+ vfree(gr->bundle64);
+ vfree(gr->bundle_veid);
vfree(gr->bundle);
vfree(gr->method);
vfree(gr->sw_ctx);
vfree(gr->sw_nonctx);
+ vfree(gr->sw_nonctx1);
+ vfree(gr->sw_nonctx2);
+ vfree(gr->sw_nonctx3);
+ vfree(gr->sw_nonctx4);
return gr;
}
-static const struct nvkm_gr_func
-gf100_gr_ = {
- .dtor = gf100_gr_dtor,
- .oneinit = gf100_gr_oneinit,
- .init = gf100_gr_init_,
- .fini = gf100_gr_fini,
- .intr = gf100_gr_intr,
- .units = gf100_gr_units,
- .chan_new = gf100_gr_chan_new,
- .object_get = gf100_gr_object_get,
- .chsw_load = gf100_gr_chsw_load,
- .ctxsw.pause = gf100_gr_fecs_stop_ctxsw,
- .ctxsw.resume = gf100_gr_fecs_start_ctxsw,
- .ctxsw.inst = gf100_gr_ctxsw_inst,
-};
-
static const struct nvkm_falcon_func
gf100_gr_flcn = {
- .fbif = 0x600,
.load_imem = nvkm_falcon_v1_load_imem,
.load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
- .bind_context = nvkm_falcon_v1_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
.start = nvkm_falcon_v1_start,
- .enable = nvkm_falcon_v1_enable,
- .disable = nvkm_falcon_v1_disable,
};
-int
-gf100_gr_new_(const struct gf100_gr_fwif *fwif, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_gr **pgr)
-{
- struct gf100_gr *gr;
- int ret;
-
- if (!(gr = kzalloc(sizeof(*gr), GFP_KERNEL)))
- return -ENOMEM;
- *pgr = &gr->base;
-
- ret = nvkm_gr_ctor(&gf100_gr_, device, type, inst, true, &gr->base);
- if (ret)
- return ret;
-
- fwif = nvkm_firmware_load(&gr->base.engine.subdev, fwif, "Gr", gr);
- if (IS_ERR(fwif))
- return PTR_ERR(fwif);
-
- gr->func = fwif->func;
-
- ret = nvkm_falcon_ctor(&gf100_gr_flcn, &gr->base.engine.subdev,
- "fecs", 0x409000, &gr->fecs.falcon);
- if (ret)
- return ret;
-
- mutex_init(&gr->fecs.mutex);
-
- ret = nvkm_falcon_ctor(&gf100_gr_flcn, &gr->base.engine.subdev,
- "gpccs", 0x41a000, &gr->gpccs.falcon);
- if (ret)
- return ret;
-
- return 0;
-}
-
void
gf100_gr_init_num_tpc_per_gpc(struct gf100_gr *gr, bool pd, bool ds)
{
@@ -2146,6 +2189,29 @@ gf100_gr_init_400054(struct gf100_gr *gr)
}
void
+gf100_gr_init_exception2(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x40011c, 0xffffffff);
+ nvkm_wr32(device, 0x400134, 0xffffffff);
+}
+
+void
+gf100_gr_init_rop_exceptions(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+ int rop;
+
+ for (rop = 0; rop < gr->rop_nr; rop++) {
+ nvkm_wr32(device, ROP_UNIT(rop, 0x144), 0x40000000);
+ nvkm_wr32(device, ROP_UNIT(rop, 0x070), 0x40000000);
+ nvkm_wr32(device, ROP_UNIT(rop, 0x204), 0xffffffff);
+ nvkm_wr32(device, ROP_UNIT(rop, 0x208), 0xffffffff);
+ }
+}
+
+void
gf100_gr_init_shader_exceptions(struct gf100_gr *gr, int gpc, int tpc)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
@@ -2252,21 +2318,47 @@ gf100_gr_init_vsc_stream_master(struct gf100_gr *gr)
nvkm_mask(device, TPC_UNIT(0, 0, 0x05c), 0x00000001, 0x00000001);
}
+static int
+gf100_gr_reset(struct nvkm_gr *base)
+{
+ struct nvkm_subdev *subdev = &base->engine.subdev;
+ struct nvkm_device *device = subdev->device;
+ struct gf100_gr *gr = gf100_gr(base);
+
+ nvkm_mask(device, 0x400500, 0x00000001, 0x00000000);
+
+ WARN_ON(gf100_gr_fecs_halt_pipeline(gr));
+
+ subdev->func->fini(subdev, false);
+ nvkm_mc_disable(device, subdev->type, subdev->inst);
+ if (gr->func->gpccs.reset)
+ gr->func->gpccs.reset(gr);
+
+ nvkm_mc_enable(device, subdev->type, subdev->inst);
+ return subdev->func->init(subdev);
+}
+
int
gf100_gr_init(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
- int gpc, tpc, rop;
+ int gpc, tpc;
- if (gr->func->init_419bd8)
- gr->func->init_419bd8(gr);
+ nvkm_mask(device, 0x400500, 0x00010001, 0x00000000);
gr->func->init_gpc_mmu(gr);
- if (gr->sw_nonctx)
+ if (gr->sw_nonctx1) {
+ gf100_gr_mmio(gr, gr->sw_nonctx1);
+ gf100_gr_mmio(gr, gr->sw_nonctx2);
+ gf100_gr_mmio(gr, gr->sw_nonctx3);
+ gf100_gr_mmio(gr, gr->sw_nonctx4);
+ } else
+ if (gr->sw_nonctx) {
gf100_gr_mmio(gr, gr->sw_nonctx);
- else
+ } else {
gf100_gr_mmio(gr, gr->func->mmio);
+ }
gf100_gr_wait_idle(gr);
@@ -2298,6 +2390,10 @@ gf100_gr_init(struct gf100_gr *gr)
nvkm_wr32(device, 0x400124, 0x00000002);
gr->func->init_fecs_exceptions(gr);
+
+ if (gr->func->init_40a790)
+ gr->func->init_40a790(gr);
+
if (gr->func->init_ds_hww_esr_2)
gr->func->init_ds_hww_esr_2(gr);
@@ -2346,19 +2442,14 @@ gf100_gr_init(struct gf100_gr *gr)
nvkm_wr32(device, GPC_UNIT(gpc, 0x2c94), 0xffffffff);
}
- for (rop = 0; rop < gr->rop_nr; rop++) {
- nvkm_wr32(device, ROP_UNIT(rop, 0x144), 0x40000000);
- nvkm_wr32(device, ROP_UNIT(rop, 0x070), 0x40000000);
- nvkm_wr32(device, ROP_UNIT(rop, 0x204), 0xffffffff);
- nvkm_wr32(device, ROP_UNIT(rop, 0x208), 0xffffffff);
- }
+ gr->func->init_rop_exceptions(gr);
nvkm_wr32(device, 0x400108, 0xffffffff);
nvkm_wr32(device, 0x400138, 0xffffffff);
nvkm_wr32(device, 0x400118, 0xffffffff);
nvkm_wr32(device, 0x400130, 0xffffffff);
- nvkm_wr32(device, 0x40011c, 0xffffffff);
- nvkm_wr32(device, 0x400134, 0xffffffff);
+ if (gr->func->init_exception2)
+ gr->func->init_exception2(gr);
if (gr->func->init_400054)
gr->func->init_400054(gr);
@@ -2371,6 +2462,18 @@ gf100_gr_init(struct gf100_gr *gr)
return gf100_gr_init_ctxctl(gr);
}
+void
+gf100_gr_fecs_reset(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x409614, 0x00000070);
+ nvkm_usec(device, 10, NVKM_DELAY);
+ nvkm_mask(device, 0x409614, 0x00000700, 0x00000700);
+ nvkm_usec(device, 10, NVKM_DELAY);
+ nvkm_rd32(device, 0x409614);
+}
+
#include "fuc/hubgf100.fuc3.h"
struct gf100_gr_ucode
@@ -2391,6 +2494,22 @@ gf100_gr_gpccs_ucode = {
.data.size = sizeof(gf100_grgpc_data),
};
+static const struct nvkm_gr_func
+gf100_gr_ = {
+ .dtor = gf100_gr_dtor,
+ .oneinit = gf100_gr_oneinit,
+ .init = gf100_gr_init_,
+ .fini = gf100_gr_fini,
+ .reset = gf100_gr_reset,
+ .units = gf100_gr_units,
+ .chan_new = gf100_gr_chan_new,
+ .object_get = gf100_gr_object_get,
+ .chsw_load = gf100_gr_chsw_load,
+ .ctxsw.pause = gf100_gr_fecs_stop_ctxsw,
+ .ctxsw.resume = gf100_gr_fecs_start_ctxsw,
+ .ctxsw.inst = gf100_gr_ctxsw_inst,
+};
+
static const struct gf100_gr_func
gf100_gr = {
.oneinit_tiles = gf100_gr_oneinit_tiles,
@@ -2406,10 +2525,13 @@ gf100_gr = {
.init_419eb4 = gf100_gr_init_419eb4,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gf100_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.grctx = &gf100_grctx,
@@ -2483,6 +2605,42 @@ gf100_gr_fwif[] = {
};
int
+gf100_gr_new_(const struct gf100_gr_fwif *fwif, struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_gr **pgr)
+{
+ struct gf100_gr *gr;
+ int ret;
+
+ if (!(gr = kzalloc(sizeof(*gr), GFP_KERNEL)))
+ return -ENOMEM;
+ *pgr = &gr->base;
+
+ ret = nvkm_gr_ctor(&gf100_gr_, device, type, inst, true, &gr->base);
+ if (ret)
+ return ret;
+
+ fwif = nvkm_firmware_load(&gr->base.engine.subdev, fwif, "Gr", gr);
+ if (IS_ERR(fwif))
+ return PTR_ERR(fwif);
+
+ gr->func = fwif->func;
+
+ ret = nvkm_falcon_ctor(&gf100_gr_flcn, &gr->base.engine.subdev,
+ "fecs", 0x409000, &gr->fecs.falcon);
+ if (ret)
+ return ret;
+
+ mutex_init(&gr->fecs.mutex);
+
+ ret = nvkm_falcon_ctor(&gf100_gr_flcn, &gr->base.engine.subdev,
+ "gpccs", 0x41a000, &gr->gpccs.falcon);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+int
gf100_gr_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, struct nvkm_gr **pgr)
{
return gf100_gr_new_(gf100_gr_fwif, device, type, inst, pgr);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h
index c0038f9..94ca7ac 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h
@@ -44,19 +44,6 @@ struct nvkm_acr_lsfw;
#define PPC_UNIT(t, m, r) (0x503000 + (t) * 0x8000 + (m) * 0x200 + (r))
#define TPC_UNIT(t, m, r) (0x504000 + (t) * 0x8000 + (m) * 0x800 + (r))
-struct gf100_gr_data {
- u32 size;
- u32 align;
- bool priv;
-};
-
-struct gf100_gr_mmio {
- u32 addr;
- u32 data;
- u32 shift;
- int buffer;
-};
-
struct gf100_gr_zbc_color {
u32 format;
u32 ds[4];
@@ -101,13 +88,19 @@ struct gf100_gr {
* using hardcoded arrays. To be allocated with vzalloc().
*/
struct gf100_gr_pack *sw_nonctx;
+ struct gf100_gr_pack *sw_nonctx1;
+ struct gf100_gr_pack *sw_nonctx2;
+ struct gf100_gr_pack *sw_nonctx3;
+ struct gf100_gr_pack *sw_nonctx4;
struct gf100_gr_pack *sw_ctx;
struct gf100_gr_pack *bundle;
+ struct gf100_gr_pack *bundle_veid;
+ struct gf100_gr_pack *bundle64;
struct gf100_gr_pack *method;
- struct gf100_gr_zbc_color zbc_color[NVKM_LTC_MAX_ZBC_CNT];
- struct gf100_gr_zbc_depth zbc_depth[NVKM_LTC_MAX_ZBC_CNT];
- struct gf100_gr_zbc_stencil zbc_stencil[NVKM_LTC_MAX_ZBC_CNT];
+ struct gf100_gr_zbc_color zbc_color[NVKM_LTC_MAX_ZBC_COLOR_CNT];
+ struct gf100_gr_zbc_depth zbc_depth[NVKM_LTC_MAX_ZBC_DEPTH_CNT];
+ struct gf100_gr_zbc_stencil zbc_stencil[NVKM_LTC_MAX_ZBC_DEPTH_CNT];
u8 rop_nr;
u8 gpc_nr;
@@ -120,6 +113,12 @@ struct gf100_gr {
u8 ppc_tpc_nr[GPC_MAX][4];
u8 ppc_tpc_min;
u8 ppc_tpc_max;
+ u8 ppc_total;
+
+ struct nvkm_memory *pagepool;
+ struct nvkm_memory *bundle_cb;
+ struct nvkm_memory *attrib_cb;
+ struct nvkm_memory *unknown;
u8 screen_tile_row_offset;
u8 tile[TPC_MAX];
@@ -130,8 +129,6 @@ struct gf100_gr {
} sm[TPC_MAX];
u8 sm_nr;
- struct gf100_gr_data mmio_data[4];
- struct gf100_gr_mmio mmio_list[4096/8];
u32 size;
u32 *data;
u32 size_zcull;
@@ -139,6 +136,7 @@ struct gf100_gr {
};
int gf100_gr_fecs_bind_pointer(struct gf100_gr *, u32 inst);
+int gf100_gr_fecs_wfi_golden_save(struct gf100_gr *, u32 inst);
struct gf100_gr_func_zbc {
void (*clear_color)(struct gf100_gr *, int zbc);
@@ -149,8 +147,9 @@ struct gf100_gr_func_zbc {
};
struct gf100_gr_func {
+ struct nvkm_intr *(*oneinit_intr)(struct gf100_gr *, enum nvkm_intr_type *);
void (*oneinit_tiles)(struct gf100_gr *);
- void (*oneinit_sm_id)(struct gf100_gr *);
+ int (*oneinit_sm_id)(struct gf100_gr *);
int (*init)(struct gf100_gr *);
void (*init_419bd8)(struct gf100_gr *);
void (*init_gpc_mmu)(struct gf100_gr *);
@@ -164,6 +163,7 @@ struct gf100_gr_func {
void (*init_swdx_pes_mask)(struct gf100_gr *);
void (*init_fs)(struct gf100_gr *);
void (*init_fecs_exceptions)(struct gf100_gr *);
+ void (*init_40a790)(struct gf100_gr *);
void (*init_ds_hww_esr_2)(struct gf100_gr *);
void (*init_40601c)(struct gf100_gr *);
void (*init_sked_hww_esr)(struct gf100_gr *);
@@ -174,6 +174,8 @@ struct gf100_gr_func {
void (*init_tex_hww_esr)(struct gf100_gr *, int gpc, int tpc);
void (*init_504430)(struct gf100_gr *, int gpc, int tpc);
void (*init_shader_exceptions)(struct gf100_gr *, int gpc, int tpc);
+ void (*init_rop_exceptions)(struct gf100_gr *);
+ void (*init_exception2)(struct gf100_gr *);
void (*init_400054)(struct gf100_gr *);
void (*init_4188a4)(struct gf100_gr *);
void (*trap_mp)(struct gf100_gr *, int gpc, int tpc);
@@ -181,9 +183,11 @@ struct gf100_gr_func {
const struct gf100_gr_pack *mmio;
struct {
struct gf100_gr_ucode *ucode;
+ void (*reset)(struct gf100_gr *);
} fecs;
struct {
struct gf100_gr_ucode *ucode;
+ void (*reset)(struct gf100_gr *);
} gpccs;
int (*rops)(struct gf100_gr *);
int gpc_nr;
@@ -197,7 +201,7 @@ struct gf100_gr_func {
int gf100_gr_rops(struct gf100_gr *);
void gf100_gr_oneinit_tiles(struct gf100_gr *);
-void gf100_gr_oneinit_sm_id(struct gf100_gr *);
+int gf100_gr_oneinit_sm_id(struct gf100_gr *);
int gf100_gr_init(struct gf100_gr *);
void gf100_gr_init_vsc_stream_master(struct gf100_gr *);
void gf100_gr_init_zcull(struct gf100_gr *);
@@ -208,9 +212,12 @@ void gf100_gr_init_419cc0(struct gf100_gr *);
void gf100_gr_init_419eb4(struct gf100_gr *);
void gf100_gr_init_tex_hww_esr(struct gf100_gr *, int, int);
void gf100_gr_init_shader_exceptions(struct gf100_gr *, int, int);
+void gf100_gr_init_rop_exceptions(struct gf100_gr *);
+void gf100_gr_init_exception2(struct gf100_gr *);
void gf100_gr_init_400054(struct gf100_gr *);
void gf100_gr_init_num_tpc_per_gpc(struct gf100_gr *, bool, bool);
extern const struct gf100_gr_func_zbc gf100_gr_zbc;
+void gf100_gr_fecs_reset(struct gf100_gr *);
void gf117_gr_init_zcull(struct gf100_gr *);
@@ -226,9 +233,13 @@ void gm107_gr_init_shader_exceptions(struct gf100_gr *, int, int);
void gm107_gr_init_400054(struct gf100_gr *);
int gk20a_gr_init(struct gf100_gr *);
+int gk20a_gr_av_to_init_(struct nvkm_blob *, u8 count, u32 pitch, struct gf100_gr_pack **);
+int gk20a_gr_av_to_init(struct nvkm_blob *, struct gf100_gr_pack **);
+int gk20a_gr_aiv_to_init(struct nvkm_blob *, struct gf100_gr_pack **);
+int gk20a_gr_av_to_method(struct nvkm_blob *, struct gf100_gr_pack **);
void gm200_gr_oneinit_tiles(struct gf100_gr *);
-void gm200_gr_oneinit_sm_id(struct gf100_gr *);
+int gm200_gr_oneinit_sm_id(struct gf100_gr *);
int gm200_gr_rops(struct gf100_gr *);
void gm200_gr_init_num_active_ltcs(struct gf100_gr *);
void gm200_gr_init_ds_hww_esr_2(struct gf100_gr *);
@@ -242,14 +253,24 @@ extern const struct gf100_gr_func_zbc gp100_gr_zbc;
void gp102_gr_init_swdx_pes_mask(struct gf100_gr *);
extern const struct gf100_gr_func_zbc gp102_gr_zbc;
+int gp102_gr_zbc_stencil_get(struct gf100_gr *, int, const u32, const u32);
+void gp102_gr_zbc_clear_stencil(struct gf100_gr *, int);
extern const struct gf100_gr_func gp107_gr;
+int gv100_gr_oneinit_sm_id(struct gf100_gr *);
+u32 gv100_gr_nonpes_aware_tpc(struct gf100_gr *gr, u32 gpc, u32 tpc);
void gv100_gr_init_419bd8(struct gf100_gr *);
void gv100_gr_init_504430(struct gf100_gr *, int, int);
void gv100_gr_init_shader_exceptions(struct gf100_gr *, int, int);
+void gv100_gr_init_4188a4(struct gf100_gr *);
void gv100_gr_trap_mp(struct gf100_gr *, int, int);
+int tu102_gr_av_to_init_veid(struct nvkm_blob *, struct gf100_gr_pack **);
+void tu102_gr_init_zcull(struct gf100_gr *);
+void tu102_gr_init_fs(struct gf100_gr *);
+void tu102_gr_init_fecs_exceptions(struct gf100_gr *);
+
#define gf100_gr_chan(p) container_of((p), struct gf100_gr_chan, object)
#include <core/object.h>
@@ -258,14 +279,14 @@ struct gf100_gr_chan {
struct gf100_gr *gr;
struct nvkm_vmm *vmm;
+ struct nvkm_vma *pagepool;
+ struct nvkm_vma *bundle_cb;
+ struct nvkm_vma *attrib_cb;
+ struct nvkm_vma *unknown;
+
struct nvkm_memory *mmio;
struct nvkm_vma *mmio_vma;
int mmio_nr;
-
- struct {
- struct nvkm_memory *mem;
- struct nvkm_vma *vma;
- } data[4];
};
void gf100_gr_ctxctl_debug(struct gf100_gr *);
@@ -279,7 +300,7 @@ struct gf100_gr_init {
u32 addr;
u8 count;
u32 pitch;
- u32 data;
+ u64 data;
};
struct gf100_gr_pack {
@@ -403,6 +424,9 @@ int gf100_gr_load(struct gf100_gr *, int, const struct gf100_gr_fwif *);
int gf100_gr_nofw(struct gf100_gr *, int, const struct gf100_gr_fwif *);
int gk20a_gr_load_sw(struct gf100_gr *, const char *path, int ver);
+int gk20a_gr_load_net(struct gf100_gr *, const char *, const char *, int,
+ int (*)(struct nvkm_blob *, struct gf100_gr_pack **),
+ struct gf100_gr_pack **);
int gm200_gr_nofw(struct gf100_gr *, int, const struct gf100_gr_fwif *);
int gm200_gr_load(struct gf100_gr *, int, const struct gf100_gr_fwif *);
@@ -415,6 +439,8 @@ void gm20b_gr_acr_bld_patch(struct nvkm_acr *, u32, s64);
extern const struct nvkm_acr_lsf_func gp108_gr_gpccs_acr;
extern const struct nvkm_acr_lsf_func gp108_gr_fecs_acr;
+void gp108_gr_acr_bld_write(struct nvkm_acr *, u32, struct nvkm_acr_lsfw *);
+void gp108_gr_acr_bld_patch(struct nvkm_acr *, u32, s64);
int gf100_gr_new_(const struct gf100_gr_fwif *, struct nvkm_device *, enum nvkm_subdev_type, int,
struct nvkm_gr **);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c
index 3acd99c..63bd29c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c
@@ -127,10 +127,13 @@ gf104_gr = {
.init_419eb4 = gf100_gr_init_419eb4,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gf104_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.grctx = &gf104_grctx,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c
index ab3760e..495a844 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c
@@ -125,10 +125,13 @@ gf108_gr = {
.init_419eb4 = gf100_gr_init_419eb4,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gf108_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.grctx = &gf108_grctx,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c
index 616e2de..70fad23 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c
@@ -99,10 +99,13 @@ gf110_gr = {
.init_419eb4 = gf100_gr_init_419eb4,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gf110_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.grctx = &gf110_grctx,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c
index 669e753..f127282 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c
@@ -125,7 +125,9 @@ gf117_gr_init_zcull(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const u32 magicgpc918 = DIV_ROUND_UP(0x00800000, gr->tpc_total);
- const u8 tile_nr = ALIGN(gr->tpc_total, 32);
+ /*TODO: fill in litter vals for gf117-gm2xx */
+ const u8 tile_nr = !gr->func->gpc_nr ? ALIGN(gr->tpc_total, 32) :
+ (gr->func->gpc_nr * gr->func->tpc_nr);
u8 bank[GPC_MAX] = {}, gpc, i, j;
u32 data;
@@ -163,10 +165,13 @@ gf117_gr = {
.init_419eb4 = gf100_gr_init_419eb4,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gf117_gr_pack_mmio,
.fecs.ucode = &gf117_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gf117_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.ppc_nr = 1,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c
index 5b09bda..75ceb51 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c
@@ -190,10 +190,13 @@ gf119_gr = {
.init_419eb4 = gf100_gr_init_419eb4,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gf119_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.grctx = &gf119_grctx,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c
index b680eaa..e53ade24 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c
@@ -418,7 +418,7 @@ gk104_gr_init_ppc_exceptions(struct gf100_gr *gr)
int gpc, ppc;
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++) {
+ for (ppc = 0; ppc < gr->func->ppc_nr; ppc++) {
if (!(gr->ppc_mask[gpc] & (1 << ppc)))
continue;
nvkm_wr32(device, PPC_UNIT(gpc, ppc, 0x038), 0xc0000000);
@@ -470,10 +470,13 @@ gk104_gr = {
.init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gk104_gr_pack_mmio,
.fecs.ucode = &gk104_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gk104_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.ppc_nr = 1,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c
index 103e06a..c7e1c5d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c
@@ -366,10 +366,13 @@ gk110_gr = {
.init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gk110_gr_pack_mmio,
.fecs.ucode = &gk110_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gk110_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.ppc_nr = 2,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c
index 034d0b1..458abae 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c
@@ -118,10 +118,13 @@ gk110b_gr = {
.init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gk110b_gr_pack_mmio,
.fecs.ucode = &gk110_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gk110_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.ppc_nr = 2,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c
index 116d682..d3f6b65c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c
@@ -176,10 +176,13 @@ gk208_gr = {
.init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_shader_exceptions = gf100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gf100_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gk208_gr_pack_mmio,
.fecs.ucode = &gk208_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gk208_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.ppc_nr = 1,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c
index be0b2ce..035ea21 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c
@@ -33,47 +33,40 @@ struct gk20a_fw_av
u32 data;
};
-static int
-gk20a_gr_av_to_init(struct gf100_gr *gr, const char *path, const char *name,
- int ver, struct gf100_gr_pack **ppack)
+int
+gk20a_gr_av_to_init_(struct nvkm_blob *blob, u8 count, u32 pitch, struct gf100_gr_pack **ppack)
{
- struct nvkm_subdev *subdev = &gr->base.engine.subdev;
- struct nvkm_blob blob;
struct gf100_gr_init *init;
struct gf100_gr_pack *pack;
int nent;
- int ret;
int i;
- ret = nvkm_firmware_load_blob(subdev, path, name, ver, &blob);
- if (ret)
- return ret;
-
- nent = (blob.size / sizeof(struct gk20a_fw_av));
+ nent = (blob->size / sizeof(struct gk20a_fw_av));
pack = vzalloc((sizeof(*pack) * 2) + (sizeof(*init) * (nent + 1)));
- if (!pack) {
- ret = -ENOMEM;
- goto end;
- }
+ if (!pack)
+ return -ENOMEM;
init = (void *)(pack + 2);
pack[0].init = init;
for (i = 0; i < nent; i++) {
struct gf100_gr_init *ent = &init[i];
- struct gk20a_fw_av *av = &((struct gk20a_fw_av *)blob.data)[i];
+ struct gk20a_fw_av *av = &((struct gk20a_fw_av *)blob->data)[i];
ent->addr = av->addr;
ent->data = av->data;
- ent->count = 1;
- ent->pitch = 1;
+ ent->count = ((ent->addr & 0xffff) != 0xe100) ? count : 1;
+ ent->pitch = pitch;
}
*ppack = pack;
+ return 0;
+}
-end:
- nvkm_blob_dtor(&blob);
- return ret;
+int
+gk20a_gr_av_to_init(struct nvkm_blob *blob, struct gf100_gr_pack **ppack)
+{
+ return gk20a_gr_av_to_init_(blob, 1, 1, ppack);
}
struct gk20a_fw_aiv
@@ -83,35 +76,25 @@ struct gk20a_fw_aiv
u32 data;
};
-static int
-gk20a_gr_aiv_to_init(struct gf100_gr *gr, const char *path, const char *name,
- int ver, struct gf100_gr_pack **ppack)
+int
+gk20a_gr_aiv_to_init(struct nvkm_blob *blob, struct gf100_gr_pack **ppack)
{
- struct nvkm_subdev *subdev = &gr->base.engine.subdev;
- struct nvkm_blob blob;
struct gf100_gr_init *init;
struct gf100_gr_pack *pack;
int nent;
- int ret;
int i;
- ret = nvkm_firmware_load_blob(subdev, path, name, ver, &blob);
- if (ret)
- return ret;
-
- nent = (blob.size / sizeof(struct gk20a_fw_aiv));
+ nent = (blob->size / sizeof(struct gk20a_fw_aiv));
pack = vzalloc((sizeof(*pack) * 2) + (sizeof(*init) * (nent + 1)));
- if (!pack) {
- ret = -ENOMEM;
- goto end;
- }
+ if (!pack)
+ return -ENOMEM;
init = (void *)(pack + 2);
pack[0].init = init;
for (i = 0; i < nent; i++) {
struct gf100_gr_init *ent = &init[i];
- struct gk20a_fw_aiv *av = &((struct gk20a_fw_aiv *)blob.data)[i];
+ struct gk20a_fw_aiv *av = &((struct gk20a_fw_aiv *)blob->data)[i];
ent->addr = av->addr;
ent->data = av->data;
@@ -120,44 +103,30 @@ gk20a_gr_aiv_to_init(struct gf100_gr *gr, const char *path, const char *name,
}
*ppack = pack;
-
-end:
- nvkm_blob_dtor(&blob);
- return ret;
+ return 0;
}
-static int
-gk20a_gr_av_to_method(struct gf100_gr *gr, const char *path, const char *name,
- int ver, struct gf100_gr_pack **ppack)
+int
+gk20a_gr_av_to_method(struct nvkm_blob *blob, struct gf100_gr_pack **ppack)
{
- struct nvkm_subdev *subdev = &gr->base.engine.subdev;
- struct nvkm_blob blob;
struct gf100_gr_init *init;
struct gf100_gr_pack *pack;
/* We don't suppose we will initialize more than 16 classes here... */
static const unsigned int max_classes = 16;
u32 classidx = 0, prevclass = 0;
int nent;
- int ret;
int i;
- ret = nvkm_firmware_load_blob(subdev, path, name, ver, &blob);
- if (ret)
- return ret;
-
- nent = (blob.size / sizeof(struct gk20a_fw_av));
-
+ nent = (blob->size / sizeof(struct gk20a_fw_av));
pack = vzalloc((sizeof(*pack) * (max_classes + 1)) +
(sizeof(*init) * (nent + max_classes + 1)));
- if (!pack) {
- ret = -ENOMEM;
- goto end;
- }
+ if (!pack)
+ return -ENOMEM;
init = (void *)(pack + max_classes + 1);
for (i = 0; i < nent; i++, init++) {
- struct gk20a_fw_av *av = &((struct gk20a_fw_av *)blob.data)[i];
+ struct gk20a_fw_av *av = &((struct gk20a_fw_av *)blob->data)[i];
u32 class = av->addr & 0xffff;
u32 addr = (av->addr & 0xffff0000) >> 14;
@@ -169,8 +138,7 @@ gk20a_gr_av_to_method(struct gf100_gr *gr, const char *path, const char *name,
prevclass = class;
if (++classidx >= max_classes) {
vfree(pack);
- ret = -ENOSPC;
- goto end;
+ return -ENOSPC;
}
}
@@ -181,10 +149,7 @@ gk20a_gr_av_to_method(struct gf100_gr *gr, const char *path, const char *name,
}
*ppack = pack;
-
-end:
- nvkm_blob_dtor(&blob);
- return ret;
+ return 0;
}
static int
@@ -294,6 +259,7 @@ gk20a_gr = {
.init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
.trap_mp = gf100_gr_trap_mp,
.set_hww_esr_report_mask = gk20a_gr_set_hww_esr_report_mask,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gf100_gr_rops,
.ppc_nr = 1,
.grctx = &gk20a_grctx,
@@ -308,12 +274,29 @@ gk20a_gr = {
};
int
+gk20a_gr_load_net(struct gf100_gr *gr, const char *path, const char *name, int ver,
+ int (*load)(struct nvkm_blob *, struct gf100_gr_pack **),
+ struct gf100_gr_pack **ppack)
+{
+ struct nvkm_blob blob;
+ int ret;
+
+ ret = nvkm_firmware_load_blob(&gr->base.engine.subdev, path, name, ver, &blob);
+ if (ret)
+ return ret;
+
+ ret = load(&blob, ppack);
+ nvkm_blob_dtor(&blob);
+ return 0;
+}
+
+int
gk20a_gr_load_sw(struct gf100_gr *gr, const char *path, int ver)
{
- if (gk20a_gr_av_to_init(gr, path, "sw_nonctx", ver, &gr->sw_nonctx) ||
- gk20a_gr_aiv_to_init(gr, path, "sw_ctx", ver, &gr->sw_ctx) ||
- gk20a_gr_av_to_init(gr, path, "sw_bundle_init", ver, &gr->bundle) ||
- gk20a_gr_av_to_method(gr, path, "sw_method_init", ver, &gr->method))
+ if (gk20a_gr_load_net(gr, path, "sw_nonctx", ver, gk20a_gr_av_to_init, &gr->sw_nonctx) ||
+ gk20a_gr_load_net(gr, path, "sw_ctx", ver, gk20a_gr_aiv_to_init, &gr->sw_ctx) ||
+ gk20a_gr_load_net(gr, path, "sw_bundle_init", ver, gk20a_gr_av_to_init, &gr->bundle) ||
+ gk20a_gr_load_net(gr, path, "sw_method_init", ver, gk20a_gr_av_to_method, &gr->method))
return -ENOENT;
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c
index 3109871..797b828 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c
@@ -411,10 +411,13 @@ gm107_gr = {
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_504430 = gm107_gr_init_504430,
.init_shader_exceptions = gm107_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gm107_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
.mmio = gm107_gr_pack_mmio,
.fecs.ucode = &gm107_gr_fecs_ucode,
+ .fecs.reset = gf100_gr_fecs_reset,
.gpccs.ucode = &gm107_gr_gpccs_ucode,
.rops = gf100_gr_rops,
.ppc_nr = 2,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c
index 385cfd9..b5210b3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c
@@ -148,11 +148,11 @@ gm200_gr_tile_map_2_8[] = {
0, 1, 1, 0, 0, 1, 1, 0,
};
-void
+int
gm200_gr_oneinit_sm_id(struct gf100_gr *gr)
{
/*XXX: There's a different algorithm here I've not yet figured out. */
- gf100_gr_oneinit_sm_id(gr);
+ return gf100_gr_oneinit_sm_id(gr);
}
void
@@ -199,8 +199,11 @@ gm200_gr = {
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_504430 = gm107_gr_init_504430,
.init_shader_exceptions = gm107_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_400054 = gm107_gr_init_400054,
.trap_mp = gf100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.tpc_nr = 4,
.ppc_nr = 2,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c
index ec1c46e..458cd1a0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c
@@ -123,6 +123,7 @@ gm20b_gr = {
.init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
.trap_mp = gf100_gr_trap_mp,
.set_hww_esr_report_mask = gm20b_gr_set_hww_esr_report_mask,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.ppc_nr = 1,
.grctx = &gm20b_grctx,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp100.c
index 0550dd6..851e743 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp100.c
@@ -87,7 +87,7 @@ gp100_gr_init_419c9c(struct gf100_gr *gr)
void
gp100_gr_init_fecs_exceptions(struct gf100_gr *gr)
{
- nvkm_wr32(gr->base.engine.subdev.device, 0x409c24, 0x000f0002);
+ nvkm_wr32(gr->base.engine.subdev.device, 0x409c24, 0x000e0002);
}
void
@@ -119,7 +119,10 @@ gp100_gr = {
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_504430 = gm107_gr_init_504430,
.init_shader_exceptions = gp100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.trap_mp = gf100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.gpc_nr = 6,
.tpc_nr = 5,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp102.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp102.c
index 5b001f3..0e223b7b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp102.c
@@ -26,7 +26,7 @@
#include <nvif/class.h>
-static void
+void
gp102_gr_zbc_clear_stencil(struct gf100_gr *gr, int zbc)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
@@ -40,14 +40,14 @@ gp102_gr_zbc_clear_stencil(struct gf100_gr *gr, int zbc)
gr->zbc_stencil[zbc].format << ((znum % 4) * 7));
}
-static int
+int
gp102_gr_zbc_stencil_get(struct gf100_gr *gr, int format,
const u32 ds, const u32 l2)
{
struct nvkm_ltc *ltc = gr->base.engine.subdev.device->ltc;
int zbc = -ENOSPC, i;
- for (i = ltc->zbc_min; i <= ltc->zbc_max; i++) {
+ for (i = ltc->zbc_depth_min; i <= ltc->zbc_depth_max; i++) {
if (gr->zbc_stencil[i].format) {
if (gr->zbc_stencil[i].format != format)
continue;
@@ -115,7 +115,10 @@ gp102_gr = {
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_504430 = gm107_gr_init_504430,
.init_shader_exceptions = gp100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.trap_mp = gf100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.gpc_nr = 6,
.tpc_nr = 5,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp104.c
index 2655574..6802cb9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp104.c
@@ -43,7 +43,10 @@ gp104_gr = {
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_504430 = gm107_gr_init_504430,
.init_shader_exceptions = gp100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.trap_mp = gf100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.gpc_nr = 6,
.tpc_nr = 5,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp107.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp107.c
index adabc04..cc2bb0d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp107.c
@@ -45,7 +45,10 @@ gp107_gr = {
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_504430 = gm107_gr_init_504430,
.init_shader_exceptions = gp100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.trap_mp = gf100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.gpc_nr = 2,
.tpc_nr = 3,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp108.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp108.c
index 7310f04..311f703 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp108.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp108.c
@@ -25,7 +25,7 @@
#include <nvfw/flcn.h>
-static void
+void
gp108_gr_acr_bld_patch(struct nvkm_acr *acr, u32 bld, s64 adjust)
{
struct flcn_bl_dmem_desc_v2 hdr;
@@ -36,7 +36,7 @@ gp108_gr_acr_bld_patch(struct nvkm_acr *acr, u32 bld, s64 adjust)
flcn_bl_dmem_desc_v2_dump(&acr->subdev, &hdr);
}
-static void
+void
gp108_gr_acr_bld_write(struct nvkm_acr *acr, u32 bld,
struct nvkm_acr_lsfw *lsfw)
{
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp10b.c
index e13683b6..5008881c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp10b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gp10b.c
@@ -55,7 +55,10 @@ gp10b_gr = {
.init_tex_hww_esr = gf100_gr_init_tex_hww_esr,
.init_504430 = gm107_gr_init_504430,
.init_shader_exceptions = gp100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.trap_mp = gf100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.gpc_nr = 1,
.tpc_nr = 2,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gv100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gv100.c
index 4d043c1..7f7404a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gv100.c
@@ -52,10 +52,11 @@ gv100_gr_trap_mp(struct gf100_gr *gr, int gpc, int tpc)
gv100_gr_trap_sm(gr, gpc, tpc, 1);
}
-static void
+void
gv100_gr_init_4188a4(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
+
nvkm_mask(device, 0x4188a4, 0x03000000, 0x03000000);
}
@@ -65,7 +66,6 @@ gv100_gr_init_shader_exceptions(struct gf100_gr *gr, int gpc, int tpc)
struct nvkm_device *device = gr->base.engine.subdev.device;
int sm;
for (sm = 0; sm < 0x100; sm += 0x80) {
- nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x728 + sm), 0x0085eb64);
nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x610), 0x00000001);
nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x72c + sm), 0x00000004);
}
@@ -85,10 +85,202 @@ gv100_gr_init_419bd8(struct gf100_gr *gr)
nvkm_mask(device, 0x419bd8, 0x00000700, 0x00000000);
}
+u32
+gv100_gr_nonpes_aware_tpc(struct gf100_gr *gr, u32 gpc, u32 tpc)
+{
+ u32 pes, temp, tpc_new = 0;
+
+ for (pes = 0; pes < gr->ppc_nr[gpc]; pes++) {
+ if (gr->ppc_tpc_mask[gpc][pes] & BIT(tpc))
+ break;
+
+ tpc_new += gr->ppc_tpc_nr[gpc][pes];
+ }
+
+ temp = (BIT(tpc) - 1) & gr->ppc_tpc_mask[gpc][pes];
+ temp = hweight32(temp);
+ return tpc_new + temp;
+}
+
+static int
+gv100_gr_scg_estimate_perf(struct gf100_gr *gr, unsigned long *gpc_tpc_mask,
+ u32 disable_gpc, u32 disable_tpc, int *perf)
+{
+ const u32 scale_factor = 512UL; /* Use fx23.9 */
+ const u32 pix_scale = 1024*1024UL; /* Pix perf in [29:20] */
+ const u32 world_scale = 1024UL; /* World performance in [19:10] */
+ const u32 tpc_scale = 1; /* TPC balancing in [9:0] */
+ u32 scg_num_pes = 0;
+ u32 min_scg_gpc_pix_perf = scale_factor; /* Init perf as maximum */
+ u32 average_tpcs = 0; /* Average of # of TPCs per GPC */
+ u32 deviation; /* absolute diff between TPC# and average_tpcs, averaged across GPCs */
+ u32 norm_tpc_deviation; /* deviation/max_tpc_per_gpc */
+ u32 tpc_balance;
+ u32 scg_gpc_pix_perf;
+ u32 scg_world_perf;
+ u32 gpc;
+ u32 pes;
+ int diff;
+ bool tpc_removed_gpc = false;
+ bool tpc_removed_pes = false;
+ u32 max_tpc_gpc = 0;
+ u32 num_tpc_mask;
+ u32 *num_tpc_gpc;
+ int ret = -EINVAL;
+
+ if (!(num_tpc_gpc = kcalloc(gr->gpc_nr, sizeof(*num_tpc_gpc), GFP_KERNEL)))
+ return -ENOMEM;
+
+ /* Calculate pix-perf-reduction-rate per GPC and find bottleneck TPC */
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
+ num_tpc_mask = gpc_tpc_mask[gpc];
+
+ if ((gpc == disable_gpc) && num_tpc_mask & BIT(disable_tpc)) {
+ /* Safety check if a TPC is removed twice */
+ if (WARN_ON(tpc_removed_gpc))
+ goto done;
+
+ /* Remove logical TPC from set */
+ num_tpc_mask &= ~BIT(disable_tpc);
+ tpc_removed_gpc = true;
+ }
+
+ /* track balancing of tpcs across gpcs */
+ num_tpc_gpc[gpc] = hweight32(num_tpc_mask);
+ average_tpcs += num_tpc_gpc[gpc];
+
+ /* save the maximum numer of gpcs */
+ max_tpc_gpc = num_tpc_gpc[gpc] > max_tpc_gpc ? num_tpc_gpc[gpc] : max_tpc_gpc;
+
+ /*
+ * Calculate ratio between TPC count and post-FS and post-SCG
+ *
+ * ratio represents relative throughput of the GPC
+ */
+ scg_gpc_pix_perf = scale_factor * num_tpc_gpc[gpc] / gr->tpc_nr[gpc];
+ if (min_scg_gpc_pix_perf > scg_gpc_pix_perf)
+ min_scg_gpc_pix_perf = scg_gpc_pix_perf;
+
+ /* Calculate # of surviving PES */
+ for (pes = 0; pes < gr->ppc_nr[gpc]; pes++) {
+ /* Count the number of TPC on the set */
+ num_tpc_mask = gr->ppc_tpc_mask[gpc][pes] & gpc_tpc_mask[gpc];
+
+ if ((gpc == disable_gpc) && (num_tpc_mask & BIT(disable_tpc))) {
+ if (WARN_ON(tpc_removed_pes))
+ goto done;
+
+ num_tpc_mask &= ~BIT(disable_tpc);
+ tpc_removed_pes = true;
+ }
+
+ if (hweight32(num_tpc_mask))
+ scg_num_pes++;
+ }
+ }
+
+ if (WARN_ON(!tpc_removed_gpc || !tpc_removed_pes))
+ goto done;
+
+ if (max_tpc_gpc == 0) {
+ *perf = 0;
+ goto done_ok;
+ }
+
+ /* Now calculate perf */
+ scg_world_perf = (scale_factor * scg_num_pes) / gr->ppc_total;
+ deviation = 0;
+ average_tpcs = scale_factor * average_tpcs / gr->gpc_nr;
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
+ diff = average_tpcs - scale_factor * num_tpc_gpc[gpc];
+ if (diff < 0)
+ diff = -diff;
+
+ deviation += diff;
+ }
+
+ deviation /= gr->gpc_nr;
+
+ norm_tpc_deviation = deviation / max_tpc_gpc;
+
+ tpc_balance = scale_factor - norm_tpc_deviation;
+
+ if ((tpc_balance > scale_factor) ||
+ (scg_world_perf > scale_factor) ||
+ (min_scg_gpc_pix_perf > scale_factor) ||
+ (norm_tpc_deviation > scale_factor)) {
+ WARN_ON(1);
+ goto done;
+ }
+
+ *perf = (pix_scale * min_scg_gpc_pix_perf) +
+ (world_scale * scg_world_perf) +
+ (tpc_scale * tpc_balance);
+done_ok:
+ ret = 0;
+done:
+ kfree(num_tpc_gpc);
+ return ret;
+}
+
+int
+gv100_gr_oneinit_sm_id(struct gf100_gr *gr)
+{
+ unsigned long *gpc_tpc_mask;
+ u32 *tpc_table, *gpc_table;
+ u32 gpc, tpc, pes, gtpc;
+ int perf, maxperf, ret = 0;
+
+ gpc_tpc_mask = kcalloc(gr->gpc_nr, sizeof(*gpc_tpc_mask), GFP_KERNEL);
+ gpc_table = kcalloc(gr->tpc_total, sizeof(*gpc_table), GFP_KERNEL);
+ tpc_table = kcalloc(gr->tpc_total, sizeof(*tpc_table), GFP_KERNEL);
+ if (!gpc_table || !tpc_table || !gpc_tpc_mask) {
+ ret = -ENOMEM;
+ goto done;
+ }
+
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
+ for (pes = 0; pes < gr->ppc_nr[gpc]; pes++)
+ gpc_tpc_mask[gpc] |= gr->ppc_tpc_mask[gpc][pes];
+ }
+
+ for (gtpc = 0; gtpc < gr->tpc_total; gtpc++) {
+ for (maxperf = -1, gpc = 0; gpc < gr->gpc_nr; gpc++) {
+ for_each_set_bit(tpc, &gpc_tpc_mask[gpc], gr->tpc_nr[gpc]) {
+ ret = gv100_gr_scg_estimate_perf(gr, gpc_tpc_mask, gpc, tpc, &perf);
+ if (ret)
+ goto done;
+
+ /* nvgpu does ">=" here, but this gets us RM's numbers. */
+ if (perf > maxperf) {
+ maxperf = perf;
+ gpc_table[gtpc] = gpc;
+ tpc_table[gtpc] = tpc;
+ }
+ }
+ }
+
+ gpc_tpc_mask[gpc_table[gtpc]] &= ~BIT(tpc_table[gtpc]);
+ }
+
+ /*TODO: build table for sm_per_tpc != 1, don't use yet, but might need later? */
+ for (gtpc = 0; gtpc < gr->tpc_total; gtpc++) {
+ gr->sm[gtpc].gpc = gpc_table[gtpc];
+ gr->sm[gtpc].tpc = tpc_table[gtpc];
+ gr->sm_nr++;
+ }
+
+done:
+ kfree(gpc_table);
+ kfree(tpc_table);
+ kfree(gpc_tpc_mask);
+ return ret;
+}
+
static const struct gf100_gr_func
gv100_gr = {
.oneinit_tiles = gm200_gr_oneinit_tiles,
- .oneinit_sm_id = gm200_gr_oneinit_sm_id,
+ .oneinit_sm_id = gv100_gr_oneinit_sm_id,
.init = gf100_gr_init,
.init_419bd8 = gv100_gr_init_419bd8,
.init_gpc_mmu = gm200_gr_init_gpc_mmu,
@@ -103,11 +295,14 @@ gv100_gr = {
.init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.init_504430 = gv100_gr_init_504430,
.init_shader_exceptions = gv100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
.init_4188a4 = gv100_gr_init_4188a4,
.trap_mp = gv100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.gpc_nr = 6,
- .tpc_nr = 5,
+ .tpc_nr = 7,
.ppc_nr = 3,
.grctx = &gv100_grctx,
.zbc = &gp102_gr_zbc,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c
index 0bc1a23..81bd682 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c
@@ -1192,7 +1192,7 @@ nv04_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv04_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
*ctx_reg(chan, NV04_PGRAPH_DEBUG_3) = 0xfad4ff31;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c
index 942450b..7fe6e58f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c
@@ -1011,7 +1011,7 @@ nv10_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv10_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
NV_WRITE_CTX(0x00400e88, 0x08000000);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c
index 6bff10c..75434f5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c
@@ -83,7 +83,7 @@ nv20_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv20_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
ret = nvkm_memory_new(gr->base.engine.subdev.device,
@@ -182,7 +182,7 @@ nv20_gr_intr(struct nvkm_gr *base)
struct nv20_gr *gr = nv20_gr(base);
struct nvkm_subdev *subdev = &gr->base.engine.subdev;
struct nvkm_device *device = subdev->device;
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
u32 stat = nvkm_rd32(device, NV03_PGRAPH_INTR);
u32 nsource = nvkm_rd32(device, NV03_PGRAPH_NSOURCE);
u32 nstatus = nvkm_rd32(device, NV03_PGRAPH_NSTATUS);
@@ -196,7 +196,7 @@ nv20_gr_intr(struct nvkm_gr *base)
char msg[128], src[128], sta[128];
unsigned long flags;
- chan = nvkm_fifo_chan_chid(device->fifo, chid, &flags);
+ chan = nvkm_chan_get_chid(&gr->base.engine, chid, &flags);
nvkm_wr32(device, NV03_PGRAPH_INTR, stat);
nvkm_wr32(device, NV04_PGRAPH_FIFO, 0x00000001);
@@ -209,11 +209,11 @@ nv20_gr_intr(struct nvkm_gr *base)
"nstatus %08x [%s] ch %d [%s] subc %d "
"class %04x mthd %04x data %08x\n",
show, msg, nsource, src, nstatus, sta, chid,
- chan ? chan->object.client->name : "unknown",
+ chan ? chan->name : "unknown",
subc, class, mthd, data);
}
- nvkm_fifo_chan_put(device->fifo, flags, &chan);
+ nvkm_chan_put(&chan, flags);
}
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv25.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv25.c
index f3a56f17..94685e4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv25.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv25.c
@@ -29,7 +29,7 @@ nv25_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv25_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
ret = nvkm_memory_new(gr->base.engine.subdev.device,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv2a.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv2a.c
index f268d26..2d62736 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv2a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv2a.c
@@ -29,7 +29,7 @@ nv2a_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv2a_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
ret = nvkm_memory_new(gr->base.engine.subdev.device,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv30.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv30.c
index e5737cd..647bd6f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv30.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv30.c
@@ -30,7 +30,7 @@ nv30_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv30_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
ret = nvkm_memory_new(gr->base.engine.subdev.device,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv34.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv34.c
index 1ab2da8..2eae3fe 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv34.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv34.c
@@ -29,7 +29,7 @@ nv34_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv34_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
ret = nvkm_memory_new(gr->base.engine.subdev.device,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv35.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv35.c
index 591260f..657d7cdb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv35.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv35.c
@@ -29,7 +29,7 @@ nv35_gr_chan_new(struct nvkm_gr *base, struct nvkm_fifo_chan *fifoch,
return -ENOMEM;
nvkm_object_ctor(&nv35_gr_chan, oclass, &chan->object);
chan->gr = gr;
- chan->chid = fifoch->chid;
+ chan->chid = fifoch->id;
*pobject = &chan->object;
ret = nvkm_memory_new(gr->base.engine.subdev.device,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c
index 67f3535..d2df097 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c
@@ -275,8 +275,8 @@ nv40_gr_intr(struct nvkm_gr *base)
"nstatus %08x [%s] ch %d [%08x %s] subc %d "
"class %04x mthd %04x data %08x\n",
show, msg, nsource, src, nstatus, sta,
- chan ? chan->fifo->chid : -1, inst << 4,
- chan ? chan->fifo->object.client->name : "unknown",
+ chan ? chan->fifo->id : -1, inst << 4,
+ chan ? chan->fifo->name : "unknown",
subc, class, mthd, data);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c
index 563a100..1ba18a8 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c
@@ -622,7 +622,7 @@ nv50_gr_intr(struct nvkm_gr *base)
struct nv50_gr *gr = nv50_gr(base);
struct nvkm_subdev *subdev = &gr->base.engine.subdev;
struct nvkm_device *device = subdev->device;
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
u32 stat = nvkm_rd32(device, 0x400100);
u32 inst = nvkm_rd32(device, 0x40032c) & 0x0fffffff;
u32 addr = nvkm_rd32(device, 0x400704);
@@ -637,10 +637,10 @@ nv50_gr_intr(struct nvkm_gr *base)
char msg[128];
int chid = -1;
- chan = nvkm_fifo_chan_inst(device->fifo, (u64)inst << 12, &flags);
+ chan = nvkm_chan_get_inst(&gr->base.engine, (u64)inst << 12, &flags);
if (chan) {
- name = chan->object.client->name;
- chid = chan->chid;
+ name = chan->name;
+ chid = chan->id;
}
if (show & 0x00100000) {
@@ -672,7 +672,7 @@ nv50_gr_intr(struct nvkm_gr *base)
if (nvkm_rd32(device, 0x400824) & (1 << 31))
nvkm_wr32(device, 0x400824, nvkm_rd32(device, 0x400824) & ~(1 << 31));
- nvkm_fifo_chan_put(device->fifo, flags, &chan);
+ nvkm_chan_put(&chan, flags);
}
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h
index 9b2c66e..08d5c96 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h
@@ -17,6 +17,7 @@ struct nvkm_gr_func {
int (*oneinit)(struct nvkm_gr *);
int (*init)(struct nvkm_gr *);
int (*fini)(struct nvkm_gr *, bool);
+ int (*reset)(struct nvkm_gr *);
void (*intr)(struct nvkm_gr *);
void (*tile)(struct nvkm_gr *, int region, struct nvkm_fb_tile *);
int (*tlb_flush)(struct nvkm_gr *);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/tu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/tu102.c
index 1a8a218..3b6c810 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/tu102.c
@@ -24,13 +24,13 @@
#include <nvif/class.h>
-static void
+void
tu102_gr_init_fecs_exceptions(struct gf100_gr *gr)
{
- nvkm_wr32(gr->base.engine.subdev.device, 0x409c24, 0x006f0002);
+ nvkm_wr32(gr->base.engine.subdev.device, 0x409c24, 0x006e0003);
}
-static void
+void
tu102_gr_init_fs(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
@@ -40,20 +40,21 @@ tu102_gr_init_fs(struct gf100_gr *gr)
gk104_grctx_generate_gpc_tpc_nr(gr);
for (sm = 0; sm < gr->sm_nr; sm++) {
- nvkm_wr32(device, GPC_UNIT(gr->sm[sm].gpc, 0x0c10 +
- gr->sm[sm].tpc * 4), sm);
+ int tpc = gv100_gr_nonpes_aware_tpc(gr, gr->sm[sm].gpc, gr->sm[sm].tpc);
+
+ nvkm_wr32(device, GPC_UNIT(gr->sm[sm].gpc, 0x0c10 + tpc * 4), sm);
}
gm200_grctx_generate_dist_skip_table(gr);
gf100_gr_init_num_tpc_per_gpc(gr, true, true);
}
-static void
+void
tu102_gr_init_zcull(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const u32 magicgpc918 = DIV_ROUND_UP(0x00800000, gr->tpc_total);
- const u8 tile_nr = ALIGN(gr->tpc_total, 64);
+ const u8 tile_nr = gr->func->gpc_nr * gr->func->tpc_nr;
u8 bank[GPC_MAX] = {}, gpc, i, j;
u32 data;
@@ -93,7 +94,7 @@ tu102_gr_init_gpc_mmu(struct gf100_gr *gr)
static const struct gf100_gr_func
tu102_gr = {
.oneinit_tiles = gm200_gr_oneinit_tiles,
- .oneinit_sm_id = gm200_gr_oneinit_sm_id,
+ .oneinit_sm_id = gv100_gr_oneinit_sm_id,
.init = gf100_gr_init,
.init_419bd8 = gv100_gr_init_419bd8,
.init_gpc_mmu = tu102_gr_init_gpc_mmu,
@@ -109,10 +110,14 @@ tu102_gr = {
.init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.init_504430 = gv100_gr_init_504430,
.init_shader_exceptions = gv100_gr_init_shader_exceptions,
+ .init_rop_exceptions = gf100_gr_init_rop_exceptions,
+ .init_exception2 = gf100_gr_init_exception2,
+ .init_4188a4 = gv100_gr_init_4188a4,
.trap_mp = gv100_gr_trap_mp,
+ .fecs.reset = gf100_gr_fecs_reset,
.rops = gm200_gr_rops,
.gpc_nr = 6,
- .tpc_nr = 5,
+ .tpc_nr = 6,
.ppc_nr = 3,
.grctx = &tu102_grctx,
.zbc = &gp102_gr_zbc,
@@ -137,6 +142,7 @@ MODULE_FIRMWARE("nvidia/tu102/gr/sw_ctx.bin");
MODULE_FIRMWARE("nvidia/tu102/gr/sw_nonctx.bin");
MODULE_FIRMWARE("nvidia/tu102/gr/sw_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu102/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/tu102/gr/sw_veid_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu104/gr/fecs_bl.bin");
MODULE_FIRMWARE("nvidia/tu104/gr/fecs_inst.bin");
@@ -150,6 +156,7 @@ MODULE_FIRMWARE("nvidia/tu104/gr/sw_ctx.bin");
MODULE_FIRMWARE("nvidia/tu104/gr/sw_nonctx.bin");
MODULE_FIRMWARE("nvidia/tu104/gr/sw_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu104/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/tu104/gr/sw_veid_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu106/gr/fecs_bl.bin");
MODULE_FIRMWARE("nvidia/tu106/gr/fecs_inst.bin");
@@ -163,6 +170,7 @@ MODULE_FIRMWARE("nvidia/tu106/gr/sw_ctx.bin");
MODULE_FIRMWARE("nvidia/tu106/gr/sw_nonctx.bin");
MODULE_FIRMWARE("nvidia/tu106/gr/sw_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu106/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/tu106/gr/sw_veid_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu117/gr/fecs_bl.bin");
MODULE_FIRMWARE("nvidia/tu117/gr/fecs_inst.bin");
@@ -176,6 +184,7 @@ MODULE_FIRMWARE("nvidia/tu117/gr/sw_ctx.bin");
MODULE_FIRMWARE("nvidia/tu117/gr/sw_nonctx.bin");
MODULE_FIRMWARE("nvidia/tu117/gr/sw_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu117/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/tu117/gr/sw_veid_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu116/gr/fecs_bl.bin");
MODULE_FIRMWARE("nvidia/tu116/gr/fecs_inst.bin");
@@ -189,6 +198,26 @@ MODULE_FIRMWARE("nvidia/tu116/gr/sw_ctx.bin");
MODULE_FIRMWARE("nvidia/tu116/gr/sw_nonctx.bin");
MODULE_FIRMWARE("nvidia/tu116/gr/sw_bundle_init.bin");
MODULE_FIRMWARE("nvidia/tu116/gr/sw_method_init.bin");
+MODULE_FIRMWARE("nvidia/tu116/gr/sw_veid_bundle_init.bin");
+
+int
+tu102_gr_av_to_init_veid(struct nvkm_blob *blob, struct gf100_gr_pack **ppack)
+{
+ return gk20a_gr_av_to_init_(blob, 64, 0x00100000, ppack);
+}
+
+int
+tu102_gr_load(struct gf100_gr *gr, int ver, const struct gf100_gr_fwif *fwif)
+{
+ int ret;
+
+ ret = gm200_gr_load(gr, ver, fwif);
+ if (ret)
+ return ret;
+
+ return gk20a_gr_load_net(gr, "gr/", "sw_veid_bundle_init", ver, tu102_gr_av_to_init_veid,
+ &gr->bundle_veid);
+}
static const struct gf100_gr_fwif
tu102_gr_fwif[] = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c
index b1054db..cb0c399 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c
@@ -213,8 +213,8 @@ nv31_mpeg_intr(struct nvkm_engine *engine)
if (show) {
nvkm_error(subdev, "ch %d [%s] %08x %08x %08x %08x\n",
- mpeg->chan ? mpeg->chan->fifo->chid : -1,
- mpeg->chan ? mpeg->chan->object.client->name :
+ mpeg->chan ? mpeg->chan->fifo->id : -1,
+ mpeg->chan ? mpeg->chan->fifo->name :
"unknown", stat, type, mthd, data);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c
index 521ce43..0890a27 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c
@@ -182,8 +182,8 @@ nv44_mpeg_intr(struct nvkm_engine *engine)
if (show) {
nvkm_error(subdev, "ch %d [%08x %s] %08x %08x %08x %08x\n",
- chan ? chan->fifo->chid : -1, inst << 4,
- chan ? chan->object.client->name : "unknown",
+ chan ? chan->fifo->id : -1, inst << 4,
+ chan ? chan->fifo->name : "unknown",
stat, type, mthd, data);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/Kbuild b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/Kbuild
index 9a0fd98..f05e796 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/Kbuild
@@ -1,3 +1,4 @@
# SPDX-License-Identifier: MIT
nvkm-y += nvkm/engine/nvdec/base.o
nvkm-y += nvkm/engine/nvdec/gm107.o
+nvkm-y += nvkm/engine/nvdec/ga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/base.c
index b0181cc..1f6e3b3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/base.c
@@ -37,7 +37,7 @@ nvkm_nvdec = {
int
nvkm_nvdec_new_(const struct nvkm_nvdec_fwif *fwif, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_nvdec **pnvdec)
+ enum nvkm_subdev_type type, int inst, u32 addr, struct nvkm_nvdec **pnvdec)
{
struct nvkm_nvdec *nvdec;
int ret;
@@ -57,5 +57,5 @@ nvkm_nvdec_new_(const struct nvkm_nvdec_fwif *fwif, struct nvkm_device *device,
nvdec->func = fwif->func;
return nvkm_falcon_ctor(nvdec->func->flcn, &nvdec->engine.subdev,
- nvdec->engine.subdev.name, 0, &nvdec->falcon);
+ nvdec->engine.subdev.name, addr, &nvdec->falcon);
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/ga102.c b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/ga102.c
new file mode 100644
index 0000000..37d8c3c
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/ga102.c
@@ -0,0 +1,61 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <subdev/mc.h>
+#include <subdev/timer.h>
+
+static const struct nvkm_falcon_func
+ga102_nvdec_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .addr2 = 0x1c00,
+ .reset_pmc = true,
+ .reset_prep = ga102_flcn_reset_prep,
+ .reset_wait_mem_scrubbing = ga102_flcn_reset_wait_mem_scrubbing,
+ .imem_dma = &ga102_flcn_dma,
+ .dmem_dma = &ga102_flcn_dma,
+};
+
+static const struct nvkm_nvdec_func
+ga102_nvdec = {
+ .flcn = &ga102_nvdec_flcn,
+};
+
+static int
+ga102_nvdec_nofw(struct nvkm_nvdec *nvdec, int ver, const struct nvkm_nvdec_fwif *fwif)
+{
+ return 0;
+}
+
+static const struct nvkm_nvdec_fwif
+ga102_nvdec_fwif[] = {
+ { -1, ga102_nvdec_nofw, &ga102_nvdec },
+ {}
+};
+
+int
+ga102_nvdec_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_nvdec **pnvdec)
+{
+ return nvkm_nvdec_new_(ga102_nvdec_fwif, device, type, inst, 0x848000, pnvdec);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/gm107.c
index 8c44ce4..564f7e8 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/gm107.c
@@ -23,18 +23,13 @@
static const struct nvkm_falcon_func
gm107_nvdec_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .reset_pmc = true,
+ .reset_wait_mem_scrubbing = gm200_flcn_reset_wait_mem_scrubbing,
.debug = 0xd00,
- .fbif = 0x600,
- .load_imem = nvkm_falcon_v1_load_imem,
- .load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
- .bind_context = nvkm_falcon_v1_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
- .start = nvkm_falcon_v1_start,
- .enable = nvkm_falcon_v1_enable,
- .disable = nvkm_falcon_v1_disable,
+ .imem_pio = &gm200_flcn_imem_pio,
+ .dmem_pio = &gm200_flcn_dmem_pio,
};
static const struct nvkm_nvdec_func
@@ -59,5 +54,5 @@ int
gm107_nvdec_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_nvdec **pnvdec)
{
- return nvkm_nvdec_new_(gm107_nvdec_fwif, device, type, inst, pnvdec);
+ return nvkm_nvdec_new_(gm107_nvdec_fwif, device, type, inst, 0, pnvdec);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/priv.h
index 0920f6a..61e1f7a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/nvdec/priv.h
@@ -15,5 +15,5 @@ struct nvkm_nvdec_fwif {
};
int nvkm_nvdec_new_(const struct nvkm_nvdec_fwif *fwif, struct nvkm_device *,
- enum nvkm_subdev_type, int, struct nvkm_nvdec **);
+ enum nvkm_subdev_type, int, u32 addr, struct nvkm_nvdec **);
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/nvenc/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/nvenc/gm107.c
index f44d41b..ad27d8b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/nvenc/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/nvenc/gm107.c
@@ -24,17 +24,6 @@
static const struct nvkm_falcon_func
gm107_nvenc_flcn = {
- .fbif = 0x800,
- .load_imem = nvkm_falcon_v1_load_imem,
- .load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
- .bind_context = nvkm_falcon_v1_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
- .start = nvkm_falcon_v1_start,
- .enable = nvkm_falcon_v1_enable,
- .disable = nvkm_falcon_v1_disable,
};
static const struct nvkm_nvenc_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c b/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c
index 1b87df0..c15b2cb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c
@@ -40,7 +40,7 @@ static const struct nvkm_enum g98_sec_isr_error_name[] = {
};
static void
-g98_sec_intr(struct nvkm_falcon *sec, struct nvkm_fifo_chan *chan)
+g98_sec_intr(struct nvkm_falcon *sec, struct nvkm_chan *chan)
{
struct nvkm_subdev *subdev = &sec->engine.subdev;
struct nvkm_device *device = subdev->device;
@@ -54,9 +54,9 @@ g98_sec_intr(struct nvkm_falcon *sec, struct nvkm_fifo_chan *chan)
nvkm_error(subdev, "DISPATCH_ERROR %04x [%s] ch %d [%010llx %s] "
"subc %d mthd %04x data %08x\n", ssta,
- en ? en->name : "UNKNOWN", chan ? chan->chid : -1,
+ en ? en->name : "UNKNOWN", chan ? chan->id : -1,
chan ? chan->inst->addr : 0,
- chan ? chan->object.client->name : "unknown",
+ chan ? chan->name : "unknown",
subc, mthd, data);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/Kbuild b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/Kbuild
index 63cd2be..19feadb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/Kbuild
@@ -3,3 +3,4 @@
nvkm-y += nvkm/engine/sec2/gp102.o
nvkm-y += nvkm/engine/sec2/gp108.o
nvkm-y += nvkm/engine/sec2/tu102.o
+nvkm-y += nvkm/engine/sec2/ga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/base.c
index 092c6d0..f2c60da 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/base.c
@@ -22,53 +22,99 @@
#include "priv.h"
#include <core/firmware.h>
-#include <subdev/top.h>
+#include <subdev/mc.h>
+#include <subdev/timer.h>
-static void
-nvkm_sec2_recv(struct work_struct *work)
+#include <nvfw/sec2.h>
+
+static int
+nvkm_sec2_finimsg(void *priv, struct nvfw_falcon_msg *hdr)
{
- struct nvkm_sec2 *sec2 = container_of(work, typeof(*sec2), work);
+ struct nvkm_sec2 *sec2 = priv;
- if (!sec2->initmsg_received) {
- int ret = sec2->func->initmsg(sec2);
- if (ret) {
- nvkm_error(&sec2->engine.subdev,
- "error parsing init message: %d\n", ret);
- return;
- }
-
- sec2->initmsg_received = true;
- }
-
- nvkm_falcon_msgq_recv(sec2->msgq);
-}
-
-static void
-nvkm_sec2_intr(struct nvkm_engine *engine)
-{
- struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
- sec2->func->intr(sec2);
+ atomic_set(&sec2->running, 0);
+ return 0;
}
static int
nvkm_sec2_fini(struct nvkm_engine *engine, bool suspend)
{
struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+ struct nvkm_subdev *subdev = &sec2->engine.subdev;
+ struct nvkm_falcon *falcon = &sec2->falcon;
+ struct nvkm_falcon_cmdq *cmdq = sec2->cmdq;
+ struct nvfw_falcon_cmd cmd = {
+ .unit_id = sec2->func->unit_unload,
+ .size = sizeof(cmd),
+ };
+ int ret;
- flush_work(&sec2->work);
+ if (!subdev->use.enabled)
+ return 0;
- if (suspend) {
- nvkm_falcon_cmdq_fini(sec2->cmdq);
- sec2->initmsg_received = false;
+ if (atomic_read(&sec2->initmsg) == 1) {
+ ret = nvkm_falcon_cmdq_send(cmdq, &cmd, nvkm_sec2_finimsg, sec2,
+ msecs_to_jiffies(1000));
+ WARN_ON(ret);
+
+ nvkm_msec(subdev->device, 2000,
+ if (nvkm_falcon_rd32(falcon, 0x100) & 0x00000010)
+ break;
+ );
}
+ nvkm_inth_block(&subdev->inth);
+
+ nvkm_falcon_cmdq_fini(cmdq);
+ falcon->func->disable(falcon);
+ nvkm_falcon_put(falcon, subdev);
return 0;
}
+static int
+nvkm_sec2_init(struct nvkm_engine *engine)
+{
+ struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+ struct nvkm_subdev *subdev = &sec2->engine.subdev;
+ struct nvkm_falcon *falcon = &sec2->falcon;
+ int ret;
+
+ ret = nvkm_falcon_get(falcon, subdev);
+ if (ret)
+ return ret;
+
+ nvkm_falcon_wr32(falcon, 0x014, 0xffffffff);
+ atomic_set(&sec2->initmsg, 0);
+ atomic_set(&sec2->running, 1);
+ nvkm_inth_allow(&subdev->inth);
+
+ nvkm_falcon_start(falcon);
+ return 0;
+}
+
+static int
+nvkm_sec2_oneinit(struct nvkm_engine *engine)
+{
+ struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+ struct nvkm_subdev *subdev = &sec2->engine.subdev;
+ struct nvkm_intr *intr = &sec2->engine.subdev.device->mc->intr;
+ enum nvkm_intr_type type = NVKM_INTR_SUBDEV;
+
+ if (sec2->func->intr_vector) {
+ intr = sec2->func->intr_vector(sec2, &type);
+ if (IS_ERR(intr))
+ return PTR_ERR(intr);
+ }
+
+ return nvkm_inth_add(intr, type, NVKM_INTR_PRIO_NORMAL, subdev, sec2->func->intr,
+ &subdev->inth);
+}
+
static void *
nvkm_sec2_dtor(struct nvkm_engine *engine)
{
struct nvkm_sec2 *sec2 = nvkm_sec2(engine);
+
nvkm_falcon_msgq_del(&sec2->msgq);
nvkm_falcon_cmdq_del(&sec2->cmdq);
nvkm_falcon_qmgr_del(&sec2->qmgr);
@@ -79,8 +125,9 @@ nvkm_sec2_dtor(struct nvkm_engine *engine)
static const struct nvkm_engine_func
nvkm_sec2 = {
.dtor = nvkm_sec2_dtor,
+ .oneinit = nvkm_sec2_oneinit,
+ .init = nvkm_sec2_init,
.fini = nvkm_sec2_fini,
- .intr = nvkm_sec2_intr,
};
int
@@ -113,6 +160,5 @@ nvkm_sec2_new_(const struct nvkm_sec2_fwif *fwif, struct nvkm_device *device,
(ret = nvkm_falcon_msgq_new(sec2->qmgr, "msgq", &sec2->msgq)))
return ret;
- INIT_WORK(&sec2->work, nvkm_sec2_recv);
return 0;
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/ga102.c b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/ga102.c
new file mode 100644
index 0000000..945abb8
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/ga102.c
@@ -0,0 +1,197 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+#include <subdev/acr.h>
+#include <subdev/vfn.h>
+
+#include <nvfw/flcn.h>
+#include <nvfw/sec2.h>
+
+static int
+ga102_sec2_initmsg(struct nvkm_sec2 *sec2)
+{
+ struct nv_sec2_init_msg_v1 msg;
+ int ret, i;
+
+ ret = nvkm_falcon_msgq_recv_initmsg(sec2->msgq, &msg, sizeof(msg));
+ if (ret)
+ return ret;
+
+ if (msg.hdr.unit_id != NV_SEC2_UNIT_INIT ||
+ msg.msg_type != NV_SEC2_INIT_MSG_INIT)
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(msg.queue_info); i++) {
+ if (msg.queue_info[i].id == NV_SEC2_INIT_MSG_QUEUE_ID_MSGQ) {
+ nvkm_falcon_msgq_init(sec2->msgq, msg.queue_info[i].index,
+ msg.queue_info[i].offset,
+ msg.queue_info[i].size);
+ } else {
+ nvkm_falcon_cmdq_init(sec2->cmdq, msg.queue_info[i].index,
+ msg.queue_info[i].offset,
+ msg.queue_info[i].size);
+ }
+ }
+
+ return 0;
+}
+
+static struct nvkm_intr *
+ga102_sec2_intr_vector(struct nvkm_sec2 *sec2, enum nvkm_intr_type *pvector)
+{
+ struct nvkm_device *device = sec2->engine.subdev.device;
+ struct nvkm_falcon *falcon = &sec2->falcon;
+ int ret;
+
+ ret = ga102_flcn_select(falcon);
+ if (ret)
+ return ERR_PTR(ret);
+
+ *pvector = nvkm_rd32(device, 0x8403e0) & 0x000000ff;
+ return &device->vfn->intr;
+}
+
+static int
+ga102_sec2_acr_bootstrap_falcon_callback(void *priv, struct nvfw_falcon_msg *hdr)
+{
+ struct nv_sec2_acr_bootstrap_falcon_msg_v1 *msg =
+ container_of(hdr, typeof(*msg), msg.hdr);
+ struct nvkm_subdev *subdev = priv;
+ const char *name = nvkm_acr_lsf_id(msg->falcon_id);
+
+ if (msg->error_code) {
+ nvkm_error(subdev, "ACR_BOOTSTRAP_FALCON failed for falcon %d [%s]: %08x %08x\n",
+ msg->falcon_id, name, msg->error_code, msg->unkn08);
+ return -EINVAL;
+ }
+
+ nvkm_debug(subdev, "%s booted\n", name);
+ return 0;
+}
+
+static int
+ga102_sec2_acr_bootstrap_falcon(struct nvkm_falcon *falcon, enum nvkm_acr_lsf_id id)
+{
+ struct nvkm_sec2 *sec2 = container_of(falcon, typeof(*sec2), falcon);
+ struct nv_sec2_acr_bootstrap_falcon_cmd_v1 cmd = {
+ .cmd.hdr.unit_id = sec2->func->unit_acr,
+ .cmd.hdr.size = sizeof(cmd),
+ .cmd.cmd_type = NV_SEC2_ACR_CMD_BOOTSTRAP_FALCON,
+ .flags = NV_SEC2_ACR_BOOTSTRAP_FALCON_FLAGS_RESET_YES,
+ .falcon_id = id,
+ };
+
+ return nvkm_falcon_cmdq_send(sec2->cmdq, &cmd.cmd.hdr,
+ ga102_sec2_acr_bootstrap_falcon_callback,
+ &sec2->engine.subdev,
+ msecs_to_jiffies(1000));
+}
+
+static const struct nvkm_acr_lsf_func
+ga102_sec2_acr_0 = {
+ .bld_size = sizeof(struct flcn_bl_dmem_desc_v2),
+ .bld_write = gp102_sec2_acr_bld_write_1,
+ .bld_patch = gp102_sec2_acr_bld_patch_1,
+ .bootstrap_falcons = BIT_ULL(NVKM_ACR_LSF_FECS) |
+ BIT_ULL(NVKM_ACR_LSF_GPCCS) |
+ BIT_ULL(NVKM_ACR_LSF_SEC2),
+ .bootstrap_falcon = ga102_sec2_acr_bootstrap_falcon,
+};
+
+static const struct nvkm_falcon_func
+ga102_sec2_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .select = ga102_flcn_select,
+ .addr2 = 0x1000,
+ .reset_pmc = true,
+ .reset_eng = gp102_flcn_reset_eng,
+ .reset_prep = ga102_flcn_reset_prep,
+ .reset_wait_mem_scrubbing = ga102_flcn_reset_wait_mem_scrubbing,
+ .imem_dma = &ga102_flcn_dma,
+ .dmem_pio = &gm200_flcn_dmem_pio,
+ .dmem_dma = &ga102_flcn_dma,
+ .emem_addr = 0x01000000,
+ .emem_pio = &gp102_flcn_emem_pio,
+ .start = nvkm_falcon_v1_start,
+ .cmdq = { 0xc00, 0xc04, 8 },
+ .msgq = { 0xc80, 0xc84, 8 },
+};
+
+static const struct nvkm_sec2_func
+ga102_sec2 = {
+ .flcn = &ga102_sec2_flcn,
+ .intr_vector = ga102_sec2_intr_vector,
+ .intr = gp102_sec2_intr,
+ .initmsg = ga102_sec2_initmsg,
+ .unit_acr = NV_SEC2_UNIT_V2_ACR,
+ .unit_unload = NV_SEC2_UNIT_V2_UNLOAD,
+};
+
+MODULE_FIRMWARE("nvidia/ga102/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/ga102/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/ga102/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/ga102/sec2/hs_bl_sig.bin");
+
+MODULE_FIRMWARE("nvidia/ga103/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/ga103/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/ga103/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/ga103/sec2/hs_bl_sig.bin");
+
+MODULE_FIRMWARE("nvidia/ga104/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/ga104/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/ga104/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/ga104/sec2/hs_bl_sig.bin");
+
+MODULE_FIRMWARE("nvidia/ga106/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/ga106/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/ga106/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/ga106/sec2/hs_bl_sig.bin");
+
+MODULE_FIRMWARE("nvidia/ga107/sec2/desc.bin");
+MODULE_FIRMWARE("nvidia/ga107/sec2/image.bin");
+MODULE_FIRMWARE("nvidia/ga107/sec2/sig.bin");
+MODULE_FIRMWARE("nvidia/ga107/sec2/hs_bl_sig.bin");
+
+static int
+ga102_sec2_load(struct nvkm_sec2 *sec2, int ver,
+ const struct nvkm_sec2_fwif *fwif)
+{
+ return nvkm_acr_lsfw_load_sig_image_desc_v2(&sec2->engine.subdev, &sec2->falcon,
+ NVKM_ACR_LSF_SEC2, "sec2/", ver, fwif->acr);
+}
+
+static const struct nvkm_sec2_fwif
+ga102_sec2_fwif[] = {
+ { 0, ga102_sec2_load, &ga102_sec2, &ga102_sec2_acr_0 },
+ { -1, gp102_sec2_nofw, &ga102_sec2 }
+};
+
+int
+ga102_sec2_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_sec2 **psec2)
+{
+ /* TOP info wasn't updated on Turing to reflect the PRI
+ * address change for some reason. We override it here.
+ */
+ return nvkm_sec2_new_(ga102_sec2_fwif, device, type, inst, 0x840000, psec2);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/gp102.c b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/gp102.c
index 44e39f5..c64013d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/gp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/gp102.c
@@ -74,16 +74,6 @@ gp102_sec2_acr_bootstrap_falcon(struct nvkm_falcon *falcon,
msecs_to_jiffies(1000));
}
-static int
-gp102_sec2_acr_boot(struct nvkm_falcon *falcon)
-{
- struct nv_sec2_args args = {};
- nvkm_falcon_load_dmem(falcon, &args,
- falcon->func->emem_addr, sizeof(args), 0);
- nvkm_falcon_start(falcon);
- return 0;
-}
-
static void
gp102_sec2_acr_bld_patch(struct nvkm_acr *acr, u32 bld, s64 adjust)
{
@@ -122,7 +112,6 @@ gp102_sec2_acr_0 = {
.bld_size = sizeof(struct loader_config_v1),
.bld_write = gp102_sec2_acr_bld_write,
.bld_patch = gp102_sec2_acr_bld_patch,
- .boot = gp102_sec2_acr_boot,
.bootstrap_falcons = BIT_ULL(NVKM_ACR_LSF_FECS) |
BIT_ULL(NVKM_ACR_LSF_GPCCS) |
BIT_ULL(NVKM_ACR_LSF_SEC2),
@@ -160,89 +149,68 @@ gp102_sec2_initmsg(struct nvkm_sec2 *sec2)
return 0;
}
-void
-gp102_sec2_intr(struct nvkm_sec2 *sec2)
+irqreturn_t
+gp102_sec2_intr(struct nvkm_inth *inth)
{
+ struct nvkm_sec2 *sec2 = container_of(inth, typeof(*sec2), engine.subdev.inth);
struct nvkm_subdev *subdev = &sec2->engine.subdev;
struct nvkm_falcon *falcon = &sec2->falcon;
u32 disp = nvkm_falcon_rd32(falcon, 0x01c);
u32 intr = nvkm_falcon_rd32(falcon, 0x008) & disp & ~(disp >> 16);
if (intr & 0x00000040) {
- schedule_work(&sec2->work);
+ if (unlikely(atomic_read(&sec2->initmsg) == 0)) {
+ int ret = sec2->func->initmsg(sec2);
+
+ if (ret)
+ nvkm_error(subdev, "error parsing init message: %d\n", ret);
+
+ atomic_set(&sec2->initmsg, ret ?: 1);
+ }
+
+ if (atomic_read(&sec2->initmsg) > 0) {
+ if (!nvkm_falcon_msgq_empty(sec2->msgq))
+ nvkm_falcon_msgq_recv(sec2->msgq);
+ }
+
nvkm_falcon_wr32(falcon, 0x004, 0x00000040);
intr &= ~0x00000040;
}
+ if (intr & 0x00000010) {
+ if (atomic_read(&sec2->running)) {
+ FLCN_ERR(falcon, "halted");
+ gm200_flcn_tracepc(falcon);
+ }
+
+ nvkm_falcon_wr32(falcon, 0x004, 0x00000010);
+ intr &= ~0x00000010;
+ }
+
if (intr) {
nvkm_error(subdev, "unhandled intr %08x\n", intr);
nvkm_falcon_wr32(falcon, 0x004, intr);
}
-}
-int
-gp102_sec2_flcn_enable(struct nvkm_falcon *falcon)
-{
- nvkm_falcon_mask(falcon, 0x3c0, 0x00000001, 0x00000001);
- udelay(10);
- nvkm_falcon_mask(falcon, 0x3c0, 0x00000001, 0x00000000);
- return nvkm_falcon_v1_enable(falcon);
-}
-
-void
-gp102_sec2_flcn_bind_context(struct nvkm_falcon *falcon,
- struct nvkm_memory *ctx)
-{
- struct nvkm_device *device = falcon->owner->device;
-
- nvkm_falcon_v1_bind_context(falcon, ctx);
- if (!ctx)
- return;
-
- /* Not sure if this is a WAR for a HW issue, or some additional
- * programming sequence that's needed to properly complete the
- * context switch we trigger above.
- *
- * Fixes unreliability of booting the SEC2 RTOS on Quadro P620,
- * particularly when resuming from suspend.
- *
- * Also removes the need for an odd workaround where we needed
- * to program SEC2's FALCON_CPUCTL_ALIAS_STARTCPU twice before
- * the SEC2 RTOS would begin executing.
- */
- nvkm_msec(device, 10,
- u32 irqstat = nvkm_falcon_rd32(falcon, 0x008);
- u32 flcn0dc = nvkm_falcon_rd32(falcon, 0x0dc);
- if ((irqstat & 0x00000008) &&
- (flcn0dc & 0x00007000) == 0x00005000)
- break;
- );
-
- nvkm_falcon_mask(falcon, 0x004, 0x00000008, 0x00000008);
- nvkm_falcon_mask(falcon, 0x058, 0x00000002, 0x00000002);
-
- nvkm_msec(device, 10,
- u32 flcn0dc = nvkm_falcon_rd32(falcon, 0x0dc);
- if ((flcn0dc & 0x00007000) == 0x00000000)
- break;
- );
+ return IRQ_HANDLED;
}
static const struct nvkm_falcon_func
gp102_sec2_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .reset_pmc = true,
+ .reset_eng = gp102_flcn_reset_eng,
+ .reset_wait_mem_scrubbing = gm200_flcn_reset_wait_mem_scrubbing,
.debug = 0x408,
- .fbif = 0x600,
- .load_imem = nvkm_falcon_v1_load_imem,
- .load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
+ .bind_inst = gm200_flcn_bind_inst,
+ .bind_stat = gm200_flcn_bind_stat,
+ .bind_intr = true,
+ .imem_pio = &gm200_flcn_imem_pio,
+ .dmem_pio = &gm200_flcn_dmem_pio,
.emem_addr = 0x01000000,
- .bind_context = gp102_sec2_flcn_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
+ .emem_pio = &gp102_flcn_emem_pio,
.start = nvkm_falcon_v1_start,
- .enable = gp102_sec2_flcn_enable,
- .disable = nvkm_falcon_v1_disable,
.cmdq = { 0xa00, 0xa04, 8 },
.msgq = { 0xa30, 0xa34, 8 },
};
@@ -250,6 +218,7 @@ gp102_sec2_flcn = {
const struct nvkm_sec2_func
gp102_sec2 = {
.flcn = &gp102_sec2_flcn,
+ .unit_unload = NV_SEC2_UNIT_UNLOAD,
.unit_acr = NV_SEC2_UNIT_ACR,
.intr = gp102_sec2_intr,
.initmsg = gp102_sec2_initmsg,
@@ -268,7 +237,7 @@ MODULE_FIRMWARE("nvidia/gp107/sec2/desc.bin");
MODULE_FIRMWARE("nvidia/gp107/sec2/image.bin");
MODULE_FIRMWARE("nvidia/gp107/sec2/sig.bin");
-static void
+void
gp102_sec2_acr_bld_patch_1(struct nvkm_acr *acr, u32 bld, s64 adjust)
{
struct flcn_bl_dmem_desc_v2 hdr;
@@ -279,7 +248,7 @@ gp102_sec2_acr_bld_patch_1(struct nvkm_acr *acr, u32 bld, s64 adjust)
flcn_bl_dmem_desc_v2_dump(&acr->subdev, &hdr);
}
-static void
+void
gp102_sec2_acr_bld_write_1(struct nvkm_acr *acr, u32 bld,
struct nvkm_acr_lsfw *lsfw)
{
@@ -304,7 +273,6 @@ gp102_sec2_acr_1 = {
.bld_size = sizeof(struct flcn_bl_dmem_desc_v2),
.bld_write = gp102_sec2_acr_bld_write_1,
.bld_patch = gp102_sec2_acr_bld_patch_1,
- .boot = gp102_sec2_acr_boot,
.bootstrap_falcons = BIT_ULL(NVKM_ACR_LSF_FECS) |
BIT_ULL(NVKM_ACR_LSF_GPCCS) |
BIT_ULL(NVKM_ACR_LSF_SEC2),
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/priv.h
index af19229..172d270 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/priv.h
@@ -2,15 +2,18 @@
#ifndef __NVKM_SEC2_PRIV_H__
#define __NVKM_SEC2_PRIV_H__
#include <engine/sec2.h>
+struct nvkm_acr_lsfw;
struct nvkm_sec2_func {
const struct nvkm_falcon_func *flcn;
+ u8 unit_unload;
u8 unit_acr;
- void (*intr)(struct nvkm_sec2 *);
+ struct nvkm_intr *(*intr_vector)(struct nvkm_sec2 *, enum nvkm_intr_type *);
+ irqreturn_t (*intr)(struct nvkm_inth *);
int (*initmsg)(struct nvkm_sec2 *);
};
-void gp102_sec2_intr(struct nvkm_sec2 *);
+irqreturn_t gp102_sec2_intr(struct nvkm_inth *);
int gp102_sec2_initmsg(struct nvkm_sec2 *);
struct nvkm_sec2_fwif {
@@ -24,6 +27,8 @@ int gp102_sec2_nofw(struct nvkm_sec2 *, int, const struct nvkm_sec2_fwif *);
int gp102_sec2_load(struct nvkm_sec2 *, int, const struct nvkm_sec2_fwif *);
extern const struct nvkm_sec2_func gp102_sec2;
extern const struct nvkm_acr_lsf_func gp102_sec2_acr_1;
+void gp102_sec2_acr_bld_write_1(struct nvkm_acr *, u32, struct nvkm_acr_lsfw *);
+void gp102_sec2_acr_bld_patch_1(struct nvkm_acr *, u32, s64);
int nvkm_sec2_new_(const struct nvkm_sec2_fwif *, struct nvkm_device *, enum nvkm_subdev_type,
int, u32 addr, struct nvkm_sec2 **);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/tu102.c b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/tu102.c
index f3faeb7..0afc4b2 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sec2/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec2/tu102.c
@@ -22,21 +22,24 @@
#include "priv.h"
#include <subdev/acr.h>
+#include <nvfw/sec2.h>
+
static const struct nvkm_falcon_func
tu102_sec2_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .reset_pmc = true,
+ .reset_eng = gp102_flcn_reset_eng,
+ .reset_wait_mem_scrubbing = gm200_flcn_reset_wait_mem_scrubbing,
.debug = 0x408,
- .fbif = 0x600,
- .load_imem = nvkm_falcon_v1_load_imem,
- .load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
+ .bind_inst = gm200_flcn_bind_inst,
+ .bind_stat = gm200_flcn_bind_stat,
+ .bind_intr = true,
+ .imem_pio = &gm200_flcn_imem_pio,
+ .dmem_pio = &gm200_flcn_dmem_pio,
.emem_addr = 0x01000000,
- .bind_context = gp102_sec2_flcn_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
+ .emem_pio = &gp102_flcn_emem_pio,
.start = nvkm_falcon_v1_start,
- .enable = nvkm_falcon_v1_enable,
- .disable = nvkm_falcon_v1_disable,
.cmdq = { 0xc00, 0xc04, 8 },
.msgq = { 0xc80, 0xc84, 8 },
};
@@ -44,7 +47,8 @@ tu102_sec2_flcn = {
static const struct nvkm_sec2_func
tu102_sec2 = {
.flcn = &tu102_sec2_flcn,
- .unit_acr = 0x07,
+ .unit_unload = NV_SEC2_UNIT_V2_UNLOAD,
+ .unit_acr = NV_SEC2_UNIT_V2_ACR,
.intr = gp102_sec2_intr,
.initmsg = gp102_sec2_initmsg,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c
index 14871d0..a9d464d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c
@@ -35,7 +35,7 @@ nvkm_sw_mthd(struct nvkm_sw *sw, int chid, int subc, u32 mthd, u32 data)
spin_lock_irqsave(&sw->engine.lock, flags);
list_for_each_entry(chan, &sw->chan, head) {
- if (chan->fifo->chid == chid) {
+ if (chan->fifo->id == chid) {
handled = nvkm_sw_chan_mthd(chan, subc, mthd, data);
list_del(&chan->head);
list_add(&chan->head, &sw->chan);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.c b/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.c
index f289670..834b8cb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.c
@@ -23,7 +23,6 @@
*/
#include "chan.h"
-#include <core/notify.h>
#include <engine/fifo.h>
#include <nvif/event.h>
@@ -36,7 +35,7 @@ nvkm_sw_chan_mthd(struct nvkm_sw_chan *chan, int subc, u32 mthd, u32 data)
case 0x0000:
return true;
case 0x0500:
- nvkm_event_send(&chan->event, 1, 0, NULL, 0);
+ nvkm_event_ntfy(&chan->event, 0, NVKM_SW_CHAN_EVENT_PAGE_FLIP);
return true;
default:
if (chan->func->mthd)
@@ -46,27 +45,8 @@ nvkm_sw_chan_mthd(struct nvkm_sw_chan *chan, int subc, u32 mthd, u32 data)
return false;
}
-static int
-nvkm_sw_chan_event_ctor(struct nvkm_object *object, void *data, u32 size,
- struct nvkm_notify *notify)
-{
- union {
- struct nvif_notify_uevent_req none;
- } *req = data;
- int ret = -ENOSYS;
-
- if (!(ret = nvif_unvers(ret, &data, &size, req->none))) {
- notify->size = sizeof(struct nvif_notify_uevent_rep);
- notify->types = 1;
- notify->index = 0;
- }
-
- return ret;
-}
-
static const struct nvkm_event_func
nvkm_sw_chan_event = {
- .ctor = nvkm_sw_chan_event_ctor,
};
static void *
@@ -107,5 +87,5 @@ nvkm_sw_chan_ctor(const struct nvkm_sw_chan_func *func, struct nvkm_sw *sw,
list_add(&chan->head, &sw->chan);
spin_unlock_irqrestore(&sw->engine.lock, flags);
- return nvkm_event_init(&nvkm_sw_chan_event, 1, 1, &chan->event);
+ return nvkm_event_init(&nvkm_sw_chan_event, &sw->engine.subdev, 1, 1, &chan->event);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.h b/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.h
index 32de534..67b2e5e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/chan.h
@@ -14,6 +14,7 @@ struct nvkm_sw_chan {
struct nvkm_fifo_chan *fifo;
struct list_head head;
+#define NVKM_SW_CHAN_EVENT_PAGE_FLIP BIT(0)
struct nvkm_event event;
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/sw/gf100.c
index 55abf83..c3cf6f2 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/gf100.c
@@ -36,10 +36,10 @@
******************************************************************************/
static int
-gf100_sw_chan_vblsem_release(struct nvkm_notify *notify)
+gf100_sw_chan_vblsem_release(struct nvkm_event_ntfy *notify, u32 bits)
{
struct nv50_sw_chan *chan =
- container_of(notify, typeof(*chan), vblank.notify[notify->index]);
+ container_of(notify, typeof(*chan), vblank.notify[notify->id]);
struct nvkm_sw *sw = chan->base.sw;
struct nvkm_device *device = sw->engine.subdev.device;
u32 inst = chan->base.fifo->inst->addr >> 12;
@@ -50,7 +50,7 @@ gf100_sw_chan_vblsem_release(struct nvkm_notify *notify)
nvkm_wr32(device, 0x060010, lower_32_bits(chan->vblank.offset));
nvkm_wr32(device, 0x060014, chan->vblank.value);
- return NVKM_NOTIFY_DROP;
+ return NVKM_EVENT_DROP;
}
static bool
@@ -73,7 +73,7 @@ gf100_sw_chan_mthd(struct nvkm_sw_chan *base, int subc, u32 mthd, u32 data)
return true;
case 0x040c:
if (data < device->disp->vblank.index_nr) {
- nvkm_notify_get(&chan->vblank.notify[data]);
+ nvkm_event_ntfy_allow(&chan->vblank.notify[data]);
return true;
}
break;
@@ -120,16 +120,8 @@ gf100_sw_chan_new(struct nvkm_sw *sw, struct nvkm_fifo_chan *fifoch,
return ret;
for (i = 0; disp && i < disp->vblank.index_nr; i++) {
- ret = nvkm_notify_init(NULL, &disp->vblank,
- gf100_sw_chan_vblsem_release, false,
- &(struct nvif_notify_head_req_v0) {
- .head = i,
- },
- sizeof(struct nvif_notify_head_req_v0),
- sizeof(struct nvif_notify_head_rep_v0),
- &chan->vblank.notify[i]);
- if (ret)
- return ret;
+ nvkm_event_ntfy_add(&disp->vblank, i, NVKM_DISP_HEAD_EVENT_VBLANK, true,
+ gf100_sw_chan_vblsem_release, &chan->vblank.notify[i]);
}
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.c
index 1fdd094..9d7a9b7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.c
@@ -36,10 +36,10 @@
******************************************************************************/
static int
-nv50_sw_chan_vblsem_release(struct nvkm_notify *notify)
+nv50_sw_chan_vblsem_release(struct nvkm_event_ntfy *notify, u32 bits)
{
struct nv50_sw_chan *chan =
- container_of(notify, typeof(*chan), vblank.notify[notify->index]);
+ container_of(notify, typeof(*chan), vblank.notify[notify->id]);
struct nvkm_sw *sw = chan->base.sw;
struct nvkm_device *device = sw->engine.subdev.device;
@@ -55,7 +55,7 @@ nv50_sw_chan_vblsem_release(struct nvkm_notify *notify)
nvkm_wr32(device, 0x060014, chan->vblank.value);
}
- return NVKM_NOTIFY_DROP;
+ return NVKM_EVENT_DROP;
}
static bool
@@ -70,7 +70,7 @@ nv50_sw_chan_mthd(struct nvkm_sw_chan *base, int subc, u32 mthd, u32 data)
case 0x0404: chan->vblank.value = data; return true;
case 0x0408:
if (data < device->disp->vblank.index_nr) {
- nvkm_notify_get(&chan->vblank.notify[data]);
+ nvkm_event_ntfy_allow(&chan->vblank.notify[data]);
return true;
}
break;
@@ -85,8 +85,10 @@ nv50_sw_chan_dtor(struct nvkm_sw_chan *base)
{
struct nv50_sw_chan *chan = nv50_sw_chan(base);
int i;
+
for (i = 0; i < ARRAY_SIZE(chan->vblank.notify); i++)
- nvkm_notify_fini(&chan->vblank.notify[i]);
+ nvkm_event_ntfy_del(&chan->vblank.notify[i]);
+
return chan;
}
@@ -113,16 +115,8 @@ nv50_sw_chan_new(struct nvkm_sw *sw, struct nvkm_fifo_chan *fifoch,
return ret;
for (i = 0; disp && i < disp->vblank.index_nr; i++) {
- ret = nvkm_notify_init(NULL, &disp->vblank,
- nv50_sw_chan_vblsem_release, false,
- &(struct nvif_notify_head_req_v0) {
- .head = i,
- },
- sizeof(struct nvif_notify_head_req_v0),
- sizeof(struct nvif_notify_head_rep_v0),
- &chan->vblank.notify[i]);
- if (ret)
- return ret;
+ nvkm_event_ntfy_add(&disp->vblank, i, NVKM_DISP_HEAD_EVENT_VBLANK, true,
+ nv50_sw_chan_vblsem_release, &chan->vblank.notify[i]);
}
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.h b/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.h
index 6d364d7..b42289c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/nv50.h
@@ -5,12 +5,12 @@
#include "priv.h"
#include "chan.h"
#include "nvsw.h"
-#include <core/notify.h>
+#include <core/event.h>
struct nv50_sw_chan {
struct nvkm_sw_chan base;
struct {
- struct nvkm_notify notify[4];
+ struct nvkm_event_ntfy notify[4];
u32 ctxdma;
u64 offset;
u32 value;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/nvsw.c b/drivers/gpu/drm/nouveau/nvkm/engine/sw/nvsw.c
index 33dd03f..f5affa1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/nvsw.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/nvsw.c
@@ -27,33 +27,34 @@
#include <nvif/if0004.h>
static int
-nvkm_nvsw_mthd_(struct nvkm_object *object, u32 mthd, void *data, u32 size)
+nvkm_nvsw_uevent(struct nvkm_object *object, void *argv, u32 argc, struct nvkm_uevent *uevent)
{
- struct nvkm_nvsw *nvsw = nvkm_nvsw(object);
- if (nvsw->func->mthd)
- return nvsw->func->mthd(nvsw, mthd, data, size);
- return -ENODEV;
+ union nv04_nvsw_event_args *args = argv;
+
+ if (!uevent)
+ return 0;
+ if (argc != sizeof(args->vn))
+ return -ENOSYS;
+
+ return nvkm_uevent_add(uevent, &nvkm_nvsw(object)->chan->event, 0,
+ NVKM_SW_CHAN_EVENT_PAGE_FLIP, NULL);
}
static int
-nvkm_nvsw_ntfy_(struct nvkm_object *object, u32 mthd,
- struct nvkm_event **pevent)
+nvkm_nvsw_mthd(struct nvkm_object *object, u32 mthd, void *data, u32 size)
{
struct nvkm_nvsw *nvsw = nvkm_nvsw(object);
- switch (mthd) {
- case NV04_NVSW_NTFY_UEVENT:
- *pevent = &nvsw->chan->event;
- return 0;
- default:
- break;
- }
- return -EINVAL;
+
+ if (nvsw->func->mthd)
+ return nvsw->func->mthd(nvsw, mthd, data, size);
+
+ return -ENODEV;
}
static const struct nvkm_object_func
nvkm_nvsw_ = {
- .mthd = nvkm_nvsw_mthd_,
- .ntfy = nvkm_nvsw_ntfy_,
+ .mthd = nvkm_nvsw_mthd,
+ .uevent = nvkm_nvsw_uevent,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/Kbuild b/drivers/gpu/drm/nouveau/nvkm/falcon/Kbuild
index d79d783..9ffe7b9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/Kbuild
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: MIT
nvkm-y += nvkm/falcon/base.o
nvkm-y += nvkm/falcon/cmdq.o
+nvkm-y += nvkm/falcon/fw.o
nvkm-y += nvkm/falcon/msgq.o
nvkm-y += nvkm/falcon/qmgr.o
nvkm-y += nvkm/falcon/v1.o
+
+nvkm-y += nvkm/falcon/gm200.o
+nvkm-y += nvkm/falcon/gp102.o
+nvkm-y += nvkm/falcon/ga100.o
+nvkm-y += nvkm/falcon/ga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/base.c b/drivers/gpu/drm/nouveau/nvkm/falcon/base.c
index f3f90c1..235149f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/base.c
@@ -22,8 +22,176 @@
#include "priv.h"
#include <subdev/mc.h>
+#include <subdev/timer.h>
#include <subdev/top.h>
+static const struct nvkm_falcon_func_dma *
+nvkm_falcon_dma(struct nvkm_falcon *falcon, enum nvkm_falcon_mem *mem_type, u32 *mem_base)
+{
+ switch (*mem_type) {
+ case IMEM: return falcon->func->imem_dma;
+ case DMEM: return falcon->func->dmem_dma;
+ default:
+ return NULL;
+ }
+}
+
+int
+nvkm_falcon_dma_wr(struct nvkm_falcon *falcon, const u8 *img, u64 dma_addr, u32 dma_base,
+ enum nvkm_falcon_mem mem_type, u32 mem_base, int len, bool sec)
+{
+ const struct nvkm_falcon_func_dma *dma = nvkm_falcon_dma(falcon, &mem_type, &mem_base);
+ const char *type = nvkm_falcon_mem(mem_type);
+ const int dmalen = 256;
+ u32 dma_start = 0;
+ u32 dst, src, cmd;
+ int ret, i;
+
+ if (WARN_ON(!dma->xfer))
+ return -EINVAL;
+
+ if (mem_type == DMEM) {
+ dma_start = dma_base;
+ dma_addr += dma_base;
+ }
+
+ FLCN_DBG(falcon, "%s %08x <- %08x bytes at %08x (%010llx %08x)",
+ type, mem_base, len, dma_base, dma_addr - dma_base, dma_start);
+ if (WARN_ON(!len || (len & (dmalen - 1))))
+ return -EINVAL;
+
+ ret = dma->init(falcon, dma_addr, dmalen, mem_type, sec, &cmd);
+ if (ret)
+ return ret;
+
+ dst = mem_base;
+ src = dma_base;
+ if (len) {
+ while (len >= dmalen) {
+ dma->xfer(falcon, dst, src - dma_start, cmd);
+
+ if (img && nvkm_printk_ok(falcon->owner, falcon->user, NV_DBG_TRACE)) {
+ for (i = 0; i < dmalen; i += 4, mem_base += 4) {
+ const int w = 8, x = (i / 4) % w;
+
+ if (x == 0)
+ printk(KERN_INFO "%s %08x <-", type, mem_base);
+ printk(KERN_CONT " %08x", *(u32 *)(img + src + i));
+ if (x == (w - 1) || ((i + 4) == dmalen))
+ printk(KERN_CONT " <- %08x+%08x", dma_base,
+ src + i - dma_base - (x * 4));
+ if (i == (7 * 4))
+ printk(KERN_CONT " *");
+ }
+ }
+
+ if (nvkm_msec(falcon->owner->device, 2000,
+ if (dma->done(falcon))
+ break;
+ ) < 0)
+ return -ETIMEDOUT;
+
+ src += dmalen;
+ dst += dmalen;
+ len -= dmalen;
+ }
+ WARN_ON(len);
+ }
+
+ return 0;
+}
+
+static const struct nvkm_falcon_func_pio *
+nvkm_falcon_pio(struct nvkm_falcon *falcon, enum nvkm_falcon_mem *mem_type, u32 *mem_base)
+{
+ switch (*mem_type) {
+ case IMEM:
+ return falcon->func->imem_pio;
+ case DMEM:
+ if (!falcon->func->emem_addr || *mem_base < falcon->func->emem_addr)
+ return falcon->func->dmem_pio;
+
+ *mem_base -= falcon->func->emem_addr;
+ fallthrough;
+ case EMEM:
+ return falcon->func->emem_pio;
+ default:
+ return NULL;
+ }
+}
+
+int
+nvkm_falcon_pio_rd(struct nvkm_falcon *falcon, u8 port, enum nvkm_falcon_mem mem_type, u32 mem_base,
+ const u8 *img, u32 img_base, int len)
+{
+ const struct nvkm_falcon_func_pio *pio = nvkm_falcon_pio(falcon, &mem_type, &mem_base);
+ const char *type = nvkm_falcon_mem(mem_type);
+ int xfer_len;
+
+ if (WARN_ON(!pio || !pio->rd))
+ return -EINVAL;
+
+ FLCN_DBG(falcon, "%s %08x -> %08x bytes at %08x", type, mem_base, len, img_base);
+ if (WARN_ON(!len || (len & (pio->min - 1))))
+ return -EINVAL;
+
+ pio->rd_init(falcon, port, mem_base);
+ do {
+ xfer_len = min(len, pio->max);
+ pio->rd(falcon, port, img, xfer_len);
+
+ if (nvkm_printk_ok(falcon->owner, falcon->user, NV_DBG_TRACE)) {
+ for (img_base = 0; img_base < xfer_len; img_base += 4, mem_base += 4) {
+ if (((img_base / 4) % 8) == 0)
+ printk(KERN_INFO "%s %08x ->", type, mem_base);
+ printk(KERN_CONT " %08x", *(u32 *)(img + img_base));
+ }
+ }
+
+ img += xfer_len;
+ len -= xfer_len;
+ } while (len);
+
+ return 0;
+}
+
+int
+nvkm_falcon_pio_wr(struct nvkm_falcon *falcon, const u8 *img, u32 img_base, u8 port,
+ enum nvkm_falcon_mem mem_type, u32 mem_base, int len, u16 tag, bool sec)
+{
+ const struct nvkm_falcon_func_pio *pio = nvkm_falcon_pio(falcon, &mem_type, &mem_base);
+ const char *type = nvkm_falcon_mem(mem_type);
+ int xfer_len;
+
+ if (WARN_ON(!pio || !pio->wr))
+ return -EINVAL;
+
+ FLCN_DBG(falcon, "%s %08x <- %08x bytes at %08x", type, mem_base, len, img_base);
+ if (WARN_ON(!len || (len & (pio->min - 1))))
+ return -EINVAL;
+
+ pio->wr_init(falcon, port, sec, mem_base);
+ do {
+ xfer_len = min(len, pio->max);
+ pio->wr(falcon, port, img, xfer_len, tag++);
+
+ if (nvkm_printk_ok(falcon->owner, falcon->user, NV_DBG_TRACE)) {
+ for (img_base = 0; img_base < xfer_len; img_base += 4, mem_base += 4) {
+ if (((img_base / 4) % 8) == 0)
+ printk(KERN_INFO "%s %08x <-", type, mem_base);
+ printk(KERN_CONT " %08x", *(u32 *)(img + img_base));
+ if ((img_base / 4) == 7 && mem_type == IMEM)
+ printk(KERN_CONT " %04x", tag - 1);
+ }
+ }
+
+ img += xfer_len;
+ len -= xfer_len;
+ } while (len);
+
+ return 0;
+}
+
void
nvkm_falcon_load_imem(struct nvkm_falcon *falcon, void *data, u32 start,
u32 size, u16 tag, u8 port, bool secure)
@@ -50,91 +218,21 @@ nvkm_falcon_load_dmem(struct nvkm_falcon *falcon, void *data, u32 start,
}
void
-nvkm_falcon_read_dmem(struct nvkm_falcon *falcon, u32 start, u32 size, u8 port,
- void *data)
-{
- mutex_lock(&falcon->dmem_mutex);
-
- falcon->func->read_dmem(falcon, start, size, port, data);
-
- mutex_unlock(&falcon->dmem_mutex);
-}
-
-void
-nvkm_falcon_bind_context(struct nvkm_falcon *falcon, struct nvkm_memory *inst)
-{
- if (!falcon->func->bind_context) {
- nvkm_error(falcon->user,
- "Context binding not supported on this falcon!\n");
- return;
- }
-
- falcon->func->bind_context(falcon, inst);
-}
-
-void
-nvkm_falcon_set_start_addr(struct nvkm_falcon *falcon, u32 start_addr)
-{
- falcon->func->set_start_addr(falcon, start_addr);
-}
-
-void
nvkm_falcon_start(struct nvkm_falcon *falcon)
{
falcon->func->start(falcon);
}
int
-nvkm_falcon_enable(struct nvkm_falcon *falcon)
-{
- struct nvkm_device *device = falcon->owner->device;
- int ret;
-
- nvkm_mc_enable(device, falcon->owner->type, falcon->owner->inst);
- ret = falcon->func->enable(falcon);
- if (ret) {
- nvkm_mc_disable(device, falcon->owner->type, falcon->owner->inst);
- return ret;
- }
-
- return 0;
-}
-
-void
-nvkm_falcon_disable(struct nvkm_falcon *falcon)
-{
- struct nvkm_device *device = falcon->owner->device;
-
- /* already disabled, return or wait_idle will timeout */
- if (!nvkm_mc_enabled(device, falcon->owner->type, falcon->owner->inst))
- return;
-
- falcon->func->disable(falcon);
-
- nvkm_mc_disable(device, falcon->owner->type, falcon->owner->inst);
-}
-
-int
nvkm_falcon_reset(struct nvkm_falcon *falcon)
{
- if (!falcon->func->reset) {
- nvkm_falcon_disable(falcon);
- return nvkm_falcon_enable(falcon);
- }
+ int ret;
- return falcon->func->reset(falcon);
-}
+ ret = falcon->func->disable(falcon);
+ if (WARN_ON(ret))
+ return ret;
-int
-nvkm_falcon_wait_for_halt(struct nvkm_falcon *falcon, u32 ms)
-{
- return falcon->func->wait_for_halt(falcon, ms);
-}
-
-int
-nvkm_falcon_clear_interrupt(struct nvkm_falcon *falcon, u32 mask)
-{
- return falcon->func->clear_interrupt(falcon, mask);
+ return nvkm_falcon_enable(falcon);
}
static int
@@ -169,7 +267,7 @@ nvkm_falcon_oneinit(struct nvkm_falcon *falcon)
}
void
-nvkm_falcon_put(struct nvkm_falcon *falcon, const struct nvkm_subdev *user)
+nvkm_falcon_put(struct nvkm_falcon *falcon, struct nvkm_subdev *user)
{
if (unlikely(!falcon))
return;
@@ -183,7 +281,7 @@ nvkm_falcon_put(struct nvkm_falcon *falcon, const struct nvkm_subdev *user)
}
int
-nvkm_falcon_get(struct nvkm_falcon *falcon, const struct nvkm_subdev *user)
+nvkm_falcon_get(struct nvkm_falcon *falcon, struct nvkm_subdev *user)
{
int ret = 0;
@@ -217,6 +315,7 @@ nvkm_falcon_ctor(const struct nvkm_falcon_func *func,
falcon->owner = subdev;
falcon->name = name;
falcon->addr = addr;
+ falcon->addr2 = func->addr2;
mutex_init(&falcon->mutex);
mutex_init(&falcon->dmem_mutex);
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/cmdq.c b/drivers/gpu/drm/nouveau/nvkm/falcon/cmdq.c
index 44cf6a88..211ebe7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/cmdq.c
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/cmdq.c
@@ -51,7 +51,7 @@ static void
nvkm_falcon_cmdq_push(struct nvkm_falcon_cmdq *cmdq, void *data, u32 size)
{
struct nvkm_falcon *falcon = cmdq->qmgr->falcon;
- nvkm_falcon_load_dmem(falcon, data, cmdq->position, size, 0);
+ nvkm_falcon_pio_wr(falcon, data, 0, 0, DMEM, cmdq->position, size, 0, false);
cmdq->position += ALIGN(size, QUEUE_ALIGNMENT);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/fw.c b/drivers/gpu/drm/nouveau/nvkm/falcon/fw.c
new file mode 100644
index 0000000..80a480b1
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/fw.c
@@ -0,0 +1,354 @@
+/*
+ * Copyright 2022 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <core/memory.h>
+#include <subdev/mmu.h>
+
+#include <nvfw/fw.h>
+#include <nvfw/hs.h>
+
+int
+nvkm_falcon_fw_patch(struct nvkm_falcon_fw *fw)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+ u32 sig_base_src = fw->sig_base_prd;
+ u32 src, dst, len, i;
+ int idx = 0;
+
+ FLCNFW_DBG(fw, "patching sigs:%d size:%d", fw->sig_nr, fw->sig_size);
+ if (fw->func->signature) {
+ idx = fw->func->signature(fw, &sig_base_src);
+ if (idx < 0)
+ return idx;
+ }
+
+ src = idx * fw->sig_size;
+ dst = fw->sig_base_img;
+ len = fw->sig_size / 4;
+ FLCNFW_DBG(fw, "patch idx:%d src:%08x dst:%08x", idx, sig_base_src + src, dst);
+ for (i = 0; i < len; i++) {
+ u32 sig = *(u32 *)(fw->sigs + src);
+
+ if (nvkm_printk_ok(falcon->owner, falcon->user, NV_DBG_TRACE)) {
+ if (i % 8 == 0)
+ printk(KERN_INFO "sig -> %08x:", dst);
+ printk(KERN_CONT " %08x", sig);
+ }
+
+ *(u32 *)(fw->fw.img + dst) = sig;
+ src += 4;
+ dst += 4;
+ }
+
+ return 0;
+}
+
+static void
+nvkm_falcon_fw_dtor_sigs(struct nvkm_falcon_fw *fw)
+{
+ kfree(fw->sigs);
+ fw->sigs = NULL;
+}
+
+int
+nvkm_falcon_fw_boot(struct nvkm_falcon_fw *fw, struct nvkm_subdev *user,
+ bool release, u32 *pmbox0, u32 *pmbox1, u32 mbox0_ok, u32 irqsclr)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+ int ret;
+
+ ret = nvkm_falcon_get(falcon, user);
+ if (ret)
+ return ret;
+
+ if (fw->sigs) {
+ ret = nvkm_falcon_fw_patch(fw);
+ if (ret)
+ goto done;
+
+ nvkm_falcon_fw_dtor_sigs(fw);
+ }
+
+ FLCNFW_DBG(fw, "resetting");
+ fw->func->reset(fw);
+
+ FLCNFW_DBG(fw, "loading");
+ if (fw->func->setup) {
+ ret = fw->func->setup(fw);
+ if (ret)
+ goto done;
+ }
+
+ ret = fw->func->load(fw);
+ if (ret)
+ goto done;
+
+ FLCNFW_DBG(fw, "booting");
+ ret = fw->func->boot(fw, pmbox0, pmbox1, mbox0_ok, irqsclr);
+ if (ret)
+ FLCNFW_ERR(fw, "boot failed: %d", ret);
+ else
+ FLCNFW_DBG(fw, "booted");
+
+done:
+ if (ret || release)
+ nvkm_falcon_put(falcon, user);
+ return ret;
+}
+
+int
+nvkm_falcon_fw_oneinit(struct nvkm_falcon_fw *fw, struct nvkm_falcon *falcon,
+ struct nvkm_vmm *vmm, struct nvkm_memory *inst)
+{
+ int ret;
+
+ fw->falcon = falcon;
+ fw->vmm = nvkm_vmm_ref(vmm);
+ fw->inst = nvkm_memory_ref(inst);
+
+ if (fw->boot) {
+ FLCN_DBG(falcon, "mapping %s fw", fw->fw.name);
+ ret = nvkm_vmm_get(fw->vmm, 12, nvkm_memory_size(&fw->fw.mem.memory), &fw->vma);
+ if (ret) {
+ FLCN_ERR(falcon, "get %d", ret);
+ return ret;
+ }
+
+ ret = nvkm_memory_map(&fw->fw.mem.memory, 0, fw->vmm, fw->vma, NULL, 0);
+ if (ret) {
+ FLCN_ERR(falcon, "map %d", ret);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+void
+nvkm_falcon_fw_dtor(struct nvkm_falcon_fw *fw)
+{
+ nvkm_vmm_put(fw->vmm, &fw->vma);
+ nvkm_vmm_unref(&fw->vmm);
+ nvkm_memory_unref(&fw->inst);
+ nvkm_falcon_fw_dtor_sigs(fw);
+ nvkm_firmware_dtor(&fw->fw);
+}
+
+static const struct nvkm_firmware_func
+nvkm_falcon_fw_dma = {
+ .type = NVKM_FIRMWARE_IMG_DMA,
+};
+
+static const struct nvkm_firmware_func
+nvkm_falcon_fw = {
+ .type = NVKM_FIRMWARE_IMG_RAM,
+};
+
+int
+nvkm_falcon_fw_sign(struct nvkm_falcon_fw *fw, u32 sig_base_img, u32 sig_size, const u8 *sigs,
+ int sig_nr_prd, u32 sig_base_prd, int sig_nr_dbg, u32 sig_base_dbg)
+{
+ fw->sig_base_prd = sig_base_prd;
+ fw->sig_base_dbg = sig_base_dbg;
+ fw->sig_base_img = sig_base_img;
+ fw->sig_size = sig_size;
+ fw->sig_nr = sig_nr_prd + sig_nr_dbg;
+
+ fw->sigs = kmalloc_array(fw->sig_nr, fw->sig_size, GFP_KERNEL);
+ if (!fw->sigs)
+ return -ENOMEM;
+
+ memcpy(fw->sigs, sigs + sig_base_prd, sig_nr_prd * fw->sig_size);
+ if (sig_nr_dbg)
+ memcpy(fw->sigs + sig_size, sigs + sig_base_dbg, sig_nr_dbg * fw->sig_size);
+
+ return 0;
+}
+
+int
+nvkm_falcon_fw_ctor(const struct nvkm_falcon_fw_func *func, const char *name,
+ struct nvkm_device *device, bool dma, const void *src, u32 len,
+ struct nvkm_falcon *falcon, struct nvkm_falcon_fw *fw)
+{
+ const struct nvkm_firmware_func *type = dma ? &nvkm_falcon_fw_dma : &nvkm_falcon_fw;
+ int ret;
+
+ fw->func = func;
+
+ ret = nvkm_firmware_ctor(type, name, device, src, len, &fw->fw);
+ if (ret)
+ return ret;
+
+ return falcon ? nvkm_falcon_fw_oneinit(fw, falcon, NULL, NULL) : 0;
+}
+
+int
+nvkm_falcon_fw_ctor_hs(const struct nvkm_falcon_fw_func *func, const char *name,
+ struct nvkm_subdev *subdev, const char *bl, const char *img, int ver,
+ struct nvkm_falcon *falcon, struct nvkm_falcon_fw *fw)
+{
+ const struct firmware *blob;
+ const struct nvfw_bin_hdr *hdr;
+ const struct nvfw_hs_header *hshdr;
+ const struct nvfw_hs_load_header *lhdr;
+ const struct nvfw_bl_desc *desc;
+ u32 loc, sig;
+ int ret;
+
+ ret = nvkm_firmware_load_name(subdev, img, "", ver, &blob);
+ if (ret)
+ return ret;
+
+ hdr = nvfw_bin_hdr(subdev, blob->data);
+ hshdr = nvfw_hs_header(subdev, blob->data + hdr->header_offset);
+
+ ret = nvkm_falcon_fw_ctor(func, name, subdev->device, bl != NULL,
+ blob->data + hdr->data_offset, hdr->data_size, falcon, fw);
+ if (ret)
+ goto done;
+
+ /* Earlier FW releases by NVIDIA for Nouveau's use aren't in NVIDIA's
+ * standard format, and don't have the indirection seen in the 0x10de
+ * case.
+ */
+ switch (hdr->bin_magic) {
+ case 0x000010de:
+ loc = *(u32 *)(blob->data + hshdr->patch_loc);
+ sig = *(u32 *)(blob->data + hshdr->patch_sig);
+ break;
+ case 0x3b1d14f0:
+ loc = hshdr->patch_loc;
+ sig = hshdr->patch_sig;
+ break;
+ default:
+ WARN_ON(1);
+ ret = -EINVAL;
+ goto done;
+ }
+
+ ret = nvkm_falcon_fw_sign(fw, loc, hshdr->sig_prod_size, blob->data,
+ 1, hshdr->sig_prod_offset + sig,
+ 1, hshdr->sig_dbg_offset + sig);
+ if (ret)
+ goto done;
+
+ lhdr = nvfw_hs_load_header(subdev, blob->data + hshdr->hdr_offset);
+
+ fw->nmem_base_img = 0;
+ fw->nmem_base = lhdr->non_sec_code_off;
+ fw->nmem_size = lhdr->non_sec_code_size;
+
+ fw->imem_base_img = lhdr->apps[0];
+ fw->imem_base = ALIGN(lhdr->apps[0], 0x100);
+ fw->imem_size = lhdr->apps[lhdr->num_apps + 0];
+
+ fw->dmem_base_img = lhdr->data_dma_base;
+ fw->dmem_base = 0;
+ fw->dmem_size = lhdr->data_size;
+ fw->dmem_sign = loc - lhdr->data_dma_base;
+
+ if (bl) {
+ nvkm_firmware_put(blob);
+
+ ret = nvkm_firmware_load_name(subdev, bl, "", ver, &blob);
+ if (ret)
+ return ret;
+
+ hdr = nvfw_bin_hdr(subdev, blob->data);
+ desc = nvfw_bl_desc(subdev, blob->data + hdr->header_offset);
+
+ fw->boot_addr = desc->start_tag << 8;
+ fw->boot_size = desc->code_size;
+ fw->boot = kmemdup(blob->data + hdr->data_offset + desc->code_off,
+ fw->boot_size, GFP_KERNEL);
+ if (!fw->boot)
+ ret = -ENOMEM;
+ } else {
+ fw->boot_addr = fw->nmem_base;
+ }
+
+done:
+ if (ret)
+ nvkm_falcon_fw_dtor(fw);
+
+ nvkm_firmware_put(blob);
+ return ret;
+}
+
+int
+nvkm_falcon_fw_ctor_hs_v2(const struct nvkm_falcon_fw_func *func, const char *name,
+ struct nvkm_subdev *subdev, const char *img, int ver,
+ struct nvkm_falcon *falcon, struct nvkm_falcon_fw *fw)
+{
+ const struct nvfw_bin_hdr *hdr;
+ const struct nvfw_hs_header_v2 *hshdr;
+ const struct nvfw_hs_load_header_v2 *lhdr;
+ const struct firmware *blob;
+ u32 loc, sig, cnt, *meta;
+ int ret;
+
+ ret = nvkm_firmware_load_name(subdev, img, "", ver, &blob);
+ if (ret)
+ return ret;
+
+ hdr = nvfw_bin_hdr(subdev, blob->data);
+ hshdr = nvfw_hs_header_v2(subdev, blob->data + hdr->header_offset);
+ meta = (u32 *)(blob->data + hshdr->meta_data_offset);
+ loc = *(u32 *)(blob->data + hshdr->patch_loc);
+ sig = *(u32 *)(blob->data + hshdr->patch_sig);
+ cnt = *(u32 *)(blob->data + hshdr->num_sig);
+
+ ret = nvkm_falcon_fw_ctor(func, name, subdev->device, true,
+ blob->data + hdr->data_offset, hdr->data_size, falcon, fw);
+ if (ret)
+ goto done;
+
+ ret = nvkm_falcon_fw_sign(fw, loc, hshdr->sig_prod_size / cnt, blob->data,
+ cnt, hshdr->sig_prod_offset + sig, 0, 0);
+ if (ret)
+ goto done;
+
+ lhdr = nvfw_hs_load_header_v2(subdev, blob->data + hshdr->header_offset);
+
+ fw->imem_base_img = lhdr->app[0].offset;
+ fw->imem_base = 0;
+ fw->imem_size = lhdr->app[0].size;
+
+ fw->dmem_base_img = lhdr->os_data_offset;
+ fw->dmem_base = 0;
+ fw->dmem_size = lhdr->os_data_size;
+ fw->dmem_sign = loc - lhdr->os_data_offset;
+
+ fw->boot_addr = lhdr->app[0].offset;
+
+ fw->fuse_ver = meta[0];
+ fw->engine_id = meta[1];
+ fw->ucode_id = meta[2];
+
+done:
+ if (ret)
+ nvkm_falcon_fw_dtor(fw);
+
+ nvkm_firmware_put(blob);
+ return ret;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/ga100.c b/drivers/gpu/drm/nouveau/nvkm/falcon/ga100.c
new file mode 100644
index 0000000..49fd329
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/ga100.c
@@ -0,0 +1,62 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+int
+ga100_flcn_fw_signature(struct nvkm_falcon_fw *fw, u32 *src_base_src)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+ struct nvkm_device *device = falcon->owner->device;
+ u32 reg_fuse_version;
+ int idx;
+
+ FLCN_DBG(falcon, "brom: %08x %08x", fw->engine_id, fw->ucode_id);
+ FLCN_DBG(falcon, "fuse_version: %d", fw->fuse_ver);
+
+ if (fw->engine_id & 0x00000001) {
+ reg_fuse_version = nvkm_rd32(device, 0x824140 + (fw->ucode_id - 1) * 4);
+ } else
+ if (fw->engine_id & 0x00000004) {
+ reg_fuse_version = nvkm_rd32(device, 0x824100 + (fw->ucode_id - 1) * 4);
+ } else
+ if (fw->engine_id & 0x00000400) {
+ reg_fuse_version = nvkm_rd32(device, 0x8241c0 + (fw->ucode_id - 1) * 4);
+ } else {
+ WARN_ON(1);
+ return -ENOSYS;
+ }
+
+ FLCN_DBG(falcon, "reg_fuse_version: %08x", reg_fuse_version);
+ if (reg_fuse_version) {
+ reg_fuse_version = fls(reg_fuse_version);
+ FLCN_DBG(falcon, "reg_fuse_version: %d", reg_fuse_version);
+
+ if (WARN_ON(fw->fuse_ver < reg_fuse_version))
+ return -EINVAL;
+
+ idx = fw->fuse_ver - reg_fuse_version;
+ } else {
+ idx = fw->sig_nr - 1;
+ }
+
+ return idx;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/ga102.c b/drivers/gpu/drm/nouveau/nvkm/falcon/ga102.c
new file mode 100644
index 0000000..0ff450fe
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/ga102.c
@@ -0,0 +1,148 @@
+/*
+ * Copyright 2022 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <subdev/mc.h>
+#include <subdev/timer.h>
+
+static bool
+ga102_flcn_dma_done(struct nvkm_falcon *falcon)
+{
+ return !!(nvkm_falcon_rd32(falcon, 0x118) & 0x00000002);
+}
+
+static void
+ga102_flcn_dma_xfer(struct nvkm_falcon *falcon, u32 mem_base, u32 dma_base, u32 cmd)
+{
+ nvkm_falcon_wr32(falcon, 0x114, mem_base);
+ nvkm_falcon_wr32(falcon, 0x11c, dma_base);
+ nvkm_falcon_wr32(falcon, 0x118, cmd);
+}
+
+static int
+ga102_flcn_dma_init(struct nvkm_falcon *falcon, u64 dma_addr, int xfer_len,
+ enum nvkm_falcon_mem mem_type, bool sec, u32 *cmd)
+{
+ *cmd = (ilog2(xfer_len) - 2) << 8;
+ if (mem_type == IMEM)
+ *cmd |= 0x00000010;
+ if (sec)
+ *cmd |= 0x00000004;
+
+ nvkm_falcon_wr32(falcon, 0x110, dma_addr >> 8);
+ nvkm_falcon_wr32(falcon, 0x128, 0x00000000);
+ return 0;
+}
+
+const struct nvkm_falcon_func_dma
+ga102_flcn_dma = {
+ .init = ga102_flcn_dma_init,
+ .xfer = ga102_flcn_dma_xfer,
+ .done = ga102_flcn_dma_done,
+};
+
+int
+ga102_flcn_reset_wait_mem_scrubbing(struct nvkm_falcon *falcon)
+{
+ nvkm_falcon_mask(falcon, 0x040, 0x00000000, 0x00000000);
+
+ if (nvkm_msec(falcon->owner->device, 20,
+ if (!(nvkm_falcon_rd32(falcon, 0x0f4) & 0x00001000))
+ break;
+ ) < 0)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+int
+ga102_flcn_reset_prep(struct nvkm_falcon *falcon)
+{
+ nvkm_falcon_rd32(falcon, 0x0f4);
+
+ nvkm_usec(falcon->owner->device, 150,
+ if (nvkm_falcon_rd32(falcon, 0x0f4) & 0x80000000)
+ break;
+ _warn = false;
+ );
+
+ return 0;
+}
+
+int
+ga102_flcn_select(struct nvkm_falcon *falcon)
+{
+ if ((nvkm_falcon_rd32(falcon, falcon->addr2 + 0x668) & 0x00000010) != 0x00000000) {
+ nvkm_falcon_wr32(falcon, falcon->addr2 + 0x668, 0x00000000);
+ if (nvkm_msec(falcon->owner->device, 10,
+ if (nvkm_falcon_rd32(falcon, falcon->addr2 + 0x668) & 0x00000001)
+ break;
+ ) < 0)
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
+int
+ga102_flcn_fw_boot(struct nvkm_falcon_fw *fw, u32 *mbox0, u32 *mbox1, u32 mbox0_ok, u32 irqsclr)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+
+ nvkm_falcon_wr32(falcon, falcon->addr2 + 0x210, fw->dmem_sign);
+ nvkm_falcon_wr32(falcon, falcon->addr2 + 0x19c, fw->engine_id);
+ nvkm_falcon_wr32(falcon, falcon->addr2 + 0x198, fw->ucode_id);
+ nvkm_falcon_wr32(falcon, falcon->addr2 + 0x180, 0x00000001);
+
+ return gm200_flcn_fw_boot(fw, mbox0, mbox1, mbox0_ok, irqsclr);
+}
+
+int
+ga102_flcn_fw_load(struct nvkm_falcon_fw *fw)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+ int ret = 0;
+
+ nvkm_falcon_mask(falcon, 0x624, 0x00000080, 0x00000080);
+ nvkm_falcon_wr32(falcon, 0x10c, 0x00000000);
+ nvkm_falcon_mask(falcon, 0x600, 0x00010007, (0 << 16) | (1 << 2) | 1);
+
+ ret = nvkm_falcon_dma_wr(falcon, fw->fw.img, fw->fw.phys, fw->imem_base_img,
+ IMEM, fw->imem_base, fw->imem_size, true);
+ if (ret)
+ return ret;
+
+ ret = nvkm_falcon_dma_wr(falcon, fw->fw.img, fw->fw.phys, fw->dmem_base_img,
+ DMEM, fw->dmem_base, fw->dmem_size, false);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+const struct nvkm_falcon_fw_func
+ga102_flcn_fw = {
+ .signature = ga100_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .load = ga102_flcn_fw_load,
+ .boot = ga102_flcn_fw_boot,
+};
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/gm200.c b/drivers/gpu/drm/nouveau/nvkm/falcon/gm200.c
new file mode 100644
index 0000000..393ade9
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/gm200.c
@@ -0,0 +1,345 @@
+/*
+ * Copyright 2022 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <core/memory.h>
+#include <subdev/mc.h>
+#include <subdev/timer.h>
+
+void
+gm200_flcn_tracepc(struct nvkm_falcon *falcon)
+{
+ u32 sctl = nvkm_falcon_rd32(falcon, 0x240);
+ u32 tidx = nvkm_falcon_rd32(falcon, 0x148);
+ int nr = (tidx & 0x00ff0000) >> 16, sp, ip;
+
+ FLCN_ERR(falcon, "TRACEPC SCTL %08x TIDX %08x", sctl, tidx);
+ for (sp = 0; sp < nr; sp++) {
+ nvkm_falcon_wr32(falcon, 0x148, sp);
+ ip = nvkm_falcon_rd32(falcon, 0x14c);
+ FLCN_ERR(falcon, "TRACEPC: %08x", ip);
+ }
+}
+
+static void
+gm200_flcn_pio_dmem_rd(struct nvkm_falcon *falcon, u8 port, const u8 *img, int len)
+{
+ while (len >= 4) {
+ *(u32 *)img = nvkm_falcon_rd32(falcon, 0x1c4 + (port * 8));
+ img += 4;
+ len -= 4;
+ }
+}
+
+static void
+gm200_flcn_pio_dmem_rd_init(struct nvkm_falcon *falcon, u8 port, u32 dmem_base)
+{
+ nvkm_falcon_wr32(falcon, 0x1c0 + (port * 8), BIT(25) | dmem_base);
+}
+
+static void
+gm200_flcn_pio_dmem_wr(struct nvkm_falcon *falcon, u8 port, const u8 *img, int len, u16 tag)
+{
+ while (len >= 4) {
+ nvkm_falcon_wr32(falcon, 0x1c4 + (port * 8), *(u32 *)img);
+ img += 4;
+ len -= 4;
+ }
+}
+
+static void
+gm200_flcn_pio_dmem_wr_init(struct nvkm_falcon *falcon, u8 port, bool sec, u32 dmem_base)
+{
+ nvkm_falcon_wr32(falcon, 0x1c0 + (port * 8), BIT(24) | dmem_base);
+}
+
+const struct nvkm_falcon_func_pio
+gm200_flcn_dmem_pio = {
+ .min = 4,
+ .max = 0x100,
+ .wr_init = gm200_flcn_pio_dmem_wr_init,
+ .wr = gm200_flcn_pio_dmem_wr,
+ .rd_init = gm200_flcn_pio_dmem_rd_init,
+ .rd = gm200_flcn_pio_dmem_rd,
+};
+
+static void
+gm200_flcn_pio_imem_wr_init(struct nvkm_falcon *falcon, u8 port, bool sec, u32 imem_base)
+{
+ nvkm_falcon_wr32(falcon, 0x180 + (port * 0x10), (sec ? BIT(28) : 0) | BIT(24) | imem_base);
+}
+
+static void
+gm200_flcn_pio_imem_wr(struct nvkm_falcon *falcon, u8 port, const u8 *img, int len, u16 tag)
+{
+ nvkm_falcon_wr32(falcon, 0x188 + (port * 0x10), tag++);
+ while (len >= 4) {
+ nvkm_falcon_wr32(falcon, 0x184 + (port * 0x10), *(u32 *)img);
+ img += 4;
+ len -= 4;
+ }
+}
+
+const struct nvkm_falcon_func_pio
+gm200_flcn_imem_pio = {
+ .min = 0x100,
+ .max = 0x100,
+ .wr_init = gm200_flcn_pio_imem_wr_init,
+ .wr = gm200_flcn_pio_imem_wr,
+};
+
+int
+gm200_flcn_bind_stat(struct nvkm_falcon *falcon, bool intr)
+{
+ if (intr && !(nvkm_falcon_rd32(falcon, 0x008) & 0x00000008))
+ return -1;
+
+ return (nvkm_falcon_rd32(falcon, 0x0dc) & 0x00007000) >> 12;
+}
+
+void
+gm200_flcn_bind_inst(struct nvkm_falcon *falcon, int target, u64 addr)
+{
+ nvkm_falcon_mask(falcon, 0x604, 0x00000007, 0x00000000); /* DMAIDX_VIRT */
+ nvkm_falcon_wr32(falcon, 0x054, (1 << 30) | (target << 28) | (addr >> 12));
+ nvkm_falcon_mask(falcon, 0x090, 0x00010000, 0x00010000);
+ nvkm_falcon_mask(falcon, 0x0a4, 0x00000008, 0x00000008);
+}
+
+int
+gm200_flcn_reset_wait_mem_scrubbing(struct nvkm_falcon *falcon)
+{
+ nvkm_falcon_mask(falcon, 0x040, 0x00000000, 0x00000000);
+
+ if (nvkm_msec(falcon->owner->device, 10,
+ if (!(nvkm_falcon_rd32(falcon, 0x10c) & 0x00000006))
+ break;
+ ) < 0)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+int
+gm200_flcn_enable(struct nvkm_falcon *falcon)
+{
+ struct nvkm_device *device = falcon->owner->device;
+ int ret;
+
+ if (falcon->func->reset_eng) {
+ ret = falcon->func->reset_eng(falcon);
+ if (ret)
+ return ret;
+ }
+
+ if (falcon->func->select) {
+ ret = falcon->func->select(falcon);
+ if (ret)
+ return ret;
+ }
+
+ if (falcon->func->reset_pmc)
+ nvkm_mc_enable(device, falcon->owner->type, falcon->owner->inst);
+
+ ret = falcon->func->reset_wait_mem_scrubbing(falcon);
+ if (ret)
+ return ret;
+
+ nvkm_falcon_wr32(falcon, 0x084, nvkm_rd32(device, 0x000000));
+ return 0;
+}
+
+int
+gm200_flcn_disable(struct nvkm_falcon *falcon)
+{
+ struct nvkm_device *device = falcon->owner->device;
+ int ret;
+
+ if (falcon->func->select) {
+ ret = falcon->func->select(falcon);
+ if (ret)
+ return ret;
+ }
+
+ nvkm_falcon_mask(falcon, 0x048, 0x00000003, 0x00000000);
+ nvkm_falcon_wr32(falcon, 0x014, 0xffffffff);
+
+ if (falcon->func->reset_pmc) {
+ if (falcon->func->reset_prep) {
+ ret = falcon->func->reset_prep(falcon);
+ if (ret)
+ return ret;
+ }
+
+ nvkm_mc_disable(device, falcon->owner->type, falcon->owner->inst);
+ }
+
+ if (falcon->func->reset_eng) {
+ ret = falcon->func->reset_eng(falcon);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+gm200_flcn_fw_boot(struct nvkm_falcon_fw *fw, u32 *pmbox0, u32 *pmbox1, u32 mbox0_ok, u32 irqsclr)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+ u32 mbox0, mbox1;
+ int ret = 0;
+
+ nvkm_falcon_wr32(falcon, 0x040, pmbox0 ? *pmbox0 : 0xcafebeef);
+ if (pmbox1)
+ nvkm_falcon_wr32(falcon, 0x044, *pmbox1);
+
+ nvkm_falcon_wr32(falcon, 0x104, fw->boot_addr);
+ nvkm_falcon_wr32(falcon, 0x100, 0x00000002);
+
+ if (nvkm_msec(falcon->owner->device, 2000,
+ if (nvkm_falcon_rd32(falcon, 0x100) & 0x00000010)
+ break;
+ ) < 0)
+ ret = -ETIMEDOUT;
+
+ mbox0 = nvkm_falcon_rd32(falcon, 0x040);
+ mbox1 = nvkm_falcon_rd32(falcon, 0x044);
+ if (FLCN_ERRON(falcon, ret || mbox0 != mbox0_ok, "mbox %08x %08x", mbox0, mbox1))
+ ret = ret ?: -EIO;
+
+ if (irqsclr)
+ nvkm_falcon_mask(falcon, 0x004, 0xffffffff, irqsclr);
+
+ return ret;
+}
+
+int
+gm200_flcn_fw_load(struct nvkm_falcon_fw *fw)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+ int target, ret;
+
+ if (fw->inst) {
+ nvkm_falcon_mask(falcon, 0x048, 0x00000001, 0x00000001);
+
+ switch (nvkm_memory_target(fw->inst)) {
+ case NVKM_MEM_TARGET_VRAM: target = 0; break;
+ case NVKM_MEM_TARGET_HOST: target = 2; break;
+ case NVKM_MEM_TARGET_NCOH: target = 3; break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ falcon->func->bind_inst(falcon, target, nvkm_memory_addr(fw->inst));
+
+ if (nvkm_msec(falcon->owner->device, 10,
+ if (falcon->func->bind_stat(falcon, falcon->func->bind_intr) == 5)
+ break;
+ ) < 0)
+ return -ETIMEDOUT;
+
+ nvkm_falcon_mask(falcon, 0x004, 0x00000008, 0x00000008);
+ nvkm_falcon_mask(falcon, 0x058, 0x00000002, 0x00000002);
+
+ if (nvkm_msec(falcon->owner->device, 10,
+ if (falcon->func->bind_stat(falcon, false) == 0)
+ break;
+ ) < 0)
+ return -ETIMEDOUT;
+ } else {
+ nvkm_falcon_mask(falcon, 0x624, 0x00000080, 0x00000080);
+ nvkm_falcon_wr32(falcon, 0x10c, 0x00000000);
+ }
+
+ if (fw->boot) {
+ switch (nvkm_memory_target(&fw->fw.mem.memory)) {
+ case NVKM_MEM_TARGET_VRAM: target = 4; break;
+ case NVKM_MEM_TARGET_HOST: target = 5; break;
+ case NVKM_MEM_TARGET_NCOH: target = 6; break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ ret = nvkm_falcon_pio_wr(falcon, fw->boot, 0, 0,
+ IMEM, falcon->code.limit - fw->boot_size, fw->boot_size,
+ fw->boot_addr >> 8, false);
+ if (ret)
+ return ret;
+
+ return fw->func->load_bld(fw);
+ }
+
+ ret = nvkm_falcon_pio_wr(falcon, fw->fw.img + fw->nmem_base_img, fw->nmem_base_img, 0,
+ IMEM, fw->nmem_base, fw->nmem_size, fw->nmem_base >> 8, false);
+ if (ret)
+ return ret;
+
+ ret = nvkm_falcon_pio_wr(falcon, fw->fw.img + fw->imem_base_img, fw->imem_base_img, 0,
+ IMEM, fw->imem_base, fw->imem_size, fw->imem_base >> 8, true);
+ if (ret)
+ return ret;
+
+ ret = nvkm_falcon_pio_wr(falcon, fw->fw.img + fw->dmem_base_img, fw->dmem_base_img, 0,
+ DMEM, fw->dmem_base, fw->dmem_size, 0, false);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+int
+gm200_flcn_fw_reset(struct nvkm_falcon_fw *fw)
+{
+ return nvkm_falcon_reset(fw->falcon);
+}
+
+int
+gm200_flcn_fw_signature(struct nvkm_falcon_fw *fw, u32 *sig_base_src)
+{
+ struct nvkm_falcon *falcon = fw->falcon;
+ u32 addr = falcon->func->debug;
+ int ret = 0;
+
+ if (addr) {
+ ret = nvkm_falcon_enable(falcon);
+ if (ret)
+ return ret;
+
+ if (nvkm_falcon_rd32(falcon, addr) & 0x00100000) {
+ *sig_base_src = fw->sig_base_dbg;
+ return 1;
+ }
+ }
+
+ return ret;
+}
+
+const struct nvkm_falcon_fw_func
+gm200_flcn_fw = {
+ .signature = gm200_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .load = gm200_flcn_fw_load,
+ .boot = gm200_flcn_fw_boot,
+};
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/gp102.c b/drivers/gpu/drm/nouveau/nvkm/falcon/gp102.c
new file mode 100644
index 0000000..c774935
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/gp102.c
@@ -0,0 +1,82 @@
+/*
+ * Copyright 2022 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+static void
+gp102_flcn_pio_emem_rd(struct nvkm_falcon *falcon, u8 port, const u8 *img, int len)
+{
+ while (len >= 4) {
+ *(u32 *)img = nvkm_falcon_rd32(falcon, 0xac4 + (port * 8));
+ img += 4;
+ len -= 4;
+ }
+}
+
+static void
+gp102_flcn_pio_emem_rd_init(struct nvkm_falcon *falcon, u8 port, u32 dmem_base)
+{
+ nvkm_falcon_wr32(falcon, 0xac0 + (port * 8), BIT(25) | dmem_base);
+}
+
+static void
+gp102_flcn_pio_emem_wr(struct nvkm_falcon *falcon, u8 port, const u8 *img, int len, u16 tag)
+{
+ while (len >= 4) {
+ nvkm_falcon_wr32(falcon, 0xac4 + (port * 8), *(u32 *)img);
+ img += 4;
+ len -= 4;
+ }
+}
+
+static void
+gp102_flcn_pio_emem_wr_init(struct nvkm_falcon *falcon, u8 port, bool sec, u32 emem_base)
+{
+ nvkm_falcon_wr32(falcon, 0xac0 + (port * 8), BIT(24) | emem_base);
+}
+
+const struct nvkm_falcon_func_pio
+gp102_flcn_emem_pio = {
+ .min = 4,
+ .max = 0x100,
+ .wr_init = gp102_flcn_pio_emem_wr_init,
+ .wr = gp102_flcn_pio_emem_wr,
+ .rd_init = gp102_flcn_pio_emem_rd_init,
+ .rd = gp102_flcn_pio_emem_rd,
+};
+
+int
+gp102_flcn_reset_eng(struct nvkm_falcon *falcon)
+{
+ int ret;
+
+ if (falcon->func->reset_prep) {
+ ret = falcon->func->reset_prep(falcon);
+ if (ret)
+ return ret;
+ }
+
+ nvkm_falcon_mask(falcon, 0x3c0, 0x00000001, 0x00000001);
+ udelay(10);
+ nvkm_falcon_mask(falcon, 0x3c0, 0x00000001, 0x00000000);
+
+ return falcon->func->reset_wait_mem_scrubbing(falcon);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/msgq.c b/drivers/gpu/drm/nouveau/nvkm/falcon/msgq.c
index e74371df..16b246f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/msgq.c
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/msgq.c
@@ -25,7 +25,7 @@
static void
nvkm_falcon_msgq_open(struct nvkm_falcon_msgq *msgq)
{
- mutex_lock(&msgq->mutex);
+ spin_lock(&msgq->lock);
msgq->position = nvkm_falcon_rd32(msgq->qmgr->falcon, msgq->tail_reg);
}
@@ -37,10 +37,10 @@ nvkm_falcon_msgq_close(struct nvkm_falcon_msgq *msgq, bool commit)
if (commit)
nvkm_falcon_wr32(falcon, msgq->tail_reg, msgq->position);
- mutex_unlock(&msgq->mutex);
+ spin_unlock(&msgq->lock);
}
-static bool
+bool
nvkm_falcon_msgq_empty(struct nvkm_falcon_msgq *msgq)
{
u32 head = nvkm_falcon_rd32(msgq->qmgr->falcon, msgq->head_reg);
@@ -68,7 +68,7 @@ nvkm_falcon_msgq_pop(struct nvkm_falcon_msgq *msgq, void *data, u32 size)
return -EINVAL;
}
- nvkm_falcon_read_dmem(falcon, tail, size, 0, data);
+ nvkm_falcon_pio_rd(falcon, 0, DMEM, tail, data, 0, size);
msgq->position += ALIGN(size, QUEUE_ALIGNMENT);
return 0;
}
@@ -208,6 +208,6 @@ nvkm_falcon_msgq_new(struct nvkm_falcon_qmgr *qmgr, const char *name,
msgq->qmgr = qmgr;
msgq->name = name;
- mutex_init(&msgq->mutex);
+ spin_lock_init(&msgq->lock);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/priv.h b/drivers/gpu/drm/nouveau/nvkm/falcon/priv.h
index 4661887..11a24b9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/priv.h
@@ -2,4 +2,12 @@
#ifndef __NVKM_FALCON_PRIV_H__
#define __NVKM_FALCON_PRIV_H__
#include <core/falcon.h>
+
+static inline int
+nvkm_falcon_enable(struct nvkm_falcon *falcon)
+{
+ if (falcon->func->enable)
+ return falcon->func->enable(falcon);
+ return 0;
+}
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/qmgr.h b/drivers/gpu/drm/nouveau/nvkm/falcon/qmgr.h
index 976cb7b..79f0da9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/qmgr.h
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/qmgr.h
@@ -73,7 +73,7 @@ struct nvkm_falcon_cmdq {
struct nvkm_falcon_msgq {
struct nvkm_falcon_qmgr *qmgr;
const char *name;
- struct mutex mutex;
+ spinlock_t lock;
u32 head_reg;
u32 tail_reg;
@@ -82,8 +82,7 @@ struct nvkm_falcon_msgq {
u32 position;
};
-#define FLCNQ_PRINTK(t,q,f,a...) \
- FLCN_PRINTK(t, (q)->qmgr->falcon, "%s: "f, (q)->name, ##a)
-#define FLCNQ_DBG(q,f,a...) FLCNQ_PRINTK(debug, (q), f, ##a)
-#define FLCNQ_ERR(q,f,a...) FLCNQ_PRINTK(error, (q), f, ##a)
+#define FLCNQ_PRINTK(q,l,p,f,a...) FLCN_PRINTK((q)->qmgr->falcon, l, p, "%s: "f, (q)->name, ##a)
+#define FLCNQ_DBG(q,f,a...) FLCNQ_PRINTK((q), DEBUG, info, f, ##a)
+#define FLCNQ_ERR(q,f,a...) FLCNQ_PRINTK((q), ERROR, err, f, ##a)
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/falcon/v1.c b/drivers/gpu/drm/nouveau/nvkm/falcon/v1.c
index b0ee4c3..dd2ddc5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/falcon/v1.c
+++ b/drivers/gpu/drm/nouveau/nvkm/falcon/v1.c
@@ -64,44 +64,13 @@ nvkm_falcon_v1_load_imem(struct nvkm_falcon *falcon, void *data, u32 start,
nvkm_falcon_wr32(falcon, 0x184 + (port * 16), 0);
}
-static void
-nvkm_falcon_v1_load_emem(struct nvkm_falcon *falcon, void *data, u32 start,
- u32 size, u8 port)
-{
- u8 rem = size % 4;
- int i;
-
- size -= rem;
-
- nvkm_falcon_wr32(falcon, 0xac0 + (port * 8), start | (0x1 << 24));
- for (i = 0; i < size / 4; i++)
- nvkm_falcon_wr32(falcon, 0xac4 + (port * 8), ((u32 *)data)[i]);
-
- /*
- * If size is not a multiple of 4, mask the last word to ensure garbage
- * does not get written
- */
- if (rem) {
- u32 extra = ((u32 *)data)[i];
-
- nvkm_falcon_wr32(falcon, 0xac4 + (port * 8),
- extra & (BIT(rem * 8) - 1));
- }
-}
-
void
nvkm_falcon_v1_load_dmem(struct nvkm_falcon *falcon, void *data, u32 start,
u32 size, u8 port)
{
- const struct nvkm_falcon_func *func = falcon->func;
u8 rem = size % 4;
int i;
- if (func->emem_addr && start >= func->emem_addr)
- return nvkm_falcon_v1_load_emem(falcon, data,
- start - func->emem_addr, size,
- port);
-
size -= rem;
nvkm_falcon_wr32(falcon, 0x1c0 + (port * 8), start | (0x1 << 24));
@@ -120,113 +89,6 @@ nvkm_falcon_v1_load_dmem(struct nvkm_falcon *falcon, void *data, u32 start,
}
}
-static void
-nvkm_falcon_v1_read_emem(struct nvkm_falcon *falcon, u32 start, u32 size,
- u8 port, void *data)
-{
- u8 rem = size % 4;
- int i;
-
- size -= rem;
-
- nvkm_falcon_wr32(falcon, 0xac0 + (port * 8), start | (0x1 << 25));
- for (i = 0; i < size / 4; i++)
- ((u32 *)data)[i] = nvkm_falcon_rd32(falcon, 0xac4 + (port * 8));
-
- /*
- * If size is not a multiple of 4, mask the last word to ensure garbage
- * does not get read
- */
- if (rem) {
- u32 extra = nvkm_falcon_rd32(falcon, 0xac4 + (port * 8));
-
- for (i = size; i < size + rem; i++) {
- ((u8 *)data)[i] = (u8)(extra & 0xff);
- extra >>= 8;
- }
- }
-}
-
-void
-nvkm_falcon_v1_read_dmem(struct nvkm_falcon *falcon, u32 start, u32 size,
- u8 port, void *data)
-{
- const struct nvkm_falcon_func *func = falcon->func;
- u8 rem = size % 4;
- int i;
-
- if (func->emem_addr && start >= func->emem_addr)
- return nvkm_falcon_v1_read_emem(falcon, start - func->emem_addr,
- size, port, data);
-
- size -= rem;
-
- nvkm_falcon_wr32(falcon, 0x1c0 + (port * 8), start | (0x1 << 25));
- for (i = 0; i < size / 4; i++)
- ((u32 *)data)[i] = nvkm_falcon_rd32(falcon, 0x1c4 + (port * 8));
-
- /*
- * If size is not a multiple of 4, mask the last word to ensure garbage
- * does not get read
- */
- if (rem) {
- u32 extra = nvkm_falcon_rd32(falcon, 0x1c4 + (port * 8));
-
- for (i = size; i < size + rem; i++) {
- ((u8 *)data)[i] = (u8)(extra & 0xff);
- extra >>= 8;
- }
- }
-}
-
-void
-nvkm_falcon_v1_bind_context(struct nvkm_falcon *falcon, struct nvkm_memory *ctx)
-{
- const u32 fbif = falcon->func->fbif;
- u32 inst_loc;
-
- /* disable instance block binding */
- if (ctx == NULL) {
- nvkm_falcon_wr32(falcon, 0x10c, 0x0);
- return;
- }
-
- nvkm_falcon_wr32(falcon, 0x10c, 0x1);
-
- /* setup apertures - virtual */
- nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_UCODE, 0x4);
- nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_VIRT, 0x0);
- /* setup apertures - physical */
- nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_PHYS_VID, 0x4);
- nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_PHYS_SYS_COH, 0x5);
- nvkm_falcon_wr32(falcon, fbif + 4 * FALCON_DMAIDX_PHYS_SYS_NCOH, 0x6);
-
- /* Set context */
- switch (nvkm_memory_target(ctx)) {
- case NVKM_MEM_TARGET_VRAM: inst_loc = 0; break;
- case NVKM_MEM_TARGET_HOST: inst_loc = 2; break;
- case NVKM_MEM_TARGET_NCOH: inst_loc = 3; break;
- default:
- WARN_ON(1);
- return;
- }
-
- /* Enable context */
- nvkm_falcon_mask(falcon, 0x048, 0x1, 0x1);
- nvkm_falcon_wr32(falcon, 0x054,
- ((nvkm_memory_addr(ctx) >> 12) & 0xfffffff) |
- (inst_loc << 28) | (1 << 30));
-
- nvkm_falcon_mask(falcon, 0x090, 0x10000, 0x10000);
- nvkm_falcon_mask(falcon, 0x0a4, 0x8, 0x8);
-}
-
-void
-nvkm_falcon_v1_set_start_addr(struct nvkm_falcon *falcon, u32 start_addr)
-{
- nvkm_falcon_wr32(falcon, 0x104, start_addr);
-}
-
void
nvkm_falcon_v1_start(struct nvkm_falcon *falcon)
{
@@ -237,75 +99,3 @@ nvkm_falcon_v1_start(struct nvkm_falcon *falcon)
else
nvkm_falcon_wr32(falcon, 0x100, 0x2);
}
-
-int
-nvkm_falcon_v1_wait_for_halt(struct nvkm_falcon *falcon, u32 ms)
-{
- struct nvkm_device *device = falcon->owner->device;
- int ret;
-
- ret = nvkm_wait_msec(device, ms, falcon->addr + 0x100, 0x10, 0x10);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-int
-nvkm_falcon_v1_clear_interrupt(struct nvkm_falcon *falcon, u32 mask)
-{
- struct nvkm_device *device = falcon->owner->device;
- int ret;
-
- /* clear interrupt(s) */
- nvkm_falcon_mask(falcon, 0x004, mask, mask);
- /* wait until interrupts are cleared */
- ret = nvkm_wait_msec(device, 10, falcon->addr + 0x008, mask, 0x0);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-static int
-falcon_v1_wait_idle(struct nvkm_falcon *falcon)
-{
- struct nvkm_device *device = falcon->owner->device;
- int ret;
-
- ret = nvkm_wait_msec(device, 10, falcon->addr + 0x04c, 0xffff, 0x0);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-int
-nvkm_falcon_v1_enable(struct nvkm_falcon *falcon)
-{
- struct nvkm_device *device = falcon->owner->device;
- int ret;
-
- ret = nvkm_wait_msec(device, 10, falcon->addr + 0x10c, 0x6, 0x0);
- if (ret < 0) {
- nvkm_error(falcon->user, "Falcon mem scrubbing timeout\n");
- return ret;
- }
-
- ret = falcon_v1_wait_idle(falcon);
- if (ret)
- return ret;
-
- /* enable IRQs */
- nvkm_falcon_wr32(falcon, 0x010, 0xff);
-
- return 0;
-}
-
-void
-nvkm_falcon_v1_disable(struct nvkm_falcon *falcon)
-{
- /* disable IRQs and wait for any previous code to complete */
- nvkm_falcon_wr32(falcon, 0x014, 0xff);
- falcon_v1_wait_idle(falcon);
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/nvfw/acr.c b/drivers/gpu/drm/nouveau/nvkm/nvfw/acr.c
index bef790a..83a9c48 100644
--- a/drivers/gpu/drm/nouveau/nvkm/nvfw/acr.c
+++ b/drivers/gpu/drm/nouveau/nvkm/nvfw/acr.c
@@ -45,6 +45,47 @@ wpr_header_v1_dump(struct nvkm_subdev *subdev, const struct wpr_header_v1 *hdr)
nvkm_debug(subdev, "\tstatus : %d\n", hdr->status);
}
+void
+wpr_generic_header_dump(struct nvkm_subdev *subdev, const struct wpr_generic_header *hdr)
+{
+ nvkm_debug(subdev, "wprGenericHeader\n");
+ nvkm_debug(subdev, "\tidentifier : %04x\n", hdr->identifier);
+ nvkm_debug(subdev, "\tversion : %04x\n", hdr->version);
+ nvkm_debug(subdev, "\tsize : %08x\n", hdr->size);
+}
+
+void
+wpr_header_v2_dump(struct nvkm_subdev *subdev, const struct wpr_header_v2 *hdr)
+{
+ wpr_generic_header_dump(subdev, &hdr->hdr);
+ wpr_header_v1_dump(subdev, &hdr->wpr);
+}
+
+void
+lsb_header_v2_dump(struct nvkm_subdev *subdev, struct lsb_header_v2 *hdr)
+{
+ wpr_generic_header_dump(subdev, &hdr->hdr);
+ nvkm_debug(subdev, "lsbHeader\n");
+ nvkm_debug(subdev, "\tucodeOff : 0x%x\n", hdr->ucode_off);
+ nvkm_debug(subdev, "\tucodeSize : 0x%x\n", hdr->ucode_size);
+ nvkm_debug(subdev, "\tdataSize : 0x%x\n", hdr->data_size);
+ nvkm_debug(subdev, "\tblCodeSize : 0x%x\n", hdr->bl_code_size);
+ nvkm_debug(subdev, "\tblImemOff : 0x%x\n", hdr->bl_imem_off);
+ nvkm_debug(subdev, "\tblDataOff : 0x%x\n", hdr->bl_data_off);
+ nvkm_debug(subdev, "\tblDataSize : 0x%x\n", hdr->bl_data_size);
+ nvkm_debug(subdev, "\treserved0 : %08x\n", hdr->rsvd0);
+ nvkm_debug(subdev, "\tappCodeOff : 0x%x\n", hdr->app_code_off);
+ nvkm_debug(subdev, "\tappCodeSize : 0x%x\n", hdr->app_code_size);
+ nvkm_debug(subdev, "\tappDataOff : 0x%x\n", hdr->app_data_off);
+ nvkm_debug(subdev, "\tappDataSize : 0x%x\n", hdr->app_data_size);
+ nvkm_debug(subdev, "\tappImemOffset : 0x%x\n", hdr->app_imem_offset);
+ nvkm_debug(subdev, "\tappDmemOffset : 0x%x\n", hdr->app_dmem_offset);
+ nvkm_debug(subdev, "\tflags : 0x%x\n", hdr->flags);
+ nvkm_debug(subdev, "\tmonitorCodeOff: 0x%x\n", hdr->monitor_code_offset);
+ nvkm_debug(subdev, "\tmonitorDataOff: 0x%x\n", hdr->monitor_data_offset);
+ nvkm_debug(subdev, "\tmanifestOffset: 0x%x\n", hdr->manifest_offset);
+}
+
static void
lsb_header_tail_dump(struct nvkm_subdev *subdev, struct lsb_header_tail *hdr)
{
diff --git a/drivers/gpu/drm/nouveau/nvkm/nvfw/hs.c b/drivers/gpu/drm/nouveau/nvkm/nvfw/hs.c
index 04ed77c..a7e0583 100644
--- a/drivers/gpu/drm/nouveau/nvkm/nvfw/hs.c
+++ b/drivers/gpu/drm/nouveau/nvkm/nvfw/hs.c
@@ -38,6 +38,24 @@ nvfw_hs_header(struct nvkm_subdev *subdev, const void *data)
return hdr;
}
+const struct nvfw_hs_header_v2 *
+nvfw_hs_header_v2(struct nvkm_subdev *subdev, const void *data)
+{
+ const struct nvfw_hs_header_v2 *hdr = data;
+
+ nvkm_debug(subdev, "hsHeader:\n");
+ nvkm_debug(subdev, "\tsigProdOffset : 0x%x\n", hdr->sig_prod_offset);
+ nvkm_debug(subdev, "\tsigProdSize : 0x%x\n", hdr->sig_prod_size);
+ nvkm_debug(subdev, "\tpatchLoc : 0x%x\n", hdr->patch_loc);
+ nvkm_debug(subdev, "\tpatchSig : 0x%x\n", hdr->patch_sig);
+ nvkm_debug(subdev, "\tmetadataOffset : 0x%x\n", hdr->meta_data_offset);
+ nvkm_debug(subdev, "\tmetadataSize : 0x%x\n", hdr->meta_data_size);
+ nvkm_debug(subdev, "\tnumSig : 0x%x\n", hdr->num_sig);
+ nvkm_debug(subdev, "\theaderOffset : 0x%x\n", hdr->header_offset);
+ nvkm_debug(subdev, "\theaderSize : 0x%x\n", hdr->header_size);
+ return hdr;
+}
+
const struct nvfw_hs_load_header *
nvfw_hs_load_header(struct nvkm_subdev *subdev, const void *data)
{
@@ -60,3 +78,24 @@ nvfw_hs_load_header(struct nvkm_subdev *subdev, const void *data)
return hdr;
}
+
+const struct nvfw_hs_load_header_v2 *
+nvfw_hs_load_header_v2(struct nvkm_subdev *subdev, const void *data)
+{
+ const struct nvfw_hs_load_header_v2 *hdr = data;
+ int i;
+
+ nvkm_debug(subdev, "hsLoadHeader:\n");
+ nvkm_debug(subdev, "\tosCodeOffset : 0x%x\n", hdr->os_code_offset);
+ nvkm_debug(subdev, "\tosCodeSize : 0x%x\n", hdr->os_code_size);
+ nvkm_debug(subdev, "\tosDataOffset : 0x%x\n", hdr->os_data_offset);
+ nvkm_debug(subdev, "\tosDataSize : 0x%x\n", hdr->os_data_size);
+ nvkm_debug(subdev, "\tnumApps : 0x%x\n", hdr->num_apps);
+ for (i = 0; i < hdr->num_apps; i++) {
+ nvkm_debug(subdev,
+ "\tApp[%d] : offset 0x%x size 0x%x\n", i,
+ hdr->app[i].offset, hdr->app[i].size);
+ }
+
+ return hdr;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/nvfw/ls.c b/drivers/gpu/drm/nouveau/nvkm/nvfw/ls.c
index b847f28..45c3a6c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/nvfw/ls.c
+++ b/drivers/gpu/drm/nouveau/nvkm/nvfw/ls.c
@@ -106,3 +106,75 @@ nvfw_ls_desc_v1(struct nvkm_subdev *subdev, const void *data)
return hdr;
}
+
+const struct nvfw_ls_desc_v2 *
+nvfw_ls_desc_v2(struct nvkm_subdev *subdev, const void *data)
+{
+ const struct nvfw_ls_desc_v2 *hdr = data;
+ char *date;
+ int i;
+
+ nvkm_debug(subdev, "lsUcodeImgDesc:\n");
+ nvkm_debug(subdev, "\tdescriptorSize : %d\n", hdr->descriptor_size);
+ nvkm_debug(subdev, "\timageSize : %d\n", hdr->image_size);
+ nvkm_debug(subdev, "\ttoolsVersion : 0x%x\n", hdr->tools_version);
+ nvkm_debug(subdev, "\tappVersion : 0x%x\n", hdr->app_version);
+
+ date = kstrndup(hdr->date, sizeof(hdr->date), GFP_KERNEL);
+ nvkm_debug(subdev, "\tdate : %s\n", date);
+ kfree(date);
+
+ nvkm_debug(subdev, "\tsecureBootloader : 0x%x\n", hdr->secure_bootloader);
+ nvkm_debug(subdev, "\tbootloaderStartOffset: 0x%x\n", hdr->bootloader_start_offset);
+ nvkm_debug(subdev, "\tbootloaderSize : 0x%x\n", hdr->bootloader_size);
+ nvkm_debug(subdev, "\tbootloaderImemOffset : 0x%x\n", hdr->bootloader_imem_offset);
+ nvkm_debug(subdev, "\tbootloaderEntryPoint : 0x%x\n", hdr->bootloader_entry_point);
+
+ nvkm_debug(subdev, "\tappStartOffset : 0x%x\n", hdr->app_start_offset);
+ nvkm_debug(subdev, "\tappSize : 0x%x\n", hdr->app_size);
+ nvkm_debug(subdev, "\tappImemOffset : 0x%x\n", hdr->app_imem_offset);
+ nvkm_debug(subdev, "\tappImemEntry : 0x%x\n", hdr->app_imem_entry);
+ nvkm_debug(subdev, "\tappDmemOffset : 0x%x\n", hdr->app_dmem_offset);
+ nvkm_debug(subdev, "\tappResidentCodeOffset: 0x%x\n", hdr->app_resident_code_offset);
+ nvkm_debug(subdev, "\tappResidentCodeSize : 0x%x\n", hdr->app_resident_code_size);
+ nvkm_debug(subdev, "\tappResidentDataOffset: 0x%x\n", hdr->app_resident_data_offset);
+ nvkm_debug(subdev, "\tappResidentDataSize : 0x%x\n", hdr->app_resident_data_size);
+
+ nvkm_debug(subdev, "\tnbImemOverlays : %d\n", hdr->nb_imem_overlays);
+ nvkm_debug(subdev, "\tnbDmemOverlays : %d\n", hdr->nb_dmem_overlays);
+ for (i = 0; i < ARRAY_SIZE(hdr->load_ovl); i++) {
+ nvkm_debug(subdev, "\tloadOvl[%d] : 0x%x %d\n", i,
+ hdr->load_ovl[i].start, hdr->load_ovl[i].size);
+ }
+
+ return hdr;
+}
+
+const struct nvfw_ls_hsbl_bin_hdr *
+nvfw_ls_hsbl_bin_hdr(struct nvkm_subdev *subdev, const void *data)
+{
+ const struct nvfw_ls_hsbl_bin_hdr *hdr = data;
+
+ nvkm_debug(subdev, "lsHsblBinHdr:\n");
+ nvkm_debug(subdev, "\tbinMagic : 0x%08x\n", hdr->bin_magic);
+ nvkm_debug(subdev, "\tbinVer : %d\n", hdr->bin_ver);
+ nvkm_debug(subdev, "\tbinSize : %d\n", hdr->bin_size);
+ nvkm_debug(subdev, "\theaderOffset : 0x%x\n", hdr->header_offset);
+ return hdr;
+}
+
+const struct nvfw_ls_hsbl_hdr *
+nvfw_ls_hsbl_hdr(struct nvkm_subdev *subdev, const void *data)
+{
+ const struct nvfw_ls_hsbl_hdr *hdr = data;
+
+ nvkm_debug(subdev, "lsHsblHdr:\n");
+ nvkm_debug(subdev, "\tsigProdOffset : 0x%x\n", hdr->sig_prod_offset);
+ nvkm_debug(subdev, "\tsigProdSize : 0x%x\n", hdr->sig_prod_size);
+ nvkm_debug(subdev, "\tpatchLoc : 0x%x\n", hdr->patch_loc);
+ nvkm_debug(subdev, "\tpatchSig : 0x%x\n", hdr->patch_sig);
+ nvkm_debug(subdev, "\tmetadataOffset : 0x%x\n", hdr->meta_data_offset);
+ nvkm_debug(subdev, "\tmetadataSize : 0x%x\n", hdr->meta_data_size);
+ nvkm_debug(subdev, "\tnumSig : 0x%x\n", hdr->num_sig);
+ return hdr;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild
index 2cb24ff..4c2f6fc 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild
@@ -23,4 +23,5 @@
include $(src)/nvkm/subdev/therm/Kbuild
include $(src)/nvkm/subdev/timer/Kbuild
include $(src)/nvkm/subdev/top/Kbuild
+include $(src)/nvkm/subdev/vfn/Kbuild
include $(src)/nvkm/subdev/volt/Kbuild
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/Kbuild
index 5b9f64a..5731f35 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/Kbuild
@@ -1,10 +1,12 @@
# SPDX-License-Identifier: MIT
nvkm-y += nvkm/subdev/acr/base.o
-nvkm-y += nvkm/subdev/acr/hsfw.o
nvkm-y += nvkm/subdev/acr/lsfw.o
nvkm-y += nvkm/subdev/acr/gm200.o
nvkm-y += nvkm/subdev/acr/gm20b.o
nvkm-y += nvkm/subdev/acr/gp102.o
nvkm-y += nvkm/subdev/acr/gp108.o
+nvkm-y += nvkm/subdev/acr/gv100.o
nvkm-y += nvkm/subdev/acr/gp10b.o
nvkm-y += nvkm/subdev/acr/tu102.o
+nvkm-y += nvkm/subdev/acr/ga100.o
+nvkm-y += nvkm/subdev/acr/ga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c
index af6cac6..795f3a64 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c
@@ -24,50 +24,63 @@
#include <core/firmware.h>
#include <core/memory.h>
#include <subdev/mmu.h>
+#include <subdev/gsp.h>
+#include <subdev/pmu.h>
+#include <engine/sec2.h>
+#include <engine/nvdec.h>
-static struct nvkm_acr_hsf *
-nvkm_acr_hsf_find(struct nvkm_acr *acr, const char *name)
+static struct nvkm_acr_hsfw *
+nvkm_acr_hsfw_find(struct nvkm_acr *acr, const char *name)
{
- struct nvkm_acr_hsf *hsf;
- list_for_each_entry(hsf, &acr->hsf, head) {
- if (!strcmp(hsf->name, name))
- return hsf;
+ struct nvkm_acr_hsfw *hsfw;
+
+ list_for_each_entry(hsfw, &acr->hsfw, head) {
+ if (!strcmp(hsfw->fw.fw.name, name))
+ return hsfw;
}
+
return NULL;
}
int
-nvkm_acr_hsf_boot(struct nvkm_acr *acr, const char *name)
+nvkm_acr_hsfw_boot(struct nvkm_acr *acr, const char *name)
{
struct nvkm_subdev *subdev = &acr->subdev;
- struct nvkm_acr_hsf *hsf;
- int ret;
+ struct nvkm_acr_hsfw *hsfw;
- hsf = nvkm_acr_hsf_find(acr, name);
- if (!hsf)
+ hsfw = nvkm_acr_hsfw_find(acr, name);
+ if (!hsfw)
return -EINVAL;
- nvkm_debug(subdev, "executing %s binary\n", hsf->name);
- ret = nvkm_falcon_get(hsf->falcon, subdev);
- if (ret)
- return ret;
+ return nvkm_falcon_fw_boot(&hsfw->fw, subdev, true, NULL, NULL,
+ hsfw->boot_mbox0, hsfw->intr_clear);
+}
- ret = hsf->func->boot(acr, hsf);
- nvkm_falcon_put(hsf->falcon, subdev);
- if (ret) {
- nvkm_error(subdev, "%s binary failed\n", hsf->name);
- return ret;
+static struct nvkm_acr_lsf *
+nvkm_acr_rtos(struct nvkm_acr *acr)
+{
+ struct nvkm_acr_lsf *lsf;
+
+ if (acr) {
+ list_for_each_entry(lsf, &acr->lsf, head) {
+ if (lsf->func->bootstrap_falcon)
+ return lsf;
+ }
}
- nvkm_debug(subdev, "%s binary completed successfully\n", hsf->name);
- return 0;
+ return NULL;
}
static void
nvkm_acr_unload(struct nvkm_acr *acr)
{
if (acr->done) {
- nvkm_acr_hsf_boot(acr, "unload");
+ if (acr->rtos) {
+ nvkm_subdev_unref(acr->rtos->falcon->owner);
+ acr->rtos = NULL;
+ }
+
+ nvkm_acr_hsfw_boot(acr, "unload");
acr->done = false;
}
}
@@ -76,7 +89,7 @@ static int
nvkm_acr_load(struct nvkm_acr *acr)
{
struct nvkm_subdev *subdev = &acr->subdev;
- struct nvkm_acr_lsf *lsf;
+ struct nvkm_acr_lsf *rtos = nvkm_acr_rtos(acr);
u64 start, limit;
int ret;
@@ -100,12 +113,12 @@ nvkm_acr_load(struct nvkm_acr *acr)
acr->done = true;
- list_for_each_entry(lsf, &acr->lsf, head) {
- if (lsf->func->boot) {
- ret = lsf->func->boot(lsf->falcon);
- if (ret)
- break;
- }
+ if (rtos) {
+ ret = nvkm_subdev_ref(rtos->falcon->owner);
+ if (ret)
+ return ret;
+
+ acr->rtos = rtos;
}
return ret;
@@ -118,33 +131,17 @@ nvkm_acr_reload(struct nvkm_acr *acr)
return nvkm_acr_load(acr);
}
-static struct nvkm_acr_lsf *
-nvkm_acr_falcon(struct nvkm_device *device)
-{
- struct nvkm_acr *acr = device->acr;
- struct nvkm_acr_lsf *lsf;
-
- if (acr) {
- list_for_each_entry(lsf, &acr->lsf, head) {
- if (lsf->func->bootstrap_falcon)
- return lsf;
- }
- }
-
- return NULL;
-}
-
int
nvkm_acr_bootstrap_falcons(struct nvkm_device *device, unsigned long mask)
{
- struct nvkm_acr_lsf *acrflcn = nvkm_acr_falcon(device);
struct nvkm_acr *acr = device->acr;
+ struct nvkm_acr_lsf *rtos = nvkm_acr_rtos(acr);
unsigned long id;
/* If there's no LS FW managing bootstrapping of other LS falcons,
* we depend on the HS firmware being able to do it instead.
*/
- if (!acrflcn) {
+ if (!rtos) {
/* Which isn't possible everywhere... */
if ((mask & acr->func->bootstrap_falcons) == mask) {
int ret = nvkm_acr_reload(acr);
@@ -156,16 +153,14 @@ nvkm_acr_bootstrap_falcons(struct nvkm_device *device, unsigned long mask)
return -ENOSYS;
}
- if ((mask & acrflcn->func->bootstrap_falcons) != mask)
+ if ((mask & rtos->func->bootstrap_falcons) != mask)
return -ENOSYS;
- if (acrflcn->func->bootstrap_multiple_falcons) {
- return acrflcn->func->
- bootstrap_multiple_falcons(acrflcn->falcon, mask);
- }
+ if (rtos->func->bootstrap_multiple_falcons)
+ return rtos->func->bootstrap_multiple_falcons(rtos->falcon, mask);
for_each_set_bit(id, &mask, NVKM_ACR_LSF_NUM) {
- int ret = acrflcn->func->bootstrap_falcon(acrflcn->falcon, id);
+ int ret = rtos->func->bootstrap_falcon(rtos->falcon, id);
if (ret)
return ret;
}
@@ -189,6 +184,9 @@ nvkm_acr_managed_falcon(struct nvkm_device *device, enum nvkm_acr_lsf_id id)
static int
nvkm_acr_fini(struct nvkm_subdev *subdev, bool suspend)
{
+ if (!subdev->use.enabled)
+ return 0;
+
nvkm_acr_unload(nvkm_acr(subdev));
return 0;
}
@@ -196,17 +194,19 @@ nvkm_acr_fini(struct nvkm_subdev *subdev, bool suspend)
static int
nvkm_acr_init(struct nvkm_subdev *subdev)
{
- if (!nvkm_acr_falcon(subdev->device))
+ struct nvkm_acr *acr = nvkm_acr(subdev);
+
+ if (!nvkm_acr_rtos(acr))
return 0;
- return nvkm_acr_load(nvkm_acr(subdev));
+ return nvkm_acr_load(acr);
}
static void
nvkm_acr_cleanup(struct nvkm_acr *acr)
{
nvkm_acr_lsfw_del_all(acr);
- nvkm_acr_hsfw_del_all(acr);
+
nvkm_firmware_put(acr->wpr_fw);
acr->wpr_fw = NULL;
}
@@ -218,7 +218,8 @@ nvkm_acr_oneinit(struct nvkm_subdev *subdev)
struct nvkm_acr *acr = nvkm_acr(subdev);
struct nvkm_acr_hsfw *hsfw;
struct nvkm_acr_lsfw *lsfw, *lsft;
- struct nvkm_acr_lsf *lsf;
+ struct nvkm_acr_lsf *lsf, *rtos;
+ struct nvkm_falcon *falcon;
u32 wpr_size = 0;
u64 falcons;
int ret, i;
@@ -260,10 +261,10 @@ nvkm_acr_oneinit(struct nvkm_subdev *subdev)
}
/* Ensure the falcon that'll provide ACR functions is booted first. */
- lsf = nvkm_acr_falcon(device);
- if (lsf) {
- falcons = lsf->func->bootstrap_falcons;
- list_move(&lsf->head, &acr->lsf);
+ rtos = nvkm_acr_rtos(acr);
+ if (rtos) {
+ falcons = rtos->func->bootstrap_falcons;
+ list_move(&rtos->head, &acr->lsf);
} else {
falcons = acr->func->bootstrap_falcons;
}
@@ -301,7 +302,7 @@ nvkm_acr_oneinit(struct nvkm_subdev *subdev)
nvkm_wobj(acr->wpr, 0, acr->wpr_fw->data, acr->wpr_fw->size);
if (!acr->wpr_fw || acr->wpr_comp)
- acr->func->wpr_build(acr, nvkm_acr_falcon(device));
+ acr->func->wpr_build(acr, rtos);
acr->func->wpr_patch(acr, (s64)acr->wpr_start - acr->wpr_prev);
if (acr->wpr_fw && acr->wpr_comp) {
@@ -336,8 +337,16 @@ nvkm_acr_oneinit(struct nvkm_subdev *subdev)
/* Load HS firmware blobs into ACR VMM. */
list_for_each_entry(hsfw, &acr->hsfw, head) {
- nvkm_debug(subdev, "loading %s fw\n", hsfw->name);
- ret = hsfw->func->load(acr, hsfw);
+ switch (hsfw->falcon_id) {
+ case NVKM_ACR_HSF_PMU : falcon = &device->pmu->falcon; break;
+ case NVKM_ACR_HSF_SEC2: falcon = &device->sec2->falcon; break;
+ case NVKM_ACR_HSF_GSP : falcon = &device->gsp->falcon; break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ ret = nvkm_falcon_fw_oneinit(&hsfw->fw, falcon, acr->vmm, acr->inst);
if (ret)
return ret;
}
@@ -351,15 +360,13 @@ static void *
nvkm_acr_dtor(struct nvkm_subdev *subdev)
{
struct nvkm_acr *acr = nvkm_acr(subdev);
- struct nvkm_acr_hsf *hsf, *hst;
+ struct nvkm_acr_hsfw *hsfw, *hsft;
struct nvkm_acr_lsf *lsf, *lst;
- list_for_each_entry_safe(hsf, hst, &acr->hsf, head) {
- nvkm_vmm_put(acr->vmm, &hsf->vma);
- nvkm_memory_unref(&hsf->ucode);
- kfree(hsf->imem);
- list_del(&hsf->head);
- kfree(hsf);
+ list_for_each_entry_safe(hsfw, hsft, &acr->hsfw, head) {
+ nvkm_falcon_fw_dtor(&hsfw->fw);
+ list_del(&hsfw->head);
+ kfree(hsfw);
}
nvkm_vmm_part(acr->vmm, acr->inst);
@@ -420,7 +427,6 @@ nvkm_acr_new_(const struct nvkm_acr_fwif *fwif, struct nvkm_device *device,
nvkm_subdev_ctor(&nvkm_acr, device, type, inst, &acr->subdev);
INIT_LIST_HEAD(&acr->hsfw);
INIT_LIST_HEAD(&acr->lsfw);
- INIT_LIST_HEAD(&acr->hsf);
INIT_LIST_HEAD(&acr->lsf);
fwif = nvkm_firmware_load(&acr->subdev, fwif, "Acr", acr);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga100.c
new file mode 100644
index 0000000..e3370c1
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga100.c
@@ -0,0 +1,49 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+void
+ga100_acr_wpr_check(struct nvkm_acr *acr, u64 *start, u64 *limit)
+{
+ struct nvkm_device *device = acr->subdev.device;
+
+ *start = (u64)(nvkm_rd32(device, 0x1fa81c) & 0xffffff00) << 8;
+ *limit = (u64)(nvkm_rd32(device, 0x1fa820) & 0xffffff00) << 8;
+ *limit = *limit + 0x20000;
+}
+
+int
+ga100_acr_hsfw_ctor(struct nvkm_acr *acr, const char *bl, const char *fw,
+ const char *name, int ver, const struct nvkm_acr_hsf_fwif *fwif)
+{
+ struct nvkm_acr_hsfw *hsfw;
+
+ if (!(hsfw = kzalloc(sizeof(*hsfw), GFP_KERNEL)))
+ return -ENOMEM;
+
+ hsfw->falcon_id = fwif->falcon_id;
+ hsfw->boot_mbox0 = fwif->boot_mbox0;
+ hsfw->intr_clear = fwif->intr_clear;
+ list_add_tail(&hsfw->head, &acr->hsfw);
+
+ return nvkm_falcon_fw_ctor_hs_v2(fwif->func, name, &acr->subdev, fw, ver, NULL, &hsfw->fw);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga102.c
new file mode 100644
index 0000000..45dcf49
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga102.c
@@ -0,0 +1,326 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <nvfw/acr.h>
+
+static int
+ga102_acr_wpr_patch(struct nvkm_acr *acr, s64 adjust)
+{
+ struct wpr_header_v2 hdr;
+ struct lsb_header_v2 *lsb;
+ struct nvkm_acr_lsfw *lsfw;
+ u32 offset = 0;
+
+ lsb = kvmalloc(sizeof(*lsb), GFP_KERNEL);
+ if (!lsb)
+ return -ENOMEM;
+
+ do {
+ nvkm_robj(acr->wpr, offset, &hdr, sizeof(hdr));
+ wpr_header_v2_dump(&acr->subdev, &hdr);
+
+ list_for_each_entry(lsfw, &acr->lsfw, head) {
+ if (lsfw->id != hdr.wpr.falcon_id)
+ continue;
+
+ nvkm_robj(acr->wpr, hdr.wpr.lsb_offset, lsb, sizeof(*lsb));
+ lsb_header_v2_dump(&acr->subdev, lsb);
+
+ lsfw->func->bld_patch(acr, lsb->bl_data_off, adjust);
+ break;
+ }
+
+ offset += sizeof(hdr);
+ } while (hdr.wpr.falcon_id != WPR_HEADER_V1_FALCON_ID_INVALID);
+
+ kvfree(lsb);
+ return 0;
+}
+
+static int
+ga102_acr_wpr_build_lsb(struct nvkm_acr *acr, struct nvkm_acr_lsfw *lsfw)
+{
+ struct lsb_header_v2 *hdr;
+ int ret = 0;
+
+ if (WARN_ON(lsfw->sig->size != sizeof(hdr->signature)))
+ return -EINVAL;
+
+ hdr = kvzalloc(sizeof(*hdr), GFP_KERNEL);
+ if (!hdr)
+ return -ENOMEM;
+
+ hdr->hdr.identifier = WPR_GENERIC_HEADER_ID_LSF_LSB_HEADER;
+ hdr->hdr.version = 2;
+ hdr->hdr.size = sizeof(*hdr);
+
+ memcpy(&hdr->signature, lsfw->sig->data, lsfw->sig->size);
+ hdr->ucode_off = lsfw->offset.img;
+ hdr->ucode_size = lsfw->ucode_size;
+ hdr->data_size = lsfw->data_size;
+ hdr->bl_code_size = lsfw->bootloader_size;
+ hdr->bl_imem_off = lsfw->bootloader_imem_offset;
+ hdr->bl_data_off = lsfw->offset.bld;
+ hdr->bl_data_size = lsfw->bl_data_size;
+ hdr->app_code_off = lsfw->app_start_offset + lsfw->app_resident_code_offset;
+ hdr->app_code_size = ALIGN(lsfw->app_resident_code_size, 0x100);
+ hdr->app_data_off = lsfw->app_start_offset + lsfw->app_resident_data_offset;
+ hdr->app_data_size = ALIGN(lsfw->app_resident_data_size, 0x100);
+ hdr->app_imem_offset = lsfw->app_imem_offset;
+ hdr->app_dmem_offset = lsfw->app_dmem_offset;
+ hdr->flags = lsfw->func->flags;
+ hdr->monitor_code_offset = 0;
+ hdr->monitor_data_offset = 0;
+ hdr->manifest_offset = 0;
+
+ if (lsfw->secure_bootloader) {
+ struct nvkm_falcon_fw fw = {
+ .fw.img = hdr->hs_fmc_params.pkc_signature,
+ .fw.name = "LSFW",
+ .func = &(const struct nvkm_falcon_fw_func) {
+ .signature = ga100_flcn_fw_signature,
+ },
+ .sig_size = lsfw->sig_size,
+ .sig_nr = lsfw->sig_nr,
+ .sigs = lsfw->sigs,
+ .fuse_ver = lsfw->fuse_ver,
+ .engine_id = lsfw->engine_id,
+ .ucode_id = lsfw->ucode_id,
+ .falcon = lsfw->falcon,
+
+ };
+
+ ret = nvkm_falcon_get(fw.falcon, &acr->subdev);
+ if (ret == 0) {
+ hdr->hs_fmc_params.hs_fmc = 1;
+ hdr->hs_fmc_params.pkc_algo = 0;
+ hdr->hs_fmc_params.pkc_algo_version = 1;
+ hdr->hs_fmc_params.engid_mask = lsfw->engine_id;
+ hdr->hs_fmc_params.ucode_id = lsfw->ucode_id;
+ hdr->hs_fmc_params.fuse_ver = lsfw->fuse_ver;
+ ret = nvkm_falcon_fw_patch(&fw);
+ nvkm_falcon_put(fw.falcon, &acr->subdev);
+ }
+ }
+
+ nvkm_wobj(acr->wpr, lsfw->offset.lsb, hdr, sizeof(*hdr));
+ kvfree(hdr);
+ return ret;
+}
+
+static int
+ga102_acr_wpr_build(struct nvkm_acr *acr, struct nvkm_acr_lsf *rtos)
+{
+ struct nvkm_acr_lsfw *lsfw;
+ struct wpr_header_v2 hdr;
+ u32 offset = 0;
+ int ret;
+
+ /*XXX: shared sub-WPR headers, fill terminator for now. */
+ nvkm_wo32(acr->wpr, 0x300, (2 << 16) | WPR_GENERIC_HEADER_ID_LSF_SHARED_SUB_WPR);
+ nvkm_wo32(acr->wpr, 0x304, 0x14);
+ nvkm_wo32(acr->wpr, 0x308, 0xffffffff);
+ nvkm_wo32(acr->wpr, 0x30c, 0);
+ nvkm_wo32(acr->wpr, 0x310, 0);
+
+ /* Fill per-LSF structures. */
+ list_for_each_entry(lsfw, &acr->lsfw, head) {
+ struct lsf_signature_v2 *sig = (void *)lsfw->sig->data;
+
+ hdr.hdr.identifier = WPR_GENERIC_HEADER_ID_LSF_WPR_HEADER;
+ hdr.hdr.version = 2;
+ hdr.hdr.size = sizeof(hdr);
+ hdr.wpr.falcon_id = lsfw->id;
+ hdr.wpr.lsb_offset = lsfw->offset.lsb;
+ hdr.wpr.bootstrap_owner = NVKM_ACR_LSF_GSPLITE;
+ hdr.wpr.lazy_bootstrap = 1;
+ hdr.wpr.bin_version = sig->ls_ucode_version;
+ hdr.wpr.status = WPR_HEADER_V1_STATUS_COPY;
+
+ /* Write WPR header. */
+ nvkm_wobj(acr->wpr, offset, &hdr, sizeof(hdr));
+ offset += sizeof(hdr);
+
+ /* Write LSB header. */
+ ret = ga102_acr_wpr_build_lsb(acr, lsfw);
+ if (ret)
+ return ret;
+
+ /* Write ucode image. */
+ nvkm_wobj(acr->wpr, lsfw->offset.img,
+ lsfw->img.data,
+ lsfw->img.size);
+
+ /* Write bootloader data. */
+ lsfw->func->bld_write(acr, lsfw->offset.bld, lsfw);
+ }
+
+ /* Finalise WPR. */
+ hdr.hdr.identifier = WPR_GENERIC_HEADER_ID_LSF_WPR_HEADER;
+ hdr.hdr.version = 2;
+ hdr.hdr.size = sizeof(hdr);
+ hdr.wpr.falcon_id = WPR_HEADER_V1_FALCON_ID_INVALID;
+ nvkm_wobj(acr->wpr, offset, &hdr, sizeof(hdr));
+ return 0;
+}
+
+static u32
+ga102_acr_wpr_layout(struct nvkm_acr *acr)
+{
+ struct nvkm_acr_lsfw *lsfw;
+ u32 wpr = 0;
+
+ wpr += 21 /* MAX_LSF */ * sizeof(struct wpr_header_v2);
+ wpr = ALIGN(wpr, 256);
+
+ wpr += 0x100; /* Shared sub-WPR headers. */
+
+ list_for_each_entry(lsfw, &acr->lsfw, head) {
+ wpr = ALIGN(wpr, 256);
+ lsfw->offset.lsb = wpr;
+ wpr += sizeof(struct lsb_header_v2);
+
+ wpr = ALIGN(wpr, 4096);
+ lsfw->offset.img = wpr;
+ wpr += lsfw->img.size;
+
+ wpr = ALIGN(wpr, 256);
+ lsfw->offset.bld = wpr;
+ lsfw->bl_data_size = ALIGN(lsfw->func->bld_size, 256);
+ wpr += lsfw->bl_data_size;
+ }
+
+ return wpr;
+}
+
+static int
+ga102_acr_wpr_parse(struct nvkm_acr *acr)
+{
+ const struct wpr_header_v2 *hdr = (void *)acr->wpr_fw->data;
+
+ while (hdr->wpr.falcon_id != WPR_HEADER_V1_FALCON_ID_INVALID) {
+ wpr_header_v2_dump(&acr->subdev, hdr);
+ if (!nvkm_acr_lsfw_add(NULL, acr, NULL, (hdr++)->wpr.falcon_id))
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+MODULE_FIRMWARE("nvidia/ga102/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/ga103/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/ga104/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/ga106/acr/ucode_unload.bin");
+MODULE_FIRMWARE("nvidia/ga107/acr/ucode_unload.bin");
+
+static const struct nvkm_acr_hsf_fwif
+ga102_acr_unload_fwif[] = {
+ { 0, ga100_acr_hsfw_ctor, &ga102_flcn_fw, NVKM_ACR_HSF_SEC2 },
+ {}
+};
+
+MODULE_FIRMWARE("nvidia/ga102/acr/ucode_asb.bin");
+MODULE_FIRMWARE("nvidia/ga103/acr/ucode_asb.bin");
+MODULE_FIRMWARE("nvidia/ga104/acr/ucode_asb.bin");
+MODULE_FIRMWARE("nvidia/ga106/acr/ucode_asb.bin");
+MODULE_FIRMWARE("nvidia/ga107/acr/ucode_asb.bin");
+
+static const struct nvkm_acr_hsf_fwif
+ga102_acr_asb_fwif[] = {
+ { 0, ga100_acr_hsfw_ctor, &ga102_flcn_fw, NVKM_ACR_HSF_GSP },
+ {}
+};
+
+static const struct nvkm_falcon_fw_func
+ga102_acr_ahesasc_0 = {
+ .signature = ga100_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .setup = gp102_acr_load_setup,
+ .load = ga102_flcn_fw_load,
+ .boot = ga102_flcn_fw_boot,
+};
+
+MODULE_FIRMWARE("nvidia/ga102/acr/ucode_ahesasc.bin");
+MODULE_FIRMWARE("nvidia/ga103/acr/ucode_ahesasc.bin");
+MODULE_FIRMWARE("nvidia/ga104/acr/ucode_ahesasc.bin");
+MODULE_FIRMWARE("nvidia/ga106/acr/ucode_ahesasc.bin");
+MODULE_FIRMWARE("nvidia/ga107/acr/ucode_ahesasc.bin");
+
+static const struct nvkm_acr_hsf_fwif
+ga102_acr_ahesasc_fwif[] = {
+ { 0, ga100_acr_hsfw_ctor, &ga102_acr_ahesasc_0, NVKM_ACR_HSF_SEC2 },
+ {}
+};
+
+static const struct nvkm_acr_func
+ga102_acr = {
+ .ahesasc = ga102_acr_ahesasc_fwif,
+ .asb = ga102_acr_asb_fwif,
+ .unload = ga102_acr_unload_fwif,
+ .wpr_parse = ga102_acr_wpr_parse,
+ .wpr_layout = ga102_acr_wpr_layout,
+ .wpr_alloc = gp102_acr_wpr_alloc,
+ .wpr_patch = ga102_acr_wpr_patch,
+ .wpr_build = ga102_acr_wpr_build,
+ .wpr_check = ga100_acr_wpr_check,
+ .init = tu102_acr_init,
+};
+
+static int
+ga102_acr_load(struct nvkm_acr *acr, int version,
+ const struct nvkm_acr_fwif *fwif)
+{
+ struct nvkm_subdev *subdev = &acr->subdev;
+ const struct nvkm_acr_hsf_fwif *hsfwif;
+
+ hsfwif = nvkm_firmware_load(subdev, fwif->func->ahesasc, "AcrAHESASC",
+ acr, NULL, "acr/ucode_ahesasc", "AHESASC");
+ if (IS_ERR(hsfwif))
+ return PTR_ERR(hsfwif);
+
+ hsfwif = nvkm_firmware_load(subdev, fwif->func->asb, "AcrASB",
+ acr, NULL, "acr/ucode_asb", "ASB");
+ if (IS_ERR(hsfwif))
+ return PTR_ERR(hsfwif);
+
+ hsfwif = nvkm_firmware_load(subdev, fwif->func->unload, "AcrUnload",
+ acr, NULL, "acr/ucode_unload", "unload");
+ if (IS_ERR(hsfwif))
+ return PTR_ERR(hsfwif);
+
+ return 0;
+}
+
+static const struct nvkm_acr_fwif
+ga102_acr_fwif[] = {
+ { 0, ga102_acr_load, &ga102_acr },
+ { -1, gm200_acr_nofw, &gm200_acr },
+ {}
+};
+
+int
+ga102_acr_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_acr **pacr)
+{
+ return nvkm_acr_new_(ga102_acr_fwif, device, type, inst, pacr);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm200.c
index 82b4c8e..31079c9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm200.c
@@ -46,7 +46,7 @@ gm200_acr_nofw(struct nvkm_acr *acr, int ver, const struct nvkm_acr_fwif *fwif)
int
gm200_acr_init(struct nvkm_acr *acr)
{
- return nvkm_acr_hsf_boot(acr, "load");
+ return nvkm_acr_hsfw_boot(acr, "load");
}
void
@@ -61,7 +61,7 @@ gm200_acr_wpr_check(struct nvkm_acr *acr, u64 *start, u64 *limit)
*limit = *limit + 0x20000;
}
-void
+int
gm200_acr_wpr_patch(struct nvkm_acr *acr, s64 adjust)
{
struct nvkm_subdev *subdev = &acr->subdev;
@@ -86,6 +86,8 @@ gm200_acr_wpr_patch(struct nvkm_acr *acr, s64 adjust)
}
offset += sizeof(hdr);
} while (hdr.falcon_id != WPR_HEADER_V0_FALCON_ID_INVALID);
+
+ return 0;
}
void
@@ -219,162 +221,50 @@ gm200_acr_wpr_parse(struct nvkm_acr *acr)
return 0;
}
-void
-gm200_acr_hsfw_bld(struct nvkm_acr *acr, struct nvkm_acr_hsf *hsf)
+int
+gm200_acr_hsfw_load_bld(struct nvkm_falcon_fw *fw)
{
struct flcn_bl_dmem_desc_v1 hsdesc = {
.ctx_dma = FALCON_DMAIDX_VIRT,
- .code_dma_base = hsf->vma->addr,
- .non_sec_code_off = hsf->non_sec_addr,
- .non_sec_code_size = hsf->non_sec_size,
- .sec_code_off = hsf->sec_addr,
- .sec_code_size = hsf->sec_size,
+ .code_dma_base = fw->vma->addr,
+ .non_sec_code_off = fw->nmem_base,
+ .non_sec_code_size = fw->nmem_size,
+ .sec_code_off = fw->imem_base,
+ .sec_code_size = fw->imem_size,
.code_entry_point = 0,
- .data_dma_base = hsf->vma->addr + hsf->data_addr,
- .data_size = hsf->data_size,
+ .data_dma_base = fw->vma->addr + fw->dmem_base_img,
+ .data_size = fw->dmem_size,
};
- flcn_bl_dmem_desc_v1_dump(&acr->subdev, &hsdesc);
+ flcn_bl_dmem_desc_v1_dump(fw->falcon->user, &hsdesc);
- nvkm_falcon_load_dmem(hsf->falcon, &hsdesc, 0, sizeof(hsdesc), 0);
+ return nvkm_falcon_pio_wr(fw->falcon, (u8 *)&hsdesc, 0, 0, DMEM, 0, sizeof(hsdesc), 0, 0);
}
int
-gm200_acr_hsfw_boot(struct nvkm_acr *acr, struct nvkm_acr_hsf *hsf,
- u32 intr_clear, u32 mbox0_ok)
+gm200_acr_hsfw_ctor(struct nvkm_acr *acr, const char *bl, const char *fw, const char *name, int ver,
+ const struct nvkm_acr_hsf_fwif *fwif)
{
- struct nvkm_subdev *subdev = &acr->subdev;
- struct nvkm_device *device = subdev->device;
- struct nvkm_falcon *falcon = hsf->falcon;
- u32 mbox0, mbox1;
- int ret;
+ struct nvkm_acr_hsfw *hsfw;
- /* Reset falcon. */
- nvkm_falcon_reset(falcon);
- nvkm_falcon_bind_context(falcon, acr->inst);
-
- /* Load bootloader into IMEM. */
- nvkm_falcon_load_imem(falcon, hsf->imem,
- falcon->code.limit - hsf->imem_size,
- hsf->imem_size,
- hsf->imem_tag,
- 0, false);
-
- /* Load bootloader data into DMEM. */
- hsf->func->bld(acr, hsf);
-
- /* Boot the falcon. */
- nvkm_mc_intr_mask(device, falcon->owner->type, falcon->owner->inst, false);
-
- nvkm_falcon_wr32(falcon, 0x040, 0xdeada5a5);
- nvkm_falcon_set_start_addr(falcon, hsf->imem_tag << 8);
- nvkm_falcon_start(falcon);
- ret = nvkm_falcon_wait_for_halt(falcon, 100);
- if (ret)
- return ret;
-
- /* Check for successful completion. */
- mbox0 = nvkm_falcon_rd32(falcon, 0x040);
- mbox1 = nvkm_falcon_rd32(falcon, 0x044);
- nvkm_debug(subdev, "mailbox %08x %08x\n", mbox0, mbox1);
- if (mbox0 && mbox0 != mbox0_ok)
- return -EIO;
-
- nvkm_falcon_clear_interrupt(falcon, intr_clear);
- nvkm_mc_intr_mask(device, falcon->owner->type, falcon->owner->inst, true);
- return ret;
-}
-
-int
-gm200_acr_hsfw_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw,
- struct nvkm_falcon *falcon)
-{
- struct nvkm_subdev *subdev = &acr->subdev;
- struct nvkm_acr_hsf *hsf;
- int ret;
-
- /* Patch the appropriate signature (production/debug) into the FW
- * image, as determined by the mode the falcon is in.
- */
- ret = nvkm_falcon_get(falcon, subdev);
- if (ret)
- return ret;
-
- if (hsfw->sig.patch_loc) {
- if (!falcon->debug) {
- nvkm_debug(subdev, "patching production signature\n");
- memcpy(hsfw->image + hsfw->sig.patch_loc,
- hsfw->sig.prod.data,
- hsfw->sig.prod.size);
- } else {
- nvkm_debug(subdev, "patching debug signature\n");
- memcpy(hsfw->image + hsfw->sig.patch_loc,
- hsfw->sig.dbg.data,
- hsfw->sig.dbg.size);
- }
- }
-
- nvkm_falcon_put(falcon, subdev);
-
- if (!(hsf = kzalloc(sizeof(*hsf), GFP_KERNEL)))
- return -ENOMEM;
- hsf->func = hsfw->func;
- hsf->name = hsfw->name;
- list_add_tail(&hsf->head, &acr->hsf);
-
- hsf->imem_size = hsfw->imem_size;
- hsf->imem_tag = hsfw->imem_tag;
- hsf->imem = kmemdup(hsfw->imem, hsfw->imem_size, GFP_KERNEL);
- if (!hsf->imem)
+ if (!(hsfw = kzalloc(sizeof(*hsfw), GFP_KERNEL)))
return -ENOMEM;
- hsf->non_sec_addr = hsfw->non_sec_addr;
- hsf->non_sec_size = hsfw->non_sec_size;
- hsf->sec_addr = hsfw->sec_addr;
- hsf->sec_size = hsfw->sec_size;
- hsf->data_addr = hsfw->data_addr;
- hsf->data_size = hsfw->data_size;
+ hsfw->falcon_id = fwif->falcon_id;
+ hsfw->boot_mbox0 = fwif->boot_mbox0;
+ hsfw->intr_clear = fwif->intr_clear;
+ list_add_tail(&hsfw->head, &acr->hsfw);
- /* Make the FW image accessible to the HS bootloader. */
- ret = nvkm_memory_new(subdev->device, NVKM_MEM_TARGET_INST,
- hsfw->image_size, 0x1000, false, &hsf->ucode);
- if (ret)
- return ret;
-
- nvkm_kmap(hsf->ucode);
- nvkm_wobj(hsf->ucode, 0, hsfw->image, hsfw->image_size);
- nvkm_done(hsf->ucode);
-
- ret = nvkm_vmm_get(acr->vmm, 12, nvkm_memory_size(hsf->ucode),
- &hsf->vma);
- if (ret)
- return ret;
-
- ret = nvkm_memory_map(hsf->ucode, 0, acr->vmm, hsf->vma, NULL, 0);
- if (ret)
- return ret;
-
- hsf->falcon = falcon;
- return 0;
+ return nvkm_falcon_fw_ctor_hs(fwif->func, name, &acr->subdev, bl, fw, ver, NULL, &hsfw->fw);
}
-int
-gm200_acr_unload_boot(struct nvkm_acr *acr, struct nvkm_acr_hsf *hsf)
-{
- return gm200_acr_hsfw_boot(acr, hsf, 0, 0x1d);
-}
-
-int
-gm200_acr_unload_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw)
-{
- return gm200_acr_hsfw_load(acr, hsfw, &acr->subdev.device->pmu->falcon);
-}
-
-const struct nvkm_acr_hsf_func
+const struct nvkm_falcon_fw_func
gm200_acr_unload_0 = {
- .load = gm200_acr_unload_load,
- .boot = gm200_acr_unload_boot,
- .bld = gm200_acr_hsfw_bld,
+ .signature = gm200_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .load = gm200_flcn_fw_load,
+ .load_bld = gm200_acr_hsfw_load_bld,
+ .boot = gm200_flcn_fw_boot,
};
MODULE_FIRMWARE("nvidia/gm200/acr/ucode_unload.bin");
@@ -384,20 +274,15 @@ MODULE_FIRMWARE("nvidia/gp100/acr/ucode_unload.bin");
static const struct nvkm_acr_hsf_fwif
gm200_acr_unload_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gm200_acr_unload_0 },
+ { 0, gm200_acr_hsfw_ctor, &gm200_acr_unload_0, NVKM_ACR_HSF_PMU, 0, 0x00000010 },
{}
};
-int
-gm200_acr_load_boot(struct nvkm_acr *acr, struct nvkm_acr_hsf *hsf)
-{
- return gm200_acr_hsfw_boot(acr, hsf, 0x10, 0);
-}
-
static int
-gm200_acr_load_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw)
+gm200_acr_load_setup(struct nvkm_falcon_fw *fw)
{
- struct flcn_acr_desc *desc = (void *)&hsfw->image[hsfw->data_addr];
+ struct flcn_acr_desc *desc = (void *)&fw->fw.img[fw->dmem_base_img];
+ struct nvkm_acr *acr = fw->falcon->owner->device->acr;
desc->wpr_region_id = 1;
desc->regions.no_regions = 2;
@@ -408,15 +293,17 @@ gm200_acr_load_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw)
desc->regions.region_props[0].write_mask = 0xc;
desc->regions.region_props[0].client_mask = 0x2;
flcn_acr_desc_dump(&acr->subdev, desc);
-
- return gm200_acr_hsfw_load(acr, hsfw, &acr->subdev.device->pmu->falcon);
+ return 0;
}
-static const struct nvkm_acr_hsf_func
+static const struct nvkm_falcon_fw_func
gm200_acr_load_0 = {
- .load = gm200_acr_load_load,
- .boot = gm200_acr_load_boot,
- .bld = gm200_acr_hsfw_bld,
+ .signature = gm200_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .setup = gm200_acr_load_setup,
+ .load = gm200_flcn_fw_load,
+ .load_bld = gm200_acr_hsfw_load_bld,
+ .boot = gm200_flcn_fw_boot,
};
MODULE_FIRMWARE("nvidia/gm200/acr/bl.bin");
@@ -433,7 +320,7 @@ MODULE_FIRMWARE("nvidia/gp100/acr/ucode_load.bin");
static const struct nvkm_acr_hsf_fwif
gm200_acr_load_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gm200_acr_load_0 },
+ { 0, gm200_acr_hsfw_ctor, &gm200_acr_load_0, NVKM_ACR_HSF_PMU, 0, 0x00000010 },
{}
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm20b.c
index 54e996f..ef5fb791 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm20b.c
@@ -45,43 +45,47 @@ gm20b_acr_wpr_alloc(struct nvkm_acr *acr, u32 wpr_size)
wpr_size, 0, true, &acr->wpr);
}
-static void
-gm20b_acr_load_bld(struct nvkm_acr *acr, struct nvkm_acr_hsf *hsf)
+static int
+gm20b_acr_hsfw_load_bld(struct nvkm_falcon_fw *fw)
{
struct flcn_bl_dmem_desc hsdesc = {
.ctx_dma = FALCON_DMAIDX_VIRT,
- .code_dma_base = hsf->vma->addr >> 8,
- .non_sec_code_off = hsf->non_sec_addr,
- .non_sec_code_size = hsf->non_sec_size,
- .sec_code_off = hsf->sec_addr,
- .sec_code_size = hsf->sec_size,
+ .code_dma_base = fw->vma->addr >> 8,
+ .non_sec_code_off = fw->nmem_base,
+ .non_sec_code_size = fw->nmem_size,
+ .sec_code_off = fw->imem_base,
+ .sec_code_size = fw->imem_size,
.code_entry_point = 0,
- .data_dma_base = (hsf->vma->addr + hsf->data_addr) >> 8,
- .data_size = hsf->data_size,
+ .data_dma_base = (fw->vma->addr + fw->dmem_base_img) >> 8,
+ .data_size = fw->dmem_size,
};
- flcn_bl_dmem_desc_dump(&acr->subdev, &hsdesc);
+ flcn_bl_dmem_desc_dump(fw->falcon->user, &hsdesc);
- nvkm_falcon_load_dmem(hsf->falcon, &hsdesc, 0, sizeof(hsdesc), 0);
+ return nvkm_falcon_pio_wr(fw->falcon, (u8 *)&hsdesc, 0, 0, DMEM, 0, sizeof(hsdesc), 0, 0);
}
+
static int
-gm20b_acr_load_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw)
+gm20b_acr_load_setup(struct nvkm_falcon_fw *fw)
{
- struct flcn_acr_desc *desc = (void *)&hsfw->image[hsfw->data_addr];
+ struct flcn_acr_desc *desc = (void *)&fw->fw.img[fw->dmem_base_img];
+ struct nvkm_acr *acr = fw->falcon->owner->device->acr;
desc->ucode_blob_base = nvkm_memory_addr(acr->wpr);
desc->ucode_blob_size = nvkm_memory_size(acr->wpr);
flcn_acr_desc_dump(&acr->subdev, desc);
-
- return gm200_acr_hsfw_load(acr, hsfw, &acr->subdev.device->pmu->falcon);
+ return 0;
}
-const struct nvkm_acr_hsf_func
+const struct nvkm_falcon_fw_func
gm20b_acr_load_0 = {
- .load = gm20b_acr_load_load,
- .boot = gm200_acr_load_boot,
- .bld = gm20b_acr_load_bld,
+ .signature = gm200_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .setup = gm20b_acr_load_setup,
+ .load = gm200_flcn_fw_load,
+ .load_bld = gm20b_acr_hsfw_load_bld,
+ .boot = gm200_flcn_fw_boot,
};
#if IS_ENABLED(CONFIG_ARCH_TEGRA_210_SOC)
@@ -91,7 +95,7 @@ MODULE_FIRMWARE("nvidia/gm20b/acr/ucode_load.bin");
static const struct nvkm_acr_hsf_fwif
gm20b_acr_load_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gm20b_acr_load_0 },
+ { 0, gm200_acr_hsfw_ctor, &gm20b_acr_load_0, NVKM_ACR_HSF_PMU, 0, 0x10 },
{}
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp102.c
index fd97a93..084f284 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp102.c
@@ -29,7 +29,7 @@
#include <nvfw/acr.h>
#include <nvfw/flcn.h>
-void
+int
gp102_acr_wpr_patch(struct nvkm_acr *acr, s64 adjust)
{
struct wpr_header_v1 hdr;
@@ -54,6 +54,8 @@ gp102_acr_wpr_patch(struct nvkm_acr *acr, s64 adjust)
offset += sizeof(hdr);
} while (hdr.falcon_id != WPR_HEADER_V1_FALCON_ID_INVALID);
+
+ return 0;
}
int
@@ -187,14 +189,15 @@ MODULE_FIRMWARE("nvidia/gp107/acr/ucode_unload.bin");
static const struct nvkm_acr_hsf_fwif
gp102_acr_unload_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gm200_acr_unload_0 },
+ { 0, gm200_acr_hsfw_ctor, &gm200_acr_unload_0, NVKM_ACR_HSF_PMU, 0x1d, 0x00000010 },
{}
};
int
-gp102_acr_load_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw)
+gp102_acr_load_setup(struct nvkm_falcon_fw *fw)
{
- struct flcn_acr_desc_v1 *desc = (void *)&hsfw->image[hsfw->data_addr];
+ struct flcn_acr_desc_v1 *desc = (void *)&fw->fw.img[fw->dmem_base_img];
+ struct nvkm_acr *acr = fw->falcon->owner->device->acr;
desc->wpr_region_id = 1;
desc->regions.no_regions = 2;
@@ -204,19 +207,19 @@ gp102_acr_load_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw)
desc->regions.region_props[0].read_mask = 0xf;
desc->regions.region_props[0].write_mask = 0xc;
desc->regions.region_props[0].client_mask = 0x2;
- desc->regions.region_props[0].shadow_mem_start_addr =
- acr->shadow_start >> 8;
+ desc->regions.region_props[0].shadow_mem_start_addr = acr->shadow_start >> 8;
flcn_acr_desc_v1_dump(&acr->subdev, desc);
-
- return gm200_acr_hsfw_load(acr, hsfw,
- &acr->subdev.device->sec2->falcon);
+ return 0;
}
-static const struct nvkm_acr_hsf_func
+static const struct nvkm_falcon_fw_func
gp102_acr_load_0 = {
- .load = gp102_acr_load_load,
- .boot = gm200_acr_load_boot,
- .bld = gm200_acr_hsfw_bld,
+ .signature = gm200_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .setup = gp102_acr_load_setup,
+ .load = gm200_flcn_fw_load,
+ .load_bld = gm200_acr_hsfw_load_bld,
+ .boot = gm200_flcn_fw_boot,
};
MODULE_FIRMWARE("nvidia/gp102/acr/bl.bin");
@@ -233,7 +236,7 @@ MODULE_FIRMWARE("nvidia/gp107/acr/ucode_load.bin");
static const struct nvkm_acr_hsf_fwif
gp102_acr_load_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gp102_acr_load_0 },
+ { 0, gm200_acr_hsfw_ctor, &gp102_acr_load_0, NVKM_ACR_HSF_SEC2, 0, 0x00000010 },
{}
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp108.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp108.c
index 373d638..6ab9d49 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp108.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp108.c
@@ -25,63 +25,62 @@
#include <nvfw/flcn.h>
-void
-gp108_acr_hsfw_bld(struct nvkm_acr *acr, struct nvkm_acr_hsf *hsf)
+int
+gp108_acr_hsfw_load_bld(struct nvkm_falcon_fw *fw)
{
struct flcn_bl_dmem_desc_v2 hsdesc = {
.ctx_dma = FALCON_DMAIDX_VIRT,
- .code_dma_base = hsf->vma->addr,
- .non_sec_code_off = hsf->non_sec_addr,
- .non_sec_code_size = hsf->non_sec_size,
- .sec_code_off = hsf->sec_addr,
- .sec_code_size = hsf->sec_size,
+ .code_dma_base = fw->vma->addr,
+ .non_sec_code_off = fw->nmem_base,
+ .non_sec_code_size = fw->nmem_size,
+ .sec_code_off = fw->imem_base,
+ .sec_code_size = fw->imem_size,
.code_entry_point = 0,
- .data_dma_base = hsf->vma->addr + hsf->data_addr,
- .data_size = hsf->data_size,
+ .data_dma_base = fw->vma->addr + fw->dmem_base_img,
+ .data_size = fw->dmem_size,
.argc = 0,
.argv = 0,
};
- flcn_bl_dmem_desc_v2_dump(&acr->subdev, &hsdesc);
+ flcn_bl_dmem_desc_v2_dump(fw->falcon->user, &hsdesc);
- nvkm_falcon_load_dmem(hsf->falcon, &hsdesc, 0, sizeof(hsdesc), 0);
+ return nvkm_falcon_pio_wr(fw->falcon, (u8 *)&hsdesc, 0, 0, DMEM, 0, sizeof(hsdesc), 0, 0);
}
-const struct nvkm_acr_hsf_func
-gp108_acr_unload_0 = {
- .load = gm200_acr_unload_load,
- .boot = gm200_acr_unload_boot,
- .bld = gp108_acr_hsfw_bld,
+const struct nvkm_falcon_fw_func
+gp108_acr_hsfw_0 = {
+ .signature = gm200_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .load = gm200_flcn_fw_load,
+ .load_bld = gp108_acr_hsfw_load_bld,
+ .boot = gm200_flcn_fw_boot,
};
MODULE_FIRMWARE("nvidia/gp108/acr/unload_bl.bin");
MODULE_FIRMWARE("nvidia/gp108/acr/ucode_unload.bin");
-MODULE_FIRMWARE("nvidia/gv100/acr/unload_bl.bin");
-MODULE_FIRMWARE("nvidia/gv100/acr/ucode_unload.bin");
-
static const struct nvkm_acr_hsf_fwif
gp108_acr_unload_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gp108_acr_unload_0 },
+ { 0, gm200_acr_hsfw_ctor, &gp108_acr_hsfw_0, NVKM_ACR_HSF_PMU, 0x1d, 0x00000010 },
{}
};
-static const struct nvkm_acr_hsf_func
+const struct nvkm_falcon_fw_func
gp108_acr_load_0 = {
- .load = gp102_acr_load_load,
- .boot = gm200_acr_load_boot,
- .bld = gp108_acr_hsfw_bld,
+ .signature = gm200_flcn_fw_signature,
+ .reset = gm200_flcn_fw_reset,
+ .setup = gp102_acr_load_setup,
+ .load = gm200_flcn_fw_load,
+ .load_bld = gp108_acr_hsfw_load_bld,
+ .boot = gm200_flcn_fw_boot,
};
MODULE_FIRMWARE("nvidia/gp108/acr/bl.bin");
MODULE_FIRMWARE("nvidia/gp108/acr/ucode_load.bin");
-MODULE_FIRMWARE("nvidia/gv100/acr/bl.bin");
-MODULE_FIRMWARE("nvidia/gv100/acr/ucode_load.bin");
-
static const struct nvkm_acr_hsf_fwif
gp108_acr_load_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gp108_acr_load_0 },
+ { 0, gm200_acr_hsfw_ctor, &gp108_acr_load_0, NVKM_ACR_HSF_SEC2, 0, 0x00000010 },
{}
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp10b.c
index f03ba02..a3422ab 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp10b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp10b.c
@@ -28,7 +28,7 @@ MODULE_FIRMWARE("nvidia/gp10b/acr/ucode_load.bin");
static const struct nvkm_acr_hsf_fwif
gp10b_acr_load_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gm20b_acr_load_0 },
+ { 0, gm200_acr_hsfw_ctor, &gm20b_acr_load_0, NVKM_ACR_HSF_PMU, 0, 0x00000010 },
{}
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gv100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gv100.c
new file mode 100644
index 0000000..4c5ca6b
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/gv100.c
@@ -0,0 +1,67 @@
+/*
+ * Copyright 2022 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+MODULE_FIRMWARE("nvidia/gv100/acr/unload_bl.bin");
+MODULE_FIRMWARE("nvidia/gv100/acr/ucode_unload.bin");
+
+static const struct nvkm_acr_hsf_fwif
+gv100_acr_unload_fwif[] = {
+ { 0, gm200_acr_hsfw_ctor, &gp108_acr_hsfw_0, NVKM_ACR_HSF_PMU, 0, 0x00000000 },
+ {}
+};
+
+MODULE_FIRMWARE("nvidia/gv100/acr/bl.bin");
+MODULE_FIRMWARE("nvidia/gv100/acr/ucode_load.bin");
+
+static const struct nvkm_acr_hsf_fwif
+gv100_acr_load_fwif[] = {
+ { 0, gm200_acr_hsfw_ctor, &gp108_acr_load_0, NVKM_ACR_HSF_SEC2, 0, 0x00000010 },
+ {}
+};
+
+static const struct nvkm_acr_func
+gv100_acr = {
+ .load = gv100_acr_load_fwif,
+ .unload = gv100_acr_unload_fwif,
+ .wpr_parse = gp102_acr_wpr_parse,
+ .wpr_layout = gp102_acr_wpr_layout,
+ .wpr_alloc = gp102_acr_wpr_alloc,
+ .wpr_build = gp102_acr_wpr_build,
+ .wpr_patch = gp102_acr_wpr_patch,
+ .wpr_check = gm200_acr_wpr_check,
+ .init = gm200_acr_init,
+};
+
+static const struct nvkm_acr_fwif
+gv100_acr_fwif[] = {
+ { 0, gp102_acr_load, &gv100_acr },
+ { -1, gm200_acr_nofw, &gm200_acr },
+ {}
+};
+
+int
+gv100_acr_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_acr **pacr)
+{
+ return nvkm_acr_new_(gv100_acr_fwif, device, type, inst, pacr);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/hsfw.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/hsfw.c
deleted file mode 100644
index a6ea89a..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/hsfw.c
+++ /dev/null
@@ -1,177 +0,0 @@
-/*
- * Copyright 2019 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- */
-#include "priv.h"
-
-#include <core/firmware.h>
-
-#include <nvfw/fw.h>
-#include <nvfw/hs.h>
-
-static void
-nvkm_acr_hsfw_del(struct nvkm_acr_hsfw *hsfw)
-{
- list_del(&hsfw->head);
- kfree(hsfw->imem);
- kfree(hsfw->image);
- kfree(hsfw->sig.prod.data);
- kfree(hsfw->sig.dbg.data);
- kfree(hsfw);
-}
-
-void
-nvkm_acr_hsfw_del_all(struct nvkm_acr *acr)
-{
- struct nvkm_acr_hsfw *hsfw, *hsft;
- list_for_each_entry_safe(hsfw, hsft, &acr->hsfw, head) {
- nvkm_acr_hsfw_del(hsfw);
- }
-}
-
-static int
-nvkm_acr_hsfw_load_image(struct nvkm_acr *acr, const char *name, int ver,
- struct nvkm_acr_hsfw *hsfw)
-{
- struct nvkm_subdev *subdev = &acr->subdev;
- const struct firmware *fw;
- const struct nvfw_bin_hdr *hdr;
- const struct nvfw_hs_header *fwhdr;
- const struct nvfw_hs_load_header *lhdr;
- u32 loc, sig;
- int ret;
-
- ret = nvkm_firmware_get(subdev, name, ver, &fw);
- if (ret < 0)
- return ret;
-
- hdr = nvfw_bin_hdr(subdev, fw->data);
- fwhdr = nvfw_hs_header(subdev, fw->data + hdr->header_offset);
-
- /* Earlier FW releases by NVIDIA for Nouveau's use aren't in NVIDIA's
- * standard format, and don't have the indirection seen in the 0x10de
- * case.
- */
- switch (hdr->bin_magic) {
- case 0x000010de:
- loc = *(u32 *)(fw->data + fwhdr->patch_loc);
- sig = *(u32 *)(fw->data + fwhdr->patch_sig);
- break;
- case 0x3b1d14f0:
- loc = fwhdr->patch_loc;
- sig = fwhdr->patch_sig;
- break;
- default:
- ret = -EINVAL;
- goto done;
- }
-
- lhdr = nvfw_hs_load_header(subdev, fw->data + fwhdr->hdr_offset);
-
- if (!(hsfw->image = kmalloc(hdr->data_size, GFP_KERNEL))) {
- ret = -ENOMEM;
- goto done;
- }
-
- memcpy(hsfw->image, fw->data + hdr->data_offset, hdr->data_size);
- hsfw->image_size = hdr->data_size;
- hsfw->non_sec_addr = lhdr->non_sec_code_off;
- hsfw->non_sec_size = lhdr->non_sec_code_size;
- hsfw->sec_addr = lhdr->apps[0];
- hsfw->sec_size = lhdr->apps[lhdr->num_apps];
- hsfw->data_addr = lhdr->data_dma_base;
- hsfw->data_size = lhdr->data_size;
-
- hsfw->sig.prod.size = fwhdr->sig_prod_size;
- hsfw->sig.prod.data = kmemdup(fw->data + fwhdr->sig_prod_offset + sig,
- hsfw->sig.prod.size, GFP_KERNEL);
- if (!hsfw->sig.prod.data) {
- ret = -ENOMEM;
- goto done;
- }
-
- hsfw->sig.dbg.size = fwhdr->sig_dbg_size;
- hsfw->sig.dbg.data = kmemdup(fw->data + fwhdr->sig_dbg_offset + sig,
- hsfw->sig.dbg.size, GFP_KERNEL);
- if (!hsfw->sig.dbg.data) {
- ret = -ENOMEM;
- goto done;
- }
-
- hsfw->sig.patch_loc = loc;
-done:
- nvkm_firmware_put(fw);
- return ret;
-}
-
-static int
-nvkm_acr_hsfw_load_bl(struct nvkm_acr *acr, const char *name, int ver,
- struct nvkm_acr_hsfw *hsfw)
-{
- struct nvkm_subdev *subdev = &acr->subdev;
- const struct nvfw_bin_hdr *hdr;
- const struct nvfw_bl_desc *desc;
- const struct firmware *fw;
- u8 *data;
- int ret;
-
- ret = nvkm_firmware_get(subdev, name, ver, &fw);
- if (ret)
- return ret;
-
- hdr = nvfw_bin_hdr(subdev, fw->data);
- desc = nvfw_bl_desc(subdev, fw->data + hdr->header_offset);
- data = (void *)fw->data + hdr->data_offset;
-
- hsfw->imem_size = desc->code_size;
- hsfw->imem_tag = desc->start_tag;
- hsfw->imem = kmemdup(data + desc->code_off, desc->code_size, GFP_KERNEL);
- nvkm_firmware_put(fw);
- if (!hsfw->imem)
- return -ENOMEM;
- else
- return 0;
-}
-
-int
-nvkm_acr_hsfw_load(struct nvkm_acr *acr, const char *bl, const char *fw,
- const char *name, int version,
- const struct nvkm_acr_hsf_fwif *fwif)
-{
- struct nvkm_acr_hsfw *hsfw;
- int ret;
-
- if (!(hsfw = kzalloc(sizeof(*hsfw), GFP_KERNEL)))
- return -ENOMEM;
-
- hsfw->func = fwif->func;
- hsfw->name = name;
- list_add_tail(&hsfw->head, &acr->hsfw);
-
- ret = nvkm_acr_hsfw_load_bl(acr, bl, version, hsfw);
- if (ret)
- goto done;
-
- ret = nvkm_acr_hsfw_load_image(acr, fw, version, hsfw);
-done:
- if (ret)
- nvkm_acr_hsfw_del(hsfw);
- return ret;
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/lsfw.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/lsfw.c
index 9b1cf67..f36a359 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/lsfw.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/lsfw.c
@@ -29,6 +29,7 @@ void
nvkm_acr_lsfw_del(struct nvkm_acr_lsfw *lsfw)
{
nvkm_blob_dtor(&lsfw->img);
+ kfree(lsfw->sigs);
nvkm_firmware_put(lsfw->sig);
list_del(&lsfw->head);
kfree(lsfw);
@@ -177,6 +178,75 @@ nvkm_acr_lsfw_load_sig_image_desc_v1(struct nvkm_subdev *subdev,
}
int
+nvkm_acr_lsfw_load_sig_image_desc_v2(struct nvkm_subdev *subdev,
+ struct nvkm_falcon *falcon,
+ enum nvkm_acr_lsf_id id,
+ const char *path, int ver,
+ const struct nvkm_acr_lsf_func *func)
+{
+ const struct firmware *fw;
+ struct nvkm_acr_lsfw *lsfw;
+ const struct nvfw_ls_desc_v2 *desc;
+ int ret = 0;
+
+ lsfw = nvkm_acr_lsfw_load_sig_image_desc_(subdev, falcon, id, path, ver, func, &fw);
+ if (IS_ERR(lsfw))
+ return PTR_ERR(lsfw);
+
+ desc = nvfw_ls_desc_v2(subdev, fw->data);
+
+ lsfw->secure_bootloader = desc->secure_bootloader;
+ lsfw->bootloader_size = ALIGN(desc->bootloader_size, 256);
+ lsfw->bootloader_imem_offset = desc->bootloader_imem_offset;
+
+ lsfw->app_size = ALIGN(desc->app_size, 256);
+ lsfw->app_start_offset = desc->app_start_offset;
+ lsfw->app_imem_entry = desc->app_imem_entry;
+ lsfw->app_resident_code_offset = desc->app_resident_code_offset;
+ lsfw->app_resident_code_size = desc->app_resident_code_size;
+ lsfw->app_resident_data_offset = desc->app_resident_data_offset;
+ lsfw->app_resident_data_size = desc->app_resident_data_size;
+ lsfw->app_imem_offset = desc->app_imem_offset;
+ lsfw->app_dmem_offset = desc->app_dmem_offset;
+
+ lsfw->ucode_size = ALIGN(lsfw->app_resident_data_offset, 256) + lsfw->bootloader_size;
+ lsfw->data_size = lsfw->app_size + lsfw->bootloader_size - lsfw->ucode_size;
+
+ nvkm_firmware_put(fw);
+
+ if (lsfw->secure_bootloader) {
+ const struct firmware *hsbl;
+ const struct nvfw_ls_hsbl_bin_hdr *hdr;
+ const struct nvfw_ls_hsbl_hdr *hshdr;
+ u32 loc, sig, cnt, *meta;
+
+ ret = nvkm_firmware_load_name(subdev, path, "hs_bl_sig", ver, &hsbl);
+ if (ret)
+ return ret;
+
+ hdr = nvfw_ls_hsbl_bin_hdr(subdev, hsbl->data);
+ hshdr = nvfw_ls_hsbl_hdr(subdev, hsbl->data + hdr->header_offset);
+ meta = (u32 *)(hsbl->data + hshdr->meta_data_offset);
+ loc = *(u32 *)(hsbl->data + hshdr->patch_loc);
+ sig = *(u32 *)(hsbl->data + hshdr->patch_sig);
+ cnt = *(u32 *)(hsbl->data + hshdr->num_sig);
+
+ lsfw->fuse_ver = meta[0];
+ lsfw->engine_id = meta[1];
+ lsfw->ucode_id = meta[2];
+ lsfw->sig_size = hshdr->sig_prod_size / cnt;
+ lsfw->sig_nr = cnt;
+ lsfw->sigs = kmemdup(hsbl->data + hshdr->sig_prod_offset + sig,
+ lsfw->sig_nr * lsfw->sig_size, GFP_KERNEL);
+ nvkm_firmware_put(hsbl);
+ if (!lsfw->sigs)
+ ret = -ENOMEM;
+ }
+
+ return ret;
+}
+
+int
nvkm_acr_lsfw_load_bl_inst_data_sig(struct nvkm_subdev *subdev,
struct nvkm_falcon *falcon,
enum nvkm_acr_lsf_id id,
@@ -251,3 +321,78 @@ nvkm_acr_lsfw_load_bl_inst_data_sig(struct nvkm_subdev *subdev,
nvkm_firmware_put(bl);
return ret;
}
+
+int
+nvkm_acr_lsfw_load_bl_sig_net(struct nvkm_subdev *subdev,
+ struct nvkm_falcon *falcon,
+ enum nvkm_acr_lsf_id id,
+ const char *path, int ver,
+ const struct nvkm_acr_lsf_func *func,
+ const void *inst_data, u32 inst_size,
+ const void *data_data, u32 data_size)
+{
+ struct nvkm_acr *acr = subdev->device->acr;
+ struct nvkm_acr_lsfw *lsfw;
+ const struct firmware _inst = { .data = inst_data, .size = inst_size };
+ const struct firmware _data = { .data = data_data, .size = data_size };
+ const struct firmware *bl = NULL, *inst = &_inst, *data = &_data;
+ const struct {
+ int bin_magic;
+ int bin_version;
+ int bin_size;
+ int header_offset;
+ int header_size;
+ } *hdr;
+ u32 *bldata;
+ int ret;
+
+ if (IS_ERR((lsfw = nvkm_acr_lsfw_add(func, acr, falcon, id))))
+ return PTR_ERR(lsfw);
+
+ ret = nvkm_firmware_load_name(subdev, path, "bl", ver, &bl);
+ if (ret)
+ goto done;
+
+ hdr = (const void *)bl->data;
+ bldata = (void *)(bl->data + hdr->header_offset);
+
+ ret = nvkm_firmware_load_name(subdev, path, "sig", ver, &lsfw->sig);
+ if (ret)
+ goto done;
+
+ lsfw->bootloader_size = ALIGN(hdr->header_size, 256);
+ lsfw->bootloader_imem_offset = func->bl_entry;
+
+ lsfw->app_start_offset = lsfw->bootloader_size;
+ lsfw->app_imem_entry = 0;
+ lsfw->app_resident_code_offset = 0;
+ lsfw->app_resident_code_size = ALIGN(inst->size, 256);
+ lsfw->app_resident_data_offset = lsfw->app_resident_code_size;
+ lsfw->app_resident_data_size = ALIGN(data->size, 256);
+ lsfw->app_imem_offset = 0;
+ lsfw->app_dmem_offset = 0;
+ lsfw->app_size = lsfw->app_resident_code_size + lsfw->app_resident_data_size;
+
+ lsfw->img.size = lsfw->bootloader_size + lsfw->app_size;
+ if (!(lsfw->img.data = kzalloc(lsfw->img.size, GFP_KERNEL))) {
+ ret = -ENOMEM;
+ goto done;
+ }
+
+ memcpy(lsfw->img.data, bldata, lsfw->bootloader_size);
+ memcpy(lsfw->img.data + lsfw->app_start_offset +
+ lsfw->app_resident_code_offset, inst->data, inst->size);
+ memcpy(lsfw->img.data + lsfw->app_start_offset +
+ lsfw->app_resident_data_offset, data->data, data->size);
+
+ lsfw->ucode_size = ALIGN(lsfw->app_resident_data_offset, 256) +
+ lsfw->bootloader_size;
+ lsfw->data_size = lsfw->app_size + lsfw->bootloader_size -
+ lsfw->ucode_size;
+
+done:
+ if (ret)
+ nvkm_acr_lsfw_del(lsfw);
+ nvkm_firmware_put(bl);
+ return ret;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/priv.h
index c30b841..4881c8b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/priv.h
@@ -24,7 +24,7 @@ struct nvkm_acr_func {
u32 (*wpr_layout)(struct nvkm_acr *);
int (*wpr_alloc)(struct nvkm_acr *, u32 wpr_size);
int (*wpr_build)(struct nvkm_acr *, struct nvkm_acr_lsf *rtos);
- void (*wpr_patch)(struct nvkm_acr *, s64 adjust);
+ int (*wpr_patch)(struct nvkm_acr *, s64 adjust);
void (*wpr_check)(struct nvkm_acr *, u64 *start, u64 *limit);
int (*init)(struct nvkm_acr *);
void (*fini)(struct nvkm_acr *);
@@ -35,7 +35,7 @@ extern const struct nvkm_acr_func gm200_acr;
int gm200_acr_wpr_parse(struct nvkm_acr *);
u32 gm200_acr_wpr_layout(struct nvkm_acr *);
int gm200_acr_wpr_build(struct nvkm_acr *, struct nvkm_acr_lsf *);
-void gm200_acr_wpr_patch(struct nvkm_acr *, s64);
+int gm200_acr_wpr_patch(struct nvkm_acr *, s64);
void gm200_acr_wpr_check(struct nvkm_acr *, u64 *, u64 *);
void gm200_acr_wpr_build_lsb_tail(struct nvkm_acr_lsfw *,
struct lsb_header_tail *);
@@ -48,96 +48,60 @@ u32 gp102_acr_wpr_layout(struct nvkm_acr *);
int gp102_acr_wpr_alloc(struct nvkm_acr *, u32 wpr_size);
int gp102_acr_wpr_build(struct nvkm_acr *, struct nvkm_acr_lsf *);
int gp102_acr_wpr_build_lsb(struct nvkm_acr *, struct nvkm_acr_lsfw *);
-void gp102_acr_wpr_patch(struct nvkm_acr *, s64);
+int gp102_acr_wpr_patch(struct nvkm_acr *, s64);
+
+int tu102_acr_init(struct nvkm_acr *);
+
+void ga100_acr_wpr_check(struct nvkm_acr *, u64 *, u64 *);
struct nvkm_acr_hsfw {
- const struct nvkm_acr_hsf_func *func;
- const char *name;
+ struct nvkm_falcon_fw fw;
+
+ enum nvkm_acr_hsf_id {
+ NVKM_ACR_HSF_PMU,
+ NVKM_ACR_HSF_SEC2,
+ NVKM_ACR_HSF_GSP,
+ } falcon_id;
+ u32 boot_mbox0;
+ u32 intr_clear;
+
struct list_head head;
-
- u32 imem_size;
- u32 imem_tag;
- u32 *imem;
-
- u8 *image;
- u32 image_size;
- u32 non_sec_addr;
- u32 non_sec_size;
- u32 sec_addr;
- u32 sec_size;
- u32 data_addr;
- u32 data_size;
-
- struct {
- struct {
- void *data;
- u32 size;
- } prod, dbg;
- u32 patch_loc;
- } sig;
};
+int nvkm_acr_hsfw_boot(struct nvkm_acr *, const char *name);
+
struct nvkm_acr_hsf_fwif {
int version;
int (*load)(struct nvkm_acr *, const char *bl, const char *fw,
const char *name, int version,
const struct nvkm_acr_hsf_fwif *);
- const struct nvkm_acr_hsf_func *func;
+ const struct nvkm_falcon_fw_func *func;
+
+ enum nvkm_acr_hsf_id falcon_id;
+ u32 boot_mbox0;
+ u32 intr_clear;
};
-int nvkm_acr_hsfw_load(struct nvkm_acr *, const char *, const char *,
- const char *, int, const struct nvkm_acr_hsf_fwif *);
-void nvkm_acr_hsfw_del_all(struct nvkm_acr *);
-struct nvkm_acr_hsf {
- const struct nvkm_acr_hsf_func *func;
- const char *name;
- struct list_head head;
+int gm200_acr_hsfw_ctor(struct nvkm_acr *, const char *, const char *, const char *, int,
+ const struct nvkm_acr_hsf_fwif *);
+int gm200_acr_hsfw_load_bld(struct nvkm_falcon_fw *);
+extern const struct nvkm_falcon_fw_func gm200_acr_unload_0;
- u32 imem_size;
- u32 imem_tag;
- u32 *imem;
+extern const struct nvkm_falcon_fw_func gm20b_acr_load_0;
- u32 non_sec_addr;
- u32 non_sec_size;
- u32 sec_addr;
- u32 sec_size;
- u32 data_addr;
- u32 data_size;
+int gp102_acr_load_setup(struct nvkm_falcon_fw *);
- struct nvkm_memory *ucode;
- struct nvkm_vma *vma;
- struct nvkm_falcon *falcon;
-};
+extern const struct nvkm_falcon_fw_func gp108_acr_load_0;
-struct nvkm_acr_hsf_func {
- int (*load)(struct nvkm_acr *, struct nvkm_acr_hsfw *);
- int (*boot)(struct nvkm_acr *, struct nvkm_acr_hsf *);
- void (*bld)(struct nvkm_acr *, struct nvkm_acr_hsf *);
-};
+extern const struct nvkm_falcon_fw_func gp108_acr_hsfw_0;
+int gp108_acr_hsfw_load_bld(struct nvkm_falcon_fw *);
-int gm200_acr_hsfw_load(struct nvkm_acr *, struct nvkm_acr_hsfw *,
- struct nvkm_falcon *);
-int gm200_acr_hsfw_boot(struct nvkm_acr *, struct nvkm_acr_hsf *,
- u32 clear_intr, u32 mbox0_ok);
-
-int gm200_acr_load_boot(struct nvkm_acr *, struct nvkm_acr_hsf *);
-
-extern const struct nvkm_acr_hsf_func gm200_acr_unload_0;
-int gm200_acr_unload_load(struct nvkm_acr *, struct nvkm_acr_hsfw *);
-int gm200_acr_unload_boot(struct nvkm_acr *, struct nvkm_acr_hsf *);
-void gm200_acr_hsfw_bld(struct nvkm_acr *, struct nvkm_acr_hsf *);
-
-extern const struct nvkm_acr_hsf_func gm20b_acr_load_0;
-
-int gp102_acr_load_load(struct nvkm_acr *, struct nvkm_acr_hsfw *);
-
-extern const struct nvkm_acr_hsf_func gp108_acr_unload_0;
-void gp108_acr_hsfw_bld(struct nvkm_acr *, struct nvkm_acr_hsf *);
+int ga100_acr_hsfw_ctor(struct nvkm_acr *, const char *, const char *, const char *, int,
+ const struct nvkm_acr_hsf_fwif *);
int nvkm_acr_new_(const struct nvkm_acr_fwif *, struct nvkm_device *, enum nvkm_subdev_type,
int inst, struct nvkm_acr **);
-int nvkm_acr_hsf_boot(struct nvkm_acr *, const char *name);
struct nvkm_acr_lsf {
const struct nvkm_acr_lsf_func *func;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/tu102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/tu102.c
index 05a87e7..c22d551 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/tu102.c
@@ -29,14 +29,14 @@
#include <nvfw/acr.h>
-static int
+int
tu102_acr_init(struct nvkm_acr *acr)
{
- int ret = nvkm_acr_hsf_boot(acr, "AHESASC");
+ int ret = nvkm_acr_hsfw_boot(acr, "AHESASC");
if (ret)
return ret;
- return nvkm_acr_hsf_boot(acr, "ASB");
+ return nvkm_acr_hsfw_boot(acr, "ASB");
}
static int
@@ -85,12 +85,6 @@ tu102_acr_wpr_build(struct nvkm_acr *acr, struct nvkm_acr_lsf *rtos)
}
static int
-tu102_acr_hsfw_boot(struct nvkm_acr *acr, struct nvkm_acr_hsf *hsf)
-{
- return gm200_acr_hsfw_boot(acr, hsf, 0, 0);
-}
-
-static int
tu102_acr_hsfw_nofw(struct nvkm_acr *acr, const char *bl, const char *fw,
const char *name, int version,
const struct nvkm_acr_hsf_fwif *fwif)
@@ -115,24 +109,11 @@ MODULE_FIRMWARE("nvidia/tu117/acr/ucode_unload.bin");
static const struct nvkm_acr_hsf_fwif
tu102_acr_unload_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &gp108_acr_unload_0 },
+ { 0, gm200_acr_hsfw_ctor, &gp108_acr_hsfw_0, NVKM_ACR_HSF_PMU, 0, 0x00000000 },
{ -1, tu102_acr_hsfw_nofw },
{}
};
-static int
-tu102_acr_asb_load(struct nvkm_acr *acr, struct nvkm_acr_hsfw *hsfw)
-{
- return gm200_acr_hsfw_load(acr, hsfw, &acr->subdev.device->gsp->falcon);
-}
-
-static const struct nvkm_acr_hsf_func
-tu102_acr_asb_0 = {
- .load = tu102_acr_asb_load,
- .boot = tu102_acr_hsfw_boot,
- .bld = gp108_acr_hsfw_bld,
-};
-
MODULE_FIRMWARE("nvidia/tu102/acr/ucode_asb.bin");
MODULE_FIRMWARE("nvidia/tu104/acr/ucode_asb.bin");
MODULE_FIRMWARE("nvidia/tu106/acr/ucode_asb.bin");
@@ -141,18 +122,11 @@ MODULE_FIRMWARE("nvidia/tu117/acr/ucode_asb.bin");
static const struct nvkm_acr_hsf_fwif
tu102_acr_asb_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &tu102_acr_asb_0 },
+ { 0, gm200_acr_hsfw_ctor, &gp108_acr_hsfw_0, NVKM_ACR_HSF_GSP, 0, 0x00000000 },
{ -1, tu102_acr_hsfw_nofw },
{}
};
-static const struct nvkm_acr_hsf_func
-tu102_acr_ahesasc_0 = {
- .load = gp102_acr_load_load,
- .boot = tu102_acr_hsfw_boot,
- .bld = gp108_acr_hsfw_bld,
-};
-
MODULE_FIRMWARE("nvidia/tu102/acr/bl.bin");
MODULE_FIRMWARE("nvidia/tu102/acr/ucode_ahesasc.bin");
@@ -170,7 +144,7 @@ MODULE_FIRMWARE("nvidia/tu117/acr/ucode_ahesasc.bin");
static const struct nvkm_acr_hsf_fwif
tu102_acr_ahesasc_fwif[] = {
- { 0, nvkm_acr_hsfw_load, &tu102_acr_ahesasc_0 },
+ { 0, gm200_acr_hsfw_ctor, &gp108_acr_load_0, NVKM_ACR_HSF_SEC2, 0, 0x00000000 },
{ -1, tu102_acr_hsfw_nofw },
{}
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c
index a308b9b..f30718d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c
@@ -26,6 +26,7 @@
#include <subdev/bios.h>
#include <subdev/bios/bit.h>
#include <subdev/bios/pmu.h>
+#include <subdev/pmu.h>
#include <subdev/timer.h>
static void
@@ -85,13 +86,18 @@ pmu_load(struct nv50_devinit *init, u8 type, bool post,
struct nvkm_subdev *subdev = &init->base.subdev;
struct nvkm_bios *bios = subdev->device->bios;
struct nvbios_pmuR pmu;
+ int ret;
if (!nvbios_pmuRm(bios, type, &pmu))
return -EINVAL;
- if (!post)
+ if (!post || !subdev->device->pmu)
return 0;
+ ret = nvkm_falcon_reset(&subdev->device->pmu->falcon);
+ if (ret)
+ return ret;
+
pmu_code(init, pmu.boot_addr_pmu, pmu.boot_addr, pmu.boot_size, false);
pmu_code(init, pmu.code_addr_pmu, pmu.code_addr, pmu.code_size, true);
pmu_data(init, pmu.data_addr_pmu, pmu.data_addr, pmu.data_size);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/base.c
index fd54fa5..b53ac9a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/base.c
@@ -22,7 +22,6 @@
#include "priv.h"
#include <core/memory.h>
-#include <core/notify.h>
static void
nvkm_fault_ntfy_fini(struct nvkm_event *event, int type, int index)
@@ -38,23 +37,8 @@ nvkm_fault_ntfy_init(struct nvkm_event *event, int type, int index)
fault->func->buffer.intr(fault->buffer[index], true);
}
-static int
-nvkm_fault_ntfy_ctor(struct nvkm_object *object, void *argv, u32 argc,
- struct nvkm_notify *notify)
-{
- struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object);
- if (argc == 0) {
- notify->size = 0;
- notify->types = 1;
- notify->index = buffer->id;
- return 0;
- }
- return -ENOSYS;
-}
-
static const struct nvkm_event_func
nvkm_fault_ntfy = {
- .ctor = nvkm_fault_ntfy_ctor,
.init = nvkm_fault_ntfy_init,
.fini = nvkm_fault_ntfy_fini,
};
@@ -130,8 +114,7 @@ nvkm_fault_oneinit(struct nvkm_subdev *subdev)
}
}
- ret = nvkm_event_init(&nvkm_fault_ntfy, 1, fault->buffer_nr,
- &fault->event);
+ ret = nvkm_event_init(&nvkm_fault_ntfy, subdev, 1, fault->buffer_nr, &fault->event);
if (ret)
return ret;
@@ -146,7 +129,7 @@ nvkm_fault_dtor(struct nvkm_subdev *subdev)
struct nvkm_fault *fault = nvkm_fault(subdev);
int i;
- nvkm_notify_fini(&fault->nrpfb);
+ nvkm_event_ntfy_del(&fault->nrpfb);
nvkm_event_fini(&fault->event);
for (i = 0; i < fault->buffer_nr; i++) {
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gp100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gp100.c
index 6af7959..04c7526 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gp100.c
@@ -65,7 +65,7 @@ gp100_fault_buffer_info(struct nvkm_fault_buffer *buffer)
void
gp100_fault_intr(struct nvkm_fault *fault)
{
- nvkm_event_send(&fault->event, 1, 0, NULL, 0);
+ nvkm_event_ntfy(&fault->event, 0, NVKM_FAULT_BUFFER_EVENT_PENDING);
}
static const struct nvkm_fault_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c
index cd9d2ad..8e34d40 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c
@@ -27,10 +27,12 @@
#include <nvif/class.h>
-static void
-gv100_fault_buffer_process(struct nvkm_fault_buffer *buffer)
+void
+gv100_fault_buffer_process(struct work_struct *work)
{
- struct nvkm_device *device = buffer->fault->subdev.device;
+ struct nvkm_fault *fault = container_of(work, typeof(*fault), nrpfb_work);
+ struct nvkm_fault_buffer *buffer = fault->buffer[0];
+ struct nvkm_device *device = fault->subdev.device;
struct nvkm_memory *mem = buffer->mem;
u32 get = nvkm_rd32(device, buffer->get);
u32 put = nvkm_rd32(device, buffer->put);
@@ -115,11 +117,12 @@ gv100_fault_buffer_info(struct nvkm_fault_buffer *buffer)
}
static int
-gv100_fault_ntfy_nrpfb(struct nvkm_notify *notify)
+gv100_fault_ntfy_nrpfb(struct nvkm_event_ntfy *ntfy, u32 bits)
{
- struct nvkm_fault *fault = container_of(notify, typeof(*fault), nrpfb);
- gv100_fault_buffer_process(fault->buffer[0]);
- return NVKM_NOTIFY_KEEP;
+ struct nvkm_fault *fault = container_of(ntfy, typeof(*fault), nrpfb);
+
+ schedule_work(&fault->nrpfb_work);
+ return NVKM_EVENT_KEEP;
}
static void
@@ -163,14 +166,14 @@ gv100_fault_intr(struct nvkm_fault *fault)
if (stat & 0x20000000) {
if (fault->buffer[0]) {
- nvkm_event_send(&fault->event, 1, 0, NULL, 0);
+ nvkm_event_ntfy(&fault->event, 0, NVKM_FAULT_BUFFER_EVENT_PENDING);
stat &= ~0x20000000;
}
}
if (stat & 0x08000000) {
if (fault->buffer[1]) {
- nvkm_event_send(&fault->event, 1, 1, NULL, 0);
+ nvkm_event_ntfy(&fault->event, 1, NVKM_FAULT_BUFFER_EVENT_PENDING);
stat &= ~0x08000000;
}
}
@@ -183,9 +186,12 @@ gv100_fault_intr(struct nvkm_fault *fault)
static void
gv100_fault_fini(struct nvkm_fault *fault)
{
- nvkm_notify_put(&fault->nrpfb);
+ nvkm_event_ntfy_block(&fault->nrpfb);
+ flush_work(&fault->nrpfb_work);
+
if (fault->buffer[0])
fault->func->buffer.fini(fault->buffer[0]);
+
nvkm_mask(fault->subdev.device, 0x100a34, 0x80000000, 0x80000000);
}
@@ -194,15 +200,15 @@ gv100_fault_init(struct nvkm_fault *fault)
{
nvkm_mask(fault->subdev.device, 0x100a2c, 0x80000000, 0x80000000);
fault->func->buffer.init(fault->buffer[0]);
- nvkm_notify_get(&fault->nrpfb);
+ nvkm_event_ntfy_allow(&fault->nrpfb);
}
int
gv100_fault_oneinit(struct nvkm_fault *fault)
{
- return nvkm_notify_init(&fault->buffer[0]->object, &fault->event,
- gv100_fault_ntfy_nrpfb, true, NULL, 0, 0,
- &fault->nrpfb);
+ nvkm_event_ntfy_add(&fault->event, 0, NVKM_FAULT_BUFFER_EVENT_PENDING, true,
+ gv100_fault_ntfy_nrpfb, &fault->nrpfb);
+ return 0;
}
static const struct nvkm_fault_func
@@ -231,5 +237,10 @@ int
gv100_fault_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fault **pfault)
{
- return nvkm_fault_new_(&gv100_fault, device, type, inst, pfault);
+ int ret = nvkm_fault_new_(&gv100_fault, device, type, inst, pfault);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&(*pfault)->nrpfb_work, gv100_fault_buffer_process);
+ return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/priv.h
index 36681c3..a551033 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/priv.h
@@ -16,6 +16,8 @@ struct nvkm_fault_buffer {
u32 put;
struct nvkm_memory *mem;
u64 addr;
+
+ struct nvkm_inth inth;
};
int nvkm_fault_new_(const struct nvkm_fault_func *, struct nvkm_device *, enum nvkm_subdev_type,
@@ -46,6 +48,7 @@ void gp100_fault_buffer_fini(struct nvkm_fault_buffer *);
void gp100_fault_buffer_init(struct nvkm_fault_buffer *);
u64 gp100_fault_buffer_pin(struct nvkm_fault_buffer *);
void gp100_fault_buffer_info(struct nvkm_fault_buffer *);
+void gv100_fault_buffer_process(struct work_struct *);
void gp100_fault_intr(struct nvkm_fault *);
u64 gp10b_fault_buffer_pin(struct nvkm_fault_buffer *);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/tu102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/tu102.c
index 91eb672..967efad 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/tu102.c
@@ -24,20 +24,27 @@
#include <core/memory.h>
#include <subdev/mc.h>
#include <subdev/mmu.h>
+#include <subdev/vfn.h>
#include <engine/fifo.h>
#include <nvif/class.h>
+static irqreturn_t
+tu102_fault_buffer_notify(struct nvkm_inth *inth)
+{
+ struct nvkm_fault_buffer *buffer = container_of(inth, typeof(*buffer), inth);
+
+ nvkm_event_ntfy(&buffer->fault->event, buffer->id, NVKM_FAULT_BUFFER_EVENT_PENDING);
+ return IRQ_HANDLED;
+}
+
static void
tu102_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable)
{
- /*XXX: Earlier versions of RM touched the old regs on Turing,
- * which don't appear to actually work anymore, but newer
- * versions of RM don't appear to touch anything at all..
- */
- struct nvkm_device *device = buffer->fault->subdev.device;
-
- nvkm_mc_intr_mask(device, NVKM_SUBDEV_FAULT, 0, enable);
+ if (enable)
+ nvkm_inth_allow(&buffer->inth);
+ else
+ nvkm_inth_block(&buffer->inth);
}
static void
@@ -46,10 +53,6 @@ tu102_fault_buffer_fini(struct nvkm_fault_buffer *buffer)
struct nvkm_device *device = buffer->fault->subdev.device;
const u32 foff = buffer->id * 0x20;
- /* Disable the fault interrupts */
- nvkm_wr32(device, 0xb81408, 0x1);
- nvkm_wr32(device, 0xb81410, 0x10);
-
nvkm_mask(device, 0xb83010 + foff, 0x80000000, 0x00000000);
}
@@ -59,10 +62,6 @@ tu102_fault_buffer_init(struct nvkm_fault_buffer *buffer)
struct nvkm_device *device = buffer->fault->subdev.device;
const u32 foff = buffer->id * 0x20;
- /* Enable the fault interrupts */
- nvkm_wr32(device, 0xb81208, 0x1);
- nvkm_wr32(device, 0xb81210, 0x10);
-
nvkm_mask(device, 0xb83010 + foff, 0xc0000000, 0x40000000);
nvkm_wr32(device, 0xb83004 + foff, upper_32_bits(buffer->addr));
nvkm_wr32(device, 0xb83000 + foff, lower_32_bits(buffer->addr));
@@ -82,9 +81,10 @@ tu102_fault_buffer_info(struct nvkm_fault_buffer *buffer)
buffer->put = 0xb8300c + foff;
}
-static void
-tu102_fault_intr_fault(struct nvkm_fault *fault)
+static irqreturn_t
+tu102_fault_info_fault(struct nvkm_inth *inth)
{
+ struct nvkm_fault *fault = container_of(inth, typeof(*fault), info_fault);
struct nvkm_subdev *subdev = &fault->subdev;
struct nvkm_device *device = subdev->device;
struct nvkm_fault_data info;
@@ -106,70 +106,61 @@ tu102_fault_intr_fault(struct nvkm_fault *fault)
info.reason = (info1 & 0x0000001f);
nvkm_fifo_fault(device->fifo, &info);
-}
-static void
-tu102_fault_intr(struct nvkm_fault *fault)
-{
- struct nvkm_subdev *subdev = &fault->subdev;
- struct nvkm_device *device = subdev->device;
- u32 stat = nvkm_rd32(device, 0xb83094);
-
- if (stat & 0x80000000) {
- tu102_fault_intr_fault(fault);
- nvkm_wr32(device, 0xb83094, 0x80000000);
- stat &= ~0x80000000;
- }
-
- if (stat & 0x00000200) {
- /* Clear the associated interrupt flag */
- nvkm_wr32(device, 0xb81010, 0x10);
-
- if (fault->buffer[0]) {
- nvkm_event_send(&fault->event, 1, 0, NULL, 0);
- stat &= ~0x00000200;
- }
- }
-
- /* Replayable MMU fault */
- if (stat & 0x00000100) {
- /* Clear the associated interrupt flag */
- nvkm_wr32(device, 0xb81008, 0x1);
-
- if (fault->buffer[1]) {
- nvkm_event_send(&fault->event, 1, 1, NULL, 0);
- stat &= ~0x00000100;
- }
- }
-
- if (stat) {
- nvkm_debug(subdev, "intr %08x\n", stat);
- }
+ nvkm_wr32(device, 0xb83094, 0x80000000);
+ return IRQ_HANDLED;
}
static void
tu102_fault_fini(struct nvkm_fault *fault)
{
- nvkm_notify_put(&fault->nrpfb);
+ nvkm_event_ntfy_block(&fault->nrpfb);
+ flush_work(&fault->nrpfb_work);
+
if (fault->buffer[0])
fault->func->buffer.fini(fault->buffer[0]);
- /*XXX: disable priv faults */
+
+ nvkm_inth_block(&fault->info_fault);
}
static void
tu102_fault_init(struct nvkm_fault *fault)
{
- /*XXX: enable priv faults */
+ nvkm_inth_allow(&fault->info_fault);
+
fault->func->buffer.init(fault->buffer[0]);
- nvkm_notify_get(&fault->nrpfb);
+ nvkm_event_ntfy_allow(&fault->nrpfb);
+}
+
+static int
+tu102_fault_oneinit(struct nvkm_fault *fault)
+{
+ struct nvkm_device *device = fault->subdev.device;
+ struct nvkm_intr *intr = &device->vfn->intr;
+ int ret, i;
+
+ ret = nvkm_inth_add(intr, nvkm_rd32(device, 0x100ee0) & 0x0000ffff,
+ NVKM_INTR_PRIO_NORMAL, &fault->subdev, tu102_fault_info_fault,
+ &fault->info_fault);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < fault->buffer_nr; i++) {
+ ret = nvkm_inth_add(intr, nvkm_rd32(device, 0x100ee4 + (i * 4)) >> 16,
+ NVKM_INTR_PRIO_NORMAL, &fault->subdev,
+ tu102_fault_buffer_notify, &fault->buffer[i]->inth);
+ if (ret)
+ return ret;
+ }
+
+ return gv100_fault_oneinit(fault);
}
static const struct nvkm_fault_func
tu102_fault = {
- .oneinit = gv100_fault_oneinit,
+ .oneinit = tu102_fault_oneinit,
.init = tu102_fault_init,
.fini = tu102_fault_fini,
- .intr = tu102_fault_intr,
.buffer.nr = 2,
.buffer.entry_size = 32,
.buffer.info = tu102_fault_buffer_info,
@@ -184,5 +175,10 @@ int
tu102_fault_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_fault **pfault)
{
- return nvkm_fault_new_(&tu102_fault, device, type, inst, pfault);
+ int ret = nvkm_fault_new_(&tu102_fault, device, type, inst, pfault);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&(*pfault)->nrpfb_work, gv100_fault_buffer_process);
+ return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/user.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/user.c
index ac835c9..c123e58 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fault/user.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fault/user.c
@@ -22,12 +22,28 @@
#include "priv.h"
#include <core/memory.h>
+#include <core/event.h>
#include <subdev/mmu.h>
#include <nvif/clb069.h>
#include <nvif/unpack.h>
static int
+nvkm_ufault_uevent(struct nvkm_object *object, void *argv, u32 argc, struct nvkm_uevent *uevent)
+{
+ struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object);
+ union nvif_clb069_event_args *args = argv;
+
+ if (!uevent)
+ return 0;
+ if (argc != sizeof(args->vn))
+ return -ENOSYS;
+
+ return nvkm_uevent_add(uevent, &buffer->fault->event, buffer->id,
+ NVKM_FAULT_BUFFER_EVENT_PENDING, NULL);
+}
+
+static int
nvkm_ufault_map(struct nvkm_object *object, void *argv, u32 argc,
enum nvkm_object_map *type, u64 *addr, u64 *size)
{
@@ -40,18 +56,6 @@ nvkm_ufault_map(struct nvkm_object *object, void *argv, u32 argc,
}
static int
-nvkm_ufault_ntfy(struct nvkm_object *object, u32 type,
- struct nvkm_event **pevent)
-{
- struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object);
- if (type == NVB069_V0_NTFY_FAULT) {
- *pevent = &buffer->fault->event;
- return 0;
- }
- return -EINVAL;
-}
-
-static int
nvkm_ufault_fini(struct nvkm_object *object, bool suspend)
{
struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object);
@@ -78,8 +82,8 @@ nvkm_ufault = {
.dtor = nvkm_ufault_dtor,
.init = nvkm_ufault_init,
.fini = nvkm_ufault_fini,
- .ntfy = nvkm_ufault_ntfy,
.map = nvkm_ufault_map,
+ .uevent = nvkm_ufault_uevent,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c
index 6faaea9..bac7dcc 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c
@@ -57,6 +57,15 @@ nvkm_fb_tile_prog(struct nvkm_fb *fb, int region, struct nvkm_fb_tile *tile)
}
}
+static void
+nvkm_fb_sysmem_flush_page_init(struct nvkm_device *device)
+{
+ struct nvkm_fb *fb = device->fb;
+
+ if (fb->func->sysmem.flush_page_init)
+ fb->func->sysmem.flush_page_init(fb);
+}
+
int
nvkm_fb_bios_memtype(struct nvkm_bios *bios)
{
@@ -125,12 +134,20 @@ nvkm_fb_oneinit(struct nvkm_subdev *subdev)
return nvkm_mm_init(&fb->tags.mm, 0, 0, tags, 1);
}
-static int
-nvkm_fb_init_scrub_vpr(struct nvkm_fb *fb)
+int
+nvkm_fb_mem_unlock(struct nvkm_fb *fb)
{
struct nvkm_subdev *subdev = &fb->subdev;
int ret;
+ if (!fb->func->vpr.scrub_required)
+ return 0;
+
+ if (!fb->func->vpr.scrub_required(fb)) {
+ nvkm_debug(subdev, "VPR not locked\n");
+ return 0;
+ }
+
nvkm_debug(subdev, "VPR locked, running scrubber binary\n");
if (!fb->vpr_scrubber.size) {
@@ -168,6 +185,8 @@ nvkm_fb_init(struct nvkm_subdev *subdev)
for (i = 0; i < fb->tile.regions; i++)
fb->func->tile.prog(fb, i, &fb->tile.region[i]);
+ nvkm_fb_sysmem_flush_page_init(subdev->device);
+
if (fb->func->init)
fb->func->init(fb);
@@ -183,13 +202,13 @@ nvkm_fb_init(struct nvkm_subdev *subdev)
if (fb->func->init_unkn)
fb->func->init_unkn(fb);
- if (fb->func->vpr.scrub_required &&
- fb->func->vpr.scrub_required(fb)) {
- ret = nvkm_fb_init_scrub_vpr(fb);
- if (ret)
- return ret;
- }
+ return 0;
+}
+static int
+nvkm_fb_preinit(struct nvkm_subdev *subdev)
+{
+ nvkm_fb_sysmem_flush_page_init(subdev->device);
return 0;
}
@@ -212,20 +231,28 @@ nvkm_fb_dtor(struct nvkm_subdev *subdev)
nvkm_blob_dtor(&fb->vpr_scrubber);
+ if (fb->sysmem.flush_page) {
+ dma_unmap_page(subdev->device->dev, fb->sysmem.flush_page_addr,
+ PAGE_SIZE, DMA_BIDIRECTIONAL);
+ __free_page(fb->sysmem.flush_page);
+ }
+
if (fb->func->dtor)
return fb->func->dtor(fb);
+
return fb;
}
static const struct nvkm_subdev_func
nvkm_fb = {
.dtor = nvkm_fb_dtor,
+ .preinit = nvkm_fb_preinit,
.oneinit = nvkm_fb_oneinit,
.init = nvkm_fb_init,
.intr = nvkm_fb_intr,
};
-void
+int
nvkm_fb_ctor(const struct nvkm_fb_func *func, struct nvkm_device *device,
enum nvkm_subdev_type type, int inst, struct nvkm_fb *fb)
{
@@ -234,6 +261,19 @@ nvkm_fb_ctor(const struct nvkm_fb_func *func, struct nvkm_device *device,
fb->tile.regions = fb->func->tile.regions;
fb->page = nvkm_longopt(device->cfgopt, "NvFbBigPage", fb->func->default_bigpage);
mutex_init(&fb->tags.mutex);
+
+ if (func->sysmem.flush_page_init) {
+ fb->sysmem.flush_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+ if (!fb->sysmem.flush_page)
+ return -ENOMEM;
+
+ fb->sysmem.flush_page_addr = dma_map_page(device->dev, fb->sysmem.flush_page,
+ 0, PAGE_SIZE, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(device->dev, fb->sysmem.flush_page_addr))
+ return -EFAULT;
+ }
+
+ return 0;
}
int
@@ -242,6 +282,5 @@ nvkm_fb_new_(const struct nvkm_fb_func *func, struct nvkm_device *device,
{
if (!(*pfb = kzalloc(sizeof(**pfb), GFP_KERNEL)))
return -ENOMEM;
- nvkm_fb_ctor(func, device, type, inst, *pfb);
- return 0;
+ return nvkm_fb_ctor(func, device, type, inst, *pfb);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga100.c
index b47bebf..5098f219 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga100.c
@@ -26,9 +26,10 @@ static const struct nvkm_fb_func
ga100_fb = {
.dtor = gf100_fb_dtor,
.oneinit = gf100_fb_oneinit,
- .init = gp100_fb_init,
+ .init = gm200_fb_init,
.init_page = gv100_fb_init_page,
.init_unkn = gp100_fb_init_unkn,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.ram_new = gp100_ram_new,
.default_bigpage = 16,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga102.c
index 6ea7908..8b7c8ea 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga102.c
@@ -22,15 +22,42 @@
#include "gf100.h"
#include "ram.h"
+#include <engine/nvdec.h>
+
+static int
+ga102_fb_vpr_scrub(struct nvkm_fb *fb)
+{
+ struct nvkm_falcon_fw fw = {};
+ int ret;
+
+ ret = nvkm_falcon_fw_ctor_hs_v2(&ga102_flcn_fw, "mem-unlock", &fb->subdev, "nvdec/scrubber",
+ 0, &fb->subdev.device->nvdec[0]->falcon, &fw);
+ if (ret)
+ return ret;
+
+ ret = nvkm_falcon_fw_boot(&fw, &fb->subdev, true, NULL, NULL, 0, 0);
+ nvkm_falcon_fw_dtor(&fw);
+ return ret;
+}
+
+static bool
+ga102_fb_vpr_scrub_required(struct nvkm_fb *fb)
+{
+ return (nvkm_rd32(fb->subdev.device, 0x1fa80c) & 0x00000010) != 0;
+}
+
static const struct nvkm_fb_func
ga102_fb = {
.dtor = gf100_fb_dtor,
.oneinit = gf100_fb_oneinit,
- .init = gp100_fb_init,
+ .init = gm200_fb_init,
.init_page = gv100_fb_init_page,
.init_unkn = gp100_fb_init_unkn,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.ram_new = ga102_ram_new,
.default_bigpage = 16,
+ .vpr.scrub_required = ga102_fb_vpr_scrub_required,
+ .vpr.scrub = ga102_fb_vpr_scrub,
};
int
@@ -38,3 +65,9 @@ ga102_fb_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, s
{
return gp102_fb_new_(&ga102_fb, device, type, inst, pfb);
}
+
+MODULE_FIRMWARE("nvidia/ga102/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/ga103/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/ga104/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/ga106/nvdec/scrubber.bin");
+MODULE_FIRMWARE("nvidia/ga107/nvdec/scrubber.bin");
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c
index 9dcc40f..07db9b3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c
@@ -61,14 +61,6 @@ gf100_fb_oneinit(struct nvkm_fb *base)
if (ret)
return ret;
- fb->r100c10_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
- if (fb->r100c10_page) {
- fb->r100c10 = dma_map_page(device->dev, fb->r100c10_page, 0,
- PAGE_SIZE, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(device->dev, fb->r100c10))
- return -EFAULT;
- }
-
return 0;
}
@@ -86,14 +78,17 @@ gf100_fb_init_page(struct nvkm_fb *fb)
}
void
+gf100_fb_sysmem_flush_page_init(struct nvkm_fb *fb)
+{
+ nvkm_wr32(fb->subdev.device, 0x100c10, fb->sysmem.flush_page_addr >> 8);
+}
+
+void
gf100_fb_init(struct nvkm_fb *base)
{
struct gf100_fb *fb = gf100_fb(base);
struct nvkm_device *device = fb->base.subdev.device;
- if (fb->r100c10_page)
- nvkm_wr32(device, 0x100c10, fb->r100c10 >> 8);
-
if (base->func->clkgate_pack) {
nvkm_therm_clkgate_init(device->therm,
base->func->clkgate_pack);
@@ -104,13 +99,6 @@ void *
gf100_fb_dtor(struct nvkm_fb *base)
{
struct gf100_fb *fb = gf100_fb(base);
- struct nvkm_device *device = fb->base.subdev.device;
-
- if (fb->r100c10_page) {
- dma_unmap_page(device->dev, fb->r100c10, PAGE_SIZE,
- DMA_BIDIRECTIONAL);
- __free_page(fb->r100c10_page);
- }
return fb;
}
@@ -136,6 +124,7 @@ gf100_fb = {
.init = gf100_fb_init,
.init_page = gf100_fb_init_page,
.intr = gf100_fb_intr,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.ram_new = gf100_ram_new,
.default_bigpage = 17,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.h b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.h
index 0cac7b06..77472b5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.h
@@ -6,8 +6,6 @@
struct gf100_fb {
struct nvkm_fb base;
- struct page *r100c10_page;
- dma_addr_t r100c10;
};
int gf100_fb_new_(const struct nvkm_fb_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
@@ -16,7 +14,5 @@ void *gf100_fb_dtor(struct nvkm_fb *);
void gf100_fb_init(struct nvkm_fb *);
void gf100_fb_intr(struct nvkm_fb *);
-void gp100_fb_init(struct nvkm_fb *);
-
void gm200_fb_init(struct nvkm_fb *base);
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.c
index 5acf8d15..fb02092 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.c
@@ -46,9 +46,6 @@ gm200_fb_init(struct nvkm_fb *base)
struct gf100_fb *fb = gf100_fb(base);
struct nvkm_device *device = fb->base.subdev.device;
- if (fb->r100c10_page)
- nvkm_wr32(device, 0x100c10, fb->r100c10 >> 8);
-
nvkm_wr32(device, 0x100cc8, nvkm_memory_addr(fb->base.mmu_wr) >> 8);
nvkm_wr32(device, 0x100ccc, nvkm_memory_addr(fb->base.mmu_rd) >> 8);
nvkm_mask(device, 0x100cc4, 0x00060000,
@@ -62,6 +59,7 @@ gm200_fb = {
.init = gm200_fb_init,
.init_page = gm200_fb_init_page,
.intr = gf100_fb_intr,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.ram_new = gm200_ram_new,
.default_bigpage = 0 /* per-instance. */,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm20b.c
index 86f61a3..50875af 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm20b.c
@@ -30,6 +30,7 @@ gm20b_fb = {
.init = gm200_fb_init,
.init_page = gm200_fb_init_page,
.intr = gf100_fb_intr,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.default_bigpage = 0 /* per-instance. */,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp100.c
index 09e943e..110c08c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp100.c
@@ -44,29 +44,15 @@ gp100_fb_init_remapper(struct nvkm_fb *fb)
nvkm_mask(device, 0x100c14, 0x00040000, 0x00000000);
}
-void
-gp100_fb_init(struct nvkm_fb *base)
-{
- struct gf100_fb *fb = gf100_fb(base);
- struct nvkm_device *device = fb->base.subdev.device;
-
- if (fb->r100c10_page)
- nvkm_wr32(device, 0x100c10, fb->r100c10 >> 8);
-
- nvkm_wr32(device, 0x100cc8, nvkm_memory_addr(fb->base.mmu_wr) >> 8);
- nvkm_wr32(device, 0x100ccc, nvkm_memory_addr(fb->base.mmu_rd) >> 8);
- nvkm_mask(device, 0x100cc4, 0x00060000,
- min(nvkm_memory_size(fb->base.mmu_rd) >> 16, (u64)2) << 17);
-}
-
static const struct nvkm_fb_func
gp100_fb = {
.dtor = gf100_fb_dtor,
.oneinit = gf100_fb_oneinit,
- .init = gp100_fb_init,
+ .init = gm200_fb_init,
.init_remapper = gp100_fb_init_remapper,
.init_page = gm200_fb_init_page,
.init_unkn = gp100_fb_init_unkn,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.ram_new = gp100_ram_new,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp102.c
index 0e78b3d..2658481 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp102.c
@@ -24,71 +24,22 @@
#include "gf100.h"
#include "ram.h"
-#include <core/firmware.h>
-#include <core/memory.h>
-#include <nvfw/fw.h>
-#include <nvfw/hs.h>
#include <engine/nvdec.h>
int
gp102_fb_vpr_scrub(struct nvkm_fb *fb)
{
struct nvkm_subdev *subdev = &fb->subdev;
- struct nvkm_device *device = subdev->device;
- struct nvkm_falcon *falcon = &device->nvdec[0]->falcon;
- struct nvkm_blob *blob = &fb->vpr_scrubber;
- const struct nvfw_bin_hdr *hsbin_hdr;
- const struct nvfw_hs_header *fw_hdr;
- const struct nvfw_hs_load_header *lhdr;
- void *scrub_data;
- u32 patch_loc, patch_sig;
+ struct nvkm_falcon_fw fw = {};
int ret;
- nvkm_falcon_get(falcon, subdev);
+ ret = nvkm_falcon_fw_ctor_hs(&gm200_flcn_fw, "mem-unlock", subdev, NULL,
+ "nvdec/scrubber", 0, &subdev->device->nvdec[0]->falcon, &fw);
+ if (ret)
+ return ret;
- hsbin_hdr = nvfw_bin_hdr(subdev, blob->data);
- fw_hdr = nvfw_hs_header(subdev, blob->data + hsbin_hdr->header_offset);
- lhdr = nvfw_hs_load_header(subdev, blob->data + fw_hdr->hdr_offset);
- scrub_data = blob->data + hsbin_hdr->data_offset;
-
- patch_loc = *(u32 *)(blob->data + fw_hdr->patch_loc);
- patch_sig = *(u32 *)(blob->data + fw_hdr->patch_sig);
- if (falcon->debug) {
- memcpy(scrub_data + patch_loc,
- blob->data + fw_hdr->sig_dbg_offset + patch_sig,
- fw_hdr->sig_dbg_size);
- } else {
- memcpy(scrub_data + patch_loc,
- blob->data + fw_hdr->sig_prod_offset + patch_sig,
- fw_hdr->sig_prod_size);
- }
-
- nvkm_falcon_reset(falcon);
- nvkm_falcon_bind_context(falcon, NULL);
-
- nvkm_falcon_load_imem(falcon, scrub_data, lhdr->non_sec_code_off,
- lhdr->non_sec_code_size,
- lhdr->non_sec_code_off >> 8, 0, false);
- nvkm_falcon_load_imem(falcon, scrub_data + lhdr->apps[0],
- ALIGN(lhdr->apps[0], 0x100),
- lhdr->apps[1],
- lhdr->apps[0] >> 8, 0, true);
- nvkm_falcon_load_dmem(falcon, scrub_data + lhdr->data_dma_base, 0,
- lhdr->data_size, 0);
-
- nvkm_falcon_set_start_addr(falcon, 0x0);
- nvkm_falcon_start(falcon);
-
- ret = nvkm_falcon_wait_for_halt(falcon, 500);
- if (ret < 0) {
- ret = -ETIMEDOUT;
- goto end;
- }
-
- /* put nvdec in clean state - without reset it will remain in HS mode */
- nvkm_falcon_reset(falcon);
-end:
- nvkm_falcon_put(falcon, subdev);
+ ret = nvkm_falcon_fw_boot(&fw, subdev, true, NULL, NULL, 0, 0x00000000);
+ nvkm_falcon_fw_dtor(&fw);
return ret;
}
@@ -104,9 +55,10 @@ static const struct nvkm_fb_func
gp102_fb = {
.dtor = gf100_fb_dtor,
.oneinit = gf100_fb_oneinit,
- .init = gp100_fb_init,
+ .init = gm200_fb_init,
.init_remapper = gp100_fb_init_remapper,
.init_page = gm200_fb_init_page,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.vpr.scrub_required = gp102_fb_vpr_scrub_required,
.vpr.scrub = gp102_fb_vpr_scrub,
.ram_new = gp100_ram_new,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp10b.c
index 84c9815..a04a5f7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp10b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp10b.c
@@ -28,6 +28,7 @@ gp10b_fb = {
.init = gm200_fb_init,
.init_page = gm200_fb_init_page,
.intr = gf100_fb_intr,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gv100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gv100.c
index 63daa83..1f01264 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gv100.c
@@ -32,9 +32,10 @@ static const struct nvkm_fb_func
gv100_fb = {
.dtor = gf100_fb_dtor,
.oneinit = gf100_fb_oneinit,
- .init = gp100_fb_init,
+ .init = gm200_fb_init,
.init_page = gv100_fb_init_page,
.init_unkn = gp100_fb_init_unkn,
+ .sysmem.flush_page_init = gf100_fb_sysmem_flush_page_init,
.vpr.scrub_required = gp102_fb_vpr_scrub_required,
.vpr.scrub = gp102_fb_vpr_scrub,
.ram_new = gp100_ram_new,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.c
index 95fd8f8..a6efbd9 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.c
@@ -137,8 +137,7 @@ nv50_fb_intr(struct nvkm_fb *base)
struct nv50_fb *fb = nv50_fb(base);
struct nvkm_subdev *subdev = &fb->base.subdev;
struct nvkm_device *device = subdev->device;
- struct nvkm_fifo *fifo = device->fifo;
- struct nvkm_fifo_chan *chan;
+ struct nvkm_chan *chan;
const struct nvkm_enum *en, *re, *cl, *sc;
u32 trap[6], idx, inst;
u8 st0, st1, st2, st3;
@@ -178,35 +177,18 @@ nv50_fb_intr(struct nvkm_fb *base)
else if (en && en->data) sc = nvkm_enum_find(en->data, st3);
else sc = NULL;
- chan = nvkm_fifo_chan_inst(fifo, inst, &flags);
+ chan = nvkm_chan_get_inst(&device->fifo->engine, inst, &flags);
nvkm_error(subdev, "trapped %s at %02x%04x%04x on channel %d [%08x %s] "
"engine %02x [%s] client %02x [%s] "
"subclient %02x [%s] reason %08x [%s]\n",
(trap[5] & 0x00000100) ? "read" : "write",
trap[5] & 0xff, trap[4] & 0xffff, trap[3] & 0xffff,
- chan ? chan->chid : -1, inst,
- chan ? chan->object.client->name : "unknown",
+ chan ? chan->id : -1, inst,
+ chan ? chan->name : "unknown",
st0, en ? en->name : "",
st2, cl ? cl->name : "", st3, sc ? sc->name : "",
st1, re ? re->name : "");
- nvkm_fifo_chan_put(fifo, flags, &chan);
-}
-
-static int
-nv50_fb_oneinit(struct nvkm_fb *base)
-{
- struct nv50_fb *fb = nv50_fb(base);
- struct nvkm_device *device = fb->base.subdev.device;
-
- fb->r100c08_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
- if (fb->r100c08_page) {
- fb->r100c08 = dma_map_page(device->dev, fb->r100c08_page, 0,
- PAGE_SIZE, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(device->dev, fb->r100c08))
- return -EFAULT;
- }
-
- return 0;
+ nvkm_chan_put(&chan, flags);
}
static void
@@ -215,12 +197,6 @@ nv50_fb_init(struct nvkm_fb *base)
struct nv50_fb *fb = nv50_fb(base);
struct nvkm_device *device = fb->base.subdev.device;
- /* Not a clue what this is exactly. Without pointing it at a
- * scratch page, VRAM->GART blits with M2MF (as in DDX DFS)
- * cause IOMMU "read from address 0" errors (rh#561267)
- */
- nvkm_wr32(device, 0x100c08, fb->r100c08 >> 8);
-
/* This is needed to get meaningful information from 100c90
* on traps. No idea what these values mean exactly. */
nvkm_wr32(device, 0x100c90, fb->func->trap);
@@ -235,17 +211,16 @@ nv50_fb_tags(struct nvkm_fb *base)
return 0;
}
+static void
+nv50_fb_sysmem_flush_page_init(struct nvkm_fb *fb)
+{
+ nvkm_wr32(fb->subdev.device, 0x100c08, fb->sysmem.flush_page_addr >> 8);
+}
+
static void *
nv50_fb_dtor(struct nvkm_fb *base)
{
struct nv50_fb *fb = nv50_fb(base);
- struct nvkm_device *device = fb->base.subdev.device;
-
- if (fb->r100c08_page) {
- dma_unmap_page(device->dev, fb->r100c08, PAGE_SIZE,
- DMA_BIDIRECTIONAL);
- __free_page(fb->r100c08_page);
- }
return fb;
}
@@ -254,9 +229,9 @@ static const struct nvkm_fb_func
nv50_fb_ = {
.dtor = nv50_fb_dtor,
.tags = nv50_fb_tags,
- .oneinit = nv50_fb_oneinit,
.init = nv50_fb_init,
.intr = nv50_fb_intr,
+ .sysmem.flush_page_init = nv50_fb_sysmem_flush_page_init,
.ram_new = nv50_fb_ram_new,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.h b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.h
index a5e6738..4f68bc4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.h
@@ -7,8 +7,6 @@
struct nv50_fb {
const struct nv50_fb_func *func;
struct nvkm_fb base;
- struct page *r100c08_page;
- dma_addr_t r100c08;
};
struct nv50_fb_func {
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h
index 3f1be97..ac03eac 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h
@@ -16,6 +16,10 @@ struct nvkm_fb_func {
void (*init_unkn)(struct nvkm_fb *);
void (*intr)(struct nvkm_fb *);
+ struct nvkm_fb_func_sysmem {
+ void (*flush_page_init)(struct nvkm_fb *);
+ } sysmem;
+
struct {
bool (*scrub_required)(struct nvkm_fb *);
int (*scrub)(struct nvkm_fb *);
@@ -37,8 +41,8 @@ struct nvkm_fb_func {
const struct nvkm_therm_clkgate_pack *clkgate_pack;
};
-void nvkm_fb_ctor(const struct nvkm_fb_func *, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_fb *);
+int nvkm_fb_ctor(const struct nvkm_fb_func *, struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_fb *);
int nvkm_fb_new_(const struct nvkm_fb_func *, struct nvkm_device *device,
enum nvkm_subdev_type type, int inst, struct nvkm_fb **);
int nvkm_fb_bios_memtype(struct nvkm_bios *);
@@ -72,6 +76,7 @@ void nv46_fb_tile_init(struct nvkm_fb *, int i, u32 addr, u32 size,
int gf100_fb_oneinit(struct nvkm_fb *);
int gf100_fb_init_page(struct nvkm_fb *);
+void gf100_fb_sysmem_flush_page_init(struct nvkm_fb *);
int gm200_fb_init_page(struct nvkm_fb *);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ram.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ram.c
index 03b1bdb..5c34416 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ram.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ram.c
@@ -25,6 +25,7 @@
#include "ram.h"
#include <core/memory.h>
+#include <subdev/instmem.h>
#include <subdev/mmu.h>
struct nvkm_vram {
@@ -35,6 +36,12 @@ struct nvkm_vram {
};
static int
+nvkm_vram_kmap(struct nvkm_memory *memory, struct nvkm_memory **pmemory)
+{
+ return nvkm_instobj_wrap(nvkm_vram(memory)->ram->fb->subdev.device, memory, pmemory);
+}
+
+static int
nvkm_vram_map(struct nvkm_memory *memory, u64 offset, struct nvkm_vmm *vmm,
struct nvkm_vma *vma, void *argv, u32 argc)
{
@@ -98,6 +105,7 @@ nvkm_vram = {
.addr = nvkm_vram_addr,
.size = nvkm_vram_size,
.map = nvkm_vram_map,
+ .kmap = nvkm_vram_kmap,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c
index 048bcc7..b196baa 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c
@@ -24,7 +24,6 @@
#include "priv.h"
#include <core/option.h>
-#include <core/notify.h>
static int
nvkm_gpio_drive(struct nvkm_gpio *gpio, int idx, int line, int dir, int out)
@@ -123,23 +122,8 @@ nvkm_gpio_intr_init(struct nvkm_event *event, int type, int index)
gpio->func->intr_mask(gpio, type, 1 << index, 1 << index);
}
-static int
-nvkm_gpio_intr_ctor(struct nvkm_object *object, void *data, u32 size,
- struct nvkm_notify *notify)
-{
- struct nvkm_gpio_ntfy_req *req = data;
- if (!WARN_ON(size != sizeof(*req))) {
- notify->size = sizeof(struct nvkm_gpio_ntfy_rep);
- notify->types = req->mask;
- notify->index = req->line;
- return 0;
- }
- return -EINVAL;
-}
-
static const struct nvkm_event_func
nvkm_gpio_intr_func = {
- .ctor = nvkm_gpio_intr_ctor,
.init = nvkm_gpio_intr_init,
.fini = nvkm_gpio_intr_fini,
};
@@ -153,11 +137,9 @@ nvkm_gpio_intr(struct nvkm_subdev *subdev)
gpio->func->intr_stat(gpio, &hi, &lo);
for (i = 0; (hi | lo) && i < gpio->func->lines; i++) {
- struct nvkm_gpio_ntfy_rep rep = {
- .mask = (NVKM_GPIO_HI * !!(hi & (1 << i))) |
- (NVKM_GPIO_LO * !!(lo & (1 << i))),
- };
- nvkm_event_send(&gpio->event, rep.mask, i, &rep, sizeof(rep));
+ u32 mask = (NVKM_GPIO_HI * !!(hi & (1 << i))) |
+ (NVKM_GPIO_LO * !!(lo & (1 << i)));
+ nvkm_event_ntfy(&gpio->event, i, mask);
}
}
@@ -251,6 +233,5 @@ nvkm_gpio_new_(const struct nvkm_gpio_func *func, struct nvkm_device *device,
nvkm_subdev_ctor(&nvkm_gpio, device, type, inst, &gpio->subdev);
gpio->func = func;
- return nvkm_event_init(&nvkm_gpio_intr_func, 2, func->lines,
- &gpio->event);
+ return nvkm_event_init(&nvkm_gpio_intr_func, &gpio->subdev, 2, func->lines, &gpio->event);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/Kbuild
index 67cc3b3..7f61a1ed 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/Kbuild
@@ -1,3 +1,4 @@
# SPDX-License-Identifier: MIT
nvkm-y += nvkm/subdev/gsp/base.o
nvkm-y += nvkm/subdev/gsp/gv100.o
+nvkm-y += nvkm/subdev/gsp/ga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c
index 2257488..591ac95 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c
@@ -53,5 +53,7 @@ nvkm_gsp_new_(const struct nvkm_gsp_fwif *fwif, struct nvkm_device *device,
if (IS_ERR(fwif))
return PTR_ERR(fwif);
- return nvkm_falcon_ctor(fwif->flcn, &gsp->subdev, gsp->subdev.name, 0, &gsp->falcon);
+ gsp->func = fwif->func;
+
+ return nvkm_falcon_ctor(gsp->func->flcn, &gsp->subdev, gsp->subdev.name, 0, &gsp->falcon);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/ga102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/ga102.c
new file mode 100644
index 0000000..5252674
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/ga102.c
@@ -0,0 +1,59 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+static const struct nvkm_falcon_func
+ga102_gsp_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .select = ga102_flcn_select,
+ .addr2 = 0x1000,
+ .reset_eng = gp102_flcn_reset_eng,
+ .reset_prep = ga102_flcn_reset_prep,
+ .reset_wait_mem_scrubbing = ga102_flcn_reset_wait_mem_scrubbing,
+ .imem_dma = &ga102_flcn_dma,
+ .dmem_dma = &ga102_flcn_dma,
+};
+
+static const struct nvkm_gsp_func
+ga102_gsp = {
+ .flcn = &ga102_gsp_flcn,
+};
+
+static int
+ga102_gsp_nofw(struct nvkm_gsp *gsp, int ver, const struct nvkm_gsp_fwif *fwif)
+{
+ return 0;
+}
+
+struct nvkm_gsp_fwif
+ga102_gsps[] = {
+ { -1, ga102_gsp_nofw, &ga102_gsp },
+ {}
+};
+
+int
+ga102_gsp_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_gsp **pgsp)
+{
+ return nvkm_gsp_new_(ga102_gsps, device, type, inst, pgsp);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/gv100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/gv100.c
index 6c4ef62a..da6a809 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/gv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/gv100.c
@@ -23,17 +23,20 @@
static const struct nvkm_falcon_func
gv100_gsp_flcn = {
- .fbif = 0x600,
- .load_imem = nvkm_falcon_v1_load_imem,
- .load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
- .bind_context = gp102_sec2_flcn_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
- .start = nvkm_falcon_v1_start,
- .enable = gp102_sec2_flcn_enable,
- .disable = nvkm_falcon_v1_disable,
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .reset_eng = gp102_flcn_reset_eng,
+ .reset_wait_mem_scrubbing = gm200_flcn_reset_wait_mem_scrubbing,
+ .bind_inst = gm200_flcn_bind_inst,
+ .bind_stat = gm200_flcn_bind_stat,
+ .bind_intr = true,
+ .imem_pio = &gm200_flcn_imem_pio,
+ .dmem_pio = &gm200_flcn_dmem_pio,
+};
+
+static const struct nvkm_gsp_func
+gv100_gsp = {
+ .flcn = &gv100_gsp_flcn,
};
static int
@@ -43,8 +46,8 @@ gv100_gsp_nofw(struct nvkm_gsp *gsp, int ver, const struct nvkm_gsp_fwif *fwif)
}
static struct nvkm_gsp_fwif
-gv100_gsp[] = {
- { -1, gv100_gsp_nofw, &gv100_gsp_flcn },
+gv100_gsps[] = {
+ { -1, gv100_gsp_nofw, &gv100_gsp },
{}
};
@@ -52,5 +55,5 @@ int
gv100_gsp_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
struct nvkm_gsp **pgsp)
{
- return nvkm_gsp_new_(gv100_gsp, device, type, inst, pgsp);
+ return nvkm_gsp_new_(gv100_gsps, device, type, inst, pgsp);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h
index 19381dd..89749a4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h
@@ -4,10 +4,14 @@
#include <subdev/gsp.h>
enum nvkm_acr_lsf_id;
+struct nvkm_gsp_func {
+ const struct nvkm_falcon_func *flcn;
+};
+
struct nvkm_gsp_fwif {
int version;
int (*load)(struct nvkm_gsp *, int ver, const struct nvkm_gsp_fwif *);
- const struct nvkm_falcon_func *flcn;
+ const struct nvkm_gsp_func *func;
};
int nvkm_gsp_new_(const struct nvkm_gsp_fwif *, struct nvkm_device *, enum nvkm_subdev_type, int,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c
index cb5cb53..976539d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c
@@ -26,7 +26,6 @@
#include "bus.h"
#include "pad.h"
-#include <core/notify.h>
#include <core/option.h>
#include <subdev/bios.h>
#include <subdev/bios/dcb.h>
@@ -104,23 +103,8 @@ nvkm_i2c_intr_init(struct nvkm_event *event, int type, int id)
i2c->func->aux_mask(i2c, type, aux->intr, aux->intr);
}
-static int
-nvkm_i2c_intr_ctor(struct nvkm_object *object, void *data, u32 size,
- struct nvkm_notify *notify)
-{
- struct nvkm_i2c_ntfy_req *req = data;
- if (!WARN_ON(size != sizeof(*req))) {
- notify->size = sizeof(struct nvkm_i2c_ntfy_rep);
- notify->types = req->mask;
- notify->index = req->port;
- return 0;
- }
- return -EINVAL;
-}
-
static const struct nvkm_event_func
nvkm_i2c_intr_func = {
- .ctor = nvkm_i2c_intr_ctor,
.init = nvkm_i2c_intr_init,
.fini = nvkm_i2c_intr_fini,
};
@@ -145,13 +129,8 @@ nvkm_i2c_intr(struct nvkm_subdev *subdev)
if (lo & aux->intr) mask |= NVKM_I2C_UNPLUG;
if (rq & aux->intr) mask |= NVKM_I2C_IRQ;
if (tx & aux->intr) mask |= NVKM_I2C_DONE;
- if (mask) {
- struct nvkm_i2c_ntfy_rep rep = {
- .mask = mask,
- };
- nvkm_event_send(&i2c->event, rep.mask, aux->id,
- &rep, sizeof(rep));
- }
+ if (mask)
+ nvkm_event_ntfy(&i2c->event, aux->id, mask);
}
}
@@ -427,5 +406,5 @@ nvkm_i2c_new_(const struct nvkm_i2c_func *func, struct nvkm_device *device,
}
}
- return nvkm_event_init(&nvkm_i2c_intr_func, 4, i, &i2c->event);
+ return nvkm_event_init(&nvkm_i2c_intr_func, &i2c->subdev, 4, i, &i2c->event);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c
index cd8163a..e0e4f97 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c
@@ -90,6 +90,18 @@ nvkm_instobj_ctor(const struct nvkm_memory_func *func,
}
int
+nvkm_instobj_wrap(struct nvkm_device *device,
+ struct nvkm_memory *memory, struct nvkm_memory **pmemory)
+{
+ struct nvkm_instmem *imem = device->imem;
+
+ if (!imem->func->memory_wrap)
+ return -ENOSYS;
+
+ return imem->func->memory_wrap(imem, memory, pmemory);
+}
+
+int
nvkm_instobj_new(struct nvkm_instmem *imem, u32 size, u32 align, bool zero,
struct nvkm_memory **pmemory)
{
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c
index c51bac7..4b2d746 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c
@@ -348,13 +348,11 @@ nv50_instobj_func = {
};
static int
-nv50_instobj_new(struct nvkm_instmem *base, u32 size, u32 align, bool zero,
- struct nvkm_memory **pmemory)
+nv50_instobj_wrap(struct nvkm_instmem *base,
+ struct nvkm_memory *memory, struct nvkm_memory **pmemory)
{
struct nv50_instmem *imem = nv50_instmem(base);
struct nv50_instobj *iobj;
- struct nvkm_device *device = imem->base.subdev.device;
- u8 page = max(order_base_2(align), 12);
if (!(iobj = kzalloc(sizeof(*iobj), GFP_KERNEL)))
return -ENOMEM;
@@ -365,7 +363,25 @@ nv50_instobj_new(struct nvkm_instmem *base, u32 size, u32 align, bool zero,
refcount_set(&iobj->maps, 0);
INIT_LIST_HEAD(&iobj->lru);
- return nvkm_ram_get(device, 0, 1, page, size, true, true, &iobj->ram);
+ iobj->ram = nvkm_memory_ref(memory);
+ return 0;
+}
+
+static int
+nv50_instobj_new(struct nvkm_instmem *imem, u32 size, u32 align, bool zero,
+ struct nvkm_memory **pmemory)
+{
+ u8 page = max(order_base_2(align), 12);
+ struct nvkm_memory *ram;
+ int ret;
+
+ ret = nvkm_ram_get(imem->subdev.device, 0, 1, page, size, true, true, &ram);
+ if (ret)
+ return ret;
+
+ ret = nv50_instobj_wrap(imem, ram, pmemory);
+ nvkm_memory_unref(&ram);
+ return ret;
}
/******************************************************************************
@@ -382,6 +398,7 @@ static const struct nvkm_instmem_func
nv50_instmem = {
.fini = nv50_instmem_fini,
.memory_new = nv50_instobj_new,
+ .memory_wrap = nv50_instobj_wrap,
.zero = false,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/priv.h
index 56c15e3..fe92986 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/priv.h
@@ -12,6 +12,7 @@ struct nvkm_instmem_func {
void (*wr32)(struct nvkm_instmem *, u32 addr, u32 data);
int (*memory_new)(struct nvkm_instmem *, u32 size, u32 align,
bool zero, struct nvkm_memory **);
+ int (*memory_wrap)(struct nvkm_instmem *, struct nvkm_memory *, struct nvkm_memory **);
bool zero;
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/Kbuild
index 728d750..0d8a915 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/Kbuild
@@ -7,3 +7,4 @@
nvkm-y += nvkm/subdev/ltc/gp100.o
nvkm-y += nvkm/subdev/ltc/gp102.o
nvkm-y += nvkm/subdev/ltc/gp10b.o
+nvkm-y += nvkm/subdev/ltc/ga102.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c
index fa683c19..f742a7b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c
@@ -97,8 +97,10 @@ nvkm_ltc_init(struct nvkm_subdev *subdev)
struct nvkm_ltc *ltc = nvkm_ltc(subdev);
int i;
- for (i = ltc->zbc_min; i <= ltc->zbc_max; i++) {
+ for (i = ltc->zbc_color_min; i <= ltc->zbc_color_max; i++)
ltc->func->zbc_clear_color(ltc, i, ltc->zbc_color[i]);
+
+ for (i = ltc->zbc_depth_min; i <= ltc->zbc_depth_max; i++) {
ltc->func->zbc_clear_depth(ltc, i, ltc->zbc_depth[i]);
if (ltc->func->zbc_clear_stencil)
ltc->func->zbc_clear_stencil(ltc, i, ltc->zbc_stencil[i]);
@@ -137,7 +139,9 @@ nvkm_ltc_new_(const struct nvkm_ltc_func *func, struct nvkm_device *device,
nvkm_subdev_ctor(&nvkm_ltc, device, type, inst, <c->subdev);
ltc->func = func;
mutex_init(<c->mutex);
- ltc->zbc_min = 1; /* reserve 0 for disabled */
- ltc->zbc_max = min(func->zbc, NVKM_LTC_MAX_ZBC_CNT) - 1;
+ ltc->zbc_color_min = 1; /* reserve 0 for disabled */
+ ltc->zbc_color_max = min(func->zbc_color, NVKM_LTC_MAX_ZBC_COLOR_CNT) - 1;
+ ltc->zbc_depth_min = 1; /* reserve 0 for disabled */
+ ltc->zbc_depth_max = min(func->zbc_depth, NVKM_LTC_MAX_ZBC_DEPTH_CNT) - 1;
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/ga102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/ga102.c
new file mode 100644
index 0000000..159d9f8
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/ga102.c
@@ -0,0 +1,57 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+static void
+ga102_ltc_zbc_clear_color(struct nvkm_ltc *ltc, int i, const u32 color[4])
+{
+ struct nvkm_device *device = ltc->subdev.device;
+
+ nvkm_mask(device, 0x17e338, 0x0000001f, i);
+ nvkm_wr32(device, 0x17e33c, color[0]);
+ nvkm_wr32(device, 0x17e340, color[1]);
+ nvkm_wr32(device, 0x17e344, color[2]);
+ nvkm_wr32(device, 0x17e348, color[3]);
+}
+
+static const struct nvkm_ltc_func
+ga102_ltc = {
+ .oneinit = gp100_ltc_oneinit,
+ .init = gp100_ltc_init,
+ .intr = gp100_ltc_intr,
+ .cbc_clear = gm107_ltc_cbc_clear,
+ .cbc_wait = gm107_ltc_cbc_wait,
+ .zbc_color = 31,
+ .zbc_depth = 16,
+ .zbc_clear_color = ga102_ltc_zbc_clear_color,
+ .zbc_clear_depth = gm107_ltc_zbc_clear_depth,
+ .zbc_clear_stencil = gp102_ltc_zbc_clear_stencil,
+ .invalidate = gf100_ltc_invalidate,
+ .flush = gf100_ltc_flush,
+};
+
+int
+ga102_ltc_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_ltc **pltc)
+{
+ return nvkm_ltc_new_(&ga102_ltc, device, type, inst, pltc);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gf100.c
index fd8aeaf..de71ba3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gf100.c
@@ -241,7 +241,8 @@ gf100_ltc = {
.intr = gf100_ltc_intr,
.cbc_clear = gf100_ltc_cbc_clear,
.cbc_wait = gf100_ltc_cbc_wait,
- .zbc = 16,
+ .zbc_color = 16,
+ .zbc_depth = 16,
.zbc_clear_color = gf100_ltc_zbc_clear_color,
.zbc_clear_depth = gf100_ltc_zbc_clear_depth,
.invalidate = gf100_ltc_invalidate,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gk104.c
index 94aa092..5d61e3c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gk104.c
@@ -42,7 +42,8 @@ gk104_ltc = {
.intr = gf100_ltc_intr,
.cbc_clear = gf100_ltc_cbc_clear,
.cbc_wait = gf100_ltc_cbc_wait,
- .zbc = 16,
+ .zbc_color = 16,
+ .zbc_depth = 16,
.zbc_clear_color = gf100_ltc_zbc_clear_color,
.zbc_clear_depth = gf100_ltc_zbc_clear_depth,
.invalidate = gf100_ltc_invalidate,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm107.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm107.c
index 54d1d65..18685d84 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm107.c
@@ -137,7 +137,8 @@ gm107_ltc = {
.intr = gm107_ltc_intr,
.cbc_clear = gm107_ltc_cbc_clear,
.cbc_wait = gm107_ltc_cbc_wait,
- .zbc = 16,
+ .zbc_color = 16,
+ .zbc_depth = 16,
.zbc_clear_color = gm107_ltc_zbc_clear_color,
.zbc_clear_depth = gm107_ltc_zbc_clear_depth,
.invalidate = gf100_ltc_invalidate,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm200.c
index 8cfdbbd..7a9464b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm200.c
@@ -49,7 +49,8 @@ gm200_ltc = {
.intr = gm107_ltc_intr,
.cbc_clear = gm107_ltc_cbc_clear,
.cbc_wait = gm107_ltc_cbc_wait,
- .zbc = 16,
+ .zbc_color = 16,
+ .zbc_depth = 16,
.zbc_clear_color = gm107_ltc_zbc_clear_color,
.zbc_clear_depth = gm107_ltc_zbc_clear_depth,
.invalidate = gf100_ltc_invalidate,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp100.c
index a4a6cd9..1a17a451 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp100.c
@@ -61,7 +61,8 @@ gp100_ltc = {
.intr = gp100_ltc_intr,
.cbc_clear = gm107_ltc_cbc_clear,
.cbc_wait = gm107_ltc_cbc_wait,
- .zbc = 16,
+ .zbc_color = 16,
+ .zbc_depth = 16,
.zbc_clear_color = gm107_ltc_zbc_clear_color,
.zbc_clear_depth = gm107_ltc_zbc_clear_depth,
.invalidate = gf100_ltc_invalidate,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp102.c
index ff05d61..265a05f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp102.c
@@ -36,7 +36,8 @@ gp102_ltc = {
.intr = gp100_ltc_intr,
.cbc_clear = gm107_ltc_cbc_clear,
.cbc_wait = gm107_ltc_cbc_wait,
- .zbc = 16,
+ .zbc_color = 16,
+ .zbc_depth = 16,
.zbc_clear_color = gm107_ltc_zbc_clear_color,
.zbc_clear_depth = gm107_ltc_zbc_clear_depth,
.zbc_clear_stencil = gp102_ltc_zbc_clear_stencil,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp10b.c
index dfebd79..e7e8fdf 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp10b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp10b.c
@@ -50,7 +50,8 @@ gp10b_ltc = {
.intr = gp100_ltc_intr,
.cbc_clear = gm107_ltc_cbc_clear,
.cbc_wait = gm107_ltc_cbc_wait,
- .zbc = 16,
+ .zbc_color = 16,
+ .zbc_depth = 16,
.zbc_clear_color = gm107_ltc_zbc_clear_color,
.zbc_clear_depth = gm107_ltc_zbc_clear_depth,
.zbc_clear_stencil = gp102_ltc_zbc_clear_stencil,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/priv.h
index 2bebe13..134e90c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/priv.h
@@ -16,7 +16,8 @@ struct nvkm_ltc_func {
void (*cbc_clear)(struct nvkm_ltc *, u32 start, u32 limit);
void (*cbc_wait)(struct nvkm_ltc *);
- int zbc;
+ int zbc_color;
+ int zbc_depth;
void (*zbc_clear_color)(struct nvkm_ltc *, int, const u32[4]);
void (*zbc_clear_depth)(struct nvkm_ltc *, int, const u32);
void (*zbc_clear_stencil)(struct nvkm_ltc *, int, const u32);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild
index ac2b34e..2a3255c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild
@@ -13,5 +13,4 @@
nvkm-y += nvkm/subdev/mc/gk20a.o
nvkm-y += nvkm/subdev/mc/gp100.o
nvkm-y += nvkm/subdev/mc/gp10b.o
-nvkm-y += nvkm/subdev/mc/tu102.o
nvkm-y += nvkm/subdev/mc/ga100.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c
index 21c4af3..c85600b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c
@@ -37,86 +37,16 @@ nvkm_mc_unk260(struct nvkm_device *device, u32 data)
void
nvkm_mc_intr_mask(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, bool en)
{
- struct nvkm_mc *mc = device->mc;
- const struct nvkm_mc_map *map;
- if (likely(mc) && mc->func->intr_mask) {
- u32 mask = nvkm_top_intr_mask(device, type, inst);
- for (map = mc->func->intr; !mask && map->stat; map++) {
- if (map->type == type && map->inst == inst)
- mask = map->stat;
- }
- mc->func->intr_mask(mc, mask, en ? mask : 0);
+ struct nvkm_subdev *subdev = nvkm_device_subdev(device, type, inst);
+
+ if (subdev) {
+ if (en)
+ nvkm_intr_allow(subdev, NVKM_INTR_SUBDEV);
+ else
+ nvkm_intr_block(subdev, NVKM_INTR_SUBDEV);
}
}
-void
-nvkm_mc_intr_unarm(struct nvkm_device *device)
-{
- struct nvkm_mc *mc = device->mc;
- if (likely(mc))
- mc->func->intr_unarm(mc);
-}
-
-void
-nvkm_mc_intr_rearm(struct nvkm_device *device)
-{
- struct nvkm_mc *mc = device->mc;
- if (likely(mc))
- mc->func->intr_rearm(mc);
-}
-
-static u32
-nvkm_mc_intr_stat(struct nvkm_mc *mc)
-{
- u32 intr = mc->func->intr_stat(mc);
- if (WARN_ON_ONCE(intr == 0xffffffff))
- intr = 0; /* likely fallen off the bus */
- return intr;
-}
-
-void
-nvkm_mc_intr(struct nvkm_device *device, bool *handled)
-{
- struct nvkm_mc *mc = device->mc;
- struct nvkm_top *top = device->top;
- struct nvkm_top_device *tdev;
- struct nvkm_subdev *subdev;
- const struct nvkm_mc_map *map;
- u32 stat, intr;
-
- if (unlikely(!mc))
- return;
-
- stat = intr = nvkm_mc_intr_stat(mc);
-
- if (top) {
- list_for_each_entry(tdev, &top->device, head) {
- if (tdev->intr >= 0 && (stat & BIT(tdev->intr))) {
- subdev = nvkm_device_subdev(device, tdev->type, tdev->inst);
- if (subdev) {
- nvkm_subdev_intr(subdev);
- stat &= ~BIT(tdev->intr);
- if (!stat)
- break;
- }
- }
- }
- }
-
- for (map = mc->func->intr; map->stat; map++) {
- if (intr & map->stat) {
- subdev = nvkm_device_subdev(device, map->type, map->inst);
- if (subdev)
- nvkm_subdev_intr(subdev);
- stat &= ~map->stat;
- }
- }
-
- if (stat)
- nvkm_error(&mc->subdev, "intr %08x\n", stat);
- *handled = intr != 0;
-}
-
static u32
nvkm_mc_reset_mask(struct nvkm_device *device, bool isauto, enum nvkm_subdev_type type, int inst)
{
@@ -143,9 +73,8 @@ nvkm_mc_reset(struct nvkm_device *device, enum nvkm_subdev_type type, int inst)
{
u64 pmc_enable = nvkm_mc_reset_mask(device, true, type, inst);
if (pmc_enable) {
- nvkm_mask(device, 0x000200, pmc_enable, 0x00000000);
- nvkm_mask(device, 0x000200, pmc_enable, pmc_enable);
- nvkm_rd32(device, 0x000200);
+ device->mc->func->device->disable(device->mc, pmc_enable);
+ device->mc->func->device->enable(device->mc, pmc_enable);
}
}
@@ -154,17 +83,15 @@ nvkm_mc_disable(struct nvkm_device *device, enum nvkm_subdev_type type, int inst
{
u64 pmc_enable = nvkm_mc_reset_mask(device, false, type, inst);
if (pmc_enable)
- nvkm_mask(device, 0x000200, pmc_enable, 0x00000000);
+ device->mc->func->device->disable(device->mc, pmc_enable);
}
void
nvkm_mc_enable(struct nvkm_device *device, enum nvkm_subdev_type type, int inst)
{
u64 pmc_enable = nvkm_mc_reset_mask(device, false, type, inst);
- if (pmc_enable) {
- nvkm_mask(device, 0x000200, pmc_enable, pmc_enable);
- nvkm_rd32(device, 0x000200);
- }
+ if (pmc_enable)
+ device->mc->func->device->enable(device->mc, pmc_enable);
}
bool
@@ -172,16 +99,7 @@ nvkm_mc_enabled(struct nvkm_device *device, enum nvkm_subdev_type type, int inst
{
u64 pmc_enable = nvkm_mc_reset_mask(device, false, type, inst);
- return (pmc_enable != 0) &&
- ((nvkm_rd32(device, 0x000200) & pmc_enable) == pmc_enable);
-}
-
-
-static int
-nvkm_mc_fini(struct nvkm_subdev *subdev, bool suspend)
-{
- nvkm_mc_intr_unarm(subdev->device);
- return 0;
+ return (pmc_enable != 0) && device->mc->func->device->enabled(device->mc, pmc_enable);
}
static int
@@ -190,7 +108,6 @@ nvkm_mc_init(struct nvkm_subdev *subdev)
struct nvkm_mc *mc = nvkm_mc(subdev);
if (mc->func->init)
mc->func->init(mc);
- nvkm_mc_intr_rearm(subdev->device);
return 0;
}
@@ -204,24 +121,27 @@ static const struct nvkm_subdev_func
nvkm_mc = {
.dtor = nvkm_mc_dtor,
.init = nvkm_mc_init,
- .fini = nvkm_mc_fini,
};
-void
-nvkm_mc_ctor(const struct nvkm_mc_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_mc *mc)
-{
- nvkm_subdev_ctor(&nvkm_mc, device, type, inst, &mc->subdev);
- mc->func = func;
-}
-
int
nvkm_mc_new_(const struct nvkm_mc_func *func, struct nvkm_device *device,
enum nvkm_subdev_type type, int inst, struct nvkm_mc **pmc)
{
struct nvkm_mc *mc;
+ int ret;
+
if (!(mc = *pmc = kzalloc(sizeof(*mc), GFP_KERNEL)))
return -ENOMEM;
- nvkm_mc_ctor(func, device, type, inst, *pmc);
+
+ nvkm_subdev_ctor(&nvkm_mc, device, type, inst, &mc->subdev);
+ mc->func = func;
+
+ if (mc->func->intr) {
+ ret = nvkm_intr_add(mc->func->intr, mc->func->intrs, &mc->subdev,
+ mc->func->intr_nonstall ? 2 : 1, &mc->intr);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.c
index 4cfc1c9..f4ee991 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.c
@@ -34,30 +34,29 @@ g84_mc_reset[] = {
{}
};
-static const struct nvkm_mc_map
-g84_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP },
- { 0x00020000, NVKM_ENGINE_VP },
- { 0x00008000, NVKM_ENGINE_BSP },
- { 0x00004000, NVKM_ENGINE_CIPHER },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00000001, NVKM_ENGINE_MPEG },
- { 0x0002d101, NVKM_SUBDEV_FB },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x00200000, NVKM_SUBDEV_GPIO },
- { 0x00200000, NVKM_SUBDEV_I2C },
- { 0x00100000, NVKM_SUBDEV_TIMER },
+static const struct nvkm_intr_data
+g84_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x04000000, true },
+ { NVKM_ENGINE_VP , 0, 0, 0x00020000, true },
+ { NVKM_ENGINE_BSP , 0, 0, 0x00008000, true },
+ { NVKM_ENGINE_CIPHER, 0, 0, 0x00004000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_ENGINE_MPEG , 0, 0, 0x00000001, true },
+ { NVKM_SUBDEV_FB , 0, 0, 0x0002d101, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_TIMER , 0, 0, 0x00100000, true },
{},
};
static const struct nvkm_mc_func
g84_mc = {
.init = nv50_mc_init,
- .intr = g84_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = g84_mc_intrs,
+ .device = &nv04_mc_device,
.reset = g84_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c
index b7e58d7..f426848 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c
@@ -34,30 +34,29 @@ g98_mc_reset[] = {
{}
};
-static const struct nvkm_mc_map
-g98_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP },
- { 0x00020000, NVKM_ENGINE_MSPDEC },
- { 0x00008000, NVKM_ENGINE_MSVLD },
- { 0x00004000, NVKM_ENGINE_SEC },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00000001, NVKM_ENGINE_MSPPP },
- { 0x0002d101, NVKM_SUBDEV_FB },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x00200000, NVKM_SUBDEV_GPIO },
- { 0x00200000, NVKM_SUBDEV_I2C },
- { 0x00100000, NVKM_SUBDEV_TIMER },
+static const struct nvkm_intr_data
+g98_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x04000000, true },
+ { NVKM_ENGINE_MSPDEC, 0, 0, 0x00020000, true },
+ { NVKM_ENGINE_MSVLD , 0, 0, 0x00008000, true },
+ { NVKM_ENGINE_SEC , 0, 0, 0x00004000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_ENGINE_MSPPP , 0, 0, 0x00000001, true },
+ { NVKM_SUBDEV_FB , 0, 0, 0x0002d101, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_TIMER , 0, 0, 0x00100000, true },
{},
};
static const struct nvkm_mc_func
g98_mc = {
.init = nv50_mc_init,
- .intr = g98_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = g98_mc_intrs,
+ .device = &nv04_mc_device,
.reset = g98_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/ga100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/ga100.c
index 4105175..1e2eabe 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/ga100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/ga100.c
@@ -22,49 +22,51 @@
#include "priv.h"
static void
-ga100_mc_intr_unarm(struct nvkm_mc *mc)
+ga100_mc_device_disable(struct nvkm_mc *mc, u32 mask)
{
- nvkm_wr32(mc->subdev.device, 0xb81610, 0x00000004);
+ struct nvkm_device *device = mc->subdev.device;
+
+ nvkm_mask(device, 0x000600, mask, 0x00000000);
+ nvkm_rd32(device, 0x000600);
+ nvkm_rd32(device, 0x000600);
}
static void
-ga100_mc_intr_rearm(struct nvkm_mc *mc)
+ga100_mc_device_enable(struct nvkm_mc *mc, u32 mask)
{
- nvkm_wr32(mc->subdev.device, 0xb81608, 0x00000004);
+ struct nvkm_device *device = mc->subdev.device;
+
+ nvkm_mask(device, 0x000600, mask, mask);
+ nvkm_rd32(device, 0x000600);
+ nvkm_rd32(device, 0x000600);
}
-static void
-ga100_mc_intr_mask(struct nvkm_mc *mc, u32 mask, u32 intr)
+static bool
+ga100_mc_device_enabled(struct nvkm_mc *mc, u32 mask)
{
- nvkm_wr32(mc->subdev.device, 0xb81210, mask & intr );
- nvkm_wr32(mc->subdev.device, 0xb81410, mask & ~(mask & intr));
+ return (nvkm_rd32(mc->subdev.device, 0x000600) & mask) == mask;
}
-static u32
-ga100_mc_intr_stat(struct nvkm_mc *mc)
-{
- u32 intr_top = nvkm_rd32(mc->subdev.device, 0xb81600), intr = 0x00000000;
- if (intr_top & 0x00000004)
- intr = nvkm_mask(mc->subdev.device, 0xb81010, 0x00000000, 0x00000000);
- return intr;
-}
+const struct nvkm_mc_device_func
+ga100_mc_device = {
+ .enabled = ga100_mc_device_enabled,
+ .enable = ga100_mc_device_enable,
+ .disable = ga100_mc_device_disable,
+};
static void
ga100_mc_init(struct nvkm_mc *mc)
{
- nv50_mc_init(mc);
- nvkm_wr32(mc->subdev.device, 0xb81210, 0xffffffff);
+ struct nvkm_device *device = mc->subdev.device;
+
+ nvkm_wr32(device, 0x000200, 0xffffffff);
+ nvkm_wr32(device, 0x000600, 0xffffffff);
}
static const struct nvkm_mc_func
ga100_mc = {
.init = ga100_mc_init,
- .intr = gp100_mc_intr,
- .intr_unarm = ga100_mc_intr_unarm,
- .intr_rearm = ga100_mc_intr_rearm,
- .intr_mask = ga100_mc_intr_mask,
- .intr_stat = ga100_mc_intr_stat,
- .reset = gk104_mc_reset,
+ .device = &ga100_mc_device,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c
index 3a589c6f..ab1eaa3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c
@@ -36,64 +36,29 @@ gf100_mc_reset[] = {
{}
};
-static const struct nvkm_mc_map
-gf100_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP },
- { 0x00020000, NVKM_ENGINE_MSPDEC },
- { 0x00008000, NVKM_ENGINE_MSVLD },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00000040, NVKM_ENGINE_CE, 1 },
- { 0x00000020, NVKM_ENGINE_CE, 0 },
- { 0x00000001, NVKM_ENGINE_MSPPP },
- { 0x40000000, NVKM_SUBDEV_PRIVRING },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x08000000, NVKM_SUBDEV_FB },
- { 0x02000000, NVKM_SUBDEV_LTC },
- { 0x01000000, NVKM_SUBDEV_PMU },
- { 0x00200000, NVKM_SUBDEV_GPIO },
- { 0x00200000, NVKM_SUBDEV_I2C },
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x00040000, NVKM_SUBDEV_THERM },
- { 0x00002000, NVKM_SUBDEV_FB },
+static const struct nvkm_intr_data
+gf100_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x04000000, true },
+ { NVKM_ENGINE_MSPDEC , 0, 0, 0x00020000, true },
+ { NVKM_ENGINE_MSVLD , 0, 0, 0x00008000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000 },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_ENGINE_CE , 1, 0, 0x00000040, true },
+ { NVKM_ENGINE_CE , 0, 0, 0x00000020, true },
+ { NVKM_ENGINE_MSPPP , 0, 0, 0x00000001, true },
+ { NVKM_SUBDEV_PRIVRING, 0, 0, 0x40000000, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_FB , 0, 0, 0x08002000, true },
+ { NVKM_SUBDEV_LTC , 0, 0, 0x02000000, true },
+ { NVKM_SUBDEV_PMU , 0, 0, 0x01000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_TIMER , 0, 0, 0x00100000, true },
+ { NVKM_SUBDEV_THERM , 0, 0, 0x00040000, true },
{},
};
void
-gf100_mc_intr_unarm(struct nvkm_mc *mc)
-{
- struct nvkm_device *device = mc->subdev.device;
- nvkm_wr32(device, 0x000140, 0x00000000);
- nvkm_wr32(device, 0x000144, 0x00000000);
- nvkm_rd32(device, 0x000140);
-}
-
-void
-gf100_mc_intr_rearm(struct nvkm_mc *mc)
-{
- struct nvkm_device *device = mc->subdev.device;
- nvkm_wr32(device, 0x000140, 0x00000001);
- nvkm_wr32(device, 0x000144, 0x00000001);
-}
-
-u32
-gf100_mc_intr_stat(struct nvkm_mc *mc)
-{
- struct nvkm_device *device = mc->subdev.device;
- u32 intr0 = nvkm_rd32(device, 0x000100);
- u32 intr1 = nvkm_rd32(device, 0x000104);
- return intr0 | intr1;
-}
-
-void
-gf100_mc_intr_mask(struct nvkm_mc *mc, u32 mask, u32 stat)
-{
- struct nvkm_device *device = mc->subdev.device;
- nvkm_mask(device, 0x000640, mask, stat);
- nvkm_mask(device, 0x000644, mask, stat);
-}
-
-void
gf100_mc_unk260(struct nvkm_mc *mc, u32 data)
{
nvkm_wr32(mc->subdev.device, 0x000260, data);
@@ -102,12 +67,11 @@ gf100_mc_unk260(struct nvkm_mc *mc, u32 data)
static const struct nvkm_mc_func
gf100_mc = {
.init = nv50_mc_init,
- .intr = gf100_mc_intr,
- .intr_unarm = gf100_mc_intr_unarm,
- .intr_rearm = gf100_mc_intr_rearm,
- .intr_mask = gf100_mc_intr_mask,
- .intr_stat = gf100_mc_intr_stat,
+ .intr = >215_mc_intr,
+ .intrs = gf100_mc_intrs,
+ .intr_nonstall = true,
.reset = gf100_mc_reset,
+ .device = &nv04_mc_device,
.unk260 = gf100_mc_unk260,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.c
index d9b9067..6682958 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.c
@@ -30,32 +30,32 @@ gk104_mc_reset[] = {
{}
};
-const struct nvkm_mc_map
-gk104_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x40000000, NVKM_SUBDEV_PRIVRING },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x08000000, NVKM_SUBDEV_FB },
- { 0x02000000, NVKM_SUBDEV_LTC },
- { 0x01000000, NVKM_SUBDEV_PMU },
- { 0x00200000, NVKM_SUBDEV_GPIO },
- { 0x00200000, NVKM_SUBDEV_I2C },
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x00040000, NVKM_SUBDEV_THERM },
- { 0x00002000, NVKM_SUBDEV_FB },
+const struct nvkm_intr_data
+gk104_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x04000000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_SUBDEV_PRIVRING, 0, 0, 0x40000000, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_FB , 0, 0, 0x08002000, true },
+ { NVKM_SUBDEV_LTC , 0, 0, 0x02000000, true },
+ { NVKM_SUBDEV_PMU , 0, 0, 0x01000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_TIMER , 0, 0, 0x00100000, true },
+ { NVKM_SUBDEV_THERM , 0, 0, 0x00040000, true },
+ { NVKM_SUBDEV_TOP , 0, 0, 0x00001000 },
+ { NVKM_SUBDEV_TOP , 0, 0, 0xffffefff, true },
{},
};
static const struct nvkm_mc_func
gk104_mc = {
.init = nv50_mc_init,
- .intr = gk104_mc_intr,
- .intr_unarm = gf100_mc_intr_unarm,
- .intr_rearm = gf100_mc_intr_rearm,
- .intr_mask = gf100_mc_intr_mask,
- .intr_stat = gf100_mc_intr_stat,
+ .intr = >215_mc_intr,
+ .intrs = gk104_mc_intrs,
+ .intr_nonstall = true,
.reset = gk104_mc_reset,
+ .device = &nv04_mc_device,
.unk260 = gf100_mc_unk260,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c
index 0359029..d98a656 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c
@@ -26,11 +26,10 @@
static const struct nvkm_mc_func
gk20a_mc = {
.init = nv50_mc_init,
- .intr = gk104_mc_intr,
- .intr_unarm = gf100_mc_intr_unarm,
- .intr_rearm = gf100_mc_intr_rearm,
- .intr_mask = gf100_mc_intr_mask,
- .intr_stat = gf100_mc_intr_stat,
+ .intr = >215_mc_intr,
+ .intrs = gk104_mc_intrs,
+ .intr_nonstall = true,
+ .device = &nv04_mc_device,
.reset = gk104_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp100.c
index 5fd1a05..eb2ab03 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp100.c
@@ -21,108 +21,82 @@
*
* Authors: Ben Skeggs
*/
-#define gp100_mc(p) container_of((p), struct gp100_mc, base)
#include "priv.h"
-struct gp100_mc {
- struct nvkm_mc base;
- spinlock_t lock;
- bool intr;
- u32 mask;
+const struct nvkm_intr_data
+gp100_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x04000000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_SUBDEV_FAULT , 0, 0, 0x00000200, true },
+ { NVKM_SUBDEV_PRIVRING, 0, 0, 0x40000000, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_FB , 0, 0, 0x08002000, true },
+ { NVKM_SUBDEV_LTC , 0, 0, 0x02000000, true },
+ { NVKM_SUBDEV_PMU , 0, 0, 0x01000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_TIMER , 0, 0, 0x00100000, true },
+ { NVKM_SUBDEV_THERM , 0, 0, 0x00040000, true },
+ { NVKM_SUBDEV_TOP , 0, 0, 0x00009000 },
+ { NVKM_SUBDEV_TOP , 0, 0, 0xffff6fff, true },
+ {},
};
static void
-gp100_mc_intr_update(struct gp100_mc *mc)
+gp100_mc_intr_allow(struct nvkm_intr *intr, int leaf, u32 mask)
{
- struct nvkm_device *device = mc->base.subdev.device;
- u32 mask = mc->intr ? mc->mask : 0, i;
- for (i = 0; i < 2; i++) {
- nvkm_wr32(device, 0x000180 + (i * 0x04), ~mask);
- nvkm_wr32(device, 0x000160 + (i * 0x04), mask);
- }
+ struct nvkm_mc *mc = container_of(intr, typeof(*mc), intr);
+
+ nvkm_wr32(mc->subdev.device, 0x000160 + (leaf * 4), mask);
}
-void
-gp100_mc_intr_unarm(struct nvkm_mc *base)
+static void
+gp100_mc_intr_block(struct nvkm_intr *intr, int leaf, u32 mask)
{
- struct gp100_mc *mc = gp100_mc(base);
- unsigned long flags;
- spin_lock_irqsave(&mc->lock, flags);
- mc->intr = false;
- gp100_mc_intr_update(mc);
- spin_unlock_irqrestore(&mc->lock, flags);
+ struct nvkm_mc *mc = container_of(intr, typeof(*mc), intr);
+
+ nvkm_wr32(mc->subdev.device, 0x000180 + (leaf * 4), mask);
}
-void
-gp100_mc_intr_rearm(struct nvkm_mc *base)
+static void
+gp100_mc_intr_rearm(struct nvkm_intr *intr)
{
- struct gp100_mc *mc = gp100_mc(base);
- unsigned long flags;
- spin_lock_irqsave(&mc->lock, flags);
- mc->intr = true;
- gp100_mc_intr_update(mc);
- spin_unlock_irqrestore(&mc->lock, flags);
+ int i;
+
+ for (i = 0; i < intr->leaves; i++)
+ intr->func->allow(intr, i, intr->mask[i]);
}
-void
-gp100_mc_intr_mask(struct nvkm_mc *base, u32 mask, u32 intr)
+static void
+gp100_mc_intr_unarm(struct nvkm_intr *intr)
{
- struct gp100_mc *mc = gp100_mc(base);
- unsigned long flags;
- spin_lock_irqsave(&mc->lock, flags);
- mc->mask = (mc->mask & ~mask) | intr;
- gp100_mc_intr_update(mc);
- spin_unlock_irqrestore(&mc->lock, flags);
+ int i;
+
+ for (i = 0; i < intr->leaves; i++)
+ intr->func->block(intr, i, 0xffffffff);
}
-const struct nvkm_mc_map
-gp100_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00000200, NVKM_SUBDEV_FAULT },
- { 0x40000000, NVKM_SUBDEV_PRIVRING },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x08000000, NVKM_SUBDEV_FB },
- { 0x02000000, NVKM_SUBDEV_LTC },
- { 0x01000000, NVKM_SUBDEV_PMU },
- { 0x00200000, NVKM_SUBDEV_GPIO },
- { 0x00200000, NVKM_SUBDEV_I2C },
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x00040000, NVKM_SUBDEV_THERM },
- { 0x00002000, NVKM_SUBDEV_FB },
- {},
+const struct nvkm_intr_func
+gp100_mc_intr = {
+ .pending = nv04_mc_intr_pending,
+ .unarm = gp100_mc_intr_unarm,
+ .rearm = gp100_mc_intr_rearm,
+ .block = gp100_mc_intr_block,
+ .allow = gp100_mc_intr_allow,
};
static const struct nvkm_mc_func
gp100_mc = {
.init = nv50_mc_init,
- .intr = gp100_mc_intr,
- .intr_unarm = gp100_mc_intr_unarm,
- .intr_rearm = gp100_mc_intr_rearm,
- .intr_mask = gp100_mc_intr_mask,
- .intr_stat = gf100_mc_intr_stat,
+ .intr = &gp100_mc_intr,
+ .intrs = gp100_mc_intrs,
+ .intr_nonstall = true,
+ .device = &nv04_mc_device,
.reset = gk104_mc_reset,
};
int
-gp100_mc_new_(const struct nvkm_mc_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_mc **pmc)
-{
- struct gp100_mc *mc;
-
- if (!(mc = kzalloc(sizeof(*mc), GFP_KERNEL)))
- return -ENOMEM;
- nvkm_mc_ctor(func, device, type, inst, &mc->base);
- *pmc = &mc->base;
-
- spin_lock_init(&mc->lock);
- mc->intr = false;
- mc->mask = 0x7fffffff;
- return 0;
-}
-
-int
gp100_mc_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, struct nvkm_mc **pmc)
{
- return gp100_mc_new_(&gp100_mc, device, type, inst, pmc);
+ return nvkm_mc_new_(&gp100_mc, device, type, inst, pmc);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp10b.c
index dd581d0..9bed9c5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp10b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp10b.c
@@ -34,16 +34,15 @@ gp10b_mc_init(struct nvkm_mc *mc)
static const struct nvkm_mc_func
gp10b_mc = {
.init = gp10b_mc_init,
- .intr = gp100_mc_intr,
- .intr_unarm = gp100_mc_intr_unarm,
- .intr_rearm = gp100_mc_intr_rearm,
- .intr_mask = gp100_mc_intr_mask,
- .intr_stat = gf100_mc_intr_stat,
+ .intr = &gp100_mc_intr,
+ .intrs = gp100_mc_intrs,
+ .intr_nonstall = true,
+ .device = &nv04_mc_device,
.reset = gk104_mc_reset,
};
int
gp10b_mc_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, struct nvkm_mc **pmc)
{
- return gp100_mc_new_(&gp10b_mc, device, type, inst, pmc);
+ return nvkm_mc_new_(&gp10b_mc, device, type, inst, pmc);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.c
index 1b4d435..3d61836 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.c
@@ -34,39 +34,56 @@ gt215_mc_reset[] = {
{}
};
-static const struct nvkm_mc_map
-gt215_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP },
- { 0x00400000, NVKM_ENGINE_CE, 0 },
- { 0x00020000, NVKM_ENGINE_MSPDEC },
- { 0x00008000, NVKM_ENGINE_MSVLD },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00000001, NVKM_ENGINE_MSPPP },
- { 0x00429101, NVKM_SUBDEV_FB },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x00200000, NVKM_SUBDEV_GPIO },
- { 0x00200000, NVKM_SUBDEV_I2C },
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x00080000, NVKM_SUBDEV_THERM },
- { 0x00040000, NVKM_SUBDEV_PMU },
+static const struct nvkm_intr_data
+gt215_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x04000000, true },
+ { NVKM_ENGINE_CE , 0, 0, 0x00400000, true },
+ { NVKM_ENGINE_MSPDEC, 0, 0, 0x00020000, true },
+ { NVKM_ENGINE_MSVLD , 0, 0, 0x00008000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_ENGINE_MSPPP , 0, 0, 0x00000001, true },
+ { NVKM_SUBDEV_FB , 0, 0, 0x00429101, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_TIMER , 0, 0, 0x00100000, true },
+ { NVKM_SUBDEV_THERM , 0, 0, 0x00080000, true },
+ { NVKM_SUBDEV_PMU , 0, 0, 0x00040000, true },
{},
};
static void
-gt215_mc_intr_mask(struct nvkm_mc *mc, u32 mask, u32 stat)
+gt215_mc_intr_allow(struct nvkm_intr *intr, int leaf, u32 mask)
{
- nvkm_mask(mc->subdev.device, 0x000640, mask, stat);
+ struct nvkm_mc *mc = container_of(intr, typeof(*mc), intr);
+
+ nvkm_mask(mc->subdev.device, 0x000640 + (leaf * 4), mask, mask);
}
+static void
+gt215_mc_intr_block(struct nvkm_intr *intr, int leaf, u32 mask)
+{
+ struct nvkm_mc *mc = container_of(intr, typeof(*mc), intr);
+
+ nvkm_mask(mc->subdev.device, 0x000640 + (leaf * 4), mask, 0);
+}
+
+const struct nvkm_intr_func
+gt215_mc_intr = {
+ .pending = nv04_mc_intr_pending,
+ .unarm = nv04_mc_intr_unarm,
+ .rearm = nv04_mc_intr_rearm,
+ .block = gt215_mc_intr_block,
+ .allow = gt215_mc_intr_allow,
+};
+
static const struct nvkm_mc_func
gt215_mc = {
.init = nv50_mc_init,
- .intr = gt215_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_mask = gt215_mc_intr_mask,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = gt215_mc_intrs,
+ .device = &nv04_mc_device,
.reset = gt215_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c
index bc0d09b..8482a55 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c
@@ -30,37 +30,89 @@ nv04_mc_reset[] = {
{}
};
-static const struct nvkm_mc_map
-nv04_mc_intr[] = {
- { 0x01010000, NVKM_ENGINE_DISP },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x00100000, NVKM_SUBDEV_TIMER },
+static void
+nv04_mc_device_disable(struct nvkm_mc *mc, u32 mask)
+{
+ nvkm_mask(mc->subdev.device, 0x000200, mask, 0x00000000);
+}
+
+static void
+nv04_mc_device_enable(struct nvkm_mc *mc, u32 mask)
+{
+ struct nvkm_device *device = mc->subdev.device;
+
+ nvkm_mask(device, 0x000200, mask, mask);
+ nvkm_rd32(device, 0x000200);
+}
+
+static bool
+nv04_mc_device_enabled(struct nvkm_mc *mc, u32 mask)
+{
+ return (nvkm_rd32(mc->subdev.device, 0x000200) & mask) == mask;
+}
+
+const struct nvkm_mc_device_func
+nv04_mc_device = {
+ .enabled = nv04_mc_device_enabled,
+ .enable = nv04_mc_device_enable,
+ .disable = nv04_mc_device_disable,
+};
+
+static const struct nvkm_intr_data
+nv04_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x01010000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_TIMER, 0, 0, 0x00100000, true },
{}
};
void
-nv04_mc_intr_unarm(struct nvkm_mc *mc)
+nv04_mc_intr_rearm(struct nvkm_intr *intr)
{
- struct nvkm_device *device = mc->subdev.device;
- nvkm_wr32(device, 0x000140, 0x00000000);
- nvkm_rd32(device, 0x000140);
+ struct nvkm_mc *mc = container_of(intr, typeof(*mc), intr);
+ int leaf;
+
+ for (leaf = 0; leaf < intr->leaves; leaf++)
+ nvkm_wr32(mc->subdev.device, 0x000140 + (leaf * 4), 0x00000001);
}
void
-nv04_mc_intr_rearm(struct nvkm_mc *mc)
+nv04_mc_intr_unarm(struct nvkm_intr *intr)
{
- struct nvkm_device *device = mc->subdev.device;
- nvkm_wr32(device, 0x000140, 0x00000001);
+ struct nvkm_mc *mc = container_of(intr, typeof(*mc), intr);
+ int leaf;
+
+ for (leaf = 0; leaf < intr->leaves; leaf++)
+ nvkm_wr32(mc->subdev.device, 0x000140 + (leaf * 4), 0x00000000);
+
+ nvkm_rd32(mc->subdev.device, 0x000140);
}
-u32
-nv04_mc_intr_stat(struct nvkm_mc *mc)
+bool
+nv04_mc_intr_pending(struct nvkm_intr *intr)
{
- return nvkm_rd32(mc->subdev.device, 0x000100);
+ struct nvkm_mc *mc = container_of(intr, typeof(*mc), intr);
+ bool pending = false;
+ int leaf;
+
+ for (leaf = 0; leaf < intr->leaves; leaf++) {
+ intr->stat[leaf] = nvkm_rd32(mc->subdev.device, 0x000100 + (leaf * 4));
+ if (intr->stat[leaf])
+ pending = true;
+ }
+
+ return pending;
}
+const struct nvkm_intr_func
+nv04_mc_intr = {
+ .pending = nv04_mc_intr_pending,
+ .unarm = nv04_mc_intr_unarm,
+ .rearm = nv04_mc_intr_rearm,
+};
+
void
nv04_mc_init(struct nvkm_mc *mc)
{
@@ -72,10 +124,9 @@ nv04_mc_init(struct nvkm_mc *mc)
static const struct nvkm_mc_func
nv04_mc = {
.init = nv04_mc_init,
- .intr = nv04_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = nv04_mc_intrs,
+ .device = &nv04_mc_device,
.reset = nv04_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.c
index ab59ca1..6d6278f4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.c
@@ -23,23 +23,22 @@
*/
#include "priv.h"
-static const struct nvkm_mc_map
-nv11_mc_intr[] = {
- { 0x03010000, NVKM_ENGINE_DISP },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x00100000, NVKM_SUBDEV_TIMER },
+static const struct nvkm_intr_data
+nv11_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x03010000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_TIMER, 0, 0, 0x00100000, true },
{}
};
static const struct nvkm_mc_func
nv11_mc = {
.init = nv04_mc_init,
- .intr = nv11_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = nv11_mc_intrs,
+ .device = &nv04_mc_device,
.reset = nv04_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.c
index 03d756e..dbad7c1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.c
@@ -31,24 +31,23 @@ nv17_mc_reset[] = {
{}
};
-const struct nvkm_mc_map
-nv17_mc_intr[] = {
- { 0x03010000, NVKM_ENGINE_DISP },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00000001, NVKM_ENGINE_MPEG },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x00100000, NVKM_SUBDEV_TIMER },
+const struct nvkm_intr_data
+nv17_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x03010000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_ENGINE_MPEG , 0, 0, 0x00000001, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_TIMER, 0, 0, 0x00100000, true },
{}
};
static const struct nvkm_mc_func
nv17_mc = {
.init = nv04_mc_init,
- .intr = nv17_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = nv17_mc_intrs,
+ .device = &nv04_mc_device,
.reset = nv17_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c
index 95f6576..649a9fc 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c
@@ -40,10 +40,9 @@ nv44_mc_init(struct nvkm_mc *mc)
static const struct nvkm_mc_func
nv44_mc = {
.init = nv44_mc_init,
- .intr = nv17_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = nv17_mc_intrs,
+ .device = &nv04_mc_device,
.reset = nv17_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c
index fce3613..d41099d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c
@@ -23,17 +23,17 @@
*/
#include "priv.h"
-static const struct nvkm_mc_map
-nv50_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00000001, NVKM_ENGINE_MPEG },
- { 0x00001101, NVKM_SUBDEV_FB },
- { 0x10000000, NVKM_SUBDEV_BUS },
- { 0x00200000, NVKM_SUBDEV_GPIO },
- { 0x00200000, NVKM_SUBDEV_I2C },
- { 0x00100000, NVKM_SUBDEV_TIMER },
+static const struct nvkm_intr_data
+nv50_mc_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 0, 0x04000000, true },
+ { NVKM_ENGINE_GR , 0, 0, 0x00001000, true },
+ { NVKM_ENGINE_FIFO , 0, 0, 0x00000100 },
+ { NVKM_ENGINE_MPEG , 0, 0, 0x00000001, true },
+ { NVKM_SUBDEV_FB , 0, 0, 0x00001101, true },
+ { NVKM_SUBDEV_BUS , 0, 0, 0x10000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 0, 0x00200000, true },
+ { NVKM_SUBDEV_TIMER, 0, 0, 0x00100000, true },
{},
};
@@ -47,10 +47,9 @@ nv50_mc_init(struct nvkm_mc *mc)
static const struct nvkm_mc_func
nv50_mc = {
.init = nv50_mc_init,
- .intr = nv50_mc_intr,
- .intr_unarm = nv04_mc_intr_unarm,
- .intr_rearm = nv04_mc_intr_rearm,
- .intr_stat = nv04_mc_intr_stat,
+ .intr = &nv04_mc_intr,
+ .intrs = nv50_mc_intrs,
+ .device = &nv04_mc_device,
.reset = nv17_mc_reset,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h
index c8bcabb..7f38d54 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h
@@ -4,8 +4,6 @@
#define nvkm_mc(p) container_of((p), struct nvkm_mc, subdev)
#include <subdev/mc.h>
-void nvkm_mc_ctor(const struct nvkm_mc_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
- struct nvkm_mc *);
int nvkm_mc_new_(const struct nvkm_mc_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
struct nvkm_mc **);
@@ -18,46 +16,44 @@ struct nvkm_mc_map {
struct nvkm_mc_func {
void (*init)(struct nvkm_mc *);
- const struct nvkm_mc_map *intr;
- /* disable reporting of interrupts to host */
- void (*intr_unarm)(struct nvkm_mc *);
- /* enable reporting of interrupts to host */
- void (*intr_rearm)(struct nvkm_mc *);
- /* (un)mask delivery of specific interrupts */
- void (*intr_mask)(struct nvkm_mc *, u32 mask, u32 stat);
- /* retrieve pending interrupt mask (NV_PMC_INTR) */
- u32 (*intr_stat)(struct nvkm_mc *);
+
+ const struct nvkm_intr_func *intr;
+ const struct nvkm_intr_data *intrs;
+ bool intr_nonstall;
+
+ const struct nvkm_mc_device_func {
+ bool (*enabled)(struct nvkm_mc *, u32 mask);
+ void (*enable)(struct nvkm_mc *, u32 mask);
+ void (*disable)(struct nvkm_mc *, u32 mask);
+ } *device;
+
const struct nvkm_mc_map *reset;
+
void (*unk260)(struct nvkm_mc *, u32);
};
void nv04_mc_init(struct nvkm_mc *);
-void nv04_mc_intr_unarm(struct nvkm_mc *);
-void nv04_mc_intr_rearm(struct nvkm_mc *);
-u32 nv04_mc_intr_stat(struct nvkm_mc *);
+extern const struct nvkm_intr_func nv04_mc_intr;
+bool nv04_mc_intr_pending(struct nvkm_intr *);
+void nv04_mc_intr_unarm(struct nvkm_intr *);
+void nv04_mc_intr_rearm(struct nvkm_intr *);
+extern const struct nvkm_mc_device_func nv04_mc_device;
extern const struct nvkm_mc_map nv04_mc_reset[];
-extern const struct nvkm_mc_map nv17_mc_intr[];
+extern const struct nvkm_intr_data nv17_mc_intrs[];
extern const struct nvkm_mc_map nv17_mc_reset[];
void nv44_mc_init(struct nvkm_mc *);
void nv50_mc_init(struct nvkm_mc *);
-void gk104_mc_init(struct nvkm_mc *);
-void gf100_mc_intr_unarm(struct nvkm_mc *);
-void gf100_mc_intr_rearm(struct nvkm_mc *);
-void gf100_mc_intr_mask(struct nvkm_mc *, u32, u32);
-u32 gf100_mc_intr_stat(struct nvkm_mc *);
+extern const struct nvkm_intr_func gt215_mc_intr;
void gf100_mc_unk260(struct nvkm_mc *, u32);
-void gp100_mc_intr_unarm(struct nvkm_mc *);
-void gp100_mc_intr_rearm(struct nvkm_mc *);
-void gp100_mc_intr_mask(struct nvkm_mc *, u32, u32);
-int gp100_mc_new_(const struct nvkm_mc_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
- struct nvkm_mc **);
-extern const struct nvkm_mc_map gk104_mc_intr[];
+void gk104_mc_init(struct nvkm_mc *);
+extern const struct nvkm_intr_data gk104_mc_intrs[];
extern const struct nvkm_mc_map gk104_mc_reset[];
-extern const struct nvkm_mc_map gp100_mc_intr[];
+extern const struct nvkm_intr_func gp100_mc_intr;
+extern const struct nvkm_intr_data gp100_mc_intrs[];
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c
deleted file mode 100644
index a96084b..0000000
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c
+++ /dev/null
@@ -1,136 +0,0 @@
-/*
- * Copyright 2018 Red Hat Inc.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- */
-#define tu102_mc(p) container_of((p), struct tu102_mc, base)
-#include "priv.h"
-
-struct tu102_mc {
- struct nvkm_mc base;
- spinlock_t lock;
- bool intr;
- u32 mask;
-};
-
-static void
-tu102_mc_intr_update(struct tu102_mc *mc)
-{
- struct nvkm_device *device = mc->base.subdev.device;
- u32 mask = mc->intr ? mc->mask : 0, i;
-
- for (i = 0; i < 2; i++) {
- nvkm_wr32(device, 0x000180 + (i * 0x04), ~mask);
- nvkm_wr32(device, 0x000160 + (i * 0x04), mask);
- }
-
- if (mask & 0x00000200)
- nvkm_wr32(device, 0xb81608, 0x6);
- else
- nvkm_wr32(device, 0xb81610, 0x6);
-}
-
-static void
-tu102_mc_intr_unarm(struct nvkm_mc *base)
-{
- struct tu102_mc *mc = tu102_mc(base);
- unsigned long flags;
-
- spin_lock_irqsave(&mc->lock, flags);
- mc->intr = false;
- tu102_mc_intr_update(mc);
- spin_unlock_irqrestore(&mc->lock, flags);
-}
-
-static void
-tu102_mc_intr_rearm(struct nvkm_mc *base)
-{
- struct tu102_mc *mc = tu102_mc(base);
- unsigned long flags;
-
- spin_lock_irqsave(&mc->lock, flags);
- mc->intr = true;
- tu102_mc_intr_update(mc);
- spin_unlock_irqrestore(&mc->lock, flags);
-}
-
-static void
-tu102_mc_intr_mask(struct nvkm_mc *base, u32 mask, u32 intr)
-{
- struct tu102_mc *mc = tu102_mc(base);
- unsigned long flags;
-
- spin_lock_irqsave(&mc->lock, flags);
- mc->mask = (mc->mask & ~mask) | intr;
- tu102_mc_intr_update(mc);
- spin_unlock_irqrestore(&mc->lock, flags);
-}
-
-static u32
-tu102_mc_intr_stat(struct nvkm_mc *mc)
-{
- struct nvkm_device *device = mc->subdev.device;
- u32 intr0 = nvkm_rd32(device, 0x000100);
- u32 intr1 = nvkm_rd32(device, 0x000104);
- u32 intr_top = nvkm_rd32(device, 0xb81600);
-
- /* Turing and above route the MMU fault interrupts via a different
- * interrupt tree with different control registers. For the moment remap
- * them back to the old PMC vector.
- */
- if (intr_top & 0x00000006)
- intr0 |= 0x00000200;
-
- return intr0 | intr1;
-}
-
-
-static const struct nvkm_mc_func
-tu102_mc = {
- .init = nv50_mc_init,
- .intr = gp100_mc_intr,
- .intr_unarm = tu102_mc_intr_unarm,
- .intr_rearm = tu102_mc_intr_rearm,
- .intr_mask = tu102_mc_intr_mask,
- .intr_stat = tu102_mc_intr_stat,
- .reset = gk104_mc_reset,
-};
-
-static int
-tu102_mc_new_(const struct nvkm_mc_func *func, struct nvkm_device *device,
- enum nvkm_subdev_type type, int inst, struct nvkm_mc **pmc)
-{
- struct tu102_mc *mc;
-
- if (!(mc = kzalloc(sizeof(*mc), GFP_KERNEL)))
- return -ENOMEM;
- nvkm_mc_ctor(func, device, type, inst, &mc->base);
- *pmc = &mc->base;
-
- spin_lock_init(&mc->lock);
- mc->intr = false;
- mc->mask = 0x7fffffff;
- return 0;
-}
-
-int
-tu102_mc_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, struct nvkm_mc **pmc)
-{
- return tu102_mc_new_(&tu102_mc, device, type, inst, pmc);
-}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/uvmm.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/uvmm.c
index 186b4e6..524cd3c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/uvmm.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/uvmm.c
@@ -39,7 +39,7 @@ nvkm_uvmm_search(struct nvkm_client *client, u64 handle)
if (IS_ERR(object))
return (void *)object;
- return nvkm_uvmm(object)->vmm;
+ return nvkm_vmm_ref(nvkm_uvmm(object)->vmm);
}
static int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c
index a7d42ea..5a0de45 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c
@@ -26,7 +26,15 @@
#include <core/option.h>
#include <core/pci.h>
-#include <subdev/mc.h>
+
+void
+nvkm_pci_msi_rearm(struct nvkm_device *device)
+{
+ struct nvkm_pci *pci = device->pci;
+
+ if (pci && pci->msi)
+ pci->func->msi_rearm(pci);
+}
u32
nvkm_pci_rd32(struct nvkm_pci *pci, u16 addr)
@@ -65,24 +73,6 @@ nvkm_pci_rom_shadow(struct nvkm_pci *pci, bool shadow)
nvkm_pci_wr32(pci, 0x0050, data);
}
-static irqreturn_t
-nvkm_pci_intr(int irq, void *arg)
-{
- struct nvkm_pci *pci = arg;
- struct nvkm_device *device = pci->subdev.device;
- bool handled = false;
-
- if (pci->irq < 0)
- return IRQ_HANDLED;
-
- nvkm_mc_intr_unarm(device);
- if (pci->msi)
- pci->func->msi_rearm(pci);
- nvkm_mc_intr(device, &handled);
- nvkm_mc_intr_rearm(device);
- return handled ? IRQ_HANDLED : IRQ_NONE;
-}
-
static int
nvkm_pci_fini(struct nvkm_subdev *subdev, bool suspend)
{
@@ -107,7 +97,6 @@ static int
nvkm_pci_oneinit(struct nvkm_subdev *subdev)
{
struct nvkm_pci *pci = nvkm_pci(subdev);
- struct pci_dev *pdev = pci->pdev;
int ret;
if (pci_is_pcie(pci->pdev)) {
@@ -116,11 +105,6 @@ nvkm_pci_oneinit(struct nvkm_subdev *subdev)
return ret;
}
- ret = request_irq(pdev->irq, nvkm_pci_intr, IRQF_SHARED, "nvkm", pci);
- if (ret)
- return ret;
-
- pci->irq = pdev->irq;
return 0;
}
@@ -157,15 +141,6 @@ nvkm_pci_dtor(struct nvkm_subdev *subdev)
nvkm_agp_dtor(pci);
- if (pci->irq >= 0) {
- /* freq_irq() will call the handler, we use pci->irq == -1
- * to signal that it's been torn down and should be a noop.
- */
- int irq = pci->irq;
- pci->irq = -1;
- free_irq(irq, pci);
- }
-
if (pci->msi)
pci_disable_msi(pci->pdev);
@@ -192,7 +167,6 @@ nvkm_pci_new_(const struct nvkm_pci_func *func, struct nvkm_device *device,
nvkm_subdev_ctor(&nvkm_pci_func, device, type, inst, &pci->subdev);
pci->func = func;
pci->pdev = device->func->pci(device)->pdev;
- pci->irq = -1;
pci->pcie.speed = -1;
pci->pcie.width = -1;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
index 455e95a..8f2f50a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
@@ -81,43 +81,12 @@ nvkm_pmu_fini(struct nvkm_subdev *subdev, bool suspend)
{
struct nvkm_pmu *pmu = nvkm_pmu(subdev);
+ if (!subdev->use.enabled)
+ return 0;
+
if (pmu->func->fini)
pmu->func->fini(pmu);
- flush_work(&pmu->recv.work);
-
- reinit_completion(&pmu->wpr_ready);
-
- nvkm_falcon_cmdq_fini(pmu->lpq);
- nvkm_falcon_cmdq_fini(pmu->hpq);
- pmu->initmsg_received = false;
- return 0;
-}
-
-static void
-nvkm_pmu_reset(struct nvkm_pmu *pmu)
-{
- struct nvkm_device *device = pmu->subdev.device;
-
- if (!pmu->func->enabled(pmu))
- return;
-
- /* Reset. */
- if (pmu->func->reset)
- pmu->func->reset(pmu);
-
- /* Wait for IMEM/DMEM scrubbing to be complete. */
- nvkm_msec(device, 2000,
- if (!(nvkm_rd32(device, 0x10a10c) & 0x00000006))
- break;
- );
-}
-
-static int
-nvkm_pmu_preinit(struct nvkm_subdev *subdev)
-{
- struct nvkm_pmu *pmu = nvkm_pmu(subdev);
- nvkm_pmu_reset(pmu);
return 0;
}
@@ -125,22 +94,10 @@ static int
nvkm_pmu_init(struct nvkm_subdev *subdev)
{
struct nvkm_pmu *pmu = nvkm_pmu(subdev);
- struct nvkm_device *device = pmu->subdev.device;
if (!pmu->func->init)
return 0;
- if (pmu->func->enabled(pmu)) {
- /* Inhibit interrupts, and wait for idle. */
- nvkm_wr32(device, 0x10a014, 0x0000ffff);
- nvkm_msec(device, 2000,
- if (!nvkm_rd32(device, 0x10a04c))
- break;
- );
-
- nvkm_pmu_reset(pmu);
- }
-
return pmu->func->init(pmu);
}
@@ -160,7 +117,6 @@ nvkm_pmu_dtor(struct nvkm_subdev *subdev)
static const struct nvkm_subdev_func
nvkm_pmu = {
.dtor = nvkm_pmu_dtor,
- .preinit = nvkm_pmu_preinit,
.init = nvkm_pmu_init,
.fini = nvkm_pmu_fini,
.intr = nvkm_pmu_intr,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c
index a67a42e..b5e52b3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c
@@ -197,7 +197,6 @@ gk20a_dvfs_data= {
static const struct nvkm_pmu_func
gk20a_pmu = {
.flcn = >215_pmu_flcn,
- .enabled = gf100_pmu_enabled,
.init = gk20a_pmu_init,
.fini = gk20a_pmu_fini,
.reset = gf100_pmu_reset,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c
index 40439e3..7359991 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.c
@@ -24,30 +24,36 @@
#include "priv.h"
static int
-gm200_pmu_flcn_reset(struct nvkm_falcon *falcon)
+gm200_pmu_flcn_bind_stat(struct nvkm_falcon *falcon, bool intr)
{
- struct nvkm_pmu *pmu = container_of(falcon, typeof(*pmu), falcon);
+ nvkm_falcon_wr32(falcon, 0x200, 0x0000030e);
+ return (nvkm_falcon_rd32(falcon, 0x20c) & 0x00007000) >> 12;
+}
- nvkm_falcon_wr32(falcon, 0x014, 0x0000ffff);
- pmu->func->reset(pmu);
- return nvkm_falcon_enable(falcon);
+void
+gm200_pmu_flcn_bind_inst(struct nvkm_falcon *falcon, int target, u64 addr)
+{
+ nvkm_falcon_wr32(falcon, 0xe00, 4); /* DMAIDX_UCODE */
+ nvkm_falcon_wr32(falcon, 0xe04, 0); /* DMAIDX_VIRT */
+ nvkm_falcon_wr32(falcon, 0xe08, 4); /* DMAIDX_PHYS_VID */
+ nvkm_falcon_wr32(falcon, 0xe0c, 5); /* DMAIDX_PHYS_SYS_COH */
+ nvkm_falcon_wr32(falcon, 0xe10, 6); /* DMAIDX_PHYS_SYS_NCOH */
+ nvkm_falcon_mask(falcon, 0x090, 0x00010000, 0x00010000);
+ nvkm_falcon_wr32(falcon, 0x480, (1 << 30) | (target << 28) | (addr >> 12));
}
const struct nvkm_falcon_func
gm200_pmu_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .reset_pmc = true,
+ .reset_wait_mem_scrubbing = gm200_flcn_reset_wait_mem_scrubbing,
.debug = 0xc08,
- .fbif = 0xe00,
- .load_imem = nvkm_falcon_v1_load_imem,
- .load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
- .bind_context = nvkm_falcon_v1_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
+ .bind_inst = gm200_pmu_flcn_bind_inst,
+ .bind_stat = gm200_pmu_flcn_bind_stat,
+ .imem_pio = &gm200_flcn_imem_pio,
+ .dmem_pio = &gm200_flcn_dmem_pio,
.start = nvkm_falcon_v1_start,
- .enable = nvkm_falcon_v1_enable,
- .disable = nvkm_falcon_v1_disable,
- .reset = gm200_pmu_flcn_reset,
.cmdq = { 0x4a0, 0x4b0, 4 },
.msgq = { 0x4c8, 0x4cc, 0 },
};
@@ -55,11 +61,9 @@ gm200_pmu_flcn = {
static const struct nvkm_pmu_func
gm200_pmu = {
.flcn = &gm200_pmu_flcn,
- .enabled = gf100_pmu_enabled,
.reset = gf100_pmu_reset,
};
-
int
gm200_pmu_nofw(struct nvkm_pmu *pmu, int ver, const struct nvkm_pmu_fwif *fwif)
{
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c
index 612310d..a724037 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.c
@@ -62,16 +62,6 @@ gm20b_pmu_acr_bootstrap_falcon(struct nvkm_falcon *falcon,
return ret;
}
-int
-gm20b_pmu_acr_boot(struct nvkm_falcon *falcon)
-{
- struct nv_pmu_args args = { .secure_mode = true };
- const u32 addr_args = falcon->data.limit - sizeof(struct nv_pmu_args);
- nvkm_falcon_load_dmem(falcon, &args, addr_args, sizeof(args), 0);
- nvkm_falcon_start(falcon);
- return 0;
-}
-
void
gm20b_pmu_acr_bld_patch(struct nvkm_acr *acr, u32 bld, s64 adjust)
{
@@ -125,7 +115,6 @@ gm20b_pmu_acr = {
.bld_size = sizeof(struct loader_config),
.bld_write = gm20b_pmu_acr_bld_write,
.bld_patch = gm20b_pmu_acr_bld_patch,
- .boot = gm20b_pmu_acr_boot,
.bootstrap_falcons = BIT_ULL(NVKM_ACR_LSF_PMU) |
BIT_ULL(NVKM_ACR_LSF_FECS) |
BIT_ULL(NVKM_ACR_LSF_GPCCS),
@@ -166,7 +155,7 @@ gm20b_pmu_acr_init_wpr(struct nvkm_pmu *pmu)
gm20b_pmu_acr_init_wpr_callback, pmu, 0);
}
-int
+static int
gm20b_pmu_initmsg(struct nvkm_pmu *pmu)
{
struct nv_pmu_init_msg msg;
@@ -192,14 +181,13 @@ gm20b_pmu_initmsg(struct nvkm_pmu *pmu)
return gm20b_pmu_acr_init_wpr(pmu);
}
-void
+static void
gm20b_pmu_recv(struct nvkm_pmu *pmu)
{
if (!pmu->initmsg_received) {
int ret = pmu->func->initmsg(pmu);
if (ret) {
- nvkm_error(&pmu->subdev,
- "error parsing init message: %d\n", ret);
+ nvkm_error(&pmu->subdev, "error parsing init message: %d\n", ret);
return;
}
@@ -209,10 +197,44 @@ gm20b_pmu_recv(struct nvkm_pmu *pmu)
nvkm_falcon_msgq_recv(pmu->msgq);
}
-static const struct nvkm_pmu_func
+static void
+gm20b_pmu_fini(struct nvkm_pmu *pmu)
+{
+ /*TODO: shutdown RTOS. */
+
+ flush_work(&pmu->recv.work);
+ nvkm_falcon_cmdq_fini(pmu->lpq);
+ nvkm_falcon_cmdq_fini(pmu->hpq);
+
+ reinit_completion(&pmu->wpr_ready);
+
+ nvkm_falcon_put(&pmu->falcon, &pmu->subdev);
+}
+
+static int
+gm20b_pmu_init(struct nvkm_pmu *pmu)
+{
+ struct nvkm_falcon *falcon = &pmu->falcon;
+ struct nv_pmu_args args = { .secure_mode = true };
+ u32 addr_args = falcon->data.limit - sizeof(args);
+ int ret;
+
+ ret = nvkm_falcon_get(&pmu->falcon, &pmu->subdev);
+ if (ret)
+ return ret;
+
+ pmu->initmsg_received = false;
+
+ nvkm_falcon_load_dmem(falcon, &args, addr_args, sizeof(args), 0);
+ nvkm_falcon_start(falcon);
+ return 0;
+}
+
+const struct nvkm_pmu_func
gm20b_pmu = {
.flcn = &gm200_pmu_flcn,
- .enabled = gf100_pmu_enabled,
+ .init = gm20b_pmu_init,
+ .fini = gm20b_pmu_fini,
.intr = gt215_pmu_intr,
.recv = gm20b_pmu_recv,
.initmsg = gm20b_pmu_initmsg,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c
index 1a6f9c3a..cd31483 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.c
@@ -23,25 +23,25 @@
*/
#include "priv.h"
-void
-gp102_pmu_reset(struct nvkm_pmu *pmu)
-{
- struct nvkm_device *device = pmu->subdev.device;
- nvkm_mask(device, 0x10a3c0, 0x00000001, 0x00000001);
- nvkm_mask(device, 0x10a3c0, 0x00000001, 0x00000000);
-}
-
-static bool
-gp102_pmu_enabled(struct nvkm_pmu *pmu)
-{
- return !(nvkm_rd32(pmu->subdev.device, 0x10a3c0) & 0x00000001);
-}
+static const struct nvkm_falcon_func
+gp102_pmu_flcn = {
+ .disable = gm200_flcn_disable,
+ .enable = gm200_flcn_enable,
+ .reset_eng = gp102_flcn_reset_eng,
+ .reset_wait_mem_scrubbing = gm200_flcn_reset_wait_mem_scrubbing,
+ .debug = 0xc08,
+ .bind_inst = gm200_pmu_flcn_bind_inst,
+ .bind_stat = gm200_flcn_bind_stat,
+ .imem_pio = &gm200_flcn_imem_pio,
+ .dmem_pio = &gm200_flcn_dmem_pio,
+ .start = nvkm_falcon_v1_start,
+ .cmdq = { 0x4a0, 0x4b0, 4 },
+ .msgq = { 0x4c8, 0x4cc, 0 },
+};
static const struct nvkm_pmu_func
gp102_pmu = {
- .flcn = &gm200_pmu_flcn,
- .enabled = gp102_pmu_enabled,
- .reset = gp102_pmu_reset,
+ .flcn = &gp102_pmu_flcn,
};
static const struct nvkm_pmu_fwif
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c
index 94cfb179..a6f410b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.c
@@ -68,7 +68,6 @@ gp10b_pmu_acr = {
.bld_size = sizeof(struct loader_config),
.bld_write = gm20b_pmu_acr_bld_write,
.bld_patch = gm20b_pmu_acr_bld_patch,
- .boot = gm20b_pmu_acr_boot,
.bootstrap_falcons = BIT_ULL(NVKM_ACR_LSF_PMU) |
BIT_ULL(NVKM_ACR_LSF_FECS) |
BIT_ULL(NVKM_ACR_LSF_GPCCS),
@@ -76,16 +75,6 @@ gp10b_pmu_acr = {
.bootstrap_multiple_falcons = gp10b_pmu_acr_bootstrap_multiple_falcons,
};
-static const struct nvkm_pmu_func
-gp10b_pmu = {
- .flcn = &gm200_pmu_flcn,
- .enabled = gf100_pmu_enabled,
- .intr = gt215_pmu_intr,
- .recv = gm20b_pmu_recv,
- .initmsg = gm20b_pmu_initmsg,
- .reset = gp102_pmu_reset,
-};
-
#if IS_ENABLED(CONFIG_ARCH_TEGRA_210_SOC)
MODULE_FIRMWARE("nvidia/gp10b/pmu/desc.bin");
MODULE_FIRMWARE("nvidia/gp10b/pmu/image.bin");
@@ -94,8 +83,8 @@ MODULE_FIRMWARE("nvidia/gp10b/pmu/sig.bin");
static const struct nvkm_pmu_fwif
gp10b_pmu_fwif[] = {
- { 0, gm20b_pmu_load, &gp10b_pmu, &gp10b_pmu_acr },
- { -1, gm200_pmu_nofw, &gp10b_pmu },
+ { 0, gm20b_pmu_load, &gm20b_pmu, &gp10b_pmu_acr },
+ { -1, gm200_pmu_nofw, &gm20b_pmu },
{}
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gt215.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gt215.c
index b0407b8..32cee21 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gt215.c
@@ -178,12 +178,14 @@ void
gt215_pmu_fini(struct nvkm_pmu *pmu)
{
nvkm_wr32(pmu->subdev.device, 0x10a014, 0x00000060);
+ flush_work(&pmu->recv.work);
}
static void
gt215_pmu_reset(struct nvkm_pmu *pmu)
{
struct nvkm_device *device = pmu->subdev.device;
+
nvkm_mask(device, 0x022210, 0x00000001, 0x00000000);
nvkm_mask(device, 0x022210, 0x00000001, 0x00000001);
nvkm_rd32(device, 0x022210);
@@ -201,6 +203,23 @@ gt215_pmu_init(struct nvkm_pmu *pmu)
struct nvkm_device *device = pmu->subdev.device;
int i;
+ /* Inhibit interrupts, and wait for idle. */
+ if (pmu->func->enabled(pmu)) {
+ nvkm_wr32(device, 0x10a014, 0x0000ffff);
+ nvkm_msec(device, 2000,
+ if (!nvkm_rd32(device, 0x10a04c))
+ break;
+ );
+ }
+
+ pmu->func->reset(pmu);
+
+ /* Wait for IMEM/DMEM scrubbing to be complete. */
+ nvkm_msec(device, 2000,
+ if (!(nvkm_rd32(device, 0x10a10c) & 0x00000006))
+ break;
+ );
+
/* upload data segment */
nvkm_wr32(device, 0x10a1c0, 0x01000000);
for (i = 0; i < pmu->func->data.size / 4; i++)
@@ -243,20 +262,6 @@ gt215_pmu_init(struct nvkm_pmu *pmu)
const struct nvkm_falcon_func
gt215_pmu_flcn = {
- .debug = 0xc08,
- .fbif = 0xe00,
- .load_imem = nvkm_falcon_v1_load_imem,
- .load_dmem = nvkm_falcon_v1_load_dmem,
- .read_dmem = nvkm_falcon_v1_read_dmem,
- .bind_context = nvkm_falcon_v1_bind_context,
- .wait_for_halt = nvkm_falcon_v1_wait_for_halt,
- .clear_interrupt = nvkm_falcon_v1_clear_interrupt,
- .set_start_addr = nvkm_falcon_v1_set_start_addr,
- .start = nvkm_falcon_v1_start,
- .enable = nvkm_falcon_v1_enable,
- .disable = nvkm_falcon_v1_disable,
- .cmdq = { 0x4a0, 0x4b0, 4 },
- .msgq = { 0x4c8, 0x4cc, 0 },
};
static const struct nvkm_pmu_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h
index 21abf31..2d0a8fa 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/priv.h
@@ -46,13 +46,12 @@ void gp102_pmu_reset(struct nvkm_pmu *pmu);
void gk110_pmu_pgob(struct nvkm_pmu *, bool);
extern const struct nvkm_falcon_func gm200_pmu_flcn;
+void gm200_pmu_flcn_bind_inst(struct nvkm_falcon *, int, u64);
+extern const struct nvkm_pmu_func gm20b_pmu;
void gm20b_pmu_acr_bld_patch(struct nvkm_acr *, u32, s64);
void gm20b_pmu_acr_bld_write(struct nvkm_acr *, u32, struct nvkm_acr_lsfw *);
-int gm20b_pmu_acr_boot(struct nvkm_falcon *);
int gm20b_pmu_acr_bootstrap_falcon(struct nvkm_falcon *, enum nvkm_acr_lsf_id);
-void gm20b_pmu_recv(struct nvkm_pmu *);
-int gm20b_pmu_initmsg(struct nvkm_pmu *);
struct nvkm_pmu_fwif {
int version;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/top/base.c
index 28d0789..eb348df 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/top/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/base.c
@@ -117,11 +117,15 @@ nvkm_top_fault(struct nvkm_device *device, int fault)
return NULL;
}
-static int
-nvkm_top_oneinit(struct nvkm_subdev *subdev)
+int
+nvkm_top_parse(struct nvkm_device *device)
{
- struct nvkm_top *top = nvkm_top(subdev);
- return top->func->oneinit(top);
+ struct nvkm_top *top = device->top;
+
+ if (!top || !list_empty(&top->device))
+ return 0;
+
+ return top->func->parse(top);
}
static void *
@@ -141,7 +145,6 @@ nvkm_top_dtor(struct nvkm_subdev *subdev)
static const struct nvkm_subdev_func
nvkm_top = {
.dtor = nvkm_top_dtor,
- .oneinit = nvkm_top_oneinit,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/ga100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/top/ga100.c
index c982d83..84790cf5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/top/ga100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/ga100.c
@@ -22,7 +22,7 @@
#include "priv.h"
static int
-ga100_top_oneinit(struct nvkm_top *top)
+ga100_top_parse(struct nvkm_top *top)
{
struct nvkm_subdev *subdev = &top->subdev;
struct nvkm_device *device = subdev->device;
@@ -97,7 +97,7 @@ ga100_top_oneinit(struct nvkm_top *top)
static const struct nvkm_top_func
ga100_top = {
- .oneinit = ga100_top_oneinit,
+ .parse = ga100_top_parse,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.c
index 4dcad97..2bbba82 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.c
@@ -24,7 +24,7 @@
#include "priv.h"
static int
-gk104_top_oneinit(struct nvkm_top *top)
+gk104_top_parse(struct nvkm_top *top)
{
struct nvkm_subdev *subdev = &top->subdev;
struct nvkm_device *device = subdev->device;
@@ -108,7 +108,7 @@ gk104_top_oneinit(struct nvkm_top *top)
static const struct nvkm_top_func
gk104_top = {
- .oneinit = gk104_top_oneinit,
+ .parse = gk104_top_parse,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/top/priv.h
index 8e103a8..532be91 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/top/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/priv.h
@@ -5,7 +5,7 @@
#include <subdev/top.h>
struct nvkm_top_func {
- int (*oneinit)(struct nvkm_top *);
+ int (*parse)(struct nvkm_top *);
};
int nvkm_top_new_(const struct nvkm_top_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/Kbuild
new file mode 100644
index 0000000..23cd21b
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/Kbuild
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: MIT
+nvkm-y += nvkm/subdev/vfn/base.o
+nvkm-y += nvkm/subdev/vfn/uvfn.o
+nvkm-y += nvkm/subdev/vfn/gv100.o
+nvkm-y += nvkm/subdev/vfn/tu102.o
+nvkm-y += nvkm/subdev/vfn/ga100.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/base.c
new file mode 100644
index 0000000..62e81d5
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/base.c
@@ -0,0 +1,60 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+static void *
+nvkm_vfn_dtor(struct nvkm_subdev *subdev)
+{
+ return nvkm_vfn(subdev);
+}
+
+static const struct nvkm_subdev_func
+nvkm_vfn = {
+ .dtor = nvkm_vfn_dtor,
+};
+
+int
+nvkm_vfn_new_(const struct nvkm_vfn_func *func, struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, u32 addr, struct nvkm_vfn **pvfn)
+{
+ struct nvkm_vfn *vfn;
+ int ret;
+
+ if (!(vfn = *pvfn = kzalloc(sizeof(*vfn), GFP_KERNEL)))
+ return -ENOMEM;
+
+ nvkm_subdev_ctor(&nvkm_vfn, device, type, inst, &vfn->subdev);
+ vfn->func = func;
+ vfn->addr.priv = addr;
+ vfn->addr.user = vfn->addr.priv + func->user.addr;
+
+ if (vfn->func->intr) {
+ ret = nvkm_intr_add(vfn->func->intr, vfn->func->intrs,
+ &vfn->subdev, 8, &vfn->intr);
+ if (ret)
+ return ret;
+ }
+
+ vfn->user.ctor = nvkm_uvfn_new;
+ vfn->user.base = func->user.base;
+ return 0;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/ga100.c
similarity index 61%
copy from drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
copy to drivers/gpu/drm/nouveau/nvkm/subdev/vfn/ga100.c
index 217268f..fd5c693 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usertu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/ga100.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2018 Red Hat Inc.
+ * Copyright 2021 Red Hat Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -19,27 +19,29 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include "user.h"
+#include "priv.h"
-static int
-tu102_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc,
- enum nvkm_object_map *type, u64 *addr, u64 *size)
-{
- struct nvkm_device *device = object->engine->subdev.device;
- *addr = 0xbb0000 + device->func->resource_addr(device, 0);
- *size = 0x010000;
- *type = NVKM_OBJECT_MAP_IO;
- return 0;
-}
+#include <nvif/class.h>
-static const struct nvkm_object_func
-tu102_fifo_user = {
- .map = tu102_fifo_user_map,
+static const struct nvkm_intr_data
+ga100_vfn_intrs[] = {
+ { NVKM_ENGINE_DISP , 0, 4, 0x04000000, true },
+ { NVKM_SUBDEV_GPIO , 0, 4, 0x00200000, true },
+ { NVKM_SUBDEV_I2C , 0, 4, 0x00200000, true },
+ { NVKM_SUBDEV_PRIVRING, 0, 4, 0x40000000, true },
+ {}
+};
+
+static const struct nvkm_vfn_func
+ga100_vfn = {
+ .intr = &tu102_vfn_intr,
+ .intrs = ga100_vfn_intrs,
+ .user = { 0x030000, 0x010000, { -1, -1, AMPERE_USERMODE_A } },
};
int
-tu102_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
- struct nvkm_object **pobject)
+ga100_vfn_new(struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_vfn **pvfn)
{
- return nvkm_object_new_(&tu102_fifo_user, oclass, argv, argc, pobject);
+ return nvkm_vfn_new_(&ga100_vfn, device, type, inst, 0xb80000, pvfn);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usergv100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/gv100.c
similarity index 63%
rename from drivers/gpu/drm/nouveau/nvkm/engine/fifo/usergv100.c
rename to drivers/gpu/drm/nouveau/nvkm/subdev/vfn/gv100.c
index 3dc3b8b..ddd39d7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/usergv100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/gv100.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2018 Red Hat Inc.
+ * Copyright 2021 Red Hat Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -19,27 +19,18 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
-#include "user.h"
+#include "priv.h"
-static int
-gv100_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc,
- enum nvkm_object_map *type, u64 *addr, u64 *size)
-{
- struct nvkm_device *device = object->engine->subdev.device;
- *addr = 0x810000 + device->func->resource_addr(device, 0);
- *size = 0x010000;
- *type = NVKM_OBJECT_MAP_IO;
- return 0;
-}
+#include <nvif/class.h>
-static const struct nvkm_object_func
-gv100_fifo_user = {
- .map = gv100_fifo_user_map,
+static const struct nvkm_vfn_func
+gv100_vfn = {
+ .user = { 0x810000, 0x010000, { -1, -1, VOLTA_USERMODE_A } },
};
int
-gv100_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc,
- struct nvkm_object **pobject)
+gv100_vfn_new(struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_vfn **pvfn)
{
- return nvkm_object_new_(&gv100_fifo_user, oclass, argv, argc, pobject);
+ return nvkm_vfn_new_(&gv100_vfn, device, type, inst, 0, pvfn);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/priv.h
new file mode 100644
index 0000000..96d53c0
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/priv.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: MIT */
+#ifndef __NVKM_VFN_PRIV_H__
+#define __NVKM_VFN_PRIV_H__
+#define nvkm_vfn(p) container_of((p), struct nvkm_vfn, subdev)
+#include <subdev/vfn.h>
+
+struct nvkm_vfn_func {
+ const struct nvkm_intr_func *intr;
+ const struct nvkm_intr_data *intrs;
+
+ struct {
+ u32 addr;
+ u32 size;
+ const struct nvkm_sclass base;
+ } user;
+};
+
+int nvkm_vfn_new_(const struct nvkm_vfn_func *, struct nvkm_device *, enum nvkm_subdev_type, int,
+ u32 addr, struct nvkm_vfn **);
+
+extern const struct nvkm_intr_func tu102_vfn_intr;
+
+int nvkm_uvfn_new(struct nvkm_device *, const struct nvkm_oclass *, void *, u32,
+ struct nvkm_object **);
+#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/tu102.c b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/tu102.c
new file mode 100644
index 0000000..3d063fb
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/tu102.c
@@ -0,0 +1,108 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "priv.h"
+
+#include <nvif/class.h>
+
+static void
+tu102_vfn_intr_reset(struct nvkm_intr *intr, int leaf, u32 mask)
+{
+ struct nvkm_vfn *vfn = container_of(intr, typeof(*vfn), intr);
+
+ nvkm_wr32(vfn->subdev.device, vfn->addr.priv + 0x1000 + (leaf * 4), mask);
+}
+
+static void
+tu102_vfn_intr_allow(struct nvkm_intr *intr, int leaf, u32 mask)
+{
+ struct nvkm_vfn *vfn = container_of(intr, typeof(*vfn), intr);
+
+ nvkm_wr32(vfn->subdev.device, vfn->addr.priv + 0x1200 + (leaf * 4), mask);
+}
+
+static void
+tu102_vfn_intr_block(struct nvkm_intr *intr, int leaf, u32 mask)
+{
+ struct nvkm_vfn *vfn = container_of(intr, typeof(*vfn), intr);
+
+ nvkm_wr32(vfn->subdev.device, vfn->addr.priv + 0x1400 + (leaf * 4), mask);
+}
+
+static void
+tu102_vfn_intr_rearm(struct nvkm_intr *intr)
+{
+ struct nvkm_vfn *vfn = container_of(intr, typeof(*vfn), intr);
+
+ nvkm_wr32(vfn->subdev.device, vfn->addr.priv + 0x1608, 0x0000000f);
+}
+
+static void
+tu102_vfn_intr_unarm(struct nvkm_intr *intr)
+{
+ struct nvkm_vfn *vfn = container_of(intr, typeof(*vfn), intr);
+
+ nvkm_wr32(vfn->subdev.device, vfn->addr.priv + 0x1610, 0x0000000f);
+}
+
+static bool
+tu102_vfn_intr_pending(struct nvkm_intr *intr)
+{
+ struct nvkm_vfn *vfn = container_of(intr, typeof(*vfn), intr);
+ struct nvkm_device *device = vfn->subdev.device;
+ u32 intr_top = nvkm_rd32(device, vfn->addr.priv + 0x1600);
+ int pending = 0, leaf;
+
+ for (leaf = 0; leaf < 8; leaf++) {
+ if (intr_top & BIT(leaf / 2)) {
+ intr->stat[leaf] = nvkm_rd32(device, vfn->addr.priv + 0x1000 + (leaf * 4));
+ if (intr->stat[leaf])
+ pending++;
+ } else {
+ intr->stat[leaf] = 0;
+ }
+ }
+
+ return pending != 0;
+}
+
+const struct nvkm_intr_func
+tu102_vfn_intr = {
+ .pending = tu102_vfn_intr_pending,
+ .unarm = tu102_vfn_intr_unarm,
+ .rearm = tu102_vfn_intr_rearm,
+ .block = tu102_vfn_intr_block,
+ .allow = tu102_vfn_intr_allow,
+ .reset = tu102_vfn_intr_reset,
+};
+
+static const struct nvkm_vfn_func
+tu102_vfn = {
+ .intr = &tu102_vfn_intr,
+ .user = { 0x030000, 0x010000, { -1, -1, TURING_USERMODE_A } },
+};
+
+int
+tu102_vfn_new(struct nvkm_device *device,
+ enum nvkm_subdev_type type, int inst, struct nvkm_vfn **pvfn)
+{
+ return nvkm_vfn_new_(&tu102_vfn, device, type, inst, 0xb80000, pvfn);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/uvfn.c b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/uvfn.c
new file mode 100644
index 0000000..c5460a1
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/vfn/uvfn.c
@@ -0,0 +1,67 @@
+/*
+ * Copyright 2021 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#define nvkm_uvfn(p) container_of((p), struct nvkm_uvfn, object)
+#include "priv.h"
+
+#include <core/object.h>
+
+struct nvkm_uvfn {
+ struct nvkm_object object;
+ struct nvkm_vfn *vfn;
+};
+
+static int
+nvkm_uvfn_map(struct nvkm_object *object, void *argv, u32 argc,
+ enum nvkm_object_map *type, u64 *addr, u64 *size)
+{
+ struct nvkm_vfn *vfn = nvkm_uvfn(object)->vfn;
+ struct nvkm_device *device = vfn->subdev.device;
+
+ *addr = device->func->resource_addr(device, 0) + vfn->addr.user;
+ *size = vfn->func->user.size;
+ *type = NVKM_OBJECT_MAP_IO;
+ return 0;
+}
+
+static const struct nvkm_object_func
+nvkm_uvfn = {
+ .map = nvkm_uvfn_map,
+};
+
+int
+nvkm_uvfn_new(struct nvkm_device *device, const struct nvkm_oclass *oclass,
+ void *argv, u32 argc, struct nvkm_object **pobject)
+{
+ struct nvkm_uvfn *uvfn;
+
+ if (argc != 0)
+ return -ENOSYS;
+
+ if (!(uvfn = kzalloc(sizeof(*uvfn), GFP_KERNEL)))
+ return -ENOMEM;
+
+ nvkm_object_ctor(&nvkm_uvfn, oclass, &uvfn->object);
+ uvfn->vfn = device->vfn;
+
+ *pobject = &uvfn->object;
+ return 0;
+}
diff --git a/drivers/gpu/drm/omapdrm/omap_fbdev.c b/drivers/gpu/drm/omapdrm/omap_fbdev.c
index 40706c5..98d8758 100644
--- a/drivers/gpu/drm/omapdrm/omap_fbdev.c
+++ b/drivers/gpu/drm/omapdrm/omap_fbdev.c
@@ -38,7 +38,7 @@ static struct drm_fb_helper *get_fb(struct fb_info *fbi);
static void pan_worker(struct work_struct *work)
{
struct omap_fbdev *fbdev = container_of(work, struct omap_fbdev, work);
- struct fb_info *fbi = fbdev->base.fbdev;
+ struct fb_info *fbi = fbdev->base.info;
int npages;
/* DMM roll shifts in 4K pages: */
@@ -161,7 +161,7 @@ static int omap_fbdev_create(struct drm_fb_helper *helper,
goto fail;
}
- fbi = drm_fb_helper_alloc_fbi(helper);
+ fbi = drm_fb_helper_alloc_info(helper);
if (IS_ERR(fbi)) {
dev_err(dev->dev, "failed to allocate fb info\n");
ret = PTR_ERR(fbi);
@@ -177,8 +177,6 @@ static int omap_fbdev_create(struct drm_fb_helper *helper,
drm_fb_helper_fill_info(fbi, helper, sizes);
- dev->mode_config.fb_base = dma_addr;
-
fbi->screen_buffer = omap_gem_vaddr(fbdev->bo);
fbi->screen_size = fbdev->bo->size;
fbi->fix.smem_start = dma_addr;
@@ -274,7 +272,7 @@ void omap_fbdev_fini(struct drm_device *dev)
if (!helper)
return;
- drm_fb_helper_unregister_fbi(helper);
+ drm_fb_helper_unregister_info(helper);
drm_fb_helper_fini(helper);
diff --git a/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c b/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c
index 393f82e..3abc475 100644
--- a/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c
+++ b/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c
@@ -66,6 +66,8 @@ static int omap_gem_dmabuf_mmap(struct dma_buf *buffer,
struct drm_gem_object *obj = buffer->priv;
int ret = 0;
+ dma_resv_assert_held(buffer->resv);
+
ret = drm_gem_mmap_obj(obj, omap_gem_mmap_size(obj), vma);
if (ret < 0)
return ret;
@@ -125,7 +127,7 @@ struct drm_gem_object *omap_gem_prime_import(struct drm_device *dev,
get_dma_buf(dma_buf);
- sgt = dma_buf_map_attachment(attach, DMA_TO_DEVICE);
+ sgt = dma_buf_map_attachment_unlocked(attach, DMA_TO_DEVICE);
if (IS_ERR(sgt)) {
ret = PTR_ERR(sgt);
goto fail_detach;
@@ -142,7 +144,7 @@ struct drm_gem_object *omap_gem_prime_import(struct drm_device *dev,
return obj;
fail_unmap:
- dma_buf_unmap_attachment(attach, sgt, DMA_TO_DEVICE);
+ dma_buf_unmap_attachment_unlocked(attach, sgt, DMA_TO_DEVICE);
fail_detach:
dma_buf_detach(dma_buf, attach);
dma_buf_put(dma_buf);
diff --git a/drivers/gpu/drm/panel/Kconfig b/drivers/gpu/drm/panel/Kconfig
index a582ddd..737edcd 100644
--- a/drivers/gpu/drm/panel/Kconfig
+++ b/drivers/gpu/drm/panel/Kconfig
@@ -203,6 +203,16 @@
24 bit RGB per pixel. It provides a MIPI DSI interface to
the host and has a built-in LED backlight.
+config DRM_PANEL_JADARD_JD9365DA_H3
+ tristate "Jadard JD9365DA-H3 WXGA DSI panel"
+ depends on OF
+ depends on DRM_MIPI_DSI
+ depends on BACKLIGHT_CLASS_DEVICE
+ help
+ Say Y here if you want to enable support for Jadard JD9365DA-H3
+ WXGA MIPI DSI panel. The panel support TFT dot matrix LCD with
+ 800RGBx1280 dots at maximum.
+
config DRM_PANEL_JDI_LT070ME05000
tristate "JDI LT070ME05000 WUXGA DSI panel"
depends on OF
@@ -296,6 +306,15 @@
panel (found on the Zoom2/3/3630 SDP boards). To compile this driver
as a module, choose M here.
+config DRM_PANEL_NEWVISION_NV3051D
+ tristate "NewVision NV3051D DSI panel"
+ depends on OF
+ depends on DRM_MIPI_DSI
+ depends on BACKLIGHT_CLASS_DEVICE
+ help
+ This driver supports the NV3051D based panel found on the Anbernic
+ RG353P and RG353V.
+
config DRM_PANEL_NEWVISION_NV3052C
tristate "NewVision NV3052C RGB/SPI panel"
depends on OF && SPI
diff --git a/drivers/gpu/drm/panel/Makefile b/drivers/gpu/drm/panel/Makefile
index 34e7173..f8f9d9f 100644
--- a/drivers/gpu/drm/panel/Makefile
+++ b/drivers/gpu/drm/panel/Makefile
@@ -18,6 +18,7 @@
obj-$(CONFIG_DRM_PANEL_ILITEK_ILI9881C) += panel-ilitek-ili9881c.o
obj-$(CONFIG_DRM_PANEL_INNOLUX_EJ030NA) += panel-innolux-ej030na.o
obj-$(CONFIG_DRM_PANEL_INNOLUX_P079ZCA) += panel-innolux-p079zca.o
+obj-$(CONFIG_DRM_PANEL_JADARD_JD9365DA_H3) += panel-jadard-jd9365da-h3.o
obj-$(CONFIG_DRM_PANEL_JDI_LT070ME05000) += panel-jdi-lt070me05000.o
obj-$(CONFIG_DRM_PANEL_JDI_R63452) += panel-jdi-fhd-r63452.o
obj-$(CONFIG_DRM_PANEL_KHADAS_TS050) += panel-khadas-ts050.o
@@ -27,6 +28,7 @@
obj-$(CONFIG_DRM_PANEL_LG_LB035Q02) += panel-lg-lb035q02.o
obj-$(CONFIG_DRM_PANEL_LG_LG4573) += panel-lg-lg4573.o
obj-$(CONFIG_DRM_PANEL_NEC_NL8048HL11) += panel-nec-nl8048hl11.o
+obj-$(CONFIG_DRM_PANEL_NEWVISION_NV3051D) += panel-newvision-nv3051d.o
obj-$(CONFIG_DRM_PANEL_NEWVISION_NV3052C) += panel-newvision-nv3052c.o
obj-$(CONFIG_DRM_PANEL_NOVATEK_NT35510) += panel-novatek-nt35510.o
obj-$(CONFIG_DRM_PANEL_NOVATEK_NT35560) += panel-novatek-nt35560.o
diff --git a/drivers/gpu/drm/panel/panel-edp.c b/drivers/gpu/drm/panel/panel-edp.c
index 4b39d1d..5cb8dc2 100644
--- a/drivers/gpu/drm/panel/panel-edp.c
+++ b/drivers/gpu/drm/panel/panel-edp.c
@@ -1883,8 +1883,10 @@ static const struct edp_panel_entry edp_panels[] = {
EDP_PANEL_ENTRY('B', 'O', 'E', 0x0a5d, &delay_200_500_e50, "NV116WHM-N45"),
EDP_PANEL_ENTRY('B', 'O', 'E', 0x0ac5, &delay_200_500_e50, "NV116WHM-N4C"),
+ EDP_PANEL_ENTRY('C', 'M', 'N', 0x1139, &delay_200_500_e80_d50, "N116BGE-EA2"),
EDP_PANEL_ENTRY('C', 'M', 'N', 0x114c, &innolux_n116bca_ea1.delay, "N116BCA-EA1"),
EDP_PANEL_ENTRY('C', 'M', 'N', 0x1152, &delay_200_500_e80_d50, "N116BCN-EA1"),
+ EDP_PANEL_ENTRY('C', 'M', 'N', 0x1153, &delay_200_500_e80_d50, "N116BGE-EA2"),
EDP_PANEL_ENTRY('C', 'M', 'N', 0x1154, &delay_200_500_e80_d50, "N116BCA-EA2"),
EDP_PANEL_ENTRY('C', 'M', 'N', 0x1247, &delay_200_500_e80_d50, "N120ACA-EA1"),
diff --git a/drivers/gpu/drm/panel/panel-ilitek-ili9341.c b/drivers/gpu/drm/panel/panel-ilitek-ili9341.c
index 39dc40c..384a724f2 100644
--- a/drivers/gpu/drm/panel/panel-ilitek-ili9341.c
+++ b/drivers/gpu/drm/panel/panel-ilitek-ili9341.c
@@ -18,6 +18,7 @@
* Copyright 2018 David Lechner <david@lechnology.com>
*/
+#include <linux/backlight.h>
#include <linux/bitops.h>
#include <linux/delay.h>
#include <linux/gpio/consumer.h>
@@ -30,7 +31,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
diff --git a/drivers/gpu/drm/panel/panel-jadard-jd9365da-h3.c b/drivers/gpu/drm/panel/panel-jadard-jd9365da-h3.c
new file mode 100644
index 0000000..48c1702
--- /dev/null
+++ b/drivers/gpu/drm/panel/panel-jadard-jd9365da-h3.c
@@ -0,0 +1,473 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (c) 2019 Radxa Limited
+ * Copyright (c) 2022 Edgeble AI Technologies Pvt. Ltd.
+ *
+ * Author:
+ * - Jagan Teki <jagan@amarulasolutions.com>
+ * - Stephen Chen <stephen@radxa.com>
+ */
+
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_panel.h>
+#include <drm/drm_print.h>
+
+#include <linux/gpio/consumer.h>
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/regulator/consumer.h>
+
+#define JD9365DA_INIT_CMD_LEN 2
+
+struct jadard_init_cmd {
+ u8 data[JD9365DA_INIT_CMD_LEN];
+};
+
+struct jadard_panel_desc {
+ const struct drm_display_mode mode;
+ unsigned int lanes;
+ enum mipi_dsi_pixel_format format;
+ const struct jadard_init_cmd *init_cmds;
+ u32 num_init_cmds;
+};
+
+struct jadard {
+ struct drm_panel panel;
+ struct mipi_dsi_device *dsi;
+ const struct jadard_panel_desc *desc;
+
+ struct regulator *vdd;
+ struct regulator *vccio;
+ struct gpio_desc *reset;
+};
+
+static inline struct jadard *panel_to_jadard(struct drm_panel *panel)
+{
+ return container_of(panel, struct jadard, panel);
+}
+
+static int jadard_enable(struct drm_panel *panel)
+{
+ struct device *dev = panel->dev;
+ struct jadard *jadard = panel_to_jadard(panel);
+ const struct jadard_panel_desc *desc = jadard->desc;
+ struct mipi_dsi_device *dsi = jadard->dsi;
+ unsigned int i;
+ int err;
+
+ msleep(10);
+
+ for (i = 0; i < desc->num_init_cmds; i++) {
+ const struct jadard_init_cmd *cmd = &desc->init_cmds[i];
+
+ err = mipi_dsi_dcs_write_buffer(dsi, cmd->data, JD9365DA_INIT_CMD_LEN);
+ if (err < 0)
+ return err;
+ }
+
+ msleep(120);
+
+ err = mipi_dsi_dcs_exit_sleep_mode(dsi);
+ if (err < 0)
+ DRM_DEV_ERROR(dev, "failed to exit sleep mode ret = %d\n", err);
+
+ err = mipi_dsi_dcs_set_display_on(dsi);
+ if (err < 0)
+ DRM_DEV_ERROR(dev, "failed to set display on ret = %d\n", err);
+
+ return 0;
+}
+
+static int jadard_disable(struct drm_panel *panel)
+{
+ struct device *dev = panel->dev;
+ struct jadard *jadard = panel_to_jadard(panel);
+ int ret;
+
+ ret = mipi_dsi_dcs_set_display_off(jadard->dsi);
+ if (ret < 0)
+ DRM_DEV_ERROR(dev, "failed to set display off: %d\n", ret);
+
+ ret = mipi_dsi_dcs_enter_sleep_mode(jadard->dsi);
+ if (ret < 0)
+ DRM_DEV_ERROR(dev, "failed to enter sleep mode: %d\n", ret);
+
+ return 0;
+}
+
+static int jadard_prepare(struct drm_panel *panel)
+{
+ struct jadard *jadard = panel_to_jadard(panel);
+ int ret;
+
+ ret = regulator_enable(jadard->vccio);
+ if (ret)
+ return ret;
+
+ ret = regulator_enable(jadard->vdd);
+ if (ret)
+ return ret;
+
+ gpiod_set_value(jadard->reset, 1);
+ msleep(5);
+
+ gpiod_set_value(jadard->reset, 0);
+ msleep(10);
+
+ gpiod_set_value(jadard->reset, 1);
+ msleep(120);
+
+ return 0;
+}
+
+static int jadard_unprepare(struct drm_panel *panel)
+{
+ struct jadard *jadard = panel_to_jadard(panel);
+
+ gpiod_set_value(jadard->reset, 1);
+ msleep(120);
+
+ regulator_disable(jadard->vdd);
+ regulator_disable(jadard->vccio);
+
+ return 0;
+}
+
+static int jadard_get_modes(struct drm_panel *panel,
+ struct drm_connector *connector)
+{
+ struct jadard *jadard = panel_to_jadard(panel);
+ const struct drm_display_mode *desc_mode = &jadard->desc->mode;
+ struct drm_display_mode *mode;
+
+ mode = drm_mode_duplicate(connector->dev, desc_mode);
+ if (!mode) {
+ DRM_DEV_ERROR(&jadard->dsi->dev, "failed to add mode %ux%ux@%u\n",
+ desc_mode->hdisplay, desc_mode->vdisplay,
+ drm_mode_vrefresh(desc_mode));
+ return -ENOMEM;
+ }
+
+ drm_mode_set_name(mode);
+ drm_mode_probed_add(connector, mode);
+
+ connector->display_info.width_mm = mode->width_mm;
+ connector->display_info.height_mm = mode->height_mm;
+
+ return 1;
+}
+
+static const struct drm_panel_funcs jadard_funcs = {
+ .disable = jadard_disable,
+ .unprepare = jadard_unprepare,
+ .prepare = jadard_prepare,
+ .enable = jadard_enable,
+ .get_modes = jadard_get_modes,
+};
+
+static const struct jadard_init_cmd cz101b4001_init_cmds[] = {
+ { .data = { 0xE0, 0x00 } },
+ { .data = { 0xE1, 0x93 } },
+ { .data = { 0xE2, 0x65 } },
+ { .data = { 0xE3, 0xF8 } },
+ { .data = { 0x80, 0x03 } },
+ { .data = { 0xE0, 0x01 } },
+ { .data = { 0x00, 0x00 } },
+ { .data = { 0x01, 0x3B } },
+ { .data = { 0x0C, 0x74 } },
+ { .data = { 0x17, 0x00 } },
+ { .data = { 0x18, 0xAF } },
+ { .data = { 0x19, 0x00 } },
+ { .data = { 0x1A, 0x00 } },
+ { .data = { 0x1B, 0xAF } },
+ { .data = { 0x1C, 0x00 } },
+ { .data = { 0x35, 0x26 } },
+ { .data = { 0x37, 0x09 } },
+ { .data = { 0x38, 0x04 } },
+ { .data = { 0x39, 0x00 } },
+ { .data = { 0x3A, 0x01 } },
+ { .data = { 0x3C, 0x78 } },
+ { .data = { 0x3D, 0xFF } },
+ { .data = { 0x3E, 0xFF } },
+ { .data = { 0x3F, 0x7F } },
+ { .data = { 0x40, 0x06 } },
+ { .data = { 0x41, 0xA0 } },
+ { .data = { 0x42, 0x81 } },
+ { .data = { 0x43, 0x14 } },
+ { .data = { 0x44, 0x23 } },
+ { .data = { 0x45, 0x28 } },
+ { .data = { 0x55, 0x02 } },
+ { .data = { 0x57, 0x69 } },
+ { .data = { 0x59, 0x0A } },
+ { .data = { 0x5A, 0x2A } },
+ { .data = { 0x5B, 0x17 } },
+ { .data = { 0x5D, 0x7F } },
+ { .data = { 0x5E, 0x6B } },
+ { .data = { 0x5F, 0x5C } },
+ { .data = { 0x60, 0x4F } },
+ { .data = { 0x61, 0x4D } },
+ { .data = { 0x62, 0x3F } },
+ { .data = { 0x63, 0x42 } },
+ { .data = { 0x64, 0x2B } },
+ { .data = { 0x65, 0x44 } },
+ { .data = { 0x66, 0x43 } },
+ { .data = { 0x67, 0x43 } },
+ { .data = { 0x68, 0x63 } },
+ { .data = { 0x69, 0x52 } },
+ { .data = { 0x6A, 0x5A } },
+ { .data = { 0x6B, 0x4F } },
+ { .data = { 0x6C, 0x4E } },
+ { .data = { 0x6D, 0x20 } },
+ { .data = { 0x6E, 0x0F } },
+ { .data = { 0x6F, 0x00 } },
+ { .data = { 0x70, 0x7F } },
+ { .data = { 0x71, 0x6B } },
+ { .data = { 0x72, 0x5C } },
+ { .data = { 0x73, 0x4F } },
+ { .data = { 0x74, 0x4D } },
+ { .data = { 0x75, 0x3F } },
+ { .data = { 0x76, 0x42 } },
+ { .data = { 0x77, 0x2B } },
+ { .data = { 0x78, 0x44 } },
+ { .data = { 0x79, 0x43 } },
+ { .data = { 0x7A, 0x43 } },
+ { .data = { 0x7B, 0x63 } },
+ { .data = { 0x7C, 0x52 } },
+ { .data = { 0x7D, 0x5A } },
+ { .data = { 0x7E, 0x4F } },
+ { .data = { 0x7F, 0x4E } },
+ { .data = { 0x80, 0x20 } },
+ { .data = { 0x81, 0x0F } },
+ { .data = { 0x82, 0x00 } },
+ { .data = { 0xE0, 0x02 } },
+ { .data = { 0x00, 0x02 } },
+ { .data = { 0x01, 0x02 } },
+ { .data = { 0x02, 0x00 } },
+ { .data = { 0x03, 0x00 } },
+ { .data = { 0x04, 0x1E } },
+ { .data = { 0x05, 0x1E } },
+ { .data = { 0x06, 0x1F } },
+ { .data = { 0x07, 0x1F } },
+ { .data = { 0x08, 0x1F } },
+ { .data = { 0x09, 0x17 } },
+ { .data = { 0x0A, 0x17 } },
+ { .data = { 0x0B, 0x37 } },
+ { .data = { 0x0C, 0x37 } },
+ { .data = { 0x0D, 0x47 } },
+ { .data = { 0x0E, 0x47 } },
+ { .data = { 0x0F, 0x45 } },
+ { .data = { 0x10, 0x45 } },
+ { .data = { 0x11, 0x4B } },
+ { .data = { 0x12, 0x4B } },
+ { .data = { 0x13, 0x49 } },
+ { .data = { 0x14, 0x49 } },
+ { .data = { 0x15, 0x1F } },
+ { .data = { 0x16, 0x01 } },
+ { .data = { 0x17, 0x01 } },
+ { .data = { 0x18, 0x00 } },
+ { .data = { 0x19, 0x00 } },
+ { .data = { 0x1A, 0x1E } },
+ { .data = { 0x1B, 0x1E } },
+ { .data = { 0x1C, 0x1F } },
+ { .data = { 0x1D, 0x1F } },
+ { .data = { 0x1E, 0x1F } },
+ { .data = { 0x1F, 0x17 } },
+ { .data = { 0x20, 0x17 } },
+ { .data = { 0x21, 0x37 } },
+ { .data = { 0x22, 0x37 } },
+ { .data = { 0x23, 0x46 } },
+ { .data = { 0x24, 0x46 } },
+ { .data = { 0x25, 0x44 } },
+ { .data = { 0x26, 0x44 } },
+ { .data = { 0x27, 0x4A } },
+ { .data = { 0x28, 0x4A } },
+ { .data = { 0x29, 0x48 } },
+ { .data = { 0x2A, 0x48 } },
+ { .data = { 0x2B, 0x1F } },
+ { .data = { 0x2C, 0x01 } },
+ { .data = { 0x2D, 0x01 } },
+ { .data = { 0x2E, 0x00 } },
+ { .data = { 0x2F, 0x00 } },
+ { .data = { 0x30, 0x1F } },
+ { .data = { 0x31, 0x1F } },
+ { .data = { 0x32, 0x1E } },
+ { .data = { 0x33, 0x1E } },
+ { .data = { 0x34, 0x1F } },
+ { .data = { 0x35, 0x17 } },
+ { .data = { 0x36, 0x17 } },
+ { .data = { 0x37, 0x37 } },
+ { .data = { 0x38, 0x37 } },
+ { .data = { 0x39, 0x08 } },
+ { .data = { 0x3A, 0x08 } },
+ { .data = { 0x3B, 0x0A } },
+ { .data = { 0x3C, 0x0A } },
+ { .data = { 0x3D, 0x04 } },
+ { .data = { 0x3E, 0x04 } },
+ { .data = { 0x3F, 0x06 } },
+ { .data = { 0x40, 0x06 } },
+ { .data = { 0x41, 0x1F } },
+ { .data = { 0x42, 0x02 } },
+ { .data = { 0x43, 0x02 } },
+ { .data = { 0x44, 0x00 } },
+ { .data = { 0x45, 0x00 } },
+ { .data = { 0x46, 0x1F } },
+ { .data = { 0x47, 0x1F } },
+ { .data = { 0x48, 0x1E } },
+ { .data = { 0x49, 0x1E } },
+ { .data = { 0x4A, 0x1F } },
+ { .data = { 0x4B, 0x17 } },
+ { .data = { 0x4C, 0x17 } },
+ { .data = { 0x4D, 0x37 } },
+ { .data = { 0x4E, 0x37 } },
+ { .data = { 0x4F, 0x09 } },
+ { .data = { 0x50, 0x09 } },
+ { .data = { 0x51, 0x0B } },
+ { .data = { 0x52, 0x0B } },
+ { .data = { 0x53, 0x05 } },
+ { .data = { 0x54, 0x05 } },
+ { .data = { 0x55, 0x07 } },
+ { .data = { 0x56, 0x07 } },
+ { .data = { 0x57, 0x1F } },
+ { .data = { 0x58, 0x40 } },
+ { .data = { 0x5B, 0x30 } },
+ { .data = { 0x5C, 0x16 } },
+ { .data = { 0x5D, 0x34 } },
+ { .data = { 0x5E, 0x05 } },
+ { .data = { 0x5F, 0x02 } },
+ { .data = { 0x63, 0x00 } },
+ { .data = { 0x64, 0x6A } },
+ { .data = { 0x67, 0x73 } },
+ { .data = { 0x68, 0x1D } },
+ { .data = { 0x69, 0x08 } },
+ { .data = { 0x6A, 0x6A } },
+ { .data = { 0x6B, 0x08 } },
+ { .data = { 0x6C, 0x00 } },
+ { .data = { 0x6D, 0x00 } },
+ { .data = { 0x6E, 0x00 } },
+ { .data = { 0x6F, 0x88 } },
+ { .data = { 0x75, 0xFF } },
+ { .data = { 0x77, 0xDD } },
+ { .data = { 0x78, 0x3F } },
+ { .data = { 0x79, 0x15 } },
+ { .data = { 0x7A, 0x17 } },
+ { .data = { 0x7D, 0x14 } },
+ { .data = { 0x7E, 0x82 } },
+ { .data = { 0xE0, 0x04 } },
+ { .data = { 0x00, 0x0E } },
+ { .data = { 0x02, 0xB3 } },
+ { .data = { 0x09, 0x61 } },
+ { .data = { 0x0E, 0x48 } },
+ { .data = { 0xE0, 0x00 } },
+ { .data = { 0xE6, 0x02 } },
+ { .data = { 0xE7, 0x0C } },
+};
+
+static const struct jadard_panel_desc cz101b4001_desc = {
+ .mode = {
+ .clock = 70000,
+
+ .hdisplay = 800,
+ .hsync_start = 800 + 40,
+ .hsync_end = 800 + 40 + 18,
+ .htotal = 800 + 40 + 18 + 20,
+
+ .vdisplay = 1280,
+ .vsync_start = 1280 + 20,
+ .vsync_end = 1280 + 20 + 4,
+ .vtotal = 1280 + 20 + 4 + 20,
+
+ .width_mm = 62,
+ .height_mm = 110,
+ .type = DRM_MODE_TYPE_DRIVER | DRM_MODE_TYPE_PREFERRED,
+ },
+ .lanes = 4,
+ .format = MIPI_DSI_FMT_RGB888,
+ .init_cmds = cz101b4001_init_cmds,
+ .num_init_cmds = ARRAY_SIZE(cz101b4001_init_cmds),
+};
+
+static int jadard_dsi_probe(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ const struct jadard_panel_desc *desc;
+ struct jadard *jadard;
+ int ret;
+
+ jadard = devm_kzalloc(&dsi->dev, sizeof(*jadard), GFP_KERNEL);
+ if (!jadard)
+ return -ENOMEM;
+
+ desc = of_device_get_match_data(dev);
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
+ MIPI_DSI_MODE_NO_EOT_PACKET;
+ dsi->format = desc->format;
+ dsi->lanes = desc->lanes;
+
+ jadard->reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+ if (IS_ERR(jadard->reset)) {
+ DRM_DEV_ERROR(&dsi->dev, "failed to get our reset GPIO\n");
+ return PTR_ERR(jadard->reset);
+ }
+
+ jadard->vdd = devm_regulator_get(dev, "vdd");
+ if (IS_ERR(jadard->vdd)) {
+ DRM_DEV_ERROR(&dsi->dev, "failed to get vdd regulator\n");
+ return PTR_ERR(jadard->vdd);
+ }
+
+ jadard->vccio = devm_regulator_get(dev, "vccio");
+ if (IS_ERR(jadard->vccio)) {
+ DRM_DEV_ERROR(&dsi->dev, "failed to get vccio regulator\n");
+ return PTR_ERR(jadard->vccio);
+ }
+
+ drm_panel_init(&jadard->panel, dev, &jadard_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+
+ ret = drm_panel_of_backlight(&jadard->panel);
+ if (ret)
+ return ret;
+
+ drm_panel_add(&jadard->panel);
+
+ mipi_dsi_set_drvdata(dsi, jadard);
+ jadard->dsi = dsi;
+ jadard->desc = desc;
+
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0)
+ drm_panel_remove(&jadard->panel);
+
+ return ret;
+}
+
+static void jadard_dsi_remove(struct mipi_dsi_device *dsi)
+{
+ struct jadard *jadard = mipi_dsi_get_drvdata(dsi);
+
+ mipi_dsi_detach(dsi);
+ drm_panel_remove(&jadard->panel);
+}
+
+static const struct of_device_id jadard_of_match[] = {
+ { .compatible = "chongzhou,cz101b4001", .data = &cz101b4001_desc },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, jadard_of_match);
+
+static struct mipi_dsi_driver jadard_driver = {
+ .probe = jadard_dsi_probe,
+ .remove = jadard_dsi_remove,
+ .driver = {
+ .name = "jadard-jd9365da",
+ .of_match_table = jadard_of_match,
+ },
+};
+module_mipi_dsi_driver(jadard_driver);
+
+MODULE_AUTHOR("Jagan Teki <jagan@edgeble.ai>");
+MODULE_AUTHOR("Stephen Chen <stephen@radxa.com>");
+MODULE_DESCRIPTION("Jadard JD9365DA-H3 WXGA DSI panel");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/panel/panel-newvision-nv3051d.c b/drivers/gpu/drm/panel/panel-newvision-nv3051d.c
new file mode 100644
index 0000000..a079580
--- /dev/null
+++ b/drivers/gpu/drm/panel/panel-newvision-nv3051d.c
@@ -0,0 +1,504 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * NV3051D MIPI-DSI panel driver for Anbernic RG353x
+ * Copyright (C) 2022 Chris Morgan
+ *
+ * based on
+ *
+ * Elida kd35t133 3.5" MIPI-DSI panel driver
+ * Copyright (C) Theobroma Systems 2020
+ */
+
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/media-bus-format.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/regulator/consumer.h>
+
+#include <video/display_timing.h>
+#include <video/mipi_display.h>
+
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_panel.h>
+
+struct nv3051d_panel_info {
+ const struct drm_display_mode *display_modes;
+ unsigned int num_modes;
+ u16 width_mm, height_mm;
+ u32 bus_flags;
+};
+
+struct panel_nv3051d {
+ struct device *dev;
+ struct drm_panel panel;
+ struct gpio_desc *reset_gpio;
+ const struct nv3051d_panel_info *panel_info;
+ struct regulator *vdd;
+};
+
+static inline struct panel_nv3051d *panel_to_panelnv3051d(struct drm_panel *panel)
+{
+ return container_of(panel, struct panel_nv3051d, panel);
+}
+
+static int panel_nv3051d_init_sequence(struct panel_nv3051d *ctx)
+{
+ struct mipi_dsi_device *dsi = to_mipi_dsi_device(ctx->dev);
+
+ /*
+ * Init sequence was supplied by device vendor with no
+ * documentation.
+ */
+
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x30);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x52);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0xE3, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0x03, 0x40);
+ mipi_dsi_dcs_write_seq(dsi, 0x04, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0x05, 0x03);
+ mipi_dsi_dcs_write_seq(dsi, 0x24, 0x12);
+ mipi_dsi_dcs_write_seq(dsi, 0x25, 0x1E);
+ mipi_dsi_dcs_write_seq(dsi, 0x26, 0x28);
+ mipi_dsi_dcs_write_seq(dsi, 0x27, 0x52);
+ mipi_dsi_dcs_write_seq(dsi, 0x28, 0x57);
+ mipi_dsi_dcs_write_seq(dsi, 0x29, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0x2A, 0xDF);
+ mipi_dsi_dcs_write_seq(dsi, 0x38, 0x9C);
+ mipi_dsi_dcs_write_seq(dsi, 0x39, 0xA7);
+ mipi_dsi_dcs_write_seq(dsi, 0x3A, 0x53);
+ mipi_dsi_dcs_write_seq(dsi, 0x44, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0x49, 0x3C);
+ mipi_dsi_dcs_write_seq(dsi, 0x59, 0xFE);
+ mipi_dsi_dcs_write_seq(dsi, 0x5C, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0x91, 0x77);
+ mipi_dsi_dcs_write_seq(dsi, 0x92, 0x77);
+ mipi_dsi_dcs_write_seq(dsi, 0xA0, 0x55);
+ mipi_dsi_dcs_write_seq(dsi, 0xA1, 0x50);
+ mipi_dsi_dcs_write_seq(dsi, 0xA4, 0x9C);
+ mipi_dsi_dcs_write_seq(dsi, 0xA7, 0x02);
+ mipi_dsi_dcs_write_seq(dsi, 0xA8, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0xA9, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0xAA, 0xFC);
+ mipi_dsi_dcs_write_seq(dsi, 0xAB, 0x28);
+ mipi_dsi_dcs_write_seq(dsi, 0xAC, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0xAD, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0xAE, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0xAF, 0x03);
+ mipi_dsi_dcs_write_seq(dsi, 0xB0, 0x08);
+ mipi_dsi_dcs_write_seq(dsi, 0xB1, 0x26);
+ mipi_dsi_dcs_write_seq(dsi, 0xB2, 0x28);
+ mipi_dsi_dcs_write_seq(dsi, 0xB3, 0x28);
+ mipi_dsi_dcs_write_seq(dsi, 0xB4, 0x33);
+ mipi_dsi_dcs_write_seq(dsi, 0xB5, 0x08);
+ mipi_dsi_dcs_write_seq(dsi, 0xB6, 0x26);
+ mipi_dsi_dcs_write_seq(dsi, 0xB7, 0x08);
+ mipi_dsi_dcs_write_seq(dsi, 0xB8, 0x26);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x30);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x52);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x02);
+ mipi_dsi_dcs_write_seq(dsi, 0xB1, 0x0E);
+ mipi_dsi_dcs_write_seq(dsi, 0xD1, 0x0E);
+ mipi_dsi_dcs_write_seq(dsi, 0xB4, 0x29);
+ mipi_dsi_dcs_write_seq(dsi, 0xD4, 0x2B);
+ mipi_dsi_dcs_write_seq(dsi, 0xB2, 0x0C);
+ mipi_dsi_dcs_write_seq(dsi, 0xD2, 0x0A);
+ mipi_dsi_dcs_write_seq(dsi, 0xB3, 0x28);
+ mipi_dsi_dcs_write_seq(dsi, 0xD3, 0x28);
+ mipi_dsi_dcs_write_seq(dsi, 0xB6, 0x11);
+ mipi_dsi_dcs_write_seq(dsi, 0xD6, 0x0D);
+ mipi_dsi_dcs_write_seq(dsi, 0xB7, 0x32);
+ mipi_dsi_dcs_write_seq(dsi, 0xD7, 0x30);
+ mipi_dsi_dcs_write_seq(dsi, 0xC1, 0x04);
+ mipi_dsi_dcs_write_seq(dsi, 0xE1, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0xB8, 0x0A);
+ mipi_dsi_dcs_write_seq(dsi, 0xD8, 0x0A);
+ mipi_dsi_dcs_write_seq(dsi, 0xB9, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0xD9, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0xBD, 0x13);
+ mipi_dsi_dcs_write_seq(dsi, 0xDD, 0x13);
+ mipi_dsi_dcs_write_seq(dsi, 0xBC, 0x11);
+ mipi_dsi_dcs_write_seq(dsi, 0xDC, 0x11);
+ mipi_dsi_dcs_write_seq(dsi, 0xBB, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xDB, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xBA, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xDA, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xBE, 0x18);
+ mipi_dsi_dcs_write_seq(dsi, 0xDE, 0x18);
+ mipi_dsi_dcs_write_seq(dsi, 0xBF, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xDF, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xC0, 0x17);
+ mipi_dsi_dcs_write_seq(dsi, 0xE0, 0x17);
+ mipi_dsi_dcs_write_seq(dsi, 0xB5, 0x3B);
+ mipi_dsi_dcs_write_seq(dsi, 0xD5, 0x3C);
+ mipi_dsi_dcs_write_seq(dsi, 0xB0, 0x0B);
+ mipi_dsi_dcs_write_seq(dsi, 0xD0, 0x0C);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x30);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x52);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x03);
+ mipi_dsi_dcs_write_seq(dsi, 0x00, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x01, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x02, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x03, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x04, 0x61);
+ mipi_dsi_dcs_write_seq(dsi, 0x05, 0x80);
+ mipi_dsi_dcs_write_seq(dsi, 0x06, 0xC7);
+ mipi_dsi_dcs_write_seq(dsi, 0x07, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0x08, 0x82);
+ mipi_dsi_dcs_write_seq(dsi, 0x09, 0x83);
+ mipi_dsi_dcs_write_seq(dsi, 0x30, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x31, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x32, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x33, 0x2A);
+ mipi_dsi_dcs_write_seq(dsi, 0x34, 0x61);
+ mipi_dsi_dcs_write_seq(dsi, 0x35, 0xC5);
+ mipi_dsi_dcs_write_seq(dsi, 0x36, 0x80);
+ mipi_dsi_dcs_write_seq(dsi, 0x37, 0x23);
+ mipi_dsi_dcs_write_seq(dsi, 0x40, 0x82);
+ mipi_dsi_dcs_write_seq(dsi, 0x41, 0x83);
+ mipi_dsi_dcs_write_seq(dsi, 0x42, 0x80);
+ mipi_dsi_dcs_write_seq(dsi, 0x43, 0x81);
+ mipi_dsi_dcs_write_seq(dsi, 0x44, 0x11);
+ mipi_dsi_dcs_write_seq(dsi, 0x45, 0xF2);
+ mipi_dsi_dcs_write_seq(dsi, 0x46, 0xF1);
+ mipi_dsi_dcs_write_seq(dsi, 0x47, 0x11);
+ mipi_dsi_dcs_write_seq(dsi, 0x48, 0xF4);
+ mipi_dsi_dcs_write_seq(dsi, 0x49, 0xF3);
+ mipi_dsi_dcs_write_seq(dsi, 0x50, 0x02);
+ mipi_dsi_dcs_write_seq(dsi, 0x51, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0x52, 0x04);
+ mipi_dsi_dcs_write_seq(dsi, 0x53, 0x03);
+ mipi_dsi_dcs_write_seq(dsi, 0x54, 0x11);
+ mipi_dsi_dcs_write_seq(dsi, 0x55, 0xF6);
+ mipi_dsi_dcs_write_seq(dsi, 0x56, 0xF5);
+ mipi_dsi_dcs_write_seq(dsi, 0x57, 0x11);
+ mipi_dsi_dcs_write_seq(dsi, 0x58, 0xF8);
+ mipi_dsi_dcs_write_seq(dsi, 0x59, 0xF7);
+ mipi_dsi_dcs_write_seq(dsi, 0x7E, 0x02);
+ mipi_dsi_dcs_write_seq(dsi, 0x7F, 0x80);
+ mipi_dsi_dcs_write_seq(dsi, 0xE0, 0x5A);
+ mipi_dsi_dcs_write_seq(dsi, 0xB1, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0xB4, 0x0E);
+ mipi_dsi_dcs_write_seq(dsi, 0xB5, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xB6, 0x04);
+ mipi_dsi_dcs_write_seq(dsi, 0xB7, 0x07);
+ mipi_dsi_dcs_write_seq(dsi, 0xB8, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0xB9, 0x05);
+ mipi_dsi_dcs_write_seq(dsi, 0xBA, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xC7, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0xCA, 0x0E);
+ mipi_dsi_dcs_write_seq(dsi, 0xCB, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0xCC, 0x04);
+ mipi_dsi_dcs_write_seq(dsi, 0xCD, 0x07);
+ mipi_dsi_dcs_write_seq(dsi, 0xCE, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0xCF, 0x05);
+ mipi_dsi_dcs_write_seq(dsi, 0xD0, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0x81, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0x84, 0x0E);
+ mipi_dsi_dcs_write_seq(dsi, 0x85, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0x86, 0x07);
+ mipi_dsi_dcs_write_seq(dsi, 0x87, 0x04);
+ mipi_dsi_dcs_write_seq(dsi, 0x88, 0x05);
+ mipi_dsi_dcs_write_seq(dsi, 0x89, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0x8A, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0x97, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0x9A, 0x0E);
+ mipi_dsi_dcs_write_seq(dsi, 0x9B, 0x0F);
+ mipi_dsi_dcs_write_seq(dsi, 0x9C, 0x07);
+ mipi_dsi_dcs_write_seq(dsi, 0x9D, 0x04);
+ mipi_dsi_dcs_write_seq(dsi, 0x9E, 0x05);
+ mipi_dsi_dcs_write_seq(dsi, 0x9F, 0x06);
+ mipi_dsi_dcs_write_seq(dsi, 0xA0, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x30);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x52);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x02);
+ mipi_dsi_dcs_write_seq(dsi, 0x01, 0x01);
+ mipi_dsi_dcs_write_seq(dsi, 0x02, 0xDA);
+ mipi_dsi_dcs_write_seq(dsi, 0x03, 0xBA);
+ mipi_dsi_dcs_write_seq(dsi, 0x04, 0xA8);
+ mipi_dsi_dcs_write_seq(dsi, 0x05, 0x9A);
+ mipi_dsi_dcs_write_seq(dsi, 0x06, 0x70);
+ mipi_dsi_dcs_write_seq(dsi, 0x07, 0xFF);
+ mipi_dsi_dcs_write_seq(dsi, 0x08, 0x91);
+ mipi_dsi_dcs_write_seq(dsi, 0x09, 0x90);
+ mipi_dsi_dcs_write_seq(dsi, 0x0A, 0xFF);
+ mipi_dsi_dcs_write_seq(dsi, 0x0B, 0x8F);
+ mipi_dsi_dcs_write_seq(dsi, 0x0C, 0x60);
+ mipi_dsi_dcs_write_seq(dsi, 0x0D, 0x58);
+ mipi_dsi_dcs_write_seq(dsi, 0x0E, 0x48);
+ mipi_dsi_dcs_write_seq(dsi, 0x0F, 0x38);
+ mipi_dsi_dcs_write_seq(dsi, 0x10, 0x2B);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x30);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x52);
+ mipi_dsi_dcs_write_seq(dsi, 0xFF, 0x00);
+ mipi_dsi_dcs_write_seq(dsi, 0x36, 0x02);
+ mipi_dsi_dcs_write_seq(dsi, 0x3A, 0x70);
+
+ dev_dbg(ctx->dev, "Panel init sequence done\n");
+
+ return 0;
+}
+
+static int panel_nv3051d_unprepare(struct drm_panel *panel)
+{
+ struct panel_nv3051d *ctx = panel_to_panelnv3051d(panel);
+ struct mipi_dsi_device *dsi = to_mipi_dsi_device(ctx->dev);
+ int ret;
+
+ ret = mipi_dsi_dcs_set_display_off(dsi);
+ if (ret < 0)
+ dev_err(ctx->dev, "failed to set display off: %d\n", ret);
+
+ msleep(20);
+
+ ret = mipi_dsi_dcs_enter_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(ctx->dev, "failed to enter sleep mode: %d\n", ret);
+ return ret;
+ }
+
+ usleep_range(10000, 15000);
+
+ regulator_disable(ctx->vdd);
+
+ return 0;
+}
+
+static int panel_nv3051d_prepare(struct drm_panel *panel)
+{
+ struct panel_nv3051d *ctx = panel_to_panelnv3051d(panel);
+ struct mipi_dsi_device *dsi = to_mipi_dsi_device(ctx->dev);
+ int ret;
+
+ dev_dbg(ctx->dev, "Resetting the panel\n");
+ ret = regulator_enable(ctx->vdd);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Failed to enable vdd supply: %d\n", ret);
+ return ret;
+ }
+
+ usleep_range(2000, 3000);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 1);
+ msleep(150);
+ gpiod_set_value_cansleep(ctx->reset_gpio, 0);
+ msleep(20);
+
+ ret = panel_nv3051d_init_sequence(ctx);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Panel init sequence failed: %d\n", ret);
+ goto disable_vdd;
+ }
+
+ ret = mipi_dsi_dcs_exit_sleep_mode(dsi);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Failed to exit sleep mode: %d\n", ret);
+ goto disable_vdd;
+ }
+
+ msleep(200);
+
+ ret = mipi_dsi_dcs_set_display_on(dsi);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Failed to set display on: %d\n", ret);
+ goto disable_vdd;
+ }
+
+ usleep_range(10000, 15000);
+
+ return 0;
+
+disable_vdd:
+ regulator_disable(ctx->vdd);
+ return ret;
+}
+
+static int panel_nv3051d_get_modes(struct drm_panel *panel,
+ struct drm_connector *connector)
+{
+ struct panel_nv3051d *ctx = panel_to_panelnv3051d(panel);
+ const struct nv3051d_panel_info *panel_info = ctx->panel_info;
+ struct drm_display_mode *mode;
+ unsigned int i;
+
+ for (i = 0; i < panel_info->num_modes; i++) {
+ mode = drm_mode_duplicate(connector->dev,
+ &panel_info->display_modes[i]);
+ if (!mode)
+ return -ENOMEM;
+
+ drm_mode_set_name(mode);
+
+ mode->type = DRM_MODE_TYPE_DRIVER;
+ if (panel_info->num_modes == 1)
+ mode->type |= DRM_MODE_TYPE_PREFERRED;
+
+ drm_mode_probed_add(connector, mode);
+ }
+
+ connector->display_info.bpc = 8;
+ connector->display_info.width_mm = panel_info->width_mm;
+ connector->display_info.height_mm = panel_info->height_mm;
+ connector->display_info.bus_flags = panel_info->bus_flags;
+
+ return panel_info->num_modes;
+}
+
+static const struct drm_panel_funcs panel_nv3051d_funcs = {
+ .unprepare = panel_nv3051d_unprepare,
+ .prepare = panel_nv3051d_prepare,
+ .get_modes = panel_nv3051d_get_modes,
+};
+
+static int panel_nv3051d_probe(struct mipi_dsi_device *dsi)
+{
+ struct device *dev = &dsi->dev;
+ struct panel_nv3051d *ctx;
+ int ret;
+
+ ctx = devm_kzalloc(dev, sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->dev = dev;
+
+ ctx->panel_info = of_device_get_match_data(dev);
+ if (!ctx->panel_info)
+ return -EINVAL;
+
+ ctx->reset_gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
+ if (IS_ERR(ctx->reset_gpio)) {
+ dev_err(dev, "cannot get reset gpio\n");
+ return PTR_ERR(ctx->reset_gpio);
+ }
+
+ ctx->vdd = devm_regulator_get(dev, "vdd");
+ if (IS_ERR(ctx->vdd)) {
+ ret = PTR_ERR(ctx->vdd);
+ if (ret != -EPROBE_DEFER)
+ dev_err(dev, "Failed to request vdd regulator: %d\n", ret);
+ return ret;
+ }
+
+ mipi_dsi_set_drvdata(dsi, ctx);
+
+ dsi->lanes = 4;
+ dsi->format = MIPI_DSI_FMT_RGB888;
+ dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
+ MIPI_DSI_MODE_LPM | MIPI_DSI_MODE_NO_EOT_PACKET;
+
+ drm_panel_init(&ctx->panel, &dsi->dev, &panel_nv3051d_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+
+ ret = drm_panel_of_backlight(&ctx->panel);
+ if (ret)
+ return ret;
+
+ drm_panel_add(&ctx->panel);
+
+ ret = mipi_dsi_attach(dsi);
+ if (ret < 0) {
+ dev_err(dev, "mipi_dsi_attach failed: %d\n", ret);
+ drm_panel_remove(&ctx->panel);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void panel_nv3051d_shutdown(struct mipi_dsi_device *dsi)
+{
+ struct panel_nv3051d *ctx = mipi_dsi_get_drvdata(dsi);
+ int ret;
+
+ ret = drm_panel_unprepare(&ctx->panel);
+ if (ret < 0)
+ dev_err(&dsi->dev, "Failed to unprepare panel: %d\n", ret);
+
+ ret = drm_panel_disable(&ctx->panel);
+ if (ret < 0)
+ dev_err(&dsi->dev, "Failed to disable panel: %d\n", ret);
+}
+
+static void panel_nv3051d_remove(struct mipi_dsi_device *dsi)
+{
+ struct panel_nv3051d *ctx = mipi_dsi_get_drvdata(dsi);
+ int ret;
+
+ panel_nv3051d_shutdown(dsi);
+
+ ret = mipi_dsi_detach(dsi);
+ if (ret < 0)
+ dev_err(&dsi->dev, "Failed to detach from DSI host: %d\n", ret);
+
+ drm_panel_remove(&ctx->panel);
+}
+
+static const struct drm_display_mode nv3051d_rgxx3_modes[] = {
+ { /* 120hz */
+ .hdisplay = 640,
+ .hsync_start = 640 + 40,
+ .hsync_end = 640 + 40 + 2,
+ .htotal = 640 + 40 + 2 + 80,
+ .vdisplay = 480,
+ .vsync_start = 480 + 18,
+ .vsync_end = 480 + 18 + 2,
+ .vtotal = 480 + 18 + 2 + 28,
+ .clock = 48300,
+ .flags = DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC,
+ },
+ { /* 100hz */
+ .hdisplay = 640,
+ .hsync_start = 640 + 40,
+ .hsync_end = 640 + 40 + 2,
+ .htotal = 640 + 40 + 2 + 80,
+ .vdisplay = 480,
+ .vsync_start = 480 + 18,
+ .vsync_end = 480 + 18 + 2,
+ .vtotal = 480 + 18 + 2 + 28,
+ .clock = 40250,
+ .flags = DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC,
+ },
+ { /* 60hz */
+ .hdisplay = 640,
+ .hsync_start = 640 + 40,
+ .hsync_end = 640 + 40 + 2,
+ .htotal = 640 + 40 + 2 + 80,
+ .vdisplay = 480,
+ .vsync_start = 480 + 18,
+ .vsync_end = 480 + 18 + 2,
+ .vtotal = 480 + 18 + 2 + 28,
+ .clock = 24150,
+ .flags = DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC,
+ },
+};
+
+static const struct nv3051d_panel_info nv3051d_rgxx3_info = {
+ .display_modes = nv3051d_rgxx3_modes,
+ .num_modes = ARRAY_SIZE(nv3051d_rgxx3_modes),
+ .width_mm = 70,
+ .height_mm = 57,
+ .bus_flags = DRM_BUS_FLAG_DE_LOW | DRM_BUS_FLAG_PIXDATA_DRIVE_NEGEDGE,
+};
+
+static const struct of_device_id newvision_nv3051d_of_match[] = {
+ { .compatible = "newvision,nv3051d", .data = &nv3051d_rgxx3_info },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, newvision_nv3051d_of_match);
+
+static struct mipi_dsi_driver newvision_nv3051d_driver = {
+ .driver = {
+ .name = "panel-newvision-nv3051d",
+ .of_match_table = newvision_nv3051d_of_match,
+ },
+ .probe = panel_nv3051d_probe,
+ .remove = panel_nv3051d_remove,
+ .shutdown = panel_nv3051d_shutdown,
+};
+module_mipi_dsi_driver(newvision_nv3051d_driver);
+
+MODULE_AUTHOR("Chris Morgan <macromorgan@hotmail.com>");
+MODULE_DESCRIPTION("DRM driver for Newvision NV3051D based MIPI DSI panels");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/panel/panel-samsung-db7430.c b/drivers/gpu/drm/panel/panel-samsung-db7430.c
index 04640c5..117b268 100644
--- a/drivers/gpu/drm/panel/panel-samsung-db7430.c
+++ b/drivers/gpu/drm/panel/panel-samsung-db7430.c
@@ -331,9 +331,16 @@ static const struct of_device_id db7430_match[] = {
};
MODULE_DEVICE_TABLE(of, db7430_match);
+static const struct spi_device_id db7430_ids[] = {
+ { "lms397kf04" },
+ { },
+};
+MODULE_DEVICE_TABLE(spi, db7430_ids);
+
static struct spi_driver db7430_driver = {
.probe = db7430_probe,
.remove = db7430_remove,
+ .id_table = db7430_ids,
.driver = {
.name = "db7430-panel",
.of_match_table = db7430_match,
diff --git a/drivers/gpu/drm/panel/panel-sitronix-st7701.c b/drivers/gpu/drm/panel/panel-sitronix-st7701.c
index c481daa..0b8cf65 100644
--- a/drivers/gpu/drm/panel/panel-sitronix-st7701.c
+++ b/drivers/gpu/drm/panel/panel-sitronix-st7701.c
@@ -19,6 +19,9 @@
/* Command2 BKx selection command */
#define DSI_CMD2BKX_SEL 0xFF
+#define DSI_CMD1 0
+#define DSI_CMD2 BIT(4)
+#define DSI_CMD2BK_MASK GENMASK(3, 0)
/* Command2, BK0 commands */
#define DSI_CMD2_BK0_PVGAMCTRL 0xB0 /* Positive Voltage Gamma Control */
@@ -39,21 +42,6 @@
#define DSI_CMD2_BK1_SPD2 0xC2 /* Source EQ2 Setting */
#define DSI_CMD2_BK1_MIPISET1 0xD0 /* MIPI Setting 1 */
-/*
- * Command2 with BK function selection.
- *
- * BIT[4].....CN2
- * BIT[1:0]...BKXSEL
- * 1:00 = CMD2BK0, Command2 BK0
- * 1:01 = CMD2BK1, Command2 BK1
- * 1:11 = CMD2BK3, Command2 BK3
- * 0:00 = Command2 disable
- */
-#define DSI_CMD2BK0_SEL 0x10
-#define DSI_CMD2BK1_SEL 0x11
-#define DSI_CMD2BK3_SEL 0x13
-#define DSI_CMD2BKX_SEL_NONE 0x00
-
/* Command2, BK0 bytes */
#define DSI_CMD2_BK0_GAMCTRL_AJ_MASK GENMASK(7, 6)
#define DSI_CMD2_BK0_GAMCTRL_VC0_MASK GENMASK(3, 0)
@@ -191,6 +179,18 @@ static u8 st7701_vgls_map(struct st7701 *st7701)
return 0;
}
+static void st7701_switch_cmd_bkx(struct st7701 *st7701, bool cmd2, u8 bkx)
+{
+ u8 val;
+
+ if (cmd2)
+ val = DSI_CMD2 | FIELD_PREP(DSI_CMD2BK_MASK, bkx);
+ else
+ val = DSI_CMD1;
+
+ ST7701_DSI(st7701, DSI_CMD2BKX_SEL, 0x77, 0x01, 0x00, 0x00, val);
+}
+
static void st7701_init_sequence(struct st7701 *st7701)
{
const struct st7701_panel_desc *desc = st7701->desc;
@@ -208,8 +208,8 @@ static void st7701_init_sequence(struct st7701 *st7701)
msleep(st7701->sleep_delay);
/* Command2, BK0 */
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BK0_SEL);
+ st7701_switch_cmd_bkx(st7701, true, 0);
+
mipi_dsi_dcs_write(st7701->dsi, DSI_CMD2_BK0_PVGAMCTRL,
desc->pv_gamma, ARRAY_SIZE(desc->pv_gamma));
mipi_dsi_dcs_write(st7701->dsi, DSI_CMD2_BK0_NVGAMCTRL,
@@ -244,11 +244,10 @@ static void st7701_init_sequence(struct st7701 *st7701)
DSI_CMD2_BK0_INVSEL_ONES_MASK |
FIELD_PREP(DSI_CMD2_BK0_INVSEL_NLINV_MASK, desc->nlinv),
FIELD_PREP(DSI_CMD2_BK0_INVSEL_RTNI_MASK,
- DIV_ROUND_UP(mode->htotal, 16)));
+ (clamp((u32)mode->htotal, 512U, 1008U) - 512) / 16));
/* Command2, BK1 */
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BK1_SEL);
+ st7701_switch_cmd_bkx(st7701, true, 1);
/* Vop = 3.5375V + (VRHA[7:0] * 0.0125V) */
ST7701_DSI(st7701, DSI_CMD2_BK1_VRHS,
@@ -373,33 +372,27 @@ static void dmt028vghmcmi_1a_gip_sequence(struct st7701 *st7701)
0x08, 0x08, 0x08, 0x40,
0x3F, 0x64);
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BKX_SEL_NONE);
+ st7701_switch_cmd_bkx(st7701, false, 0);
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BK3_SEL);
+ st7701_switch_cmd_bkx(st7701, true, 3);
ST7701_DSI(st7701, 0xE6, 0x7C);
ST7701_DSI(st7701, 0xE8, 0x00, 0x0E);
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BKX_SEL_NONE);
+ st7701_switch_cmd_bkx(st7701, false, 0);
ST7701_DSI(st7701, 0x11);
msleep(120);
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BK3_SEL);
+ st7701_switch_cmd_bkx(st7701, true, 3);
ST7701_DSI(st7701, 0xE8, 0x00, 0x0C);
msleep(10);
ST7701_DSI(st7701, 0xE8, 0x00, 0x00);
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BKX_SEL_NONE);
+ st7701_switch_cmd_bkx(st7701, false, 0);
ST7701_DSI(st7701, 0x11);
msleep(120);
ST7701_DSI(st7701, 0xE8, 0x00, 0x00);
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BKX_SEL_NONE);
+ st7701_switch_cmd_bkx(st7701, false, 0);
ST7701_DSI(st7701, 0x3A, 0x70);
}
@@ -426,8 +419,7 @@ static int st7701_prepare(struct drm_panel *panel)
st7701->desc->gip_sequence(st7701);
/* Disable Command2 */
- ST7701_DSI(st7701, DSI_CMD2BKX_SEL,
- 0x77, 0x01, 0x00, 0x00, DSI_CMD2BKX_SEL_NONE);
+ st7701_switch_cmd_bkx(st7701, false, 0);
return 0;
}
@@ -762,7 +754,15 @@ static int st7701_dsi_probe(struct mipi_dsi_device *dsi)
st7701->dsi = dsi;
st7701->desc = desc;
- return mipi_dsi_attach(dsi);
+ ret = mipi_dsi_attach(dsi);
+ if (ret)
+ goto err_attach;
+
+ return 0;
+
+err_attach:
+ drm_panel_remove(&st7701->panel);
+ return ret;
}
static void st7701_dsi_remove(struct mipi_dsi_device *dsi)
diff --git a/drivers/gpu/drm/panel/panel-tpo-tpg110.c b/drivers/gpu/drm/panel/panel-tpo-tpg110.c
index 0b1f5a11..8453044 100644
--- a/drivers/gpu/drm/panel/panel-tpo-tpg110.c
+++ b/drivers/gpu/drm/panel/panel-tpo-tpg110.c
@@ -463,9 +463,16 @@ static const struct of_device_id tpg110_match[] = {
};
MODULE_DEVICE_TABLE(of, tpg110_match);
+static const struct spi_device_id tpg110_ids[] = {
+ { "tpg110" },
+ { },
+};
+MODULE_DEVICE_TABLE(spi, tpg110_ids);
+
static struct spi_driver tpg110_driver = {
.probe = tpg110_probe,
.remove = tpg110_remove,
+ .id_table = tpg110_ids,
.driver = {
.name = "tpo-tpg110-panel",
.of_match_table = tpg110_match,
diff --git a/drivers/gpu/drm/panel/panel-widechips-ws2401.c b/drivers/gpu/drm/panel/panel-widechips-ws2401.c
index 236f3cb..2591ff8 100644
--- a/drivers/gpu/drm/panel/panel-widechips-ws2401.c
+++ b/drivers/gpu/drm/panel/panel-widechips-ws2401.c
@@ -425,9 +425,16 @@ static const struct of_device_id ws2401_match[] = {
};
MODULE_DEVICE_TABLE(of, ws2401_match);
+static const struct spi_device_id ws2401_ids[] = {
+ { "lms380kf01" },
+ { },
+};
+MODULE_DEVICE_TABLE(spi, ws2401_ids);
+
static struct spi_driver ws2401_driver = {
.probe = ws2401_probe,
.remove = ws2401_remove,
+ .id_table = ws2401_ids,
.driver = {
.name = "ws2401-panel",
.of_match_table = ws2401_match,
diff --git a/drivers/gpu/drm/panfrost/panfrost_dump.c b/drivers/gpu/drm/panfrost/panfrost_dump.c
index 6bd0634..e7942ac 100644
--- a/drivers/gpu/drm/panfrost/panfrost_dump.c
+++ b/drivers/gpu/drm/panfrost/panfrost_dump.c
@@ -209,7 +209,7 @@ void panfrost_core_dump(struct panfrost_job *job)
goto dump_header;
}
- ret = drm_gem_shmem_vmap(&bo->base, &map);
+ ret = drm_gem_vmap_unlocked(&bo->base.base, &map);
if (ret) {
dev_err(pfdev->dev, "Panfrost Dump: couldn't map Buffer Object\n");
iter.hdr->bomap.valid = 0;
@@ -236,7 +236,7 @@ void panfrost_core_dump(struct panfrost_job *job)
vaddr = map.vaddr;
memcpy(iter.data, vaddr, bo->base.base.size);
- drm_gem_shmem_vunmap(&bo->base, &map);
+ drm_gem_vunmap_unlocked(&bo->base.base, &map);
iter.hdr->bomap.valid = 1;
diff --git a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
index bc0df93..ba9b6e2 100644
--- a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
+++ b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
@@ -106,7 +106,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
goto err_close_bo;
}
- ret = drm_gem_shmem_vmap(bo, &map);
+ ret = drm_gem_vmap_unlocked(&bo->base, &map);
if (ret)
goto err_put_mapping;
perfcnt->buf = map.vaddr;
@@ -165,7 +165,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
return 0;
err_vunmap:
- drm_gem_shmem_vunmap(bo, &map);
+ drm_gem_vunmap_unlocked(&bo->base, &map);
err_put_mapping:
panfrost_gem_mapping_put(perfcnt->mapping);
err_close_bo:
@@ -195,7 +195,7 @@ static int panfrost_perfcnt_disable_locked(struct panfrost_device *pfdev,
GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_OFF));
perfcnt->user = NULL;
- drm_gem_shmem_vunmap(&perfcnt->mapping->obj->base, &map);
+ drm_gem_vunmap_unlocked(&perfcnt->mapping->obj->base.base, &map);
perfcnt->buf = NULL;
panfrost_gem_close(&perfcnt->mapping->obj->base.base, file_priv);
panfrost_mmu_as_put(pfdev, perfcnt->mapping->mmu);
diff --git a/drivers/gpu/drm/pl111/pl111_drv.c b/drivers/gpu/drm/pl111/pl111_drv.c
index eb25eed..00deba0 100644
--- a/drivers/gpu/drm/pl111/pl111_drv.c
+++ b/drivers/gpu/drm/pl111/pl111_drv.c
@@ -48,7 +48,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
diff --git a/drivers/gpu/drm/qxl/qxl_display.c b/drivers/gpu/drm/qxl/qxl_display.c
index a152a7c..6492a70 100644
--- a/drivers/gpu/drm/qxl/qxl_display.c
+++ b/drivers/gpu/drm/qxl/qxl_display.c
@@ -1261,8 +1261,6 @@ int qxl_modeset_init(struct qxl_device *qdev)
qdev->ddev.mode_config.max_width = 8192;
qdev->ddev.mode_config.max_height = 8192;
- qdev->ddev.mode_config.fb_base = qdev->vram_base;
-
drm_mode_create_suggested_offset_properties(&qdev->ddev);
qxl_mode_create_hotplug_mode_update_property(qdev);
diff --git a/drivers/gpu/drm/qxl/qxl_drv.c b/drivers/gpu/drm/qxl/qxl_drv.c
index 3044ca9..a3b83f8 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.c
+++ b/drivers/gpu/drm/qxl/qxl_drv.c
@@ -37,6 +37,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include <drm/drm_gem_ttm_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/qxl/qxl_drv.h b/drivers/gpu/drm/qxl/qxl_drv.h
index 432758a..76f0608 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.h
+++ b/drivers/gpu/drm/qxl/qxl_drv.h
@@ -38,7 +38,6 @@
#include <drm/drm_crtc.h>
#include <drm/drm_encoder.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_gem_ttm_helper.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_gem.h>
diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
index 695d930..06a58da 100644
--- a/drivers/gpu/drm/qxl/qxl_object.c
+++ b/drivers/gpu/drm/qxl/qxl_object.c
@@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
bo->map_count++;
goto out;
}
- r = ttm_bo_vmap(&bo->tbo, &bo->map);
+
+ r = __qxl_bo_pin(bo);
if (r)
return r;
+
+ r = ttm_bo_vmap(&bo->tbo, &bo->map);
+ if (r) {
+ __qxl_bo_unpin(bo);
+ return r;
+ }
bo->map_count = 1;
/* TODO: Remove kptr in favor of map everywhere. */
@@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
if (r)
return r;
- r = __qxl_bo_pin(bo);
- if (r) {
- qxl_bo_unreserve(bo);
- return r;
- }
-
r = qxl_bo_vmap_locked(bo, map);
qxl_bo_unreserve(bo);
return r;
@@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
return;
bo->kptr = NULL;
ttm_bo_vunmap(&bo->tbo, &bo->map);
+ __qxl_bo_unpin(bo);
}
int qxl_bo_vunmap(struct qxl_bo *bo)
@@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
return r;
qxl_bo_vunmap_locked(bo);
- __qxl_bo_unpin(bo);
qxl_bo_unreserve(bo);
return 0;
}
diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
index 142d014..9169c26 100644
--- a/drivers/gpu/drm/qxl/qxl_prime.c
+++ b/drivers/gpu/drm/qxl/qxl_prime.c
@@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
struct qxl_bo *bo = gem_to_qxl_bo(obj);
int ret;
- ret = qxl_bo_vmap(bo, map);
+ ret = qxl_bo_vmap_locked(bo, map);
if (ret < 0)
return ret;
@@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
{
struct qxl_bo *bo = gem_to_qxl_bo(obj);
- qxl_bo_vunmap(bo);
+ qxl_bo_vunmap_locked(bo);
}
diff --git a/drivers/gpu/drm/radeon/Kconfig b/drivers/gpu/drm/radeon/Kconfig
index 52819e7..97a277f 100644
--- a/drivers/gpu/drm/radeon/Kconfig
+++ b/drivers/gpu/drm/radeon/Kconfig
@@ -1,4 +1,34 @@
# SPDX-License-Identifier: MIT
+
+config DRM_RADEON
+ tristate "ATI Radeon"
+ depends on DRM && PCI && MMU
+ depends on AGP || !AGP
+ select FW_LOADER
+ select DRM_DISPLAY_DP_HELPER
+ select DRM_DISPLAY_HELPER
+ select DRM_KMS_HELPER
+ select DRM_TTM
+ select DRM_TTM_HELPER
+ select SND_HDA_COMPONENT if SND_HDA_CORE
+ select POWER_SUPPLY
+ select HWMON
+ select BACKLIGHT_CLASS_DEVICE
+ select INTERVAL_TREE
+ # radeon depends on ACPI_VIDEO when ACPI is enabled, for select to work
+ # ACPI_VIDEO's dependencies must also be selected.
+ select INPUT if ACPI
+ select ACPI_VIDEO if ACPI
+ # On x86 ACPI_VIDEO also needs ACPI_WMI
+ select X86_PLATFORM_DEVICES if ACPI && X86
+ select ACPI_WMI if ACPI && X86
+ help
+ Choose this option if you have an ATI Radeon graphics card. There
+ are both PCI and AGP versions. You don't need to choose this to
+ run the Radeon in plain VGA mode.
+
+ If M is selected, the module will be called radeon.
+
config DRM_RADEON_USERPTR
bool "Always enable userptr support"
depends on DRM_RADEON
diff --git a/drivers/gpu/drm/radeon/atombios.h b/drivers/gpu/drm/radeon/atombios.h
index da35a970..235e59b5 100644
--- a/drivers/gpu/drm/radeon/atombios.h
+++ b/drivers/gpu/drm/radeon/atombios.h
@@ -3615,7 +3615,7 @@ typedef struct _ATOM_FAKE_EDID_PATCH_RECORD
{
UCHAR ucRecordType;
UCHAR ucFakeEDIDLength;
- UCHAR ucFakeEDIDString[1]; // This actually has ucFakeEdidLength elements.
+ UCHAR ucFakeEDIDString[]; // This actually has ucFakeEdidLength elements.
} ATOM_FAKE_EDID_PATCH_RECORD;
typedef struct _ATOM_PANEL_RESOLUTION_PATCH_RECORD
diff --git a/drivers/gpu/drm/radeon/radeon.h b/drivers/gpu/drm/radeon/radeon.h
index 166c18d..2e7161a 100644
--- a/drivers/gpu/drm/radeon/radeon.h
+++ b/drivers/gpu/drm/radeon/radeon.h
@@ -79,6 +79,7 @@
#include <drm/ttm/ttm_execbuf_util.h>
#include <drm/drm_gem.h>
+#include <drm/drm_audio_component.h>
#include "radeon_family.h"
#include "radeon_mode.h"
@@ -1796,6 +1797,9 @@ struct r600_audio {
struct radeon_audio_funcs *hdmi_funcs;
struct radeon_audio_funcs *dp_funcs;
struct radeon_audio_basic_funcs *funcs;
+ struct drm_audio_component *component;
+ bool component_registered;
+ struct mutex component_mutex;
};
/*
@@ -2994,6 +2998,10 @@ void radeon_irq_kms_set_irq_n_enabled(struct radeon_device *rdev,
bool enable, const char *name,
unsigned n);
+/* Audio component binding */
+void radeon_audio_component_init(struct radeon_device *rdev);
+void radeon_audio_component_fini(struct radeon_device *rdev);
+
#include "radeon_object.h"
#endif
diff --git a/drivers/gpu/drm/radeon/radeon_atombios.c b/drivers/gpu/drm/radeon/radeon_atombios.c
index 204127b..4ad5a328 100644
--- a/drivers/gpu/drm/radeon/radeon_atombios.c
+++ b/drivers/gpu/drm/radeon/radeon_atombios.c
@@ -1727,8 +1727,11 @@ struct radeon_encoder_atom_dig *radeon_atombios_get_lvds_info(struct
}
}
record += fake_edid_record->ucFakeEDIDLength ?
- fake_edid_record->ucFakeEDIDLength + 2 :
- sizeof(ATOM_FAKE_EDID_PATCH_RECORD);
+ struct_size(fake_edid_record,
+ ucFakeEDIDString,
+ fake_edid_record->ucFakeEDIDLength) :
+ /* empty fake edid record must be 3 bytes long */
+ sizeof(ATOM_FAKE_EDID_PATCH_RECORD) + 1;
break;
case LCD_PANEL_RESOLUTION_RECORD_TYPE:
panel_res_record = (ATOM_PANEL_RESOLUTION_PATCH_RECORD *)record;
diff --git a/drivers/gpu/drm/radeon/radeon_audio.c b/drivers/gpu/drm/radeon/radeon_audio.c
index 7c5e80d..d6ccaf2 100644
--- a/drivers/gpu/drm/radeon/radeon_audio.c
+++ b/drivers/gpu/drm/radeon/radeon_audio.c
@@ -23,6 +23,7 @@
*/
#include <linux/gcd.h>
+#include <linux/component.h>
#include <drm/drm_crtc.h>
#include "dce6_afmt.h"
@@ -180,6 +181,8 @@ static struct radeon_audio_funcs dce6_dp_funcs = {
.dpms = evergreen_dp_enable,
};
+static void radeon_audio_component_notify(struct radeon_device *rdev, int port);
+
static void radeon_audio_enable(struct radeon_device *rdev,
struct r600_audio_pin *pin, u8 enable_mask)
{
@@ -207,6 +210,8 @@ static void radeon_audio_enable(struct radeon_device *rdev,
if (rdev->audio.funcs->enable)
rdev->audio.funcs->enable(rdev, pin, enable_mask);
+
+ radeon_audio_component_notify(rdev, pin->id);
}
static void radeon_audio_interface_init(struct radeon_device *rdev)
@@ -721,3 +726,115 @@ unsigned int radeon_audio_decode_dfs_div(unsigned int div)
else
return 0;
}
+
+/*
+ * Audio component support
+ */
+static void radeon_audio_component_notify(struct radeon_device *rdev, int port)
+{
+ struct drm_audio_component *acomp;
+
+ mutex_lock(&rdev->audio.component_mutex);
+ acomp = rdev->audio.component;
+ if (acomp && acomp->audio_ops && acomp->audio_ops->pin_eld_notify)
+ acomp->audio_ops->pin_eld_notify(acomp->audio_ops->audio_ptr,
+ port, -1);
+ mutex_unlock(&rdev->audio.component_mutex);
+}
+
+static int radeon_audio_component_get_eld(struct device *kdev, int port,
+ int pipe, bool *enabled,
+ unsigned char *buf, int max_bytes)
+{
+ struct drm_device *dev = dev_get_drvdata(kdev);
+ struct radeon_device *rdev = dev->dev_private;
+ struct drm_encoder *encoder;
+ struct radeon_encoder *radeon_encoder;
+ struct radeon_encoder_atom_dig *dig;
+ struct drm_connector *connector;
+ int ret = 0;
+
+ *enabled = false;
+ if (!rdev->audio.enabled || !rdev->mode_info.mode_config_initialized)
+ return 0;
+
+ list_for_each_entry(encoder, &rdev->ddev->mode_config.encoder_list, head) {
+ if (!radeon_encoder_is_digital(encoder))
+ continue;
+ radeon_encoder = to_radeon_encoder(encoder);
+ dig = radeon_encoder->enc_priv;
+ if (!dig->pin || dig->pin->id != port)
+ continue;
+ connector = radeon_get_connector_for_encoder(encoder);
+ if (!connector)
+ continue;
+ *enabled = true;
+ ret = drm_eld_size(connector->eld);
+ memcpy(buf, connector->eld, min(max_bytes, ret));
+ break;
+ }
+
+ return ret;
+}
+
+static const struct drm_audio_component_ops radeon_audio_component_ops = {
+ .get_eld = radeon_audio_component_get_eld,
+};
+
+static int radeon_audio_component_bind(struct device *kdev,
+ struct device *hda_kdev, void *data)
+{
+ struct drm_device *dev = dev_get_drvdata(kdev);
+ struct radeon_device *rdev = dev->dev_private;
+ struct drm_audio_component *acomp = data;
+
+ if (WARN_ON(!device_link_add(hda_kdev, kdev, DL_FLAG_STATELESS)))
+ return -ENOMEM;
+
+ mutex_lock(&rdev->audio.component_mutex);
+ acomp->ops = &radeon_audio_component_ops;
+ acomp->dev = kdev;
+ rdev->audio.component = acomp;
+ mutex_unlock(&rdev->audio.component_mutex);
+
+ return 0;
+}
+
+static void radeon_audio_component_unbind(struct device *kdev,
+ struct device *hda_kdev, void *data)
+{
+ struct drm_device *dev = dev_get_drvdata(kdev);
+ struct radeon_device *rdev = dev->dev_private;
+ struct drm_audio_component *acomp = data;
+
+ device_link_remove(hda_kdev, kdev);
+
+ mutex_lock(&rdev->audio.component_mutex);
+ rdev->audio.component = NULL;
+ acomp->ops = NULL;
+ acomp->dev = NULL;
+ mutex_unlock(&rdev->audio.component_mutex);
+}
+
+static const struct component_ops radeon_audio_component_bind_ops = {
+ .bind = radeon_audio_component_bind,
+ .unbind = radeon_audio_component_unbind,
+};
+
+void radeon_audio_component_init(struct radeon_device *rdev)
+{
+ if (rdev->audio.component_registered ||
+ !radeon_audio || !radeon_audio_chipset_supported(rdev))
+ return;
+
+ if (!component_add(rdev->dev, &radeon_audio_component_bind_ops))
+ rdev->audio.component_registered = true;
+}
+
+void radeon_audio_component_fini(struct radeon_device *rdev)
+{
+ if (rdev->audio.component_registered) {
+ component_del(rdev->dev, &radeon_audio_component_bind_ops);
+ rdev->audio.component_registered = false;
+ }
+}
diff --git a/drivers/gpu/drm/radeon/radeon_bios.c b/drivers/gpu/drm/radeon/radeon_bios.c
index 3312165..63bdc9f 100644
--- a/drivers/gpu/drm/radeon/radeon_bios.c
+++ b/drivers/gpu/drm/radeon/radeon_bios.c
@@ -227,6 +227,7 @@ static bool radeon_atrm_get_bios(struct radeon_device *rdev)
if (!found)
return false;
+ pci_dev_put(pdev);
rdev->bios = kmalloc(size, GFP_KERNEL);
if (!rdev->bios) {
@@ -612,13 +613,14 @@ static bool radeon_acpi_vfct_bios(struct radeon_device *rdev)
acpi_size tbl_size;
UEFI_ACPI_VFCT *vfct;
unsigned offset;
+ bool r = false;
if (!ACPI_SUCCESS(acpi_get_table("VFCT", 1, &hdr)))
return false;
tbl_size = hdr->length;
if (tbl_size < sizeof(UEFI_ACPI_VFCT)) {
DRM_ERROR("ACPI VFCT table present but broken (too short #1)\n");
- return false;
+ goto out;
}
vfct = (UEFI_ACPI_VFCT *)hdr;
@@ -631,13 +633,13 @@ static bool radeon_acpi_vfct_bios(struct radeon_device *rdev)
offset += sizeof(VFCT_IMAGE_HEADER);
if (offset > tbl_size) {
DRM_ERROR("ACPI VFCT image header truncated\n");
- return false;
+ goto out;
}
offset += vhdr->ImageLength;
if (offset > tbl_size) {
DRM_ERROR("ACPI VFCT image truncated\n");
- return false;
+ goto out;
}
if (vhdr->ImageLength &&
@@ -649,15 +651,18 @@ static bool radeon_acpi_vfct_bios(struct radeon_device *rdev)
rdev->bios = kmemdup(&vbios->VbiosContent,
vhdr->ImageLength,
GFP_KERNEL);
+ if (rdev->bios)
+ r = true;
- if (!rdev->bios)
- return false;
- return true;
+ goto out;
}
}
DRM_ERROR("ACPI VFCT table present but broken (too short #2)\n");
- return false;
+
+out:
+ acpi_put_table(hdr);
+ return r;
}
#else
static inline bool radeon_acpi_vfct_bios(struct radeon_device *rdev)
diff --git a/drivers/gpu/drm/radeon/radeon_cs.c b/drivers/gpu/drm/radeon/radeon_cs.c
index 446f7ba..46a27eb 100644
--- a/drivers/gpu/drm/radeon/radeon_cs.c
+++ b/drivers/gpu/drm/radeon/radeon_cs.c
@@ -400,8 +400,11 @@ static int cmp_size_smaller_first(void *priv, const struct list_head *a,
struct radeon_bo_list *lb = list_entry(b, struct radeon_bo_list, tv.head);
/* Sort A before B if A is smaller. */
- return (int)la->robj->tbo.resource->num_pages -
- (int)lb->robj->tbo.resource->num_pages;
+ if (la->robj->tbo.base.size > lb->robj->tbo.base.size)
+ return 1;
+ if (la->robj->tbo.base.size < lb->robj->tbo.base.size)
+ return -1;
+ return 0;
}
/**
diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index a556b6b..6344454 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1207,7 +1207,7 @@ static void radeon_check_arguments(struct radeon_device *rdev)
* @pdev: pci dev pointer
* @state: vga_switcheroo state
*
- * Callback for the switcheroo driver. Suspends or resumes the
+ * Callback for the switcheroo driver. Suspends or resumes
* the asics before or after it is powered up using ACPI methods.
*/
static void radeon_switcheroo_set_state(struct pci_dev *pdev, enum vga_switcheroo_state state)
@@ -1312,6 +1312,7 @@ int radeon_device_init(struct radeon_device *rdev,
mutex_init(&rdev->pm.mutex);
mutex_init(&rdev->gpu_clock_mutex);
mutex_init(&rdev->srbm_mutex);
+ mutex_init(&rdev->audio.component_mutex);
init_rwsem(&rdev->pm.mclk_lock);
init_rwsem(&rdev->exclusive_lock);
init_waitqueue_head(&rdev->irq.vblank_queue);
@@ -1451,6 +1452,8 @@ int radeon_device_init(struct radeon_device *rdev,
goto failed;
}
+ radeon_audio_component_init(rdev);
+
r = radeon_ib_ring_tests(rdev);
if (r)
DRM_ERROR("ib ring test failed (%d).\n", r);
@@ -1513,6 +1516,7 @@ void radeon_device_fini(struct radeon_device *rdev)
rdev->shutdown = true;
/* evict vram memory */
radeon_bo_evict_vram(rdev);
+ radeon_audio_component_fini(rdev);
radeon_fini(rdev);
if (!pci_is_thunderbolt_attached(rdev->pdev))
vga_switcheroo_unregister_client(rdev->pdev);
diff --git a/drivers/gpu/drm/radeon/radeon_display.c b/drivers/gpu/drm/radeon/radeon_display.c
index ca5598a..9bed1a6 100644
--- a/drivers/gpu/drm/radeon/radeon_display.c
+++ b/drivers/gpu/drm/radeon/radeon_display.c
@@ -1604,8 +1604,6 @@ int radeon_modeset_init(struct radeon_device *rdev)
rdev->ddev->mode_config.fb_modifiers_not_supported = true;
- rdev->ddev->mode_config.fb_base = rdev->mc.aper_base;
-
ret = radeon_modeset_create_props(rdev);
if (ret) {
return ret;
diff --git a/drivers/gpu/drm/radeon/radeon_fb.c b/drivers/gpu/drm/radeon/radeon_fb.c
index 6ccea51..c1710ed 100644
--- a/drivers/gpu/drm/radeon/radeon_fb.c
+++ b/drivers/gpu/drm/radeon/radeon_fb.c
@@ -80,6 +80,8 @@ static const struct fb_ops radeonfb_ops = {
DRM_FB_HELPER_DEFAULT_OPS,
.fb_open = radeonfb_open,
.fb_release = radeonfb_release,
+ .fb_read = drm_fb_helper_cfb_read,
+ .fb_write = drm_fb_helper_cfb_write,
.fb_fillrect = drm_fb_helper_cfb_fillrect,
.fb_copyarea = drm_fb_helper_cfb_copyarea,
.fb_imageblit = drm_fb_helper_cfb_imageblit,
@@ -243,7 +245,7 @@ static int radeonfb_create(struct drm_fb_helper *helper,
rbo = gem_to_radeon_bo(gobj);
/* okay we have an object now allocate the framebuffer */
- info = drm_fb_helper_alloc_fbi(helper);
+ info = drm_fb_helper_alloc_info(helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
goto out;
@@ -276,7 +278,7 @@ static int radeonfb_create(struct drm_fb_helper *helper,
drm_fb_helper_fill_info(info, &rfbdev->helper, sizes);
/* setup aperture base/size for vesafb takeover */
- info->apertures->ranges[0].base = rdev->ddev->mode_config.fb_base;
+ info->apertures->ranges[0].base = rdev->mc.aper_base;
info->apertures->ranges[0].size = rdev->mc.aper_size;
/* Use default scratch pixmap (info->pixmap.flags = FB_PIXMAP_SYSTEM) */
@@ -309,7 +311,7 @@ static int radeon_fbdev_destroy(struct drm_device *dev, struct radeon_fbdev *rfb
{
struct drm_framebuffer *fb = &rfbdev->fb;
- drm_fb_helper_unregister_fbi(&rfbdev->helper);
+ drm_fb_helper_unregister_info(&rfbdev->helper);
if (fb->obj[0]) {
radeonfb_destroy_pinned_object(fb->obj[0]);
diff --git a/drivers/gpu/drm/radeon/radeon_object.c b/drivers/gpu/drm/radeon/radeon_object.c
index 00c33b2..10c0fbd 100644
--- a/drivers/gpu/drm/radeon/radeon_object.c
+++ b/drivers/gpu/drm/radeon/radeon_object.c
@@ -232,7 +232,7 @@ int radeon_bo_kmap(struct radeon_bo *bo, void **ptr)
}
return 0;
}
- r = ttm_bo_kmap(&bo->tbo, 0, bo->tbo.resource->num_pages, &bo->kmap);
+ r = ttm_bo_kmap(&bo->tbo, 0, PFN_UP(bo->tbo.base.size), &bo->kmap);
if (r) {
return r;
}
@@ -737,7 +737,7 @@ vm_fault_t radeon_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
if (bo->resource->mem_type != TTM_PL_VRAM)
return 0;
- size = bo->resource->num_pages << PAGE_SHIFT;
+ size = bo->resource->size;
offset = bo->resource->start << PAGE_SHIFT;
if ((offset + size) <= rdev->mc.visible_vram_size)
return 0;
diff --git a/drivers/gpu/drm/radeon/radeon_trace.h b/drivers/gpu/drm/radeon/radeon_trace.h
index c9fed5f..2267661 100644
--- a/drivers/gpu/drm/radeon/radeon_trace.h
+++ b/drivers/gpu/drm/radeon/radeon_trace.h
@@ -22,7 +22,7 @@ TRACE_EVENT(radeon_bo_create,
TP_fast_assign(
__entry->bo = bo;
- __entry->pages = bo->tbo.resource->num_pages;
+ __entry->pages = PFN_UP(bo->tbo.resource->size);
),
TP_printk("bo=%p, pages=%u", __entry->bo, __entry->pages)
);
diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c
index d33fec4..30402b5 100644
--- a/drivers/gpu/drm/radeon/radeon_ttm.c
+++ b/drivers/gpu/drm/radeon/radeon_ttm.c
@@ -181,7 +181,7 @@ static int radeon_move_blit(struct ttm_buffer_object *bo,
BUILD_BUG_ON((PAGE_SIZE % RADEON_GPU_PAGE_SIZE) != 0);
- num_pages = new_mem->num_pages * (PAGE_SIZE / RADEON_GPU_PAGE_SIZE);
+ num_pages = PFN_UP(new_mem->size) * (PAGE_SIZE / RADEON_GPU_PAGE_SIZE);
fence = radeon_copy(rdev, old_start, new_start, num_pages, bo->base.resv);
if (IS_ERR(fence))
return PTR_ERR(fence);
@@ -268,7 +268,7 @@ static int radeon_bo_move(struct ttm_buffer_object *bo, bool evict,
static int radeon_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem)
{
struct radeon_device *rdev = radeon_get_rdev(bdev);
- size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+ size_t bus_size = (size_t)mem->size;
switch (mem->mem_type) {
case TTM_PL_SYSTEM:
@@ -869,11 +869,11 @@ static ssize_t radeon_ttm_gtt_read(struct file *f, char __user *buf,
page = rdev->gart.pages[p];
if (page) {
- ptr = kmap(page);
+ ptr = kmap_local_page(page);
ptr += off;
r = copy_to_user(buf, ptr, cur_size);
- kunmap(rdev->gart.pages[p]);
+ kunmap_local(ptr);
} else
r = clear_user(buf, cur_size);
diff --git a/drivers/gpu/drm/rcar-du/Kconfig b/drivers/gpu/drm/rcar-du/Kconfig
index fd2c2ea..b2bddbe 100644
--- a/drivers/gpu/drm/rcar-du/Kconfig
+++ b/drivers/gpu/drm/rcar-du/Kconfig
@@ -41,8 +41,6 @@
depends on DRM_RCAR_USE_LVDS
select DRM_KMS_HELPER
select DRM_PANEL
- select OF_FLATTREE
- select OF_OVERLAY
config DRM_RCAR_USE_MIPI_DSI
bool "R-Car DU MIPI DSI Encoder Support"
@@ -56,6 +54,14 @@
depends on DRM_RCAR_USE_MIPI_DSI
select DRM_MIPI_DSI
+config DRM_RZG2L_MIPI_DSI
+ tristate "RZ/G2L MIPI DSI Encoder Support"
+ depends on DRM && DRM_BRIDGE && OF
+ depends on ARCH_RENESAS || COMPILE_TEST
+ select DRM_MIPI_DSI
+ help
+ Enable support for the RZ/G2L Display Unit embedded MIPI DSI encoders.
+
config DRM_RCAR_VSP
bool "R-Car DU VSP Compositor Support" if ARM
default y if ARM64
diff --git a/drivers/gpu/drm/rcar-du/Makefile b/drivers/gpu/drm/rcar-du/Makefile
index 6f13232..b8f2c82 100644
--- a/drivers/gpu/drm/rcar-du/Makefile
+++ b/drivers/gpu/drm/rcar-du/Makefile
@@ -14,3 +14,5 @@
obj-$(CONFIG_DRM_RCAR_DW_HDMI) += rcar_dw_hdmi.o
obj-$(CONFIG_DRM_RCAR_LVDS) += rcar_lvds.o
obj-$(CONFIG_DRM_RCAR_MIPI_DSI) += rcar_mipi_dsi.o
+
+obj-$(CONFIG_DRM_RZG2L_MIPI_DSI) += rzg2l_mipi_dsi.o
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.c b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
index a2776f1..d003e8d 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_drv.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
@@ -20,7 +20,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
#include <drm/drm_probe_helper.h>
diff --git a/drivers/gpu/drm/rcar-du/rzg2l_mipi_dsi.c b/drivers/gpu/drm/rcar-du/rzg2l_mipi_dsi.c
new file mode 100644
index 0000000..aa95b85
--- /dev/null
+++ b/drivers/gpu/drm/rcar-du/rzg2l_mipi_dsi.c
@@ -0,0 +1,816 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * RZ/G2L MIPI DSI Encoder Driver
+ *
+ * Copyright (C) 2022 Renesas Electronics Corporation
+ */
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/reset.h>
+#include <linux/slab.h>
+
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_bridge.h>
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_of.h>
+#include <drm/drm_panel.h>
+#include <drm/drm_probe_helper.h>
+
+#include "rzg2l_mipi_dsi_regs.h"
+
+struct rzg2l_mipi_dsi {
+ struct device *dev;
+ void __iomem *mmio;
+
+ struct reset_control *rstc;
+ struct reset_control *arstc;
+ struct reset_control *prstc;
+
+ struct mipi_dsi_host host;
+ struct drm_bridge bridge;
+ struct drm_bridge *next_bridge;
+
+ struct clk *vclk;
+
+ enum mipi_dsi_pixel_format format;
+ unsigned int num_data_lanes;
+ unsigned int lanes;
+ unsigned long mode_flags;
+};
+
+static inline struct rzg2l_mipi_dsi *
+bridge_to_rzg2l_mipi_dsi(struct drm_bridge *bridge)
+{
+ return container_of(bridge, struct rzg2l_mipi_dsi, bridge);
+}
+
+static inline struct rzg2l_mipi_dsi *
+host_to_rzg2l_mipi_dsi(struct mipi_dsi_host *host)
+{
+ return container_of(host, struct rzg2l_mipi_dsi, host);
+}
+
+struct rzg2l_mipi_dsi_timings {
+ unsigned long hsfreq_max;
+ u32 t_init;
+ u32 tclk_prepare;
+ u32 ths_prepare;
+ u32 tclk_zero;
+ u32 tclk_pre;
+ u32 tclk_post;
+ u32 tclk_trail;
+ u32 ths_zero;
+ u32 ths_trail;
+ u32 ths_exit;
+ u32 tlpx;
+};
+
+static const struct rzg2l_mipi_dsi_timings rzg2l_mipi_dsi_global_timings[] = {
+ {
+ .hsfreq_max = 80000,
+ .t_init = 79801,
+ .tclk_prepare = 8,
+ .ths_prepare = 13,
+ .tclk_zero = 33,
+ .tclk_pre = 24,
+ .tclk_post = 94,
+ .tclk_trail = 10,
+ .ths_zero = 23,
+ .ths_trail = 17,
+ .ths_exit = 13,
+ .tlpx = 6,
+ },
+ {
+ .hsfreq_max = 125000,
+ .t_init = 79801,
+ .tclk_prepare = 8,
+ .ths_prepare = 12,
+ .tclk_zero = 33,
+ .tclk_pre = 15,
+ .tclk_post = 94,
+ .tclk_trail = 10,
+ .ths_zero = 23,
+ .ths_trail = 17,
+ .ths_exit = 13,
+ .tlpx = 6,
+ },
+ {
+ .hsfreq_max = 250000,
+ .t_init = 79801,
+ .tclk_prepare = 8,
+ .ths_prepare = 12,
+ .tclk_zero = 33,
+ .tclk_pre = 13,
+ .tclk_post = 94,
+ .tclk_trail = 10,
+ .ths_zero = 23,
+ .ths_trail = 16,
+ .ths_exit = 13,
+ .tlpx = 6,
+ },
+ {
+ .hsfreq_max = 360000,
+ .t_init = 79801,
+ .tclk_prepare = 8,
+ .ths_prepare = 10,
+ .tclk_zero = 33,
+ .tclk_pre = 4,
+ .tclk_post = 35,
+ .tclk_trail = 7,
+ .ths_zero = 16,
+ .ths_trail = 9,
+ .ths_exit = 13,
+ .tlpx = 6,
+ },
+ {
+ .hsfreq_max = 720000,
+ .t_init = 79801,
+ .tclk_prepare = 8,
+ .ths_prepare = 9,
+ .tclk_zero = 33,
+ .tclk_pre = 4,
+ .tclk_post = 35,
+ .tclk_trail = 7,
+ .ths_zero = 16,
+ .ths_trail = 9,
+ .ths_exit = 13,
+ .tlpx = 6,
+ },
+ {
+ .hsfreq_max = 1500000,
+ .t_init = 79801,
+ .tclk_prepare = 8,
+ .ths_prepare = 9,
+ .tclk_zero = 33,
+ .tclk_pre = 4,
+ .tclk_post = 35,
+ .tclk_trail = 7,
+ .ths_zero = 16,
+ .ths_trail = 9,
+ .ths_exit = 13,
+ .tlpx = 6,
+ },
+};
+
+static void rzg2l_mipi_dsi_phy_write(struct rzg2l_mipi_dsi *dsi, u32 reg, u32 data)
+{
+ iowrite32(data, dsi->mmio + reg);
+}
+
+static void rzg2l_mipi_dsi_link_write(struct rzg2l_mipi_dsi *dsi, u32 reg, u32 data)
+{
+ iowrite32(data, dsi->mmio + LINK_REG_OFFSET + reg);
+}
+
+static u32 rzg2l_mipi_dsi_phy_read(struct rzg2l_mipi_dsi *dsi, u32 reg)
+{
+ return ioread32(dsi->mmio + reg);
+}
+
+static u32 rzg2l_mipi_dsi_link_read(struct rzg2l_mipi_dsi *dsi, u32 reg)
+{
+ return ioread32(dsi->mmio + LINK_REG_OFFSET + reg);
+}
+
+/* -----------------------------------------------------------------------------
+ * Hardware Setup
+ */
+
+static int rzg2l_mipi_dsi_dphy_init(struct rzg2l_mipi_dsi *dsi,
+ unsigned long hsfreq)
+{
+ const struct rzg2l_mipi_dsi_timings *dphy_timings;
+ unsigned int i;
+ u32 dphyctrl0;
+ u32 dphytim0;
+ u32 dphytim1;
+ u32 dphytim2;
+ u32 dphytim3;
+ int ret;
+
+ /* All DSI global operation timings are set with recommended setting */
+ for (i = 0; i < ARRAY_SIZE(rzg2l_mipi_dsi_global_timings); ++i) {
+ dphy_timings = &rzg2l_mipi_dsi_global_timings[i];
+ if (hsfreq <= dphy_timings->hsfreq_max)
+ break;
+ }
+
+ /* Initializing DPHY before accessing LINK */
+ dphyctrl0 = DSIDPHYCTRL0_CAL_EN_HSRX_OFS | DSIDPHYCTRL0_CMN_MASTER_EN |
+ DSIDPHYCTRL0_RE_VDD_DETVCCQLV18 | DSIDPHYCTRL0_EN_BGR;
+
+ rzg2l_mipi_dsi_phy_write(dsi, DSIDPHYCTRL0, dphyctrl0);
+ usleep_range(20, 30);
+
+ dphyctrl0 |= DSIDPHYCTRL0_EN_LDO1200;
+ rzg2l_mipi_dsi_phy_write(dsi, DSIDPHYCTRL0, dphyctrl0);
+ usleep_range(10, 20);
+
+ dphytim0 = DSIDPHYTIM0_TCLK_MISS(0) |
+ DSIDPHYTIM0_T_INIT(dphy_timings->t_init);
+ dphytim1 = DSIDPHYTIM1_THS_PREPARE(dphy_timings->ths_prepare) |
+ DSIDPHYTIM1_TCLK_PREPARE(dphy_timings->tclk_prepare) |
+ DSIDPHYTIM1_THS_SETTLE(0) |
+ DSIDPHYTIM1_TCLK_SETTLE(0);
+ dphytim2 = DSIDPHYTIM2_TCLK_TRAIL(dphy_timings->tclk_trail) |
+ DSIDPHYTIM2_TCLK_POST(dphy_timings->tclk_post) |
+ DSIDPHYTIM2_TCLK_PRE(dphy_timings->tclk_pre) |
+ DSIDPHYTIM2_TCLK_ZERO(dphy_timings->tclk_zero);
+ dphytim3 = DSIDPHYTIM3_TLPX(dphy_timings->tlpx) |
+ DSIDPHYTIM3_THS_EXIT(dphy_timings->ths_exit) |
+ DSIDPHYTIM3_THS_TRAIL(dphy_timings->ths_trail) |
+ DSIDPHYTIM3_THS_ZERO(dphy_timings->ths_zero);
+
+ rzg2l_mipi_dsi_phy_write(dsi, DSIDPHYTIM0, dphytim0);
+ rzg2l_mipi_dsi_phy_write(dsi, DSIDPHYTIM1, dphytim1);
+ rzg2l_mipi_dsi_phy_write(dsi, DSIDPHYTIM2, dphytim2);
+ rzg2l_mipi_dsi_phy_write(dsi, DSIDPHYTIM3, dphytim3);
+
+ ret = reset_control_deassert(dsi->rstc);
+ if (ret < 0)
+ return ret;
+
+ udelay(1);
+
+ return 0;
+}
+
+static void rzg2l_mipi_dsi_dphy_exit(struct rzg2l_mipi_dsi *dsi)
+{
+ u32 dphyctrl0;
+
+ dphyctrl0 = rzg2l_mipi_dsi_phy_read(dsi, DSIDPHYCTRL0);
+
+ dphyctrl0 &= ~(DSIDPHYCTRL0_EN_LDO1200 | DSIDPHYCTRL0_EN_BGR);
+ rzg2l_mipi_dsi_phy_write(dsi, DSIDPHYCTRL0, dphyctrl0);
+
+ reset_control_assert(dsi->rstc);
+}
+
+static int rzg2l_mipi_dsi_startup(struct rzg2l_mipi_dsi *dsi,
+ const struct drm_display_mode *mode)
+{
+ unsigned long hsfreq;
+ unsigned int bpp;
+ u32 txsetr;
+ u32 clstptsetr;
+ u32 lptrnstsetr;
+ u32 clkkpt;
+ u32 clkbfht;
+ u32 clkstpt;
+ u32 golpbkt;
+ int ret;
+
+ /*
+ * Relationship between hsclk and vclk must follow
+ * vclk * bpp = hsclk * 8 * lanes
+ * where vclk: video clock (Hz)
+ * bpp: video pixel bit depth
+ * hsclk: DSI HS Byte clock frequency (Hz)
+ * lanes: number of data lanes
+ *
+ * hsclk(bit) = hsclk(byte) * 8
+ */
+ bpp = mipi_dsi_pixel_format_to_bpp(dsi->format);
+ hsfreq = (mode->clock * bpp * 8) / (8 * dsi->lanes);
+
+ ret = pm_runtime_resume_and_get(dsi->dev);
+ if (ret < 0)
+ return ret;
+
+ clk_set_rate(dsi->vclk, mode->clock * 1000);
+
+ ret = rzg2l_mipi_dsi_dphy_init(dsi, hsfreq);
+ if (ret < 0)
+ goto err_phy;
+
+ /* Enable Data lanes and Clock lanes */
+ txsetr = TXSETR_DLEN | TXSETR_NUMLANEUSE(dsi->lanes - 1) | TXSETR_CLEN;
+ rzg2l_mipi_dsi_link_write(dsi, TXSETR, txsetr);
+
+ /*
+ * Global timings characteristic depends on high speed Clock Frequency
+ * Currently MIPI DSI-IF just supports maximum FHD@60 with:
+ * - videoclock = 148.5 (MHz)
+ * - bpp: maximum 24bpp
+ * - data lanes: maximum 4 lanes
+ * Therefore maximum hsclk will be 891 Mbps.
+ */
+ if (hsfreq > 445500) {
+ clkkpt = 12;
+ clkbfht = 15;
+ clkstpt = 48;
+ golpbkt = 75;
+ } else if (hsfreq > 250000) {
+ clkkpt = 7;
+ clkbfht = 8;
+ clkstpt = 27;
+ golpbkt = 40;
+ } else {
+ clkkpt = 8;
+ clkbfht = 6;
+ clkstpt = 24;
+ golpbkt = 29;
+ }
+
+ clstptsetr = CLSTPTSETR_CLKKPT(clkkpt) | CLSTPTSETR_CLKBFHT(clkbfht) |
+ CLSTPTSETR_CLKSTPT(clkstpt);
+ rzg2l_mipi_dsi_link_write(dsi, CLSTPTSETR, clstptsetr);
+
+ lptrnstsetr = LPTRNSTSETR_GOLPBKT(golpbkt);
+ rzg2l_mipi_dsi_link_write(dsi, LPTRNSTSETR, lptrnstsetr);
+
+ return 0;
+
+err_phy:
+ rzg2l_mipi_dsi_dphy_exit(dsi);
+ pm_runtime_put(dsi->dev);
+
+ return ret;
+}
+
+static void rzg2l_mipi_dsi_stop(struct rzg2l_mipi_dsi *dsi)
+{
+ rzg2l_mipi_dsi_dphy_exit(dsi);
+ pm_runtime_put(dsi->dev);
+}
+
+static void rzg2l_mipi_dsi_set_display_timing(struct rzg2l_mipi_dsi *dsi,
+ const struct drm_display_mode *mode)
+{
+ u32 vich1ppsetr;
+ u32 vich1vssetr;
+ u32 vich1vpsetr;
+ u32 vich1hssetr;
+ u32 vich1hpsetr;
+ int dsi_format;
+ u32 delay[2];
+ u8 index;
+
+ /* Configuration for Pixel Packet */
+ dsi_format = mipi_dsi_pixel_format_to_bpp(dsi->format);
+ switch (dsi_format) {
+ case 24:
+ vich1ppsetr = VICH1PPSETR_DT_RGB24;
+ break;
+ case 18:
+ vich1ppsetr = VICH1PPSETR_DT_RGB18;
+ break;
+ }
+
+ if ((dsi->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE) &&
+ !(dsi->mode_flags & MIPI_DSI_MODE_VIDEO_BURST))
+ vich1ppsetr |= VICH1PPSETR_TXESYNC_PULSE;
+
+ rzg2l_mipi_dsi_link_write(dsi, VICH1PPSETR, vich1ppsetr);
+
+ /* Configuration for Video Parameters */
+ vich1vssetr = VICH1VSSETR_VACTIVE(mode->vdisplay) |
+ VICH1VSSETR_VSA(mode->vsync_end - mode->vsync_start);
+ vich1vssetr |= (mode->flags & DRM_MODE_FLAG_PVSYNC) ?
+ VICH1VSSETR_VSPOL_HIGH : VICH1VSSETR_VSPOL_LOW;
+
+ vich1vpsetr = VICH1VPSETR_VFP(mode->vsync_start - mode->vdisplay) |
+ VICH1VPSETR_VBP(mode->vtotal - mode->vsync_end);
+
+ vich1hssetr = VICH1HSSETR_HACTIVE(mode->hdisplay) |
+ VICH1HSSETR_HSA(mode->hsync_end - mode->hsync_start);
+ vich1hssetr |= (mode->flags & DRM_MODE_FLAG_PHSYNC) ?
+ VICH1HSSETR_HSPOL_HIGH : VICH1HSSETR_HSPOL_LOW;
+
+ vich1hpsetr = VICH1HPSETR_HFP(mode->hsync_start - mode->hdisplay) |
+ VICH1HPSETR_HBP(mode->htotal - mode->hsync_end);
+
+ rzg2l_mipi_dsi_link_write(dsi, VICH1VSSETR, vich1vssetr);
+ rzg2l_mipi_dsi_link_write(dsi, VICH1VPSETR, vich1vpsetr);
+ rzg2l_mipi_dsi_link_write(dsi, VICH1HSSETR, vich1hssetr);
+ rzg2l_mipi_dsi_link_write(dsi, VICH1HPSETR, vich1hpsetr);
+
+ /*
+ * Configuration for Delay Value
+ * Delay value based on 2 ranges of video clock.
+ * 74.25MHz is videoclock of HD@60p or FHD@30p
+ */
+ if (mode->clock > 74250) {
+ delay[0] = 231;
+ delay[1] = 216;
+ } else {
+ delay[0] = 220;
+ delay[1] = 212;
+ }
+
+ if (dsi->mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS)
+ index = 0;
+ else
+ index = 1;
+
+ rzg2l_mipi_dsi_link_write(dsi, VICH1SET1R,
+ VICH1SET1R_DLY(delay[index]));
+}
+
+static int rzg2l_mipi_dsi_start_hs_clock(struct rzg2l_mipi_dsi *dsi)
+{
+ bool is_clk_cont;
+ u32 hsclksetr;
+ u32 status;
+ int ret;
+
+ is_clk_cont = !(dsi->mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS);
+
+ /* Start HS clock */
+ hsclksetr = HSCLKSETR_HSCLKRUN_HS | (is_clk_cont ?
+ HSCLKSETR_HSCLKMODE_CONT :
+ HSCLKSETR_HSCLKMODE_NON_CONT);
+ rzg2l_mipi_dsi_link_write(dsi, HSCLKSETR, hsclksetr);
+
+ if (is_clk_cont) {
+ ret = read_poll_timeout(rzg2l_mipi_dsi_link_read, status,
+ status & PLSR_CLLP2HS,
+ 2000, 20000, false, dsi, PLSR);
+ if (ret < 0) {
+ dev_err(dsi->dev, "failed to start HS clock\n");
+ return ret;
+ }
+ }
+
+ dev_dbg(dsi->dev, "Start High Speed Clock with %s clock mode",
+ is_clk_cont ? "continuous" : "non-continuous");
+
+ return 0;
+}
+
+static int rzg2l_mipi_dsi_stop_hs_clock(struct rzg2l_mipi_dsi *dsi)
+{
+ bool is_clk_cont;
+ u32 status;
+ int ret;
+
+ is_clk_cont = !(dsi->mode_flags & MIPI_DSI_CLOCK_NON_CONTINUOUS);
+
+ /* Stop HS clock */
+ rzg2l_mipi_dsi_link_write(dsi, HSCLKSETR,
+ is_clk_cont ? HSCLKSETR_HSCLKMODE_CONT :
+ HSCLKSETR_HSCLKMODE_NON_CONT);
+
+ if (is_clk_cont) {
+ ret = read_poll_timeout(rzg2l_mipi_dsi_link_read, status,
+ status & PLSR_CLHS2LP,
+ 2000, 20000, false, dsi, PLSR);
+ if (ret < 0) {
+ dev_err(dsi->dev, "failed to stop HS clock\n");
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static int rzg2l_mipi_dsi_start_video(struct rzg2l_mipi_dsi *dsi)
+{
+ u32 vich1set0r;
+ u32 status;
+ int ret;
+
+ /* Configuration for Blanking sequence and start video input*/
+ vich1set0r = VICH1SET0R_HFPNOLP | VICH1SET0R_HBPNOLP |
+ VICH1SET0R_HSANOLP | VICH1SET0R_VSTART;
+ rzg2l_mipi_dsi_link_write(dsi, VICH1SET0R, vich1set0r);
+
+ ret = read_poll_timeout(rzg2l_mipi_dsi_link_read, status,
+ status & VICH1SR_VIRDY,
+ 2000, 20000, false, dsi, VICH1SR);
+ if (ret < 0)
+ dev_err(dsi->dev, "Failed to start video signal input\n");
+
+ return ret;
+}
+
+static int rzg2l_mipi_dsi_stop_video(struct rzg2l_mipi_dsi *dsi)
+{
+ u32 status;
+ int ret;
+
+ rzg2l_mipi_dsi_link_write(dsi, VICH1SET0R, VICH1SET0R_VSTPAFT);
+ ret = read_poll_timeout(rzg2l_mipi_dsi_link_read, status,
+ (status & VICH1SR_STOP) && (!(status & VICH1SR_RUNNING)),
+ 2000, 20000, false, dsi, VICH1SR);
+ if (ret < 0)
+ goto err;
+
+ ret = read_poll_timeout(rzg2l_mipi_dsi_link_read, status,
+ !(status & LINKSR_HSBUSY),
+ 2000, 20000, false, dsi, LINKSR);
+ if (ret < 0)
+ goto err;
+
+ return 0;
+
+err:
+ dev_err(dsi->dev, "Failed to stop video signal input\n");
+ return ret;
+}
+
+/* -----------------------------------------------------------------------------
+ * Bridge
+ */
+
+static int rzg2l_mipi_dsi_attach(struct drm_bridge *bridge,
+ enum drm_bridge_attach_flags flags)
+{
+ struct rzg2l_mipi_dsi *dsi = bridge_to_rzg2l_mipi_dsi(bridge);
+
+ return drm_bridge_attach(bridge->encoder, dsi->next_bridge, bridge,
+ flags);
+}
+
+static void rzg2l_mipi_dsi_atomic_enable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_bridge_state)
+{
+ struct drm_atomic_state *state = old_bridge_state->base.state;
+ struct rzg2l_mipi_dsi *dsi = bridge_to_rzg2l_mipi_dsi(bridge);
+ const struct drm_display_mode *mode;
+ struct drm_connector *connector;
+ struct drm_crtc *crtc;
+ int ret;
+
+ connector = drm_atomic_get_new_connector_for_encoder(state, bridge->encoder);
+ crtc = drm_atomic_get_new_connector_state(state, connector)->crtc;
+ mode = &drm_atomic_get_new_crtc_state(state, crtc)->adjusted_mode;
+
+ ret = rzg2l_mipi_dsi_startup(dsi, mode);
+ if (ret < 0)
+ return;
+
+ rzg2l_mipi_dsi_set_display_timing(dsi, mode);
+
+ ret = rzg2l_mipi_dsi_start_hs_clock(dsi);
+ if (ret < 0)
+ goto err_stop;
+
+ ret = rzg2l_mipi_dsi_start_video(dsi);
+ if (ret < 0)
+ goto err_stop_clock;
+
+ return;
+
+err_stop_clock:
+ rzg2l_mipi_dsi_stop_hs_clock(dsi);
+err_stop:
+ rzg2l_mipi_dsi_stop(dsi);
+}
+
+static void rzg2l_mipi_dsi_atomic_disable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_bridge_state)
+{
+ struct rzg2l_mipi_dsi *dsi = bridge_to_rzg2l_mipi_dsi(bridge);
+
+ rzg2l_mipi_dsi_stop_video(dsi);
+ rzg2l_mipi_dsi_stop_hs_clock(dsi);
+ rzg2l_mipi_dsi_stop(dsi);
+}
+
+static enum drm_mode_status
+rzg2l_mipi_dsi_bridge_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *info,
+ const struct drm_display_mode *mode)
+{
+ if (mode->clock > 148500)
+ return MODE_CLOCK_HIGH;
+
+ return MODE_OK;
+}
+
+static const struct drm_bridge_funcs rzg2l_mipi_dsi_bridge_ops = {
+ .attach = rzg2l_mipi_dsi_attach,
+ .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
+ .atomic_reset = drm_atomic_helper_bridge_reset,
+ .atomic_enable = rzg2l_mipi_dsi_atomic_enable,
+ .atomic_disable = rzg2l_mipi_dsi_atomic_disable,
+ .mode_valid = rzg2l_mipi_dsi_bridge_mode_valid,
+};
+
+/* -----------------------------------------------------------------------------
+ * Host setting
+ */
+
+static int rzg2l_mipi_dsi_host_attach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *device)
+{
+ struct rzg2l_mipi_dsi *dsi = host_to_rzg2l_mipi_dsi(host);
+ int ret;
+
+ if (device->lanes > dsi->num_data_lanes) {
+ dev_err(dsi->dev,
+ "Number of lines of device (%u) exceeds host (%u)\n",
+ device->lanes, dsi->num_data_lanes);
+ return -EINVAL;
+ }
+
+ switch (mipi_dsi_pixel_format_to_bpp(device->format)) {
+ case 24:
+ case 18:
+ break;
+ default:
+ dev_err(dsi->dev, "Unsupported format 0x%04x\n", device->format);
+ return -EINVAL;
+ }
+
+ dsi->lanes = device->lanes;
+ dsi->format = device->format;
+ dsi->mode_flags = device->mode_flags;
+
+ dsi->next_bridge = devm_drm_of_get_bridge(dsi->dev, dsi->dev->of_node,
+ 1, 0);
+ if (IS_ERR(dsi->next_bridge)) {
+ ret = PTR_ERR(dsi->next_bridge);
+ dev_err(dsi->dev, "failed to get next bridge: %d\n", ret);
+ return ret;
+ }
+
+ drm_bridge_add(&dsi->bridge);
+
+ return 0;
+}
+
+static int rzg2l_mipi_dsi_host_detach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *device)
+{
+ struct rzg2l_mipi_dsi *dsi = host_to_rzg2l_mipi_dsi(host);
+
+ drm_bridge_remove(&dsi->bridge);
+
+ return 0;
+}
+
+static const struct mipi_dsi_host_ops rzg2l_mipi_dsi_host_ops = {
+ .attach = rzg2l_mipi_dsi_host_attach,
+ .detach = rzg2l_mipi_dsi_host_detach,
+};
+
+/* -----------------------------------------------------------------------------
+ * Power Management
+ */
+
+static int __maybe_unused rzg2l_mipi_pm_runtime_suspend(struct device *dev)
+{
+ struct rzg2l_mipi_dsi *dsi = dev_get_drvdata(dev);
+
+ reset_control_assert(dsi->prstc);
+ reset_control_assert(dsi->arstc);
+
+ return 0;
+}
+
+static int __maybe_unused rzg2l_mipi_pm_runtime_resume(struct device *dev)
+{
+ struct rzg2l_mipi_dsi *dsi = dev_get_drvdata(dev);
+ int ret;
+
+ ret = reset_control_deassert(dsi->arstc);
+ if (ret < 0)
+ return ret;
+
+ ret = reset_control_deassert(dsi->prstc);
+ if (ret < 0)
+ reset_control_assert(dsi->arstc);
+
+ return ret;
+}
+
+static const struct dev_pm_ops rzg2l_mipi_pm_ops = {
+ SET_RUNTIME_PM_OPS(rzg2l_mipi_pm_runtime_suspend, rzg2l_mipi_pm_runtime_resume, NULL)
+};
+
+/* -----------------------------------------------------------------------------
+ * Probe & Remove
+ */
+
+static int rzg2l_mipi_dsi_probe(struct platform_device *pdev)
+{
+ unsigned int num_data_lanes;
+ struct rzg2l_mipi_dsi *dsi;
+ u32 txsetr;
+ int ret;
+
+ dsi = devm_kzalloc(&pdev->dev, sizeof(*dsi), GFP_KERNEL);
+ if (!dsi)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, dsi);
+ dsi->dev = &pdev->dev;
+
+ ret = drm_of_get_data_lanes_count_ep(dsi->dev->of_node, 1, 0, 1, 4);
+ if (ret < 0)
+ return dev_err_probe(dsi->dev, ret,
+ "missing or invalid data-lanes property\n");
+
+ num_data_lanes = ret;
+
+ dsi->mmio = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(dsi->mmio))
+ return PTR_ERR(dsi->mmio);
+
+ dsi->vclk = devm_clk_get(dsi->dev, "vclk");
+ if (IS_ERR(dsi->vclk))
+ return PTR_ERR(dsi->vclk);
+
+ dsi->rstc = devm_reset_control_get_exclusive(dsi->dev, "rst");
+ if (IS_ERR(dsi->rstc))
+ return dev_err_probe(dsi->dev, PTR_ERR(dsi->rstc),
+ "failed to get rst\n");
+
+ dsi->arstc = devm_reset_control_get_exclusive(dsi->dev, "arst");
+ if (IS_ERR(dsi->arstc))
+ return dev_err_probe(&pdev->dev, PTR_ERR(dsi->arstc),
+ "failed to get arst\n");
+
+ dsi->prstc = devm_reset_control_get_exclusive(dsi->dev, "prst");
+ if (IS_ERR(dsi->prstc))
+ return dev_err_probe(dsi->dev, PTR_ERR(dsi->prstc),
+ "failed to get prst\n");
+
+ platform_set_drvdata(pdev, dsi);
+
+ pm_runtime_enable(dsi->dev);
+
+ ret = pm_runtime_resume_and_get(dsi->dev);
+ if (ret < 0)
+ goto err_pm_disable;
+
+ /*
+ * TXSETR register can be read only after DPHY init. But during probe
+ * mode->clock and format are not available. So initialize DPHY with
+ * timing parameters for 80Mbps.
+ */
+ ret = rzg2l_mipi_dsi_dphy_init(dsi, 80000);
+ if (ret < 0)
+ goto err_phy;
+
+ txsetr = rzg2l_mipi_dsi_link_read(dsi, TXSETR);
+ dsi->num_data_lanes = min(((txsetr >> 16) & 3) + 1, num_data_lanes);
+ rzg2l_mipi_dsi_dphy_exit(dsi);
+ pm_runtime_put(dsi->dev);
+
+ /* Initialize the DRM bridge. */
+ dsi->bridge.funcs = &rzg2l_mipi_dsi_bridge_ops;
+ dsi->bridge.of_node = dsi->dev->of_node;
+
+ /* Init host device */
+ dsi->host.dev = dsi->dev;
+ dsi->host.ops = &rzg2l_mipi_dsi_host_ops;
+ ret = mipi_dsi_host_register(&dsi->host);
+ if (ret < 0)
+ goto err_pm_disable;
+
+ return 0;
+
+err_phy:
+ rzg2l_mipi_dsi_dphy_exit(dsi);
+ pm_runtime_put(dsi->dev);
+err_pm_disable:
+ pm_runtime_disable(dsi->dev);
+ return ret;
+}
+
+static int rzg2l_mipi_dsi_remove(struct platform_device *pdev)
+{
+ struct rzg2l_mipi_dsi *dsi = platform_get_drvdata(pdev);
+
+ mipi_dsi_host_unregister(&dsi->host);
+ pm_runtime_disable(&pdev->dev);
+
+ return 0;
+}
+
+static const struct of_device_id rzg2l_mipi_dsi_of_table[] = {
+ { .compatible = "renesas,rzg2l-mipi-dsi" },
+ { /* sentinel */ }
+};
+
+MODULE_DEVICE_TABLE(of, rzg2l_mipi_dsi_of_table);
+
+static struct platform_driver rzg2l_mipi_dsi_platform_driver = {
+ .probe = rzg2l_mipi_dsi_probe,
+ .remove = rzg2l_mipi_dsi_remove,
+ .driver = {
+ .name = "rzg2l-mipi-dsi",
+ .pm = &rzg2l_mipi_pm_ops,
+ .of_match_table = rzg2l_mipi_dsi_of_table,
+ },
+};
+
+module_platform_driver(rzg2l_mipi_dsi_platform_driver);
+
+MODULE_AUTHOR("Biju Das <biju.das.jz@bp.renesas.com>");
+MODULE_DESCRIPTION("Renesas RZ/G2L MIPI DSI Encoder Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/rcar-du/rzg2l_mipi_dsi_regs.h b/drivers/gpu/drm/rcar-du/rzg2l_mipi_dsi_regs.h
new file mode 100644
index 0000000..1dbc16e
--- /dev/null
+++ b/drivers/gpu/drm/rcar-du/rzg2l_mipi_dsi_regs.h
@@ -0,0 +1,151 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * RZ/G2L MIPI DSI Interface Registers Definitions
+ *
+ * Copyright (C) 2022 Renesas Electronics Corporation
+ */
+
+#ifndef __RZG2L_MIPI_DSI_REGS_H__
+#define __RZG2L_MIPI_DSI_REGS_H__
+
+#include <linux/bits.h>
+
+/* DPHY Registers */
+#define DSIDPHYCTRL0 0x00
+#define DSIDPHYCTRL0_CAL_EN_HSRX_OFS BIT(16)
+#define DSIDPHYCTRL0_CMN_MASTER_EN BIT(8)
+#define DSIDPHYCTRL0_RE_VDD_DETVCCQLV18 BIT(2)
+#define DSIDPHYCTRL0_EN_LDO1200 BIT(1)
+#define DSIDPHYCTRL0_EN_BGR BIT(0)
+
+#define DSIDPHYTIM0 0x04
+#define DSIDPHYTIM0_TCLK_MISS(x) ((x) << 24)
+#define DSIDPHYTIM0_T_INIT(x) ((x) << 0)
+
+#define DSIDPHYTIM1 0x08
+#define DSIDPHYTIM1_THS_PREPARE(x) ((x) << 24)
+#define DSIDPHYTIM1_TCLK_PREPARE(x) ((x) << 16)
+#define DSIDPHYTIM1_THS_SETTLE(x) ((x) << 8)
+#define DSIDPHYTIM1_TCLK_SETTLE(x) ((x) << 0)
+
+#define DSIDPHYTIM2 0x0c
+#define DSIDPHYTIM2_TCLK_TRAIL(x) ((x) << 24)
+#define DSIDPHYTIM2_TCLK_POST(x) ((x) << 16)
+#define DSIDPHYTIM2_TCLK_PRE(x) ((x) << 8)
+#define DSIDPHYTIM2_TCLK_ZERO(x) ((x) << 0)
+
+#define DSIDPHYTIM3 0x10
+#define DSIDPHYTIM3_TLPX(x) ((x) << 24)
+#define DSIDPHYTIM3_THS_EXIT(x) ((x) << 16)
+#define DSIDPHYTIM3_THS_TRAIL(x) ((x) << 8)
+#define DSIDPHYTIM3_THS_ZERO(x) ((x) << 0)
+
+/* --------------------------------------------------------*/
+/* Link Registers */
+#define LINK_REG_OFFSET 0x10000
+
+/* Link Status Register */
+#define LINKSR 0x10
+#define LINKSR_LPBUSY BIT(13)
+#define LINKSR_HSBUSY BIT(12)
+#define LINKSR_VICHRUN1 BIT(8)
+#define LINKSR_SQCHRUN1 BIT(4)
+#define LINKSR_SQCHRUN0 BIT(0)
+
+/* Tx Set Register */
+#define TXSETR 0x100
+#define TXSETR_NUMLANECAP (0x3 << 16)
+#define TXSETR_DLEN (1 << 9)
+#define TXSETR_CLEN (1 << 8)
+#define TXSETR_NUMLANEUSE(x) (((x) & 0x3) << 0)
+
+/* HS Clock Set Register */
+#define HSCLKSETR 0x104
+#define HSCLKSETR_HSCLKMODE_CONT (1 << 1)
+#define HSCLKSETR_HSCLKMODE_NON_CONT (0 << 1)
+#define HSCLKSETR_HSCLKRUN_HS (1 << 0)
+#define HSCLKSETR_HSCLKRUN_LP (0 << 0)
+
+/* Reset Control Register */
+#define RSTCR 0x110
+#define RSTCR_SWRST BIT(0)
+#define RSTCR_FCETXSTP BIT(16)
+
+/* Reset Status Register */
+#define RSTSR 0x114
+#define RSTSR_DL0DIR (1 << 15)
+#define RSTSR_DLSTPST (0xf << 8)
+#define RSTSR_SWRSTV1 (1 << 4)
+#define RSTSR_SWRSTIB (1 << 3)
+#define RSTSR_SWRSTAPB (1 << 2)
+#define RSTSR_SWRSTLP (1 << 1)
+#define RSTSR_SWRSTHS (1 << 0)
+
+/* Clock Lane Stop Time Set Register */
+#define CLSTPTSETR 0x314
+#define CLSTPTSETR_CLKKPT(x) ((x) << 24)
+#define CLSTPTSETR_CLKBFHT(x) ((x) << 16)
+#define CLSTPTSETR_CLKSTPT(x) ((x) << 2)
+
+/* LP Transition Time Set Register */
+#define LPTRNSTSETR 0x318
+#define LPTRNSTSETR_GOLPBKT(x) ((x) << 0)
+
+/* Physical Lane Status Register */
+#define PLSR 0x320
+#define PLSR_CLHS2LP BIT(27)
+#define PLSR_CLLP2HS BIT(26)
+
+/* Video-Input Channel 1 Set 0 Register */
+#define VICH1SET0R 0x400
+#define VICH1SET0R_VSEN BIT(12)
+#define VICH1SET0R_HFPNOLP BIT(10)
+#define VICH1SET0R_HBPNOLP BIT(9)
+#define VICH1SET0R_HSANOLP BIT(8)
+#define VICH1SET0R_VSTPAFT BIT(1)
+#define VICH1SET0R_VSTART BIT(0)
+
+/* Video-Input Channel 1 Set 1 Register */
+#define VICH1SET1R 0x404
+#define VICH1SET1R_DLY(x) (((x) & 0xfff) << 2)
+
+/* Video-Input Channel 1 Status Register */
+#define VICH1SR 0x410
+#define VICH1SR_VIRDY BIT(3)
+#define VICH1SR_RUNNING BIT(2)
+#define VICH1SR_STOP BIT(1)
+#define VICH1SR_START BIT(0)
+
+/* Video-Input Channel 1 Pixel Packet Set Register */
+#define VICH1PPSETR 0x420
+#define VICH1PPSETR_DT_RGB18 (0x1e << 16)
+#define VICH1PPSETR_DT_RGB18_LS (0x2e << 16)
+#define VICH1PPSETR_DT_RGB24 (0x3e << 16)
+#define VICH1PPSETR_TXESYNC_PULSE (1 << 15)
+#define VICH1PPSETR_VC(x) ((x) << 22)
+
+/* Video-Input Channel 1 Vertical Size Set Register */
+#define VICH1VSSETR 0x428
+#define VICH1VSSETR_VACTIVE(x) (((x) & 0x7fff) << 16)
+#define VICH1VSSETR_VSPOL_LOW (1 << 15)
+#define VICH1VSSETR_VSPOL_HIGH (0 << 15)
+#define VICH1VSSETR_VSA(x) (((x) & 0xfff) << 0)
+
+/* Video-Input Channel 1 Vertical Porch Set Register */
+#define VICH1VPSETR 0x42c
+#define VICH1VPSETR_VFP(x) (((x) & 0x1fff) << 16)
+#define VICH1VPSETR_VBP(x) (((x) & 0x1fff) << 0)
+
+/* Video-Input Channel 1 Horizontal Size Set Register */
+#define VICH1HSSETR 0x430
+#define VICH1HSSETR_HACTIVE(x) (((x) & 0x7fff) << 16)
+#define VICH1HSSETR_HSPOL_LOW (1 << 15)
+#define VICH1HSSETR_HSPOL_HIGH (0 << 15)
+#define VICH1HSSETR_HSA(x) (((x) & 0xfff) << 0)
+
+/* Video-Input Channel 1 Horizontal Porch Set Register */
+#define VICH1HPSETR 0x434
+#define VICH1HPSETR_HFP(x) (((x) & 0x1fff) << 16)
+#define VICH1HPSETR_HBP(x) (((x) & 0x1fff) << 0)
+
+#endif /* __RZG2L_MIPI_DSI_REGS_H__ */
diff --git a/drivers/gpu/drm/rockchip/cdn-dp-core.c b/drivers/gpu/drm/rockchip/cdn-dp-core.c
index 518ee13..8526dda 100644
--- a/drivers/gpu/drm/rockchip/cdn-dp-core.c
+++ b/drivers/gpu/drm/rockchip/cdn-dp-core.c
@@ -571,7 +571,7 @@ static void cdn_dp_encoder_mode_set(struct drm_encoder *encoder,
video->v_sync_polarity = !!(mode->flags & DRM_MODE_FLAG_NVSYNC);
video->h_sync_polarity = !!(mode->flags & DRM_MODE_FLAG_NHSYNC);
- memcpy(&dp->mode, adjusted, sizeof(*mode));
+ drm_mode_copy(&dp->mode, adjusted);
}
static bool cdn_dp_check_link_status(struct cdn_dp_device *dp)
diff --git a/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c b/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c
index f4df982..7901c3b 100644
--- a/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c
+++ b/drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c
@@ -364,12 +364,6 @@ static inline u32 dsi_read(struct dw_mipi_dsi_rockchip *dsi, u32 reg)
return readl(dsi->base + reg);
}
-static inline void dsi_update_bits(struct dw_mipi_dsi_rockchip *dsi, u32 reg,
- u32 mask, u32 val)
-{
- dsi_write(dsi, reg, (dsi_read(dsi, reg) & ~mask) | val);
-}
-
static void dw_mipi_dsi_phy_write(struct dw_mipi_dsi_rockchip *dsi,
u8 test_code,
u8 test_data)
@@ -1221,7 +1215,7 @@ static int dw_mipi_dsi_dphy_power_on(struct phy *phy)
return i;
}
- ret = pm_runtime_get_sync(dsi->dev);
+ ret = pm_runtime_resume_and_get(dsi->dev);
if (ret < 0) {
DRM_DEV_ERROR(dsi->dev, "failed to enable device: %d\n", ret);
return ret;
diff --git a/drivers/gpu/drm/rockchip/inno_hdmi.c b/drivers/gpu/drm/rockchip/inno_hdmi.c
index 87b2243..f517748 100644
--- a/drivers/gpu/drm/rockchip/inno_hdmi.c
+++ b/drivers/gpu/drm/rockchip/inno_hdmi.c
@@ -499,7 +499,7 @@ static void inno_hdmi_encoder_mode_set(struct drm_encoder *encoder,
inno_hdmi_setup(hdmi, adj_mode);
/* Store the display mode for plugin/DPMS poweron events */
- memcpy(&hdmi->previous_mode, adj_mode, sizeof(hdmi->previous_mode));
+ drm_mode_copy(&hdmi->previous_mode, adj_mode);
}
static void inno_hdmi_encoder_enable(struct drm_encoder *encoder)
diff --git a/drivers/gpu/drm/rockchip/rk3066_hdmi.c b/drivers/gpu/drm/rockchip/rk3066_hdmi.c
index cf2cf51..90145ad 100644
--- a/drivers/gpu/drm/rockchip/rk3066_hdmi.c
+++ b/drivers/gpu/drm/rockchip/rk3066_hdmi.c
@@ -395,7 +395,7 @@ rk3066_hdmi_encoder_mode_set(struct drm_encoder *encoder,
struct rk3066_hdmi *hdmi = encoder_to_rk3066_hdmi(encoder);
/* Store the display mode for plugin/DPMS poweron events. */
- memcpy(&hdmi->previous_mode, adj_mode, sizeof(hdmi->previous_mode));
+ drm_mode_copy(&hdmi->previous_mode, adj_mode);
}
static void rk3066_hdmi_encoder_enable(struct drm_encoder *encoder)
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_drv.c b/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
index 813f9f8..6e0788d 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
@@ -17,7 +17,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_of.h>
#include <drm/drm_probe_helper.h>
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_drv.h b/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
index 1641440..aeb03a5 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
@@ -9,10 +9,10 @@
#ifndef _ROCKCHIP_DRM_DRV_H
#define _ROCKCHIP_DRM_DRV_H
-#include <drm/drm_fb_helper.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_gem.h>
+#include <linux/i2c.h>
#include <linux/module.h>
#include <linux/component.h>
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_fb.c b/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
index 092bf86..cfe8b79 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
@@ -9,7 +9,6 @@
#include <drm/drm.h>
#include <drm/drm_atomic.h>
#include <drm/drm_damage_helper.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_framebuffer_helper.h>
@@ -25,35 +24,6 @@ static const struct drm_framebuffer_funcs rockchip_drm_fb_funcs = {
.dirty = drm_atomic_helper_dirtyfb,
};
-static struct drm_framebuffer *
-rockchip_fb_alloc(struct drm_device *dev, const struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_gem_object **obj, unsigned int num_planes)
-{
- struct drm_framebuffer *fb;
- int ret;
- int i;
-
- fb = kzalloc(sizeof(*fb), GFP_KERNEL);
- if (!fb)
- return ERR_PTR(-ENOMEM);
-
- drm_helper_mode_fill_fb_struct(dev, fb, mode_cmd);
-
- for (i = 0; i < num_planes; i++)
- fb->obj[i] = obj[i];
-
- ret = drm_framebuffer_init(dev, fb, &rockchip_drm_fb_funcs);
- if (ret) {
- DRM_DEV_ERROR(dev->dev,
- "Failed to initialize framebuffer: %d\n",
- ret);
- kfree(fb);
- return ERR_PTR(ret);
- }
-
- return fb;
-}
-
static const struct drm_mode_config_helper_funcs rockchip_mode_config_helpers = {
.atomic_commit_tail = drm_atomic_helper_commit_tail_rpm,
};
@@ -101,25 +71,10 @@ rockchip_fb_create(struct drm_device *dev, struct drm_file *file,
static const struct drm_mode_config_funcs rockchip_drm_mode_config_funcs = {
.fb_create = rockchip_fb_create,
- .output_poll_changed = drm_fb_helper_output_poll_changed,
.atomic_check = drm_atomic_helper_check,
.atomic_commit = drm_atomic_helper_commit,
};
-struct drm_framebuffer *
-rockchip_drm_framebuffer_init(struct drm_device *dev,
- const struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_gem_object *obj)
-{
- struct drm_framebuffer *fb;
-
- fb = rockchip_fb_alloc(dev, mode_cmd, &obj, 1);
- if (IS_ERR(fb))
- return ERR_CAST(fb);
-
- return fb;
-}
-
void rockchip_drm_mode_config_init(struct drm_device *dev)
{
dev->mode_config.min_width = 0;
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_fb.h b/drivers/gpu/drm/rockchip/rockchip_drm_fb.h
index 1a69652..bae4e07 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_fb.h
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_fb.h
@@ -7,11 +7,5 @@
#ifndef _ROCKCHIP_DRM_FB_H
#define _ROCKCHIP_DRM_FB_H
-struct drm_framebuffer *
-rockchip_drm_framebuffer_init(struct drm_device *dev,
- const struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_gem_object *obj);
-void rockchip_drm_framebuffer_fini(struct drm_framebuffer *fb);
-
void rockchip_drm_mode_config_init(struct drm_device *dev);
#endif /* _ROCKCHIP_DRM_FB_H */
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
index da8a699..6edb7c5 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
@@ -9,6 +9,7 @@
#include <linux/vmalloc.h>
#include <drm/drm.h>
+#include <drm/drm_fb_helper.h>
#include <drm/drm_gem.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_prime.h>
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
index c356de5..fa1f4ee 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
@@ -602,7 +602,7 @@ static int vop_enable(struct drm_crtc *crtc, struct drm_crtc_state *old_state)
struct vop *vop = to_vop(crtc);
int ret, i;
- ret = pm_runtime_get_sync(vop->dev);
+ ret = pm_runtime_resume_and_get(vop->dev);
if (ret < 0) {
DRM_DEV_ERROR(vop->dev, "failed to get pm runtime: %d\n", ret);
return ret;
@@ -1983,7 +1983,7 @@ static int vop_initial(struct vop *vop)
return PTR_ERR(vop->dclk);
}
- ret = pm_runtime_get_sync(vop->dev);
+ ret = pm_runtime_resume_and_get(vop->dev);
if (ret < 0) {
DRM_DEV_ERROR(vop->dev, "failed to get pm runtime: %d\n", ret);
return ret;
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c b/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
index 105a548..8cecf81 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
@@ -822,7 +822,7 @@ static void vop2_enable(struct vop2 *vop2)
{
int ret;
- ret = pm_runtime_get_sync(vop2->dev);
+ ret = pm_runtime_resume_and_get(vop2->dev);
if (ret < 0) {
drm_err(vop2->drm, "failed to get pm runtime: %d\n", ret);
return;
diff --git a/drivers/gpu/drm/rockchip/rockchip_lvds.c b/drivers/gpu/drm/rockchip/rockchip_lvds.c
index 5a28433..68f6ebb 100644
--- a/drivers/gpu/drm/rockchip/rockchip_lvds.c
+++ b/drivers/gpu/drm/rockchip/rockchip_lvds.c
@@ -152,7 +152,7 @@ static int rk3288_lvds_poweron(struct rockchip_lvds *lvds)
DRM_DEV_ERROR(lvds->dev, "failed to enable lvds pclk %d\n", ret);
return ret;
}
- ret = pm_runtime_get_sync(lvds->dev);
+ ret = pm_runtime_resume_and_get(lvds->dev);
if (ret < 0) {
DRM_DEV_ERROR(lvds->dev, "failed to get pm runtime: %d\n", ret);
clk_disable(lvds->pclk);
@@ -336,16 +336,20 @@ static int px30_lvds_poweron(struct rockchip_lvds *lvds)
{
int ret;
- ret = pm_runtime_get_sync(lvds->dev);
+ ret = pm_runtime_resume_and_get(lvds->dev);
if (ret < 0) {
DRM_DEV_ERROR(lvds->dev, "failed to get pm runtime: %d\n", ret);
return ret;
}
/* Enable LVDS mode */
- return regmap_update_bits(lvds->grf, PX30_LVDS_GRF_PD_VO_CON1,
+ ret = regmap_update_bits(lvds->grf, PX30_LVDS_GRF_PD_VO_CON1,
PX30_LVDS_MODE_EN(1) | PX30_LVDS_P2S_EN(1),
PX30_LVDS_MODE_EN(1) | PX30_LVDS_P2S_EN(1));
+ if (ret)
+ pm_runtime_put(lvds->dev);
+
+ return ret;
}
static void px30_lvds_poweroff(struct rockchip_lvds *lvds)
diff --git a/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c b/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c
index 9bf0637b..9d30aa7 100644
--- a/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c
+++ b/drivers/gpu/drm/rockchip/rockchip_vop2_reg.c
@@ -188,7 +188,7 @@ static const struct vop2_win_data rk3568_vop_win_data[] = {
.base = 0x1800,
.layer_sel_id = 2,
.supported_rotations = DRM_MODE_REFLECT_Y,
- .type = DRM_PLANE_TYPE_OVERLAY,
+ .type = DRM_PLANE_TYPE_PRIMARY,
.max_upscale_factor = 8,
.max_downscale_factor = 8,
.dly = { 20, 47, 41 },
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 4b913db..fe09e5b 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -73,6 +73,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
entity->priority = priority;
entity->sched_list = num_sched_list > 1 ? sched_list : NULL;
entity->last_scheduled = NULL;
+ RB_CLEAR_NODE(&entity->rb_tree_node);
if(num_sched_list)
entity->rq = &sched_list[0]->sched_rq[entity->priority];
@@ -139,6 +140,73 @@ bool drm_sched_entity_is_ready(struct drm_sched_entity *entity)
return true;
}
+static void drm_sched_entity_kill_jobs_work(struct work_struct *wrk)
+{
+ struct drm_sched_job *job = container_of(wrk, typeof(*job), work);
+
+ drm_sched_fence_finished(job->s_fence);
+ WARN_ON(job->s_fence->parent);
+ job->sched->ops->free_job(job);
+}
+
+/* Signal the scheduler finished fence when the entity in question is killed. */
+static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
+ struct dma_fence_cb *cb)
+{
+ struct drm_sched_job *job = container_of(cb, struct drm_sched_job,
+ finish_cb);
+ int r;
+
+ dma_fence_put(f);
+
+ /* Wait for all dependencies to avoid data corruptions */
+ while (!xa_empty(&job->dependencies)) {
+ f = xa_erase(&job->dependencies, job->last_dependency++);
+ r = dma_fence_add_callback(f, &job->finish_cb,
+ drm_sched_entity_kill_jobs_cb);
+ if (!r)
+ return;
+
+ dma_fence_put(f);
+ }
+
+ INIT_WORK(&job->work, drm_sched_entity_kill_jobs_work);
+ schedule_work(&job->work);
+}
+
+/* Remove the entity from the scheduler and kill all pending jobs */
+static void drm_sched_entity_kill(struct drm_sched_entity *entity)
+{
+ struct drm_sched_job *job;
+ struct dma_fence *prev;
+
+ if (!entity->rq)
+ return;
+
+ spin_lock(&entity->rq_lock);
+ entity->stopped = true;
+ drm_sched_rq_remove_entity(entity->rq, entity);
+ spin_unlock(&entity->rq_lock);
+
+ /* Make sure this entity is not used by the scheduler at the moment */
+ wait_for_completion(&entity->entity_idle);
+
+ prev = dma_fence_get(entity->last_scheduled);
+ while ((job = to_drm_sched_job(spsc_queue_pop(&entity->job_queue)))) {
+ struct drm_sched_fence *s_fence = job->s_fence;
+
+ dma_fence_set_error(&s_fence->finished, -ESRCH);
+
+ dma_fence_get(&s_fence->finished);
+ if (!prev || dma_fence_add_callback(prev, &job->finish_cb,
+ drm_sched_entity_kill_jobs_cb))
+ drm_sched_entity_kill_jobs_cb(NULL, &job->finish_cb);
+
+ prev = &s_fence->finished;
+ }
+ dma_fence_put(prev);
+}
+
/**
* drm_sched_entity_flush - Flush a context entity
*
@@ -179,91 +247,13 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)
/* For killed process disable any more IBs enqueue right now */
last_user = cmpxchg(&entity->last_user, current->group_leader, NULL);
if ((!last_user || last_user == current->group_leader) &&
- (current->flags & PF_EXITING) && (current->exit_code == SIGKILL)) {
- spin_lock(&entity->rq_lock);
- entity->stopped = true;
- drm_sched_rq_remove_entity(entity->rq, entity);
- spin_unlock(&entity->rq_lock);
- }
+ (current->flags & PF_EXITING) && (current->exit_code == SIGKILL))
+ drm_sched_entity_kill(entity);
return ret;
}
EXPORT_SYMBOL(drm_sched_entity_flush);
-static void drm_sched_entity_kill_jobs_work(struct work_struct *wrk)
-{
- struct drm_sched_job *job = container_of(wrk, typeof(*job), work);
-
- drm_sched_fence_finished(job->s_fence);
- WARN_ON(job->s_fence->parent);
- job->sched->ops->free_job(job);
-}
-
-
-/* Signal the scheduler finished fence when the entity in question is killed. */
-static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
- struct dma_fence_cb *cb)
-{
- struct drm_sched_job *job = container_of(cb, struct drm_sched_job,
- finish_cb);
-
- dma_fence_put(f);
- INIT_WORK(&job->work, drm_sched_entity_kill_jobs_work);
- schedule_work(&job->work);
-}
-
-static struct dma_fence *
-drm_sched_job_dependency(struct drm_sched_job *job,
- struct drm_sched_entity *entity)
-{
- if (!xa_empty(&job->dependencies))
- return xa_erase(&job->dependencies, job->last_dependency++);
-
- if (job->sched->ops->dependency)
- return job->sched->ops->dependency(job, entity);
-
- return NULL;
-}
-
-static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
-{
- struct drm_sched_job *job;
- struct dma_fence *f;
- int r;
-
- while ((job = to_drm_sched_job(spsc_queue_pop(&entity->job_queue)))) {
- struct drm_sched_fence *s_fence = job->s_fence;
-
- /* Wait for all dependencies to avoid data corruptions */
- while ((f = drm_sched_job_dependency(job, entity))) {
- dma_fence_wait(f, false);
- dma_fence_put(f);
- }
-
- drm_sched_fence_scheduled(s_fence);
- dma_fence_set_error(&s_fence->finished, -ESRCH);
-
- /*
- * When pipe is hanged by older entity, new entity might
- * not even have chance to submit it's first job to HW
- * and so entity->last_scheduled will remain NULL
- */
- if (!entity->last_scheduled) {
- drm_sched_entity_kill_jobs_cb(NULL, &job->finish_cb);
- continue;
- }
-
- dma_fence_get(entity->last_scheduled);
- r = dma_fence_add_callback(entity->last_scheduled,
- &job->finish_cb,
- drm_sched_entity_kill_jobs_cb);
- if (r == -ENOENT)
- drm_sched_entity_kill_jobs_cb(NULL, &job->finish_cb);
- else if (r)
- DRM_ERROR("fence add callback failed (%d)\n", r);
- }
-}
-
/**
* drm_sched_entity_fini - Destroy a context entity
*
@@ -277,33 +267,17 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
*/
void drm_sched_entity_fini(struct drm_sched_entity *entity)
{
- struct drm_gpu_scheduler *sched = NULL;
-
- if (entity->rq) {
- sched = entity->rq->sched;
- drm_sched_rq_remove_entity(entity->rq, entity);
- }
-
- /* Consumption of existing IBs wasn't completed. Forcefully
- * remove them here.
+ /*
+ * If consumption of existing IBs wasn't completed. Forcefully remove
+ * them here. Also makes sure that the scheduler won't touch this entity
+ * any more.
*/
- if (spsc_queue_count(&entity->job_queue)) {
- if (sched) {
- /*
- * Wait for thread to idle to make sure it isn't processing
- * this entity.
- */
- wait_for_completion(&entity->entity_idle);
+ drm_sched_entity_kill(entity);
- }
- if (entity->dependency) {
- dma_fence_remove_callback(entity->dependency,
- &entity->cb);
- dma_fence_put(entity->dependency);
- entity->dependency = NULL;
- }
-
- drm_sched_entity_kill_jobs(entity);
+ if (entity->dependency) {
+ dma_fence_remove_callback(entity->dependency, &entity->cb);
+ dma_fence_put(entity->dependency);
+ entity->dependency = NULL;
}
dma_fence_put(entity->last_scheduled);
@@ -416,6 +390,19 @@ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
return false;
}
+static struct dma_fence *
+drm_sched_job_dependency(struct drm_sched_job *job,
+ struct drm_sched_entity *entity)
+{
+ if (!xa_empty(&job->dependencies))
+ return xa_erase(&job->dependencies, job->last_dependency++);
+
+ if (job->sched->ops->prepare_job)
+ return job->sched->ops->prepare_job(job, entity);
+
+ return NULL;
+}
+
struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
{
struct drm_sched_job *sched_job;
@@ -448,6 +435,19 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
smp_wmb();
spsc_queue_pop(&entity->job_queue);
+
+ /*
+ * Update the entity's location in the min heap according to
+ * the timestamp of the next job, if any.
+ */
+ if (drm_sched_policy == DRM_SCHED_POLICY_FIFO) {
+ struct drm_sched_job *next;
+
+ next = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));
+ if (next)
+ drm_sched_rq_update_fifo(entity, next->submit_ts);
+ }
+
return sched_job;
}
@@ -512,6 +512,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
atomic_inc(entity->rq->sched->score);
WRITE_ONCE(entity->last_user, current->group_leader);
first = spsc_queue_push(&entity->job_queue, &sched_job->queue_node);
+ sched_job->submit_ts = ktime_get();
/* first job wakes up scheduler */
if (first) {
@@ -523,8 +524,13 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
DRM_ERROR("Trying to push to a killed entity\n");
return;
}
+
drm_sched_rq_add_entity(entity->rq, entity);
spin_unlock(&entity->rq_lock);
+
+ if (drm_sched_policy == DRM_SCHED_POLICY_FIFO)
+ drm_sched_rq_update_fifo(entity, sched_job->submit_ts);
+
drm_sched_wakeup(entity->rq->sched);
}
}
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index e5a4ecd..31f3a12 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -62,6 +62,55 @@
#define to_drm_sched_job(sched_job) \
container_of((sched_job), struct drm_sched_job, queue_node)
+int drm_sched_policy = DRM_SCHED_POLICY_FIFO;
+
+/**
+ * DOC: sched_policy (int)
+ * Used to override default entities scheduling policy in a run queue.
+ */
+MODULE_PARM_DESC(sched_policy, "Specify the scheduling policy for entities on a run-queue, " __stringify(DRM_SCHED_POLICY_RR) " = Round Robin, " __stringify(DRM_SCHED_POLICY_FIFO) " = FIFO (default).");
+module_param_named(sched_policy, drm_sched_policy, int, 0444);
+
+static __always_inline bool drm_sched_entity_compare_before(struct rb_node *a,
+ const struct rb_node *b)
+{
+ struct drm_sched_entity *ent_a = rb_entry((a), struct drm_sched_entity, rb_tree_node);
+ struct drm_sched_entity *ent_b = rb_entry((b), struct drm_sched_entity, rb_tree_node);
+
+ return ktime_before(ent_a->oldest_job_waiting, ent_b->oldest_job_waiting);
+}
+
+static inline void drm_sched_rq_remove_fifo_locked(struct drm_sched_entity *entity)
+{
+ struct drm_sched_rq *rq = entity->rq;
+
+ if (!RB_EMPTY_NODE(&entity->rb_tree_node)) {
+ rb_erase_cached(&entity->rb_tree_node, &rq->rb_tree_root);
+ RB_CLEAR_NODE(&entity->rb_tree_node);
+ }
+}
+
+void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts)
+{
+ /*
+ * Both locks need to be grabbed, one to protect from entity->rq change
+ * for entity from within concurrent drm_sched_entity_select_rq and the
+ * other to update the rb tree structure.
+ */
+ spin_lock(&entity->rq_lock);
+ spin_lock(&entity->rq->lock);
+
+ drm_sched_rq_remove_fifo_locked(entity);
+
+ entity->oldest_job_waiting = ts;
+
+ rb_add_cached(&entity->rb_tree_node, &entity->rq->rb_tree_root,
+ drm_sched_entity_compare_before);
+
+ spin_unlock(&entity->rq->lock);
+ spin_unlock(&entity->rq_lock);
+}
+
/**
* drm_sched_rq_init - initialize a given run queue struct
*
@@ -75,6 +124,7 @@ static void drm_sched_rq_init(struct drm_gpu_scheduler *sched,
{
spin_lock_init(&rq->lock);
INIT_LIST_HEAD(&rq->entities);
+ rq->rb_tree_root = RB_ROOT_CACHED;
rq->current_entity = NULL;
rq->sched = sched;
}
@@ -92,9 +142,12 @@ void drm_sched_rq_add_entity(struct drm_sched_rq *rq,
{
if (!list_empty(&entity->list))
return;
+
spin_lock(&rq->lock);
+
atomic_inc(rq->sched->score);
list_add_tail(&entity->list, &rq->entities);
+
spin_unlock(&rq->lock);
}
@@ -111,23 +164,30 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,
{
if (list_empty(&entity->list))
return;
+
spin_lock(&rq->lock);
+
atomic_dec(rq->sched->score);
list_del_init(&entity->list);
+
if (rq->current_entity == entity)
rq->current_entity = NULL;
+
+ if (drm_sched_policy == DRM_SCHED_POLICY_FIFO)
+ drm_sched_rq_remove_fifo_locked(entity);
+
spin_unlock(&rq->lock);
}
/**
- * drm_sched_rq_select_entity - Select an entity which could provide a job to run
+ * drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run
*
* @rq: scheduler run queue to check.
*
* Try to find a ready entity, returns NULL if none found.
*/
static struct drm_sched_entity *
-drm_sched_rq_select_entity(struct drm_sched_rq *rq)
+drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)
{
struct drm_sched_entity *entity;
@@ -164,6 +224,34 @@ drm_sched_rq_select_entity(struct drm_sched_rq *rq)
}
/**
+ * drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run
+ *
+ * @rq: scheduler run queue to check.
+ *
+ * Find oldest waiting ready entity, returns NULL if none found.
+ */
+static struct drm_sched_entity *
+drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq)
+{
+ struct rb_node *rb;
+
+ spin_lock(&rq->lock);
+ for (rb = rb_first_cached(&rq->rb_tree_root); rb; rb = rb_next(rb)) {
+ struct drm_sched_entity *entity;
+
+ entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);
+ if (drm_sched_entity_is_ready(entity)) {
+ rq->current_entity = entity;
+ reinit_completion(&entity->entity_idle);
+ break;
+ }
+ }
+ spin_unlock(&rq->lock);
+
+ return rb ? rb_entry(rb, struct drm_sched_entity, rb_tree_node) : NULL;
+}
+
+/**
* drm_sched_job_done - complete a job
* @s_job: pointer to the job which is done
*
@@ -198,32 +286,6 @@ static void drm_sched_job_done_cb(struct dma_fence *f, struct dma_fence_cb *cb)
}
/**
- * drm_sched_dependency_optimized - test if the dependency can be optimized
- *
- * @fence: the dependency fence
- * @entity: the entity which depends on the above fence
- *
- * Returns true if the dependency can be optimized and false otherwise
- */
-bool drm_sched_dependency_optimized(struct dma_fence* fence,
- struct drm_sched_entity *entity)
-{
- struct drm_gpu_scheduler *sched = entity->rq->sched;
- struct drm_sched_fence *s_fence;
-
- if (!fence || dma_fence_is_signaled(fence))
- return false;
- if (fence->context == entity->fence_context)
- return true;
- s_fence = to_drm_sched_fence(fence);
- if (s_fence && s_fence->sched == sched)
- return true;
-
- return false;
-}
-EXPORT_SYMBOL(drm_sched_dependency_optimized);
-
-/**
* drm_sched_start_timeout - start timeout for reset worker
*
* @sched: scheduler instance to start the worker for
@@ -355,27 +417,6 @@ static void drm_sched_job_timedout(struct work_struct *work)
}
}
- /**
- * drm_sched_increase_karma - Update sched_entity guilty flag
- *
- * @bad: The job guilty of time out
- *
- * Increment on every hang caused by the 'bad' job. If this exceeds the hang
- * limit of the scheduler then the respective sched entity is marked guilty and
- * jobs from it will not be scheduled further
- */
-void drm_sched_increase_karma(struct drm_sched_job *bad)
-{
- drm_sched_increase_karma_ext(bad, 1);
-}
-EXPORT_SYMBOL(drm_sched_increase_karma);
-
-void drm_sched_reset_karma(struct drm_sched_job *bad)
-{
- drm_sched_increase_karma_ext(bad, 0);
-}
-EXPORT_SYMBOL(drm_sched_reset_karma);
-
/**
* drm_sched_stop - stop the scheduler
*
@@ -517,31 +558,14 @@ EXPORT_SYMBOL(drm_sched_start);
*/
void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched)
{
- drm_sched_resubmit_jobs_ext(sched, INT_MAX);
-}
-EXPORT_SYMBOL(drm_sched_resubmit_jobs);
-
-/**
- * drm_sched_resubmit_jobs_ext - helper to relunch certain number of jobs from mirror ring list
- *
- * @sched: scheduler instance
- * @max: job numbers to relaunch
- *
- */
-void drm_sched_resubmit_jobs_ext(struct drm_gpu_scheduler *sched, int max)
-{
struct drm_sched_job *s_job, *tmp;
uint64_t guilty_context;
bool found_guilty = false;
struct dma_fence *fence;
- int i = 0;
list_for_each_entry_safe(s_job, tmp, &sched->pending_list, list) {
struct drm_sched_fence *s_fence = s_job->s_fence;
- if (i >= max)
- break;
-
if (!found_guilty && atomic_read(&s_job->karma) > sched->hang_limit) {
found_guilty = true;
guilty_context = s_job->s_fence->scheduled.context;
@@ -551,7 +575,6 @@ void drm_sched_resubmit_jobs_ext(struct drm_gpu_scheduler *sched, int max)
dma_fence_set_error(&s_fence->finished, -ECANCELED);
fence = sched->ops->run_job(s_job);
- i++;
if (IS_ERR_OR_NULL(fence)) {
if (IS_ERR(fence))
@@ -567,7 +590,7 @@ void drm_sched_resubmit_jobs_ext(struct drm_gpu_scheduler *sched, int max)
}
}
}
-EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
+EXPORT_SYMBOL(drm_sched_resubmit_jobs);
/**
* drm_sched_job_init - init a scheduler job
@@ -685,6 +708,41 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
EXPORT_SYMBOL(drm_sched_job_add_dependency);
/**
+ * drm_sched_job_add_resv_dependencies - add all fences from the resv to the job
+ * @job: scheduler job to add the dependencies to
+ * @resv: the dma_resv object to get the fences from
+ * @usage: the dma_resv_usage to use to filter the fences
+ *
+ * This adds all fences matching the given usage from @resv to @job.
+ * Must be called with the @resv lock held.
+ *
+ * Returns:
+ * 0 on success, or an error on failing to expand the array.
+ */
+int drm_sched_job_add_resv_dependencies(struct drm_sched_job *job,
+ struct dma_resv *resv,
+ enum dma_resv_usage usage)
+{
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
+ int ret;
+
+ dma_resv_assert_held(resv);
+
+ dma_resv_for_each_fence(&cursor, resv, usage, fence) {
+ /* Make sure to grab an additional ref on the added fence */
+ dma_fence_get(fence);
+ ret = drm_sched_job_add_dependency(job, fence);
+ if (ret) {
+ dma_fence_put(fence);
+ return ret;
+ }
+ }
+ return 0;
+}
+EXPORT_SYMBOL(drm_sched_job_add_resv_dependencies);
+
+/**
* drm_sched_job_add_implicit_dependencies - adds implicit dependencies as job
* dependencies
* @job: scheduler job to add the dependencies to
@@ -703,27 +761,11 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
struct drm_gem_object *obj,
bool write)
{
- struct dma_resv_iter cursor;
- struct dma_fence *fence;
- int ret;
-
- dma_resv_assert_held(obj->resv);
-
- dma_resv_for_each_fence(&cursor, obj->resv, dma_resv_usage_rw(write),
- fence) {
- /* Make sure to grab an additional ref on the added fence */
- dma_fence_get(fence);
- ret = drm_sched_job_add_dependency(job, fence);
- if (ret) {
- dma_fence_put(fence);
- return ret;
- }
- }
- return 0;
+ return drm_sched_job_add_resv_dependencies(job, obj->resv,
+ dma_resv_usage_rw(write));
}
EXPORT_SYMBOL(drm_sched_job_add_implicit_dependencies);
-
/**
* drm_sched_job_cleanup - clean up scheduler job resources
* @job: scheduler job to clean up
@@ -803,7 +845,9 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)
/* Kernel run queue has higher priority than normal run queue*/
for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {
- entity = drm_sched_rq_select_entity(&sched->sched_rq[i]);
+ entity = drm_sched_policy == DRM_SCHED_POLICY_FIFO ?
+ drm_sched_rq_select_entity_fifo(&sched->sched_rq[i]) :
+ drm_sched_rq_select_entity_rr(&sched->sched_rq[i]);
if (entity)
break;
}
@@ -1082,13 +1126,15 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)
EXPORT_SYMBOL(drm_sched_fini);
/**
- * drm_sched_increase_karma_ext - Update sched_entity guilty flag
+ * drm_sched_increase_karma - Update sched_entity guilty flag
*
* @bad: The job guilty of time out
- * @type: type for increase/reset karma
*
+ * Increment on every hang caused by the 'bad' job. If this exceeds the hang
+ * limit of the scheduler then the respective sched entity is marked guilty and
+ * jobs from it will not be scheduled further
*/
-void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type)
+void drm_sched_increase_karma(struct drm_sched_job *bad)
{
int i;
struct drm_sched_entity *tmp;
@@ -1100,10 +1146,7 @@ void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type)
* corrupt but keep in mind that kernel jobs always considered good.
*/
if (bad->s_priority != DRM_SCHED_PRIORITY_KERNEL) {
- if (type == 0)
- atomic_set(&bad->karma, 0);
- else if (type == 1)
- atomic_inc(&bad->karma);
+ atomic_inc(&bad->karma);
for (i = DRM_SCHED_PRIORITY_MIN; i < DRM_SCHED_PRIORITY_KERNEL;
i++) {
@@ -1114,7 +1157,7 @@ void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type)
if (bad->s_fence->scheduled.context ==
entity->fence_context) {
if (entity->guilty)
- atomic_set(entity->guilty, type);
+ atomic_set(entity->guilty, 1);
break;
}
}
@@ -1124,4 +1167,4 @@ void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type)
}
}
}
-EXPORT_SYMBOL(drm_sched_increase_karma_ext);
+EXPORT_SYMBOL(drm_sched_increase_karma);
diff --git a/drivers/gpu/drm/solomon/ssd130x.c b/drivers/gpu/drm/solomon/ssd130x.c
index bc41a5a..53464af 100644
--- a/drivers/gpu/drm/solomon/ssd130x.c
+++ b/drivers/gpu/drm/solomon/ssd130x.c
@@ -20,9 +20,10 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
#include <drm/drm_damage_helper.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_atomic_helper.h>
@@ -578,21 +579,24 @@ static void ssd130x_primary_plane_helper_atomic_update(struct drm_plane *plane,
struct drm_plane_state *plane_state = drm_atomic_get_new_plane_state(state, plane);
struct drm_plane_state *old_plane_state = drm_atomic_get_old_plane_state(state, plane);
struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(plane_state);
+ struct drm_atomic_helper_damage_iter iter;
struct drm_device *drm = plane->dev;
- struct drm_rect src_clip, dst_clip;
+ struct drm_rect dst_clip;
+ struct drm_rect damage;
int idx;
- if (!drm_atomic_helper_damage_merged(old_plane_state, plane_state, &src_clip))
- return;
-
- dst_clip = plane_state->dst;
- if (!drm_rect_intersect(&dst_clip, &src_clip))
- return;
-
if (!drm_dev_enter(drm, &idx))
return;
- ssd130x_fb_blit_rect(plane_state->fb, &shadow_plane_state->data[0], &dst_clip);
+ drm_atomic_helper_damage_iter_init(&iter, old_plane_state, plane_state);
+ drm_atomic_for_each_plane_damage(&iter, &damage) {
+ dst_clip = plane_state->dst;
+
+ if (!drm_rect_intersect(&dst_clip, &damage))
+ continue;
+
+ ssd130x_fb_blit_rect(plane_state->fb, &shadow_plane_state->data[0], &dst_clip);
+ }
drm_dev_exit(idx);
}
@@ -642,19 +646,6 @@ static enum drm_mode_status ssd130x_crtc_helper_mode_valid(struct drm_crtc *crtc
return MODE_OK;
}
-static int ssd130x_crtc_helper_atomic_check(struct drm_crtc *crtc,
- struct drm_atomic_state *new_state)
-{
- struct drm_crtc_state *new_crtc_state = drm_atomic_get_new_crtc_state(new_state, crtc);
- int ret;
-
- ret = drm_atomic_helper_check_crtc_state(new_crtc_state, false);
- if (ret)
- return ret;
-
- return drm_atomic_add_affected_planes(new_state, crtc);
-}
-
/*
* The CRTC is always enabled. Screen updates are performed by
* the primary plane's atomic_update function. Disabling clears
@@ -662,7 +653,7 @@ static int ssd130x_crtc_helper_atomic_check(struct drm_crtc *crtc,
*/
static const struct drm_crtc_helper_funcs ssd130x_crtc_helper_funcs = {
.mode_valid = ssd130x_crtc_helper_mode_valid,
- .atomic_check = ssd130x_crtc_helper_atomic_check,
+ .atomic_check = drm_crtc_helper_atomic_check,
};
static void ssd130x_crtc_reset(struct drm_crtc *crtc)
diff --git a/drivers/gpu/drm/sti/sti_drv.c b/drivers/gpu/drm/sti/sti_drv.c
index 7abf010..ef6a4e63 100644
--- a/drivers/gpu/drm/sti/sti_drv.c
+++ b/drivers/gpu/drm/sti/sti_drv.c
@@ -14,7 +14,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_debugfs.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_of.h>
diff --git a/drivers/gpu/drm/sti/sti_dvo.c b/drivers/gpu/drm/sti/sti_dvo.c
index b6ee8a8..f3a5616 100644
--- a/drivers/gpu/drm/sti/sti_dvo.c
+++ b/drivers/gpu/drm/sti/sti_dvo.c
@@ -288,7 +288,7 @@ static void sti_dvo_set_mode(struct drm_bridge *bridge,
DRM_DEBUG_DRIVER("\n");
- memcpy(&dvo->mode, mode, sizeof(struct drm_display_mode));
+ drm_mode_copy(&dvo->mode, mode);
/* According to the path used (main or aux), the dvo clocks should
* have a different parent clock. */
diff --git a/drivers/gpu/drm/sti/sti_hda.c b/drivers/gpu/drm/sti/sti_hda.c
index 03cc401..ec6656b 100644
--- a/drivers/gpu/drm/sti/sti_hda.c
+++ b/drivers/gpu/drm/sti/sti_hda.c
@@ -524,7 +524,7 @@ static void sti_hda_set_mode(struct drm_bridge *bridge,
DRM_DEBUG_DRIVER("\n");
- memcpy(&hda->mode, mode, sizeof(struct drm_display_mode));
+ drm_mode_copy(&hda->mode, mode);
if (!hda_get_mode_idx(hda->mode, &mode_idx)) {
DRM_ERROR("Undefined mode\n");
diff --git a/drivers/gpu/drm/sti/sti_hdmi.c b/drivers/gpu/drm/sti/sti_hdmi.c
index cb82622..fcc2194 100644
--- a/drivers/gpu/drm/sti/sti_hdmi.c
+++ b/drivers/gpu/drm/sti/sti_hdmi.c
@@ -941,7 +941,7 @@ static void sti_hdmi_set_mode(struct drm_bridge *bridge,
DRM_DEBUG_DRIVER("\n");
/* Copy the drm display mode in the connector local structure */
- memcpy(&hdmi->mode, mode, sizeof(struct drm_display_mode));
+ drm_mode_copy(&hdmi->mode, mode);
/* Update clock framerate according to the selected mode */
ret = clk_set_rate(hdmi->clk_pix, mode->clock * 1000);
diff --git a/drivers/gpu/drm/stm/drv.c b/drivers/gpu/drm/stm/drv.c
index d7914f5..50410bd 100644
--- a/drivers/gpu/drm/stm/drv.c
+++ b/drivers/gpu/drm/stm/drv.c
@@ -18,7 +18,7 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/sun4i/sun4i_drv.c b/drivers/gpu/drm/sun4i/sun4i_drv.c
index d06ffd9..cc94efb 100644
--- a/drivers/gpu/drm/sun4i/sun4i_drv.c
+++ b/drivers/gpu/drm/sun4i/sun4i_drv.c
@@ -17,7 +17,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_module.h>
#include <drm/drm_of.h>
diff --git a/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c b/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c
index 34234a1..760ff05 100644
--- a/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c
+++ b/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c
@@ -1101,12 +1101,16 @@ static const struct component_ops sun6i_dsi_ops = {
static int sun6i_dsi_probe(struct platform_device *pdev)
{
+ const struct sun6i_dsi_variant *variant;
struct device *dev = &pdev->dev;
- const char *bus_clk_name = NULL;
struct sun6i_dsi *dsi;
void __iomem *base;
int ret;
+ variant = device_get_match_data(dev);
+ if (!variant)
+ return -EINVAL;
+
dsi = devm_kzalloc(dev, sizeof(*dsi), GFP_KERNEL);
if (!dsi)
return -ENOMEM;
@@ -1114,10 +1118,7 @@ static int sun6i_dsi_probe(struct platform_device *pdev)
dsi->dev = dev;
dsi->host.ops = &sun6i_dsi_host_ops;
dsi->host.dev = dev;
-
- if (of_device_is_compatible(dev->of_node,
- "allwinner,sun6i-a31-mipi-dsi"))
- bus_clk_name = "bus";
+ dsi->variant = variant;
base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(base)) {
@@ -1142,7 +1143,7 @@ static int sun6i_dsi_probe(struct platform_device *pdev)
return PTR_ERR(dsi->regs);
}
- dsi->bus_clk = devm_clk_get(dev, bus_clk_name);
+ dsi->bus_clk = devm_clk_get(dev, variant->has_mod_clk ? "bus" : NULL);
if (IS_ERR(dsi->bus_clk))
return dev_err_probe(dev, PTR_ERR(dsi->bus_clk),
"Couldn't get the DSI bus clock\n");
@@ -1151,21 +1152,21 @@ static int sun6i_dsi_probe(struct platform_device *pdev)
if (ret)
return ret;
- if (of_device_is_compatible(dev->of_node,
- "allwinner,sun6i-a31-mipi-dsi")) {
+ if (variant->has_mod_clk) {
dsi->mod_clk = devm_clk_get(dev, "mod");
if (IS_ERR(dsi->mod_clk)) {
dev_err(dev, "Couldn't get the DSI mod clock\n");
ret = PTR_ERR(dsi->mod_clk);
goto err_attach_clk;
}
- }
- /*
- * In order to operate properly, that clock seems to be always
- * set to 297MHz.
- */
- clk_set_rate_exclusive(dsi->mod_clk, 297000000);
+ /*
+ * In order to operate properly, the module clock on the
+ * A31 variant always seems to be set to 297MHz.
+ */
+ if (variant->set_mod_clk)
+ clk_set_rate_exclusive(dsi->mod_clk, 297000000);
+ }
dsi->dphy = devm_phy_get(dev, "dphy");
if (IS_ERR(dsi->dphy)) {
@@ -1191,7 +1192,8 @@ static int sun6i_dsi_probe(struct platform_device *pdev)
err_remove_dsi_host:
mipi_dsi_host_unregister(&dsi->host);
err_unprotect_clk:
- clk_rate_exclusive_put(dsi->mod_clk);
+ if (dsi->variant->has_mod_clk && dsi->variant->set_mod_clk)
+ clk_rate_exclusive_put(dsi->mod_clk);
err_attach_clk:
regmap_mmio_detach_clk(dsi->regs);
@@ -1205,16 +1207,39 @@ static int sun6i_dsi_remove(struct platform_device *pdev)
component_del(&pdev->dev, &sun6i_dsi_ops);
mipi_dsi_host_unregister(&dsi->host);
- clk_rate_exclusive_put(dsi->mod_clk);
+ if (dsi->variant->has_mod_clk && dsi->variant->set_mod_clk)
+ clk_rate_exclusive_put(dsi->mod_clk);
regmap_mmio_detach_clk(dsi->regs);
return 0;
}
+static const struct sun6i_dsi_variant sun6i_a31_mipi_dsi_variant = {
+ .has_mod_clk = true,
+ .set_mod_clk = true,
+};
+
+static const struct sun6i_dsi_variant sun50i_a64_mipi_dsi_variant = {
+};
+
+static const struct sun6i_dsi_variant sun50i_a100_mipi_dsi_variant = {
+ .has_mod_clk = true,
+};
+
static const struct of_device_id sun6i_dsi_of_table[] = {
- { .compatible = "allwinner,sun6i-a31-mipi-dsi" },
- { .compatible = "allwinner,sun50i-a64-mipi-dsi" },
+ {
+ .compatible = "allwinner,sun6i-a31-mipi-dsi",
+ .data = &sun6i_a31_mipi_dsi_variant,
+ },
+ {
+ .compatible = "allwinner,sun50i-a64-mipi-dsi",
+ .data = &sun50i_a64_mipi_dsi_variant,
+ },
+ {
+ .compatible = "allwinner,sun50i-a100-mipi-dsi",
+ .data = &sun50i_a100_mipi_dsi_variant,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, sun6i_dsi_of_table);
diff --git a/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.h b/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.h
index c863900..f1ddefe 100644
--- a/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.h
+++ b/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.h
@@ -15,6 +15,11 @@
#define SUN6I_DSI_TCON_DIV 4
+struct sun6i_dsi_variant {
+ bool has_mod_clk;
+ bool set_mod_clk;
+};
+
struct sun6i_dsi {
struct drm_connector connector;
struct drm_encoder encoder;
@@ -31,6 +36,8 @@ struct sun6i_dsi {
struct mipi_dsi_device *device;
struct drm_device *drm;
struct drm_panel *panel;
+
+ const struct sun6i_dsi_variant *variant;
};
static inline struct sun6i_dsi *host_to_sun6i_dsi(struct mipi_dsi_host *host)
diff --git a/drivers/gpu/drm/tegra/Makefile b/drivers/gpu/drm/tegra/Makefile
index df6cc98..bb0d2c1 100644
--- a/drivers/gpu/drm/tegra/Makefile
+++ b/drivers/gpu/drm/tegra/Makefile
@@ -24,7 +24,8 @@
gr3d.o \
falcon.o \
vic.o \
- nvdec.o
+ nvdec.o \
+ riscv.o
tegra-drm-y += trace.o
diff --git a/drivers/gpu/drm/tegra/dc.c b/drivers/gpu/drm/tegra/dc.c
index bd0f607..a67453c 100644
--- a/drivers/gpu/drm/tegra/dc.c
+++ b/drivers/gpu/drm/tegra/dc.c
@@ -3205,8 +3205,10 @@ static int tegra_dc_probe(struct platform_device *pdev)
usleep_range(2000, 4000);
err = reset_control_assert(dc->rst);
- if (err < 0)
+ if (err < 0) {
+ clk_disable_unprepare(dc->clk);
return err;
+ }
usleep_range(2000, 4000);
diff --git a/drivers/gpu/drm/tegra/drm.c b/drivers/gpu/drm/tegra/drm.c
index a1f909d..7bd2e65 100644
--- a/drivers/gpu/drm/tegra/drm.c
+++ b/drivers/gpu/drm/tegra/drm.c
@@ -1386,6 +1386,7 @@ static const struct of_device_id host1x_drm_subdevs[] = {
{ .compatible = "nvidia,tegra194-vic", },
{ .compatible = "nvidia,tegra194-nvdec", },
{ .compatible = "nvidia,tegra234-vic", },
+ { .compatible = "nvidia,tegra234-nvdec", },
{ /* sentinel */ }
};
diff --git a/drivers/gpu/drm/tegra/fb.c b/drivers/gpu/drm/tegra/fb.c
index 9291209..a900300 100644
--- a/drivers/gpu/drm/tegra/fb.c
+++ b/drivers/gpu/drm/tegra/fb.c
@@ -206,6 +206,8 @@ static int tegra_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
static const struct fb_ops tegra_fb_ops = {
.owner = THIS_MODULE,
DRM_FB_HELPER_DEFAULT_OPS,
+ .fb_read = drm_fb_helper_sys_read,
+ .fb_write = drm_fb_helper_sys_write,
.fb_fillrect = drm_fb_helper_sys_fillrect,
.fb_copyarea = drm_fb_helper_sys_copyarea,
.fb_imageblit = drm_fb_helper_sys_imageblit,
@@ -243,7 +245,7 @@ static int tegra_fbdev_probe(struct drm_fb_helper *helper,
if (IS_ERR(bo))
return PTR_ERR(bo);
- info = drm_fb_helper_alloc_fbi(helper);
+ info = drm_fb_helper_alloc_info(helper);
if (IS_ERR(info)) {
dev_err(drm->dev, "failed to allocate framebuffer info\n");
drm_gem_object_put(&bo->gem);
@@ -261,7 +263,7 @@ static int tegra_fbdev_probe(struct drm_fb_helper *helper,
fb = fbdev->fb;
helper->fb = fb;
- helper->fbdev = info;
+ helper->info = info;
info->fbops = &tegra_fb_ops;
@@ -280,7 +282,6 @@ static int tegra_fbdev_probe(struct drm_fb_helper *helper,
}
}
- drm->mode_config.fb_base = (resource_size_t)bo->iova;
info->screen_base = (void __iomem *)bo->vaddr + offset;
info->screen_size = size;
info->fix.smem_start = (unsigned long)(bo->iova + offset);
@@ -348,7 +349,7 @@ static int tegra_fbdev_init(struct tegra_fbdev *fbdev,
static void tegra_fbdev_exit(struct tegra_fbdev *fbdev)
{
- drm_fb_helper_unregister_fbi(&fbdev->base);
+ drm_fb_helper_unregister_info(&fbdev->base);
if (fbdev->fb) {
struct tegra_bo *bo = tegra_fb_get_plane(fbdev->fb, 0);
diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
index 8199109..979e7bc 100644
--- a/drivers/gpu/drm/tegra/gem.c
+++ b/drivers/gpu/drm/tegra/gem.c
@@ -84,7 +84,7 @@ static struct host1x_bo_mapping *tegra_bo_pin(struct device *dev, struct host1x_
goto free;
}
- map->sgt = dma_buf_map_attachment(map->attach, direction);
+ map->sgt = dma_buf_map_attachment_unlocked(map->attach, direction);
if (IS_ERR(map->sgt)) {
dma_buf_detach(buf, map->attach);
err = PTR_ERR(map->sgt);
@@ -160,7 +160,8 @@ static struct host1x_bo_mapping *tegra_bo_pin(struct device *dev, struct host1x_
static void tegra_bo_unpin(struct host1x_bo_mapping *map)
{
if (map->attach) {
- dma_buf_unmap_attachment(map->attach, map->sgt, map->direction);
+ dma_buf_unmap_attachment_unlocked(map->attach, map->sgt,
+ map->direction);
dma_buf_detach(map->attach->dmabuf, map->attach);
} else {
dma_unmap_sgtable(map->dev, map->sgt, map->direction, 0);
@@ -181,7 +182,7 @@ static void *tegra_bo_mmap(struct host1x_bo *bo)
if (obj->vaddr) {
return obj->vaddr;
} else if (obj->gem.import_attach) {
- ret = dma_buf_vmap(obj->gem.import_attach->dmabuf, &map);
+ ret = dma_buf_vmap_unlocked(obj->gem.import_attach->dmabuf, &map);
return ret ? NULL : map.vaddr;
} else {
return vmap(obj->pages, obj->num_pages, VM_MAP,
@@ -197,7 +198,7 @@ static void tegra_bo_munmap(struct host1x_bo *bo, void *addr)
if (obj->vaddr)
return;
else if (obj->gem.import_attach)
- dma_buf_vunmap(obj->gem.import_attach->dmabuf, &map);
+ dma_buf_vunmap_unlocked(obj->gem.import_attach->dmabuf, &map);
else
vunmap(addr);
}
@@ -461,7 +462,7 @@ static struct tegra_bo *tegra_bo_import(struct drm_device *drm,
get_dma_buf(buf);
- bo->sgt = dma_buf_map_attachment(attach, DMA_TO_DEVICE);
+ bo->sgt = dma_buf_map_attachment_unlocked(attach, DMA_TO_DEVICE);
if (IS_ERR(bo->sgt)) {
err = PTR_ERR(bo->sgt);
goto detach;
@@ -479,7 +480,7 @@ static struct tegra_bo *tegra_bo_import(struct drm_device *drm,
detach:
if (!IS_ERR_OR_NULL(bo->sgt))
- dma_buf_unmap_attachment(attach, bo->sgt, DMA_TO_DEVICE);
+ dma_buf_unmap_attachment_unlocked(attach, bo->sgt, DMA_TO_DEVICE);
dma_buf_detach(buf, attach);
dma_buf_put(buf);
@@ -508,8 +509,8 @@ void tegra_bo_free_object(struct drm_gem_object *gem)
tegra_bo_iommu_unmap(tegra, bo);
if (gem->import_attach) {
- dma_buf_unmap_attachment(gem->import_attach, bo->sgt,
- DMA_TO_DEVICE);
+ dma_buf_unmap_attachment_unlocked(gem->import_attach, bo->sgt,
+ DMA_TO_DEVICE);
drm_prime_gem_destroy(gem, NULL);
} else {
tegra_bo_free(gem->dev, bo);
@@ -693,6 +694,8 @@ static int tegra_gem_prime_mmap(struct dma_buf *buf, struct vm_area_struct *vma)
struct drm_gem_object *gem = buf->priv;
int err;
+ dma_resv_assert_held(buf->resv);
+
err = drm_gem_mmap_obj(gem, gem->size, vma);
if (err < 0)
return err;
diff --git a/drivers/gpu/drm/tegra/hdmi.c b/drivers/gpu/drm/tegra/hdmi.c
index bf24076..40ec3e6 100644
--- a/drivers/gpu/drm/tegra/hdmi.c
+++ b/drivers/gpu/drm/tegra/hdmi.c
@@ -867,14 +867,7 @@ static int tegra_hdmi_reconfigure_audio(struct tegra_hdmi *hdmi)
static bool tegra_output_is_hdmi(struct tegra_output *output)
{
- struct edid *edid;
-
- if (!output->connector.edid_blob_ptr)
- return false;
-
- edid = (struct edid *)output->connector.edid_blob_ptr->data;
-
- return drm_detect_hdmi_monitor(edid);
+ return output->connector.display_info.is_hdmi;
}
static enum drm_connector_status
diff --git a/drivers/gpu/drm/tegra/nvdec.c b/drivers/gpu/drm/tegra/nvdec.c
index 276fe04..10fd215 100644
--- a/drivers/gpu/drm/tegra/nvdec.c
+++ b/drivers/gpu/drm/tegra/nvdec.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
- * Copyright (c) 2015-2021, NVIDIA Corporation.
+ * Copyright (c) 2015-2022, NVIDIA Corporation.
*/
#include <linux/clk.h>
@@ -8,6 +8,7 @@
#include <linux/dma-mapping.h>
#include <linux/host1x.h>
#include <linux/iommu.h>
+#include <linux/iopoll.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_device.h>
@@ -16,18 +17,22 @@
#include <linux/pm_runtime.h>
#include <linux/reset.h>
-#include <soc/tegra/pmc.h>
+#include <soc/tegra/mc.h>
#include "drm.h"
#include "falcon.h"
+#include "riscv.h"
#include "vic.h"
+#define NVDEC_FALCON_DEBUGINFO 0x1094
#define NVDEC_TFBIF_TRANSCFG 0x2c44
struct nvdec_config {
const char *firmware;
unsigned int version;
bool supports_sid;
+ bool has_riscv;
+ bool has_extra_clocks;
};
struct nvdec {
@@ -37,10 +42,16 @@ struct nvdec {
struct tegra_drm_client client;
struct host1x_channel *channel;
struct device *dev;
- struct clk *clk;
+ struct clk_bulk_data clks[3];
+ unsigned int num_clks;
+ struct reset_control *reset;
/* Platform configuration */
const struct nvdec_config *config;
+
+ /* RISC-V specific data */
+ struct tegra_drm_riscv riscv;
+ phys_addr_t carveout_base;
};
static inline struct nvdec *to_nvdec(struct tegra_drm_client *client)
@@ -54,7 +65,7 @@ static inline void nvdec_writel(struct nvdec *nvdec, u32 value,
writel(value, nvdec->regs + offset);
}
-static int nvdec_boot(struct nvdec *nvdec)
+static int nvdec_boot_falcon(struct nvdec *nvdec)
{
#ifdef CONFIG_IOMMU_API
struct iommu_fwspec *spec = dev_iommu_fwspec_get(nvdec->dev);
@@ -90,6 +101,64 @@ static int nvdec_boot(struct nvdec *nvdec)
return 0;
}
+static int nvdec_wait_debuginfo(struct nvdec *nvdec, const char *phase)
+{
+ int err;
+ u32 val;
+
+ err = readl_poll_timeout(nvdec->regs + NVDEC_FALCON_DEBUGINFO, val, val == 0x0, 10, 100000);
+ if (err) {
+ dev_err(nvdec->dev, "failed to boot %s, debuginfo=0x%x\n", phase, val);
+ return err;
+ }
+
+ return 0;
+}
+
+static int nvdec_boot_riscv(struct nvdec *nvdec)
+{
+ int err;
+
+ err = reset_control_acquire(nvdec->reset);
+ if (err)
+ return err;
+
+ nvdec_writel(nvdec, 0xabcd1234, NVDEC_FALCON_DEBUGINFO);
+
+ err = tegra_drm_riscv_boot_bootrom(&nvdec->riscv, nvdec->carveout_base, 1,
+ &nvdec->riscv.bl_desc);
+ if (err) {
+ dev_err(nvdec->dev, "failed to execute bootloader\n");
+ goto release_reset;
+ }
+
+ err = nvdec_wait_debuginfo(nvdec, "bootloader");
+ if (err)
+ goto release_reset;
+
+ err = reset_control_reset(nvdec->reset);
+ if (err)
+ goto release_reset;
+
+ nvdec_writel(nvdec, 0xabcd1234, NVDEC_FALCON_DEBUGINFO);
+
+ err = tegra_drm_riscv_boot_bootrom(&nvdec->riscv, nvdec->carveout_base, 1,
+ &nvdec->riscv.os_desc);
+ if (err) {
+ dev_err(nvdec->dev, "failed to execute firmware\n");
+ goto release_reset;
+ }
+
+ err = nvdec_wait_debuginfo(nvdec, "firmware");
+ if (err)
+ goto release_reset;
+
+release_reset:
+ reset_control_release(nvdec->reset);
+
+ return err;
+}
+
static int nvdec_init(struct host1x_client *client)
{
struct tegra_drm_client *drm = host1x_to_drm_client(client);
@@ -189,7 +258,7 @@ static const struct host1x_client_ops nvdec_client_ops = {
.exit = nvdec_exit,
};
-static int nvdec_load_firmware(struct nvdec *nvdec)
+static int nvdec_load_falcon_firmware(struct nvdec *nvdec)
{
struct host1x_client *client = &nvdec->client.base;
struct tegra_drm *tegra = nvdec->client.drm;
@@ -252,30 +321,35 @@ static int nvdec_load_firmware(struct nvdec *nvdec)
return err;
}
-
static __maybe_unused int nvdec_runtime_resume(struct device *dev)
{
struct nvdec *nvdec = dev_get_drvdata(dev);
int err;
- err = clk_prepare_enable(nvdec->clk);
+ err = clk_bulk_prepare_enable(nvdec->num_clks, nvdec->clks);
if (err < 0)
return err;
usleep_range(10, 20);
- err = nvdec_load_firmware(nvdec);
- if (err < 0)
- goto disable;
+ if (nvdec->config->has_riscv) {
+ err = nvdec_boot_riscv(nvdec);
+ if (err < 0)
+ goto disable;
+ } else {
+ err = nvdec_load_falcon_firmware(nvdec);
+ if (err < 0)
+ goto disable;
- err = nvdec_boot(nvdec);
- if (err < 0)
- goto disable;
+ err = nvdec_boot_falcon(nvdec);
+ if (err < 0)
+ goto disable;
+ }
return 0;
disable:
- clk_disable_unprepare(nvdec->clk);
+ clk_bulk_disable_unprepare(nvdec->num_clks, nvdec->clks);
return err;
}
@@ -285,7 +359,7 @@ static __maybe_unused int nvdec_runtime_suspend(struct device *dev)
host1x_channel_stop(nvdec->channel);
- clk_disable_unprepare(nvdec->clk);
+ clk_bulk_disable_unprepare(nvdec->num_clks, nvdec->clks);
return 0;
}
@@ -346,10 +420,18 @@ static const struct nvdec_config nvdec_t194_config = {
.supports_sid = true,
};
+static const struct nvdec_config nvdec_t234_config = {
+ .version = 0x23,
+ .supports_sid = true,
+ .has_riscv = true,
+ .has_extra_clocks = true,
+};
+
static const struct of_device_id tegra_nvdec_of_match[] = {
{ .compatible = "nvidia,tegra210-nvdec", .data = &nvdec_t210_config },
{ .compatible = "nvidia,tegra186-nvdec", .data = &nvdec_t186_config },
{ .compatible = "nvidia,tegra194-nvdec", .data = &nvdec_t194_config },
+ { .compatible = "nvidia,tegra234-nvdec", .data = &nvdec_t234_config },
{ },
};
MODULE_DEVICE_TABLE(of, tegra_nvdec_of_match);
@@ -383,13 +465,22 @@ static int nvdec_probe(struct platform_device *pdev)
if (IS_ERR(nvdec->regs))
return PTR_ERR(nvdec->regs);
- nvdec->clk = devm_clk_get(dev, NULL);
- if (IS_ERR(nvdec->clk)) {
- dev_err(&pdev->dev, "failed to get clock\n");
- return PTR_ERR(nvdec->clk);
+ nvdec->clks[0].id = "nvdec";
+ nvdec->num_clks = 1;
+
+ if (nvdec->config->has_extra_clocks) {
+ nvdec->num_clks = 3;
+ nvdec->clks[1].id = "fuse";
+ nvdec->clks[2].id = "tsec_pka";
}
- err = clk_set_rate(nvdec->clk, ULONG_MAX);
+ err = devm_clk_bulk_get(dev, nvdec->num_clks, nvdec->clks);
+ if (err) {
+ dev_err(&pdev->dev, "failed to get clock(s)\n");
+ return err;
+ }
+
+ err = clk_set_rate(nvdec->clks[0].clk, ULONG_MAX);
if (err < 0) {
dev_err(&pdev->dev, "failed to set clock rate\n");
return err;
@@ -399,12 +490,42 @@ static int nvdec_probe(struct platform_device *pdev)
if (err < 0)
host_class = HOST1X_CLASS_NVDEC;
- nvdec->falcon.dev = dev;
- nvdec->falcon.regs = nvdec->regs;
+ if (nvdec->config->has_riscv) {
+ struct tegra_mc *mc;
- err = falcon_init(&nvdec->falcon);
- if (err < 0)
- return err;
+ mc = devm_tegra_memory_controller_get(dev);
+ if (IS_ERR(mc)) {
+ dev_err_probe(dev, PTR_ERR(mc),
+ "failed to get memory controller handle\n");
+ return PTR_ERR(mc);
+ }
+
+ err = tegra_mc_get_carveout_info(mc, 1, &nvdec->carveout_base, NULL);
+ if (err) {
+ dev_err(dev, "failed to get carveout info: %d\n", err);
+ return err;
+ }
+
+ nvdec->reset = devm_reset_control_get_exclusive_released(dev, "nvdec");
+ if (IS_ERR(nvdec->reset)) {
+ dev_err_probe(dev, PTR_ERR(nvdec->reset), "failed to get reset\n");
+ return PTR_ERR(nvdec->reset);
+ }
+
+ nvdec->riscv.dev = dev;
+ nvdec->riscv.regs = nvdec->regs;
+
+ err = tegra_drm_riscv_read_descriptors(&nvdec->riscv);
+ if (err < 0)
+ return err;
+ } else {
+ nvdec->falcon.dev = dev;
+ nvdec->falcon.regs = nvdec->regs;
+
+ err = falcon_init(&nvdec->falcon);
+ if (err < 0)
+ return err;
+ }
platform_set_drvdata(pdev, nvdec);
diff --git a/drivers/gpu/drm/tegra/output.c b/drivers/gpu/drm/tegra/output.c
index 47d26b5..a8925dc 100644
--- a/drivers/gpu/drm/tegra/output.c
+++ b/drivers/gpu/drm/tegra/output.c
@@ -133,11 +133,11 @@ int tegra_output_probe(struct tegra_output *output)
}
}
- output->hpd_gpio = devm_gpiod_get_from_of_node(output->dev,
- output->of_node,
- "nvidia,hpd-gpio", 0,
- GPIOD_IN,
- "HDMI hotplug detect");
+ output->hpd_gpio = devm_fwnode_gpiod_get(output->dev,
+ of_fwnode_handle(output->of_node),
+ "nvidia,hpd",
+ GPIOD_IN,
+ "HDMI hotplug detect");
if (IS_ERR(output->hpd_gpio)) {
if (PTR_ERR(output->hpd_gpio) != -ENOENT)
return PTR_ERR(output->hpd_gpio);
diff --git a/drivers/gpu/drm/tegra/riscv.c b/drivers/gpu/drm/tegra/riscv.c
new file mode 100644
index 0000000..6580416
--- /dev/null
+++ b/drivers/gpu/drm/tegra/riscv.c
@@ -0,0 +1,106 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022, NVIDIA Corporation.
+ */
+
+#include <linux/dev_printk.h>
+#include <linux/device.h>
+#include <linux/iopoll.h>
+#include <linux/of.h>
+
+#include "riscv.h"
+
+#define RISCV_CPUCTL 0x4388
+#define RISCV_CPUCTL_STARTCPU_TRUE (1 << 0)
+#define RISCV_BR_RETCODE 0x465c
+#define RISCV_BR_RETCODE_RESULT_V(x) ((x) & 0x3)
+#define RISCV_BR_RETCODE_RESULT_PASS_V 3
+#define RISCV_BCR_CTRL 0x4668
+#define RISCV_BCR_CTRL_CORE_SELECT_RISCV (1 << 4)
+#define RISCV_BCR_DMACFG 0x466c
+#define RISCV_BCR_DMACFG_TARGET_LOCAL_FB (0 << 0)
+#define RISCV_BCR_DMACFG_LOCK_LOCKED (1 << 31)
+#define RISCV_BCR_DMAADDR_PKCPARAM_LO 0x4670
+#define RISCV_BCR_DMAADDR_PKCPARAM_HI 0x4674
+#define RISCV_BCR_DMAADDR_FMCCODE_LO 0x4678
+#define RISCV_BCR_DMAADDR_FMCCODE_HI 0x467c
+#define RISCV_BCR_DMAADDR_FMCDATA_LO 0x4680
+#define RISCV_BCR_DMAADDR_FMCDATA_HI 0x4684
+#define RISCV_BCR_DMACFG_SEC 0x4694
+#define RISCV_BCR_DMACFG_SEC_GSCID(v) ((v) << 16)
+
+static void riscv_writel(struct tegra_drm_riscv *riscv, u32 value, u32 offset)
+{
+ writel(value, riscv->regs + offset);
+}
+
+int tegra_drm_riscv_read_descriptors(struct tegra_drm_riscv *riscv)
+{
+ struct tegra_drm_riscv_descriptor *bl = &riscv->bl_desc;
+ struct tegra_drm_riscv_descriptor *os = &riscv->os_desc;
+ const struct device_node *np = riscv->dev->of_node;
+ int err;
+
+#define READ_PROP(name, location) \
+ err = of_property_read_u32(np, name, location); \
+ if (err) { \
+ dev_err(riscv->dev, "failed to read " name ": %d\n", err); \
+ return err; \
+ }
+
+ READ_PROP("nvidia,bl-manifest-offset", &bl->manifest_offset);
+ READ_PROP("nvidia,bl-code-offset", &bl->code_offset);
+ READ_PROP("nvidia,bl-data-offset", &bl->data_offset);
+ READ_PROP("nvidia,os-manifest-offset", &os->manifest_offset);
+ READ_PROP("nvidia,os-code-offset", &os->code_offset);
+ READ_PROP("nvidia,os-data-offset", &os->data_offset);
+#undef READ_PROP
+
+ if (bl->manifest_offset == 0 && bl->code_offset == 0 &&
+ bl->data_offset == 0 && os->manifest_offset == 0 &&
+ os->code_offset == 0 && os->data_offset == 0) {
+ dev_err(riscv->dev, "descriptors not available\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int tegra_drm_riscv_boot_bootrom(struct tegra_drm_riscv *riscv, phys_addr_t image_address,
+ u32 gscid, const struct tegra_drm_riscv_descriptor *desc)
+{
+ phys_addr_t addr;
+ int err;
+ u32 val;
+
+ riscv_writel(riscv, RISCV_BCR_CTRL_CORE_SELECT_RISCV, RISCV_BCR_CTRL);
+
+ addr = image_address + desc->manifest_offset;
+ riscv_writel(riscv, lower_32_bits(addr >> 8), RISCV_BCR_DMAADDR_PKCPARAM_LO);
+ riscv_writel(riscv, upper_32_bits(addr >> 8), RISCV_BCR_DMAADDR_PKCPARAM_HI);
+
+ addr = image_address + desc->code_offset;
+ riscv_writel(riscv, lower_32_bits(addr >> 8), RISCV_BCR_DMAADDR_FMCCODE_LO);
+ riscv_writel(riscv, upper_32_bits(addr >> 8), RISCV_BCR_DMAADDR_FMCCODE_HI);
+
+ addr = image_address + desc->data_offset;
+ riscv_writel(riscv, lower_32_bits(addr >> 8), RISCV_BCR_DMAADDR_FMCDATA_LO);
+ riscv_writel(riscv, upper_32_bits(addr >> 8), RISCV_BCR_DMAADDR_FMCDATA_HI);
+
+ riscv_writel(riscv, RISCV_BCR_DMACFG_SEC_GSCID(gscid), RISCV_BCR_DMACFG_SEC);
+ riscv_writel(riscv,
+ RISCV_BCR_DMACFG_TARGET_LOCAL_FB | RISCV_BCR_DMACFG_LOCK_LOCKED, RISCV_BCR_DMACFG);
+
+ riscv_writel(riscv, RISCV_CPUCTL_STARTCPU_TRUE, RISCV_CPUCTL);
+
+ err = readl_poll_timeout(
+ riscv->regs + RISCV_BR_RETCODE, val,
+ RISCV_BR_RETCODE_RESULT_V(val) == RISCV_BR_RETCODE_RESULT_PASS_V,
+ 10, 100000);
+ if (err) {
+ dev_err(riscv->dev, "error during bootrom execution. BR_RETCODE=%d\n", val);
+ return err;
+ }
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/tegra/riscv.h b/drivers/gpu/drm/tegra/riscv.h
new file mode 100644
index 0000000..bbeb2db
--- /dev/null
+++ b/drivers/gpu/drm/tegra/riscv.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022, NVIDIA Corporation.
+ */
+
+#ifndef DRM_TEGRA_RISCV_H
+#define DRM_TEGRA_RISCV_H
+
+struct tegra_drm_riscv_descriptor {
+ u32 manifest_offset;
+ u32 code_offset;
+ u32 code_size;
+ u32 data_offset;
+ u32 data_size;
+};
+
+struct tegra_drm_riscv {
+ /* User initializes */
+ struct device *dev;
+ void __iomem *regs;
+
+ struct tegra_drm_riscv_descriptor bl_desc;
+ struct tegra_drm_riscv_descriptor os_desc;
+};
+
+int tegra_drm_riscv_read_descriptors(struct tegra_drm_riscv *riscv);
+int tegra_drm_riscv_boot_bootrom(struct tegra_drm_riscv *riscv, phys_addr_t image_address,
+ u32 gscid, const struct tegra_drm_riscv_descriptor *desc);
+
+#endif
diff --git a/drivers/gpu/drm/tegra/submit.c b/drivers/gpu/drm/tegra/submit.c
index b24738bd..066f885 100644
--- a/drivers/gpu/drm/tegra/submit.c
+++ b/drivers/gpu/drm/tegra/submit.c
@@ -133,7 +133,7 @@ static void gather_bo_munmap(struct host1x_bo *host_bo, void *addr)
{
}
-const struct host1x_bo_ops gather_bo_ops = {
+static const struct host1x_bo_ops gather_bo_ops = {
.get = gather_bo_get,
.put = gather_bo_put,
.pin = gather_bo_pin,
@@ -169,14 +169,9 @@ static void *alloc_copy_user_array(void __user *from, size_t count, size_t size)
if (copy_len > 0x4000)
return ERR_PTR(-E2BIG);
- data = kvmalloc(copy_len, GFP_KERNEL);
- if (!data)
- return ERR_PTR(-ENOMEM);
-
- if (copy_from_user(data, from, copy_len)) {
- kvfree(data);
- return ERR_PTR(-EFAULT);
- }
+ data = vmemdup_user(from, copy_len);
+ if (IS_ERR(data))
+ return ERR_CAST(data);
return data;
}
diff --git a/drivers/gpu/drm/tegra/uapi.c b/drivers/gpu/drm/tegra/uapi.c
index a98239c..5adab6b 100644
--- a/drivers/gpu/drm/tegra/uapi.c
+++ b/drivers/gpu/drm/tegra/uapi.c
@@ -116,7 +116,7 @@ int tegra_drm_ioctl_channel_open(struct drm_device *drm, void *data, struct drm_
if (supported)
context->memory_context = host1x_memory_context_alloc(
- host, get_task_pid(current, PIDTYPE_TGID));
+ host, client->base.dev, get_task_pid(current, PIDTYPE_TGID));
if (IS_ERR(context->memory_context)) {
if (PTR_ERR(context->memory_context) != -EOPNOTSUPP) {
diff --git a/drivers/gpu/drm/tests/Makefile b/drivers/gpu/drm/tests/Makefile
index 91b70f7..b29ef10 100644
--- a/drivers/gpu/drm/tests/Makefile
+++ b/drivers/gpu/drm/tests/Makefile
@@ -1,5 +1,14 @@
# SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_DRM_KUNIT_TEST) += drm_format_helper_test.o drm_damage_helper_test.o \
- drm_cmdline_parser_test.o drm_rect_test.o drm_format_test.o drm_plane_helper_test.o \
- drm_dp_mst_helper_test.o drm_framebuffer_test.o drm_buddy_test.o drm_mm_test.o
+obj-$(CONFIG_DRM_KUNIT_TEST) += \
+ drm_buddy_test.o \
+ drm_cmdline_parser_test.o \
+ drm_damage_helper_test.o \
+ drm_dp_mst_helper_test.o \
+ drm_format_helper_test.o \
+ drm_format_test.o \
+ drm_framebuffer_test.o \
+ drm_kunit_helpers.o \
+ drm_mm_test.o \
+ drm_plane_helper_test.o \
+ drm_rect_test.o
diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c
index 62f6958..f8ee714 100644
--- a/drivers/gpu/drm/tests/drm_buddy_test.c
+++ b/drivers/gpu/drm/tests/drm_buddy_test.c
@@ -726,11 +726,13 @@ static void drm_test_buddy_alloc_limit(struct kunit *test)
drm_buddy_fini(&mm);
}
-static int drm_buddy_init_test(struct kunit *test)
+static int drm_buddy_suite_init(struct kunit_suite *suite)
{
while (!random_seed)
random_seed = get_random_u32();
+ kunit_info(suite, "Testing DRM buddy manager, with random_seed=0x%x\n", random_seed);
+
return 0;
}
@@ -746,7 +748,7 @@ static struct kunit_case drm_buddy_tests[] = {
static struct kunit_suite drm_buddy_test_suite = {
.name = "drm_buddy",
- .init = drm_buddy_init_test,
+ .suite_init = drm_buddy_suite_init,
.test_cases = drm_buddy_tests,
};
diff --git a/drivers/gpu/drm/tests/drm_client_modeset_test.c b/drivers/gpu/drm/tests/drm_client_modeset_test.c
new file mode 100644
index 0000000..362a5fb
--- /dev/null
+++ b/drivers/gpu/drm/tests/drm_client_modeset_test.c
@@ -0,0 +1,98 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 Maxime Ripard <mripard@kernel.org>
+ */
+
+#include <kunit/test.h>
+
+#include <drm/drm_connector.h>
+#include <drm/drm_edid.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_modeset_helper_vtables.h>
+#include <drm/drm_probe_helper.h>
+
+#include "drm_kunit_helpers.h"
+
+struct drm_client_modeset_test_priv {
+ struct drm_device *drm;
+ struct drm_connector connector;
+};
+
+static int drm_client_modeset_connector_get_modes(struct drm_connector *connector)
+{
+ return drm_add_modes_noedid(connector, 1920, 1200);
+}
+
+static const struct drm_connector_helper_funcs drm_client_modeset_connector_helper_funcs = {
+ .get_modes = drm_client_modeset_connector_get_modes,
+};
+
+static const struct drm_connector_funcs drm_client_modeset_connector_funcs = {
+};
+
+static int drm_client_modeset_test_init(struct kunit *test)
+{
+ struct drm_client_modeset_test_priv *priv;
+ int ret;
+
+ priv = kunit_kzalloc(test, sizeof(*priv), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, priv);
+
+ test->priv = priv;
+
+ priv->drm = drm_kunit_device_init(test, DRIVER_MODESET, "drm-client-modeset-test");
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, priv->drm);
+
+ ret = drmm_connector_init(priv->drm, &priv->connector,
+ &drm_client_modeset_connector_funcs,
+ DRM_MODE_CONNECTOR_Unknown,
+ NULL);
+ KUNIT_ASSERT_EQ(test, ret, 0);
+
+ drm_connector_helper_add(&priv->connector, &drm_client_modeset_connector_helper_funcs);
+
+ return 0;
+}
+
+static void drm_test_pick_cmdline_res_1920_1080_60(struct kunit *test)
+{
+ struct drm_client_modeset_test_priv *priv = test->priv;
+ struct drm_device *drm = priv->drm;
+ struct drm_connector *connector = &priv->connector;
+ struct drm_cmdline_mode *cmdline_mode = &connector->cmdline_mode;
+ struct drm_display_mode *expected_mode, *mode;
+ const char *cmdline = "1920x1080@60";
+ int ret;
+
+ expected_mode = drm_mode_find_dmt(priv->drm, 1920, 1080, 60, false);
+ KUNIT_ASSERT_NOT_NULL(test, expected_mode);
+
+ KUNIT_ASSERT_TRUE(test,
+ drm_mode_parse_command_line_for_connector(cmdline,
+ connector,
+ cmdline_mode));
+
+ mutex_lock(&drm->mode_config.mutex);
+ ret = drm_helper_probe_single_connector_modes(connector, 1920, 1080);
+ mutex_unlock(&drm->mode_config.mutex);
+ KUNIT_ASSERT_GT(test, ret, 0);
+
+ mode = drm_connector_pick_cmdline_mode(connector);
+ KUNIT_ASSERT_NOT_NULL(test, mode);
+
+ KUNIT_EXPECT_TRUE(test, drm_mode_equal(expected_mode, mode));
+}
+
+static struct kunit_case drm_test_pick_cmdline_tests[] = {
+ KUNIT_CASE(drm_test_pick_cmdline_res_1920_1080_60),
+ {}
+};
+
+static struct kunit_suite drm_test_pick_cmdline_test_suite = {
+ .name = "drm_test_pick_cmdline",
+ .init = drm_client_modeset_test_init,
+ .test_cases = drm_test_pick_cmdline_tests
+};
+
+kunit_test_suite(drm_test_pick_cmdline_test_suite);
diff --git a/drivers/gpu/drm/tests/drm_dp_mst_helper_test.c b/drivers/gpu/drm/tests/drm_dp_mst_helper_test.c
index 65c9d22..545beea 100644
--- a/drivers/gpu/drm/tests/drm_dp_mst_helper_test.c
+++ b/drivers/gpu/drm/tests/drm_dp_mst_helper_test.c
@@ -5,44 +5,280 @@
* Copyright (c) 2022 Maíra Canal <mairacanal@riseup.net>
*/
-#define PREFIX_STR "[drm_dp_mst_helper]"
-
#include <kunit/test.h>
-#include <linux/random.h>
-
#include <drm/display/drm_dp_mst_helper.h>
#include <drm/drm_print.h>
#include "../display/drm_dp_mst_topology_internal.h"
+struct drm_dp_mst_calc_pbn_mode_test {
+ const int clock;
+ const int bpp;
+ const bool dsc;
+ const int expected;
+};
+
+static const struct drm_dp_mst_calc_pbn_mode_test drm_dp_mst_calc_pbn_mode_cases[] = {
+ {
+ .clock = 154000,
+ .bpp = 30,
+ .dsc = false,
+ .expected = 689
+ },
+ {
+ .clock = 234000,
+ .bpp = 30,
+ .dsc = false,
+ .expected = 1047
+ },
+ {
+ .clock = 297000,
+ .bpp = 24,
+ .dsc = false,
+ .expected = 1063
+ },
+ {
+ .clock = 332880,
+ .bpp = 24,
+ .dsc = true,
+ .expected = 50
+ },
+ {
+ .clock = 324540,
+ .bpp = 24,
+ .dsc = true,
+ .expected = 49
+ },
+};
+
static void drm_test_dp_mst_calc_pbn_mode(struct kunit *test)
{
- int pbn, i;
- const struct {
- int rate;
- int bpp;
- int expected;
- bool dsc;
- } test_params[] = {
- { 154000, 30, 689, false },
- { 234000, 30, 1047, false },
- { 297000, 24, 1063, false },
- { 332880, 24, 50, true },
- { 324540, 24, 49, true },
- };
+ const struct drm_dp_mst_calc_pbn_mode_test *params = test->param_value;
- for (i = 0; i < ARRAY_SIZE(test_params); i++) {
- pbn = drm_dp_calc_pbn_mode(test_params[i].rate,
- test_params[i].bpp,
- test_params[i].dsc);
- KUNIT_EXPECT_EQ_MSG(test, pbn, test_params[i].expected,
- "Expected PBN %d for clock %d bpp %d, got %d\n",
- test_params[i].expected, test_params[i].rate,
- test_params[i].bpp, pbn);
- }
+ KUNIT_EXPECT_EQ(test, drm_dp_calc_pbn_mode(params->clock, params->bpp, params->dsc),
+ params->expected);
}
+static void dp_mst_calc_pbn_mode_desc(const struct drm_dp_mst_calc_pbn_mode_test *t, char *desc)
+{
+ sprintf(desc, "Clock %d BPP %d DSC %s", t->clock, t->bpp, t->dsc ? "enabled" : "disabled");
+}
+
+KUNIT_ARRAY_PARAM(drm_dp_mst_calc_pbn_mode, drm_dp_mst_calc_pbn_mode_cases,
+ dp_mst_calc_pbn_mode_desc);
+
+static u8 data[] = { 0xff, 0x00, 0xdd };
+
+struct drm_dp_mst_sideband_msg_req_test {
+ const char *desc;
+ const struct drm_dp_sideband_msg_req_body in;
+};
+
+static const struct drm_dp_mst_sideband_msg_req_test drm_dp_mst_sideband_msg_req_cases[] = {
+ {
+ .desc = "DP_ENUM_PATH_RESOURCES with port number",
+ .in = {
+ .req_type = DP_ENUM_PATH_RESOURCES,
+ .u.port_num.port_number = 5,
+ },
+ },
+ {
+ .desc = "DP_POWER_UP_PHY with port number",
+ .in = {
+ .req_type = DP_POWER_UP_PHY,
+ .u.port_num.port_number = 5,
+ },
+ },
+ {
+ .desc = "DP_POWER_DOWN_PHY with port number",
+ .in = {
+ .req_type = DP_POWER_DOWN_PHY,
+ .u.port_num.port_number = 5,
+ },
+ },
+ {
+ .desc = "DP_ALLOCATE_PAYLOAD with SDP stream sinks",
+ .in = {
+ .req_type = DP_ALLOCATE_PAYLOAD,
+ .u.allocate_payload.number_sdp_streams = 3,
+ .u.allocate_payload.sdp_stream_sink = { 1, 2, 3 },
+ },
+ },
+ {
+ .desc = "DP_ALLOCATE_PAYLOAD with port number",
+ .in = {
+ .req_type = DP_ALLOCATE_PAYLOAD,
+ .u.allocate_payload.port_number = 0xf,
+ },
+ },
+ {
+ .desc = "DP_ALLOCATE_PAYLOAD with VCPI",
+ .in = {
+ .req_type = DP_ALLOCATE_PAYLOAD,
+ .u.allocate_payload.vcpi = 0x7f,
+ },
+ },
+ {
+ .desc = "DP_ALLOCATE_PAYLOAD with PBN",
+ .in = {
+ .req_type = DP_ALLOCATE_PAYLOAD,
+ .u.allocate_payload.pbn = U16_MAX,
+ },
+ },
+ {
+ .desc = "DP_QUERY_PAYLOAD with port number",
+ .in = {
+ .req_type = DP_QUERY_PAYLOAD,
+ .u.query_payload.port_number = 0xf,
+ },
+ },
+ {
+ .desc = "DP_QUERY_PAYLOAD with VCPI",
+ .in = {
+ .req_type = DP_QUERY_PAYLOAD,
+ .u.query_payload.vcpi = 0x7f,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_DPCD_READ with port number",
+ .in = {
+ .req_type = DP_REMOTE_DPCD_READ,
+ .u.dpcd_read.port_number = 0xf,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_DPCD_READ with DPCD address",
+ .in = {
+ .req_type = DP_REMOTE_DPCD_READ,
+ .u.dpcd_read.dpcd_address = 0xfedcb,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_DPCD_READ with max number of bytes",
+ .in = {
+ .req_type = DP_REMOTE_DPCD_READ,
+ .u.dpcd_read.num_bytes = U8_MAX,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_DPCD_WRITE with port number",
+ .in = {
+ .req_type = DP_REMOTE_DPCD_WRITE,
+ .u.dpcd_write.port_number = 0xf,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_DPCD_WRITE with DPCD address",
+ .in = {
+ .req_type = DP_REMOTE_DPCD_WRITE,
+ .u.dpcd_write.dpcd_address = 0xfedcb,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_DPCD_WRITE with data array",
+ .in = {
+ .req_type = DP_REMOTE_DPCD_WRITE,
+ .u.dpcd_write.num_bytes = ARRAY_SIZE(data),
+ .u.dpcd_write.bytes = data,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_I2C_READ with port number",
+ .in = {
+ .req_type = DP_REMOTE_I2C_READ,
+ .u.i2c_read.port_number = 0xf,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_I2C_READ with I2C device ID",
+ .in = {
+ .req_type = DP_REMOTE_I2C_READ,
+ .u.i2c_read.read_i2c_device_id = 0x7f,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_I2C_READ with transactions array",
+ .in = {
+ .req_type = DP_REMOTE_I2C_READ,
+ .u.i2c_read.num_transactions = 3,
+ .u.i2c_read.num_bytes_read = ARRAY_SIZE(data) * 3,
+ .u.i2c_read.transactions = {
+ { .bytes = data, .num_bytes = ARRAY_SIZE(data), .i2c_dev_id = 0x7f,
+ .i2c_transaction_delay = 0xf, },
+ { .bytes = data, .num_bytes = ARRAY_SIZE(data), .i2c_dev_id = 0x7e,
+ .i2c_transaction_delay = 0xe, },
+ { .bytes = data, .num_bytes = ARRAY_SIZE(data), .i2c_dev_id = 0x7d,
+ .i2c_transaction_delay = 0xd, },
+ },
+ },
+ },
+ {
+ .desc = "DP_REMOTE_I2C_WRITE with port number",
+ .in = {
+ .req_type = DP_REMOTE_I2C_WRITE,
+ .u.i2c_write.port_number = 0xf,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_I2C_WRITE with I2C device ID",
+ .in = {
+ .req_type = DP_REMOTE_I2C_WRITE,
+ .u.i2c_write.write_i2c_device_id = 0x7f,
+ },
+ },
+ {
+ .desc = "DP_REMOTE_I2C_WRITE with data array",
+ .in = {
+ .req_type = DP_REMOTE_I2C_WRITE,
+ .u.i2c_write.num_bytes = ARRAY_SIZE(data),
+ .u.i2c_write.bytes = data,
+ },
+ },
+ {
+ .desc = "DP_QUERY_STREAM_ENC_STATUS with stream ID",
+ .in = {
+ .req_type = DP_QUERY_STREAM_ENC_STATUS,
+ .u.enc_status.stream_id = 1,
+ },
+ },
+ {
+ .desc = "DP_QUERY_STREAM_ENC_STATUS with client ID",
+ .in = {
+ .req_type = DP_QUERY_STREAM_ENC_STATUS,
+ .u.enc_status.client_id = { 0x4f, 0x7f, 0xb4, 0x00, 0x8c, 0x0d, 0x67 },
+ },
+ },
+ {
+ .desc = "DP_QUERY_STREAM_ENC_STATUS with stream event",
+ .in = {
+ .req_type = DP_QUERY_STREAM_ENC_STATUS,
+ .u.enc_status.stream_event = 3,
+ },
+ },
+ {
+ .desc = "DP_QUERY_STREAM_ENC_STATUS with valid stream event",
+ .in = {
+ .req_type = DP_QUERY_STREAM_ENC_STATUS,
+ .u.enc_status.valid_stream_event = 0,
+ },
+ },
+ {
+ .desc = "DP_QUERY_STREAM_ENC_STATUS with stream behavior",
+ .in = {
+ .req_type = DP_QUERY_STREAM_ENC_STATUS,
+ .u.enc_status.stream_behavior = 3,
+ },
+ },
+ {
+ .desc = "DP_QUERY_STREAM_ENC_STATUS with a valid stream behavior",
+ .in = {
+ .req_type = DP_QUERY_STREAM_ENC_STATUS,
+ .u.enc_status.valid_stream_behavior = 1,
+ }
+ },
+};
+
static bool
sideband_msg_req_equal(const struct drm_dp_sideband_msg_req_body *in,
const struct drm_dp_sideband_msg_req_body *out)
@@ -118,41 +354,41 @@ sideband_msg_req_equal(const struct drm_dp_sideband_msg_req_body *in,
return true;
}
-static bool
-sideband_msg_req_encode_decode(struct drm_dp_sideband_msg_req_body *in)
+static void drm_test_dp_mst_msg_printf(struct drm_printer *p, struct va_format *vaf)
{
+ struct kunit *test = p->arg;
+
+ kunit_err(test, "%pV", vaf);
+}
+
+static void drm_test_dp_mst_sideband_msg_req_decode(struct kunit *test)
+{
+ const struct drm_dp_mst_sideband_msg_req_test *params = test->param_value;
+ const struct drm_dp_sideband_msg_req_body *in = ¶ms->in;
struct drm_dp_sideband_msg_req_body *out;
- struct drm_printer p = drm_err_printer(PREFIX_STR);
struct drm_dp_sideband_msg_tx *txmsg;
- int i, ret;
- bool result = true;
+ struct drm_printer p = {
+ .printfn = drm_test_dp_mst_msg_printf,
+ .arg = test
+ };
+ int i;
- out = kzalloc(sizeof(*out), GFP_KERNEL);
- if (!out)
- return false;
+ out = kunit_kzalloc(test, sizeof(*out), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, out);
- txmsg = kzalloc(sizeof(*txmsg), GFP_KERNEL);
- if (!txmsg) {
- kfree(out);
- return false;
- }
+ txmsg = kunit_kzalloc(test, sizeof(*txmsg), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, txmsg);
drm_dp_encode_sideband_req(in, txmsg);
- ret = drm_dp_decode_sideband_req(txmsg, out);
- if (ret < 0) {
- drm_printf(&p, "Failed to decode sideband request: %d\n",
- ret);
- result = false;
- goto out;
- }
+ KUNIT_EXPECT_GE_MSG(test, drm_dp_decode_sideband_req(txmsg, out), 0,
+ "Failed to decode sideband request");
if (!sideband_msg_req_equal(in, out)) {
- drm_printf(&p, "Encode/decode failed, expected:\n");
+ KUNIT_FAIL(test, "Encode/decode failed");
+ kunit_err(test, "Expected:");
drm_dp_dump_sideband_msg_req_body(in, 1, &p);
- drm_printf(&p, "Got:\n");
+ kunit_err(test, "Got:");
drm_dp_dump_sideband_msg_req_body(out, 1, &p);
- result = false;
- goto out;
}
switch (in->req_type) {
@@ -167,112 +403,21 @@ sideband_msg_req_encode_decode(struct drm_dp_sideband_msg_req_body *in)
kfree(out->u.i2c_write.bytes);
break;
}
-
- /* Clear everything but the req_type for the input */
- memset(&in->u, 0, sizeof(in->u));
-
-out:
- kfree(out);
- kfree(txmsg);
- return result;
}
-static void drm_test_dp_mst_sideband_msg_req_decode(struct kunit *test)
+static void
+drm_dp_mst_sideband_msg_req_desc(const struct drm_dp_mst_sideband_msg_req_test *t, char *desc)
{
- struct drm_dp_sideband_msg_req_body in = { 0 };
- u8 data[] = { 0xff, 0x0, 0xdd };
- int i;
-
- in.req_type = DP_ENUM_PATH_RESOURCES;
- in.u.port_num.port_number = 5;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_POWER_UP_PHY;
- in.u.port_num.port_number = 5;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_POWER_DOWN_PHY;
- in.u.port_num.port_number = 5;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_ALLOCATE_PAYLOAD;
- in.u.allocate_payload.number_sdp_streams = 3;
- for (i = 0; i < in.u.allocate_payload.number_sdp_streams; i++)
- in.u.allocate_payload.sdp_stream_sink[i] = i + 1;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.allocate_payload.port_number = 0xf;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.allocate_payload.vcpi = 0x7f;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.allocate_payload.pbn = U16_MAX;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_QUERY_PAYLOAD;
- in.u.query_payload.port_number = 0xf;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.query_payload.vcpi = 0x7f;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_REMOTE_DPCD_READ;
- in.u.dpcd_read.port_number = 0xf;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.dpcd_read.dpcd_address = 0xfedcb;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.dpcd_read.num_bytes = U8_MAX;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_REMOTE_DPCD_WRITE;
- in.u.dpcd_write.port_number = 0xf;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.dpcd_write.dpcd_address = 0xfedcb;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.dpcd_write.num_bytes = ARRAY_SIZE(data);
- in.u.dpcd_write.bytes = data;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_REMOTE_I2C_READ;
- in.u.i2c_read.port_number = 0xf;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.i2c_read.read_i2c_device_id = 0x7f;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.i2c_read.num_transactions = 3;
- in.u.i2c_read.num_bytes_read = ARRAY_SIZE(data) * 3;
- for (i = 0; i < in.u.i2c_read.num_transactions; i++) {
- in.u.i2c_read.transactions[i].bytes = data;
- in.u.i2c_read.transactions[i].num_bytes = ARRAY_SIZE(data);
- in.u.i2c_read.transactions[i].i2c_dev_id = 0x7f & ~i;
- in.u.i2c_read.transactions[i].i2c_transaction_delay = 0xf & ~i;
- }
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_REMOTE_I2C_WRITE;
- in.u.i2c_write.port_number = 0xf;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.i2c_write.write_i2c_device_id = 0x7f;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.i2c_write.num_bytes = ARRAY_SIZE(data);
- in.u.i2c_write.bytes = data;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
-
- in.req_type = DP_QUERY_STREAM_ENC_STATUS;
- in.u.enc_status.stream_id = 1;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- get_random_bytes(in.u.enc_status.client_id,
- sizeof(in.u.enc_status.client_id));
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.enc_status.stream_event = 3;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.enc_status.valid_stream_event = 0;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.enc_status.stream_behavior = 3;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
- in.u.enc_status.valid_stream_behavior = 1;
- KUNIT_EXPECT_TRUE(test, sideband_msg_req_encode_decode(&in));
+ strcpy(desc, t->desc);
}
+KUNIT_ARRAY_PARAM(drm_dp_mst_sideband_msg_req, drm_dp_mst_sideband_msg_req_cases,
+ drm_dp_mst_sideband_msg_req_desc);
+
static struct kunit_case drm_dp_mst_helper_tests[] = {
- KUNIT_CASE(drm_test_dp_mst_calc_pbn_mode),
- KUNIT_CASE(drm_test_dp_mst_sideband_msg_req_decode),
+ KUNIT_CASE_PARAM(drm_test_dp_mst_calc_pbn_mode, drm_dp_mst_calc_pbn_mode_gen_params),
+ KUNIT_CASE_PARAM(drm_test_dp_mst_sideband_msg_req_decode,
+ drm_dp_mst_sideband_msg_req_gen_params),
{ }
};
diff --git a/drivers/gpu/drm/tests/drm_kunit_helpers.c b/drivers/gpu/drm/tests/drm_kunit_helpers.c
new file mode 100644
index 0000000..f166209
--- /dev/null
+++ b/drivers/gpu/drm/tests/drm_kunit_helpers.c
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+
+#include <kunit/resource.h>
+
+#include <linux/device.h>
+
+#include "drm_kunit_helpers.h"
+
+struct kunit_dev {
+ struct drm_device base;
+};
+
+static const struct drm_mode_config_funcs drm_mode_config_funcs = {
+};
+
+static int dev_init(struct kunit_resource *res, void *ptr)
+{
+ char *name = ptr;
+ struct device *dev;
+
+ dev = root_device_register(name);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
+
+ res->data = dev;
+ return 0;
+}
+
+static void dev_free(struct kunit_resource *res)
+{
+ struct device *dev = res->data;
+
+ root_device_unregister(dev);
+}
+
+struct drm_device *drm_kunit_device_init(struct kunit *test, u32 features, char *name)
+{
+ struct kunit_dev *kdev;
+ struct drm_device *drm;
+ struct drm_driver *driver;
+ struct device *dev;
+ int ret;
+
+ dev = kunit_alloc_resource(test, dev_init, dev_free, GFP_KERNEL, name);
+ if (!dev)
+ return ERR_PTR(-ENOMEM);
+
+ driver = kunit_kzalloc(test, sizeof(*driver), GFP_KERNEL);
+ if (!driver)
+ return ERR_PTR(-ENOMEM);
+
+ driver->driver_features = features;
+ kdev = devm_drm_dev_alloc(dev, driver, struct kunit_dev, base);
+ if (IS_ERR(kdev))
+ return ERR_CAST(kdev);
+
+ drm = &kdev->base;
+ drm->mode_config.funcs = &drm_mode_config_funcs;
+
+ ret = drmm_mode_config_init(drm);
+ if (ret)
+ return ERR_PTR(ret);
+
+ return drm;
+}
+
+MODULE_AUTHOR("Maxime Ripard <maxime@cerno.tech>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/tests/drm_kunit_helpers.h b/drivers/gpu/drm/tests/drm_kunit_helpers.h
new file mode 100644
index 0000000..20ab6ee
--- /dev/null
+++ b/drivers/gpu/drm/tests/drm_kunit_helpers.h
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#ifndef DRM_KUNIT_HELPERS_H_
+#define DRM_KUNIT_HELPERS_H_
+
+struct drm_device;
+struct kunit;
+
+struct drm_device *drm_kunit_device_init(struct kunit *test, u32 features, char *name);
+
+#endif // DRM_KUNIT_HELPERS_H_
diff --git a/drivers/gpu/drm/tests/drm_mm_test.c b/drivers/gpu/drm/tests/drm_mm_test.c
index c4b66ee..89f12d3 100644
--- a/drivers/gpu/drm/tests/drm_mm_test.c
+++ b/drivers/gpu/drm/tests/drm_mm_test.c
@@ -2209,11 +2209,15 @@ static void drm_test_mm_color_evict_range(struct kunit *test)
vfree(nodes);
}
-static int drm_mm_init_test(struct kunit *test)
+static int drm_mm_suite_init(struct kunit_suite *suite)
{
while (!random_seed)
random_seed = get_random_u32();
+ kunit_info(suite,
+ "Testing DRM range manager, with random_seed=0x%x max_iterations=%u max_prime=%u\n",
+ random_seed, max_iterations, max_prime);
+
return 0;
}
@@ -2246,7 +2250,7 @@ static struct kunit_case drm_mm_tests[] = {
static struct kunit_suite drm_mm_test_suite = {
.name = "drm_mm",
- .init = drm_mm_init_test,
+ .suite_init = drm_mm_suite_init,
.test_cases = drm_mm_tests,
};
diff --git a/drivers/gpu/drm/tests/drm_plane_helper_test.c b/drivers/gpu/drm/tests/drm_plane_helper_test.c
index ec71af7..0f39214 100644
--- a/drivers/gpu/drm/tests/drm_plane_helper_test.c
+++ b/drivers/gpu/drm/tests/drm_plane_helper_test.c
@@ -10,225 +10,306 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_modes.h>
+#include <drm/drm_rect.h>
-static void set_src(struct drm_plane_state *plane_state,
- unsigned int src_x, unsigned int src_y,
- unsigned int src_w, unsigned int src_h)
+static const struct drm_crtc_state crtc_state = {
+ .crtc = ZERO_SIZE_PTR,
+ .enable = true,
+ .active = true,
+ .mode = {
+ DRM_MODE("1024x768", 0, 65000, 1024, 1048,
+ 1184, 1344, 0, 768, 771, 777, 806, 0,
+ DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC)
+ },
+};
+
+struct drm_check_plane_state_test {
+ const char *name;
+ const char *msg;
+ struct {
+ unsigned int x;
+ unsigned int y;
+ unsigned int w;
+ unsigned int h;
+ } src, src_expected;
+ struct {
+ int x;
+ int y;
+ unsigned int w;
+ unsigned int h;
+ } crtc, crtc_expected;
+ unsigned int rotation;
+ int min_scale;
+ int max_scale;
+ bool can_position;
+};
+
+static int drm_plane_helper_init(struct kunit *test)
{
- plane_state->src_x = src_x;
- plane_state->src_y = src_y;
- plane_state->src_w = src_w;
- plane_state->src_h = src_h;
+ const struct drm_check_plane_state_test *params = test->param_value;
+ struct drm_plane *plane;
+ struct drm_framebuffer *fb;
+ struct drm_plane_state *mock;
+
+ plane = kunit_kzalloc(test, sizeof(*plane), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, plane);
+
+ fb = kunit_kzalloc(test, sizeof(*fb), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, fb);
+ fb->width = 2048;
+ fb->height = 2048;
+
+ mock = kunit_kzalloc(test, sizeof(*mock), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, mock);
+ mock->plane = plane;
+ mock->crtc = ZERO_SIZE_PTR;
+ mock->fb = fb;
+ mock->rotation = params->rotation;
+ mock->src_x = params->src.x;
+ mock->src_y = params->src.y;
+ mock->src_w = params->src.w;
+ mock->src_h = params->src.h;
+ mock->crtc_x = params->crtc.x;
+ mock->crtc_y = params->crtc.y;
+ mock->crtc_w = params->crtc.w;
+ mock->crtc_h = params->crtc.h;
+
+ test->priv = mock;
+
+ return 0;
}
-static bool check_src_eq(struct drm_plane_state *plane_state,
+static void check_src_eq(struct kunit *test, struct drm_plane_state *plane_state,
unsigned int src_x, unsigned int src_y,
unsigned int src_w, unsigned int src_h)
{
- if (plane_state->src.x1 < 0) {
- pr_err("src x coordinate %x should never be below 0.\n", plane_state->src.x1);
- drm_rect_debug_print("src: ", &plane_state->src, true);
- return false;
- }
- if (plane_state->src.y1 < 0) {
- pr_err("src y coordinate %x should never be below 0.\n", plane_state->src.y1);
- drm_rect_debug_print("src: ", &plane_state->src, true);
- return false;
- }
+ struct drm_rect expected = DRM_RECT_INIT(src_x, src_y, src_w, src_h);
- if (plane_state->src.x1 != src_x ||
- plane_state->src.y1 != src_y ||
- drm_rect_width(&plane_state->src) != src_w ||
- drm_rect_height(&plane_state->src) != src_h) {
- drm_rect_debug_print("src: ", &plane_state->src, true);
- return false;
- }
+ KUNIT_ASSERT_GE_MSG(test, plane_state->src.x1, 0,
+ "src x coordinate %x should never be below 0, src: " DRM_RECT_FP_FMT,
+ plane_state->src.x1, DRM_RECT_FP_ARG(&plane_state->src));
- return true;
+ KUNIT_ASSERT_GE_MSG(test, plane_state->src.y1, 0,
+ "src y coordinate %x should never be below 0, src: " DRM_RECT_FP_FMT,
+ plane_state->src.y1, DRM_RECT_FP_ARG(&plane_state->src));
+
+ KUNIT_EXPECT_TRUE_MSG(test, drm_rect_equals(&plane_state->src, &expected),
+ "dst: " DRM_RECT_FP_FMT ", expected: " DRM_RECT_FP_FMT,
+ DRM_RECT_FP_ARG(&plane_state->src), DRM_RECT_FP_ARG(&expected));
}
-static void set_crtc(struct drm_plane_state *plane_state,
- int crtc_x, int crtc_y,
- unsigned int crtc_w, unsigned int crtc_h)
-{
- plane_state->crtc_x = crtc_x;
- plane_state->crtc_y = crtc_y;
- plane_state->crtc_w = crtc_w;
- plane_state->crtc_h = crtc_h;
-}
-
-static bool check_crtc_eq(struct drm_plane_state *plane_state,
+static void check_crtc_eq(struct kunit *test, struct drm_plane_state *plane_state,
int crtc_x, int crtc_y,
unsigned int crtc_w, unsigned int crtc_h)
{
- if (plane_state->dst.x1 != crtc_x ||
- plane_state->dst.y1 != crtc_y ||
- drm_rect_width(&plane_state->dst) != crtc_w ||
- drm_rect_height(&plane_state->dst) != crtc_h) {
- drm_rect_debug_print("dst: ", &plane_state->dst, false);
+ struct drm_rect expected = DRM_RECT_INIT(crtc_x, crtc_y, crtc_w, crtc_h);
- return false;
- }
-
- return true;
+ KUNIT_EXPECT_TRUE_MSG(test, drm_rect_equals(&plane_state->dst, &expected),
+ "dst: " DRM_RECT_FMT ", expected: " DRM_RECT_FMT,
+ DRM_RECT_ARG(&plane_state->dst), DRM_RECT_ARG(&expected));
}
static void drm_test_check_plane_state(struct kunit *test)
{
- int ret;
+ const struct drm_check_plane_state_test *params = test->param_value;
+ struct drm_plane_state *plane_state = test->priv;
- static const struct drm_crtc_state crtc_state = {
- .crtc = ZERO_SIZE_PTR,
- .enable = true,
- .active = true,
- .mode = {
- DRM_MODE("1024x768", 0, 65000, 1024, 1048, 1184, 1344, 0, 768, 771,
- 777, 806, 0, DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC)
- },
- };
- static struct drm_plane plane = {
- .dev = NULL
- };
- static struct drm_framebuffer fb = {
- .width = 2048,
- .height = 2048
- };
- static struct drm_plane_state plane_state = {
- .plane = &plane,
- .crtc = ZERO_SIZE_PTR,
- .fb = &fb,
- .rotation = DRM_MODE_ROTATE_0
- };
-
- /* Simple clipping, no scaling. */
- set_src(&plane_state, 0, 0, fb.width << 16, fb.height << 16);
- set_crtc(&plane_state, 0, 0, fb.width, fb.height);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- DRM_PLANE_NO_SCALING,
- false, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Simple clipping check should pass\n");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0, 0, 1024 << 16, 768 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 0, 0, 1024, 768));
-
- /* Rotated clipping + reflection, no scaling. */
- plane_state.rotation = DRM_MODE_ROTATE_90 | DRM_MODE_REFLECT_X;
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- DRM_PLANE_NO_SCALING,
- false, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Rotated clipping check should pass\n");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0, 0, 768 << 16, 1024 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 0, 0, 1024, 768));
- plane_state.rotation = DRM_MODE_ROTATE_0;
-
- /* Check whether positioning works correctly. */
- set_src(&plane_state, 0, 0, 1023 << 16, 767 << 16);
- set_crtc(&plane_state, 0, 0, 1023, 767);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- DRM_PLANE_NO_SCALING,
- false, false);
- KUNIT_EXPECT_TRUE_MSG(test, ret,
- "Should not be able to position on the crtc with can_position=false\n");
-
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- DRM_PLANE_NO_SCALING,
- true, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Simple positioning should work\n");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0, 0, 1023 << 16, 767 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 0, 0, 1023, 767));
-
- /* Simple scaling tests. */
- set_src(&plane_state, 0, 0, 512 << 16, 384 << 16);
- set_crtc(&plane_state, 0, 0, 1024, 768);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- 0x8001,
- DRM_PLANE_NO_SCALING,
- false, false);
- KUNIT_EXPECT_TRUE_MSG(test, ret, "Upscaling out of range should fail.\n");
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- 0x8000,
- DRM_PLANE_NO_SCALING,
- false, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Upscaling exactly 2x should work\n");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0, 0, 512 << 16, 384 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 0, 0, 1024, 768));
-
- set_src(&plane_state, 0, 0, 2048 << 16, 1536 << 16);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- 0x1ffff, false, false);
- KUNIT_EXPECT_TRUE_MSG(test, ret, "Downscaling out of range should fail.\n");
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- 0x20000, false, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Should succeed with exact scaling limit\n");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0, 0, 2048 << 16, 1536 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 0, 0, 1024, 768));
-
- /* Testing rounding errors. */
- set_src(&plane_state, 0, 0, 0x40001, 0x40001);
- set_crtc(&plane_state, 1022, 766, 4, 4);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- 0x10001,
- true, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Should succeed by clipping to exact multiple");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0, 0, 2 << 16, 2 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 1022, 766, 2, 2));
-
- set_src(&plane_state, 0x20001, 0x20001, 0x4040001, 0x3040001);
- set_crtc(&plane_state, -2, -2, 1028, 772);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- DRM_PLANE_NO_SCALING,
- 0x10001,
- false, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Should succeed by clipping to exact multiple");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0x40002, 0x40002,
- 1024 << 16, 768 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 0, 0, 1024, 768));
-
- set_src(&plane_state, 0, 0, 0x3ffff, 0x3ffff);
- set_crtc(&plane_state, 1022, 766, 4, 4);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- 0xffff,
- DRM_PLANE_NO_SCALING,
- true, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Should succeed by clipping to exact multiple");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- /* Should not be rounded to 0x20001, which would be upscaling. */
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0, 0, 2 << 16, 2 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 1022, 766, 2, 2));
-
- set_src(&plane_state, 0x1ffff, 0x1ffff, 0x403ffff, 0x303ffff);
- set_crtc(&plane_state, -2, -2, 1028, 772);
- ret = drm_atomic_helper_check_plane_state(&plane_state, &crtc_state,
- 0xffff,
- DRM_PLANE_NO_SCALING,
- false, false);
- KUNIT_EXPECT_FALSE_MSG(test, ret, 0, "Should succeed by clipping to exact multiple");
- KUNIT_EXPECT_TRUE(test, plane_state.visible);
- KUNIT_EXPECT_TRUE(test, check_src_eq(&plane_state, 0x3fffe, 0x3fffe,
- 1024 << 16, 768 << 16));
- KUNIT_EXPECT_TRUE(test, check_crtc_eq(&plane_state, 0, 0, 1024, 768));
+ KUNIT_ASSERT_EQ_MSG(test,
+ drm_atomic_helper_check_plane_state(plane_state, &crtc_state,
+ params->min_scale,
+ params->max_scale,
+ params->can_position, false),
+ 0, params->msg);
+ KUNIT_EXPECT_TRUE(test, plane_state->visible);
+ check_src_eq(test, plane_state, params->src_expected.x, params->src_expected.y,
+ params->src_expected.w, params->src_expected.h);
+ check_crtc_eq(test, plane_state, params->crtc_expected.x, params->crtc_expected.y,
+ params->crtc_expected.w, params->crtc_expected.h);
}
+static void drm_check_plane_state_desc(const struct drm_check_plane_state_test *t,
+ char *desc)
+{
+ sprintf(desc, "%s", t->name);
+}
+
+static const struct drm_check_plane_state_test drm_check_plane_state_tests[] = {
+ {
+ .name = "clipping_simple",
+ .msg = "Simple clipping check should pass",
+ .src = { 0, 0,
+ 2048 << 16,
+ 2048 << 16 },
+ .crtc = { 0, 0, 2048, 2048 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = false,
+ .src_expected = { 0, 0, 1024 << 16, 768 << 16 },
+ .crtc_expected = { 0, 0, 1024, 768 },
+ },
+ {
+ .name = "clipping_rotate_reflect",
+ .msg = "Rotated clipping check should pass",
+ .src = { 0, 0,
+ 2048 << 16,
+ 2048 << 16 },
+ .crtc = { 0, 0, 2048, 2048 },
+ .rotation = DRM_MODE_ROTATE_90 | DRM_MODE_REFLECT_X,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = false,
+ .src_expected = { 0, 0, 768 << 16, 1024 << 16 },
+ .crtc_expected = { 0, 0, 1024, 768 },
+ },
+ {
+ .name = "positioning_simple",
+ .msg = "Simple positioning should work",
+ .src = { 0, 0, 1023 << 16, 767 << 16 },
+ .crtc = { 0, 0, 1023, 767 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = true,
+ .src_expected = { 0, 0, 1023 << 16, 767 << 16 },
+ .crtc_expected = { 0, 0, 1023, 767 },
+ },
+ {
+ .name = "upscaling",
+ .msg = "Upscaling exactly 2x should work",
+ .src = { 0, 0, 512 << 16, 384 << 16 },
+ .crtc = { 0, 0, 1024, 768 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = 0x8000,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = false,
+ .src_expected = { 0, 0, 512 << 16, 384 << 16 },
+ .crtc_expected = { 0, 0, 1024, 768 },
+ },
+ {
+ .name = "downscaling",
+ .msg = "Should succeed with exact scaling limit",
+ .src = { 0, 0, 2048 << 16, 1536 << 16 },
+ .crtc = { 0, 0, 1024, 768 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = 0x20000,
+ .can_position = false,
+ .src_expected = { 0, 0, 2048 << 16, 1536 << 16 },
+ .crtc_expected = { 0, 0, 1024, 768 },
+ },
+ {
+ .name = "rounding1",
+ .msg = "Should succeed by clipping to exact multiple",
+ .src = { 0, 0, 0x40001, 0x40001 },
+ .crtc = { 1022, 766, 4, 4 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = 0x10001,
+ .can_position = true,
+ .src_expected = { 0, 0, 2 << 16, 2 << 16 },
+ .crtc_expected = { 1022, 766, 2, 2 },
+ },
+ {
+ .name = "rounding2",
+ .msg = "Should succeed by clipping to exact multiple",
+ .src = { 0x20001, 0x20001, 0x4040001, 0x3040001 },
+ .crtc = { -2, -2, 1028, 772 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = 0x10001,
+ .can_position = false,
+ .src_expected = { 0x40002, 0x40002, 1024 << 16, 768 << 16 },
+ .crtc_expected = { 0, 0, 1024, 768 },
+ },
+ {
+ .name = "rounding3",
+ .msg = "Should succeed by clipping to exact multiple",
+ .src = { 0, 0, 0x3ffff, 0x3ffff },
+ .crtc = { 1022, 766, 4, 4 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = 0xffff,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = true,
+ /* Should not be rounded to 0x20001, which would be upscaling. */
+ .src_expected = { 0, 0, 2 << 16, 2 << 16 },
+ .crtc_expected = { 1022, 766, 2, 2 },
+ },
+ {
+ .name = "rounding4",
+ .msg = "Should succeed by clipping to exact multiple",
+ .src = { 0x1ffff, 0x1ffff, 0x403ffff, 0x303ffff },
+ .crtc = { -2, -2, 1028, 772 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = 0xffff,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = false,
+ .src_expected = { 0x3fffe, 0x3fffe, 1024 << 16, 768 << 16 },
+ .crtc_expected = { 0, 0, 1024, 768 },
+ },
+};
+
+KUNIT_ARRAY_PARAM(drm_check_plane_state, drm_check_plane_state_tests, drm_check_plane_state_desc);
+
+static void drm_test_check_invalid_plane_state(struct kunit *test)
+{
+ const struct drm_check_plane_state_test *params = test->param_value;
+ struct drm_plane_state *plane_state = test->priv;
+
+ KUNIT_ASSERT_LT_MSG(test,
+ drm_atomic_helper_check_plane_state(plane_state, &crtc_state,
+ params->min_scale,
+ params->max_scale,
+ params->can_position, false),
+ 0, params->msg);
+}
+
+static const struct drm_check_plane_state_test drm_check_invalid_plane_state_tests[] = {
+ {
+ .name = "positioning_invalid",
+ .msg = "Should not be able to position on the crtc with can_position=false",
+ .src = { 0, 0, 1023 << 16, 767 << 16 },
+ .crtc = { 0, 0, 1023, 767 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = false,
+ },
+ {
+ .name = "upscaling_invalid",
+ .msg = "Upscaling out of range should fail",
+ .src = { 0, 0, 512 << 16, 384 << 16 },
+ .crtc = { 0, 0, 1024, 768 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = 0x8001,
+ .max_scale = DRM_PLANE_NO_SCALING,
+ .can_position = false,
+ },
+ {
+ .name = "downscaling_invalid",
+ .msg = "Downscaling out of range should fail",
+ .src = { 0, 0, 2048 << 16, 1536 << 16 },
+ .crtc = { 0, 0, 1024, 768 },
+ .rotation = DRM_MODE_ROTATE_0,
+ .min_scale = DRM_PLANE_NO_SCALING,
+ .max_scale = 0x1ffff,
+ .can_position = false,
+ },
+};
+
+KUNIT_ARRAY_PARAM(drm_check_invalid_plane_state, drm_check_invalid_plane_state_tests,
+ drm_check_plane_state_desc);
+
static struct kunit_case drm_plane_helper_test[] = {
- KUNIT_CASE(drm_test_check_plane_state),
+ KUNIT_CASE_PARAM(drm_test_check_plane_state, drm_check_plane_state_gen_params),
+ KUNIT_CASE_PARAM(drm_test_check_invalid_plane_state,
+ drm_check_invalid_plane_state_gen_params),
{}
};
static struct kunit_suite drm_plane_helper_test_suite = {
.name = "drm_plane_helper",
+ .init = drm_plane_helper_init,
.test_cases = drm_plane_helper_test,
};
diff --git a/drivers/gpu/drm/tidss/tidss_drv.c b/drivers/gpu/drm/tidss/tidss_drv.c
index 15cd9b9..07d94b1 100644
--- a/drivers/gpu/drm/tidss/tidss_drv.c
+++ b/drivers/gpu/drm/tidss/tidss_drv.c
@@ -14,7 +14,7 @@
#include <drm/drm_crtc.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/tidss/tidss_kms.c b/drivers/gpu/drm/tidss/tidss_kms.c
index afb2879..345bcc3 100644
--- a/drivers/gpu/drm/tidss/tidss_kms.c
+++ b/drivers/gpu/drm/tidss/tidss_kms.c
@@ -10,7 +10,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_crtc_helper.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_of.h>
#include <drm/drm_panel.h>
diff --git a/drivers/gpu/drm/tilcdc/tilcdc_drv.c b/drivers/gpu/drm/tilcdc/tilcdc_drv.c
index f72755b..80615ec 100644
--- a/drivers/gpu/drm/tilcdc/tilcdc_drv.c
+++ b/drivers/gpu/drm/tilcdc/tilcdc_drv.c
@@ -16,7 +16,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_debugfs.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
diff --git a/drivers/gpu/drm/tiny/Kconfig b/drivers/gpu/drm/tiny/Kconfig
index 56595726..f6889f6 100644
--- a/drivers/gpu/drm/tiny/Kconfig
+++ b/drivers/gpu/drm/tiny/Kconfig
@@ -51,6 +51,19 @@
This is a KMS driver for projectors which use the GM12U320 chipset
for video transfer over USB2/3, such as the Acer C120 mini projector.
+config DRM_OFDRM
+ tristate "Open Firmware display driver"
+ depends on DRM && MMU && OF && (PPC || COMPILE_TEST)
+ select APERTURE_HELPERS
+ select DRM_GEM_SHMEM_HELPER
+ select DRM_KMS_HELPER
+ help
+ DRM driver for Open Firmware framebuffers.
+
+ This driver assumes that the display hardware has been initialized
+ by the Open Firmware before the kernel boots. Scanout buffer, size,
+ and display format must be provided via device tree.
+
config DRM_PANEL_MIPI_DBI
tristate "DRM support for MIPI DBI compatible panels"
depends on DRM && SPI
diff --git a/drivers/gpu/drm/tiny/Makefile b/drivers/gpu/drm/tiny/Makefile
index 1d9d622..76dde89 100644
--- a/drivers/gpu/drm/tiny/Makefile
+++ b/drivers/gpu/drm/tiny/Makefile
@@ -4,6 +4,7 @@
obj-$(CONFIG_DRM_BOCHS) += bochs.o
obj-$(CONFIG_DRM_CIRRUS_QEMU) += cirrus.o
obj-$(CONFIG_DRM_GM12U320) += gm12u320.o
+obj-$(CONFIG_DRM_OFDRM) += ofdrm.o
obj-$(CONFIG_DRM_PANEL_MIPI_DBI) += panel-mipi-dbi.o
obj-$(CONFIG_DRM_SIMPLEDRM) += simpledrm.o
obj-$(CONFIG_TINYDRM_HX8357D) += hx8357d.o
diff --git a/drivers/gpu/drm/tiny/arcpgu.c b/drivers/gpu/drm/tiny/arcpgu.c
index bb302a3..611bbee 100644
--- a/drivers/gpu/drm/tiny/arcpgu.c
+++ b/drivers/gpu/drm/tiny/arcpgu.c
@@ -12,7 +12,7 @@
#include <drm/drm_drv.h>
#include <drm/drm_edid.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_dma_helper.h>
diff --git a/drivers/gpu/drm/tiny/bochs.c b/drivers/gpu/drm/tiny/bochs.c
index a512622..0243460 100644
--- a/drivers/gpu/drm/tiny/bochs.c
+++ b/drivers/gpu/drm/tiny/bochs.c
@@ -7,7 +7,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_framebuffer_helper.h>
@@ -543,7 +543,6 @@ static int bochs_kms_init(struct bochs_device *bochs)
bochs->dev->mode_config.max_width = 8192;
bochs->dev->mode_config.max_height = 8192;
- bochs->dev->mode_config.fb_base = bochs->fb_base;
bochs->dev->mode_config.preferred_depth = 24;
bochs->dev->mode_config.prefer_shadow = 0;
bochs->dev->mode_config.prefer_shadow_fbdev = 1;
diff --git a/drivers/gpu/drm/tiny/cirrus.c b/drivers/gpu/drm/tiny/cirrus.c
index 354d5e8..678c2ef 100644
--- a/drivers/gpu/drm/tiny/cirrus.c
+++ b/drivers/gpu/drm/tiny/cirrus.c
@@ -30,7 +30,7 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_fourcc.h>
diff --git a/drivers/gpu/drm/tiny/gm12u320.c b/drivers/gpu/drm/tiny/gm12u320.c
index 7441d99..130fd07 100644
--- a/drivers/gpu/drm/tiny/gm12u320.c
+++ b/drivers/gpu/drm/tiny/gm12u320.c
@@ -12,7 +12,7 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_edid.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_fourcc.h>
diff --git a/drivers/gpu/drm/tiny/hx8357d.c b/drivers/gpu/drm/tiny/hx8357d.c
index 48c24aa..9f634f7 100644
--- a/drivers/gpu/drm/tiny/hx8357d.c
+++ b/drivers/gpu/drm/tiny/hx8357d.c
@@ -18,7 +18,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/tiny/ili9163.c b/drivers/gpu/drm/tiny/ili9163.c
index 9a1a594..ca0451f 100644
--- a/drivers/gpu/drm/tiny/ili9163.c
+++ b/drivers/gpu/drm/tiny/ili9163.c
@@ -9,7 +9,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_mipi_dbi.h>
diff --git a/drivers/gpu/drm/tiny/ili9225.c b/drivers/gpu/drm/tiny/ili9225.c
index a79da2b..815bab2 100644
--- a/drivers/gpu/drm/tiny/ili9225.c
+++ b/drivers/gpu/drm/tiny/ili9225.c
@@ -20,7 +20,7 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_atomic_helper.h>
diff --git a/drivers/gpu/drm/tiny/ili9341.c b/drivers/gpu/drm/tiny/ili9341.c
index 69b265e..420f600 100644
--- a/drivers/gpu/drm/tiny/ili9341.c
+++ b/drivers/gpu/drm/tiny/ili9341.c
@@ -17,7 +17,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/tiny/ili9486.c b/drivers/gpu/drm/tiny/ili9486.c
index c80028bb..1bb8474 100644
--- a/drivers/gpu/drm/tiny/ili9486.c
+++ b/drivers/gpu/drm/tiny/ili9486.c
@@ -16,7 +16,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/tiny/mi0283qt.c b/drivers/gpu/drm/tiny/mi0283qt.c
index bc522fb..47df2b5 100644
--- a/drivers/gpu/drm/tiny/mi0283qt.c
+++ b/drivers/gpu/drm/tiny/mi0283qt.c
@@ -15,7 +15,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/tiny/ofdrm.c b/drivers/gpu/drm/tiny/ofdrm.c
new file mode 100644
index 0000000..dc9e4d7
--- /dev/null
+++ b/drivers/gpu/drm/tiny/ofdrm.c
@@ -0,0 +1,1429 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <linux/of_address.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+
+#include <drm/drm_aperture.h>
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_state_helper.h>
+#include <drm/drm_connector.h>
+#include <drm/drm_damage_helper.h>
+#include <drm/drm_device.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_fbdev_generic.h>
+#include <drm/drm_format_helper.h>
+#include <drm/drm_framebuffer.h>
+#include <drm/drm_gem_atomic_helper.h>
+#include <drm/drm_gem_framebuffer_helper.h>
+#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_managed.h>
+#include <drm/drm_modeset_helper_vtables.h>
+#include <drm/drm_plane_helper.h>
+#include <drm/drm_probe_helper.h>
+#include <drm/drm_simple_kms_helper.h>
+
+#define DRIVER_NAME "ofdrm"
+#define DRIVER_DESC "DRM driver for OF platform devices"
+#define DRIVER_DATE "20220501"
+#define DRIVER_MAJOR 1
+#define DRIVER_MINOR 0
+
+#define PCI_VENDOR_ID_ATI_R520 0x7100
+#define PCI_VENDOR_ID_ATI_R600 0x9400
+
+#define OFDRM_GAMMA_LUT_SIZE 256
+
+/* Definitions used by the Avivo palette */
+#define AVIVO_DC_LUT_RW_SELECT 0x6480
+#define AVIVO_DC_LUT_RW_MODE 0x6484
+#define AVIVO_DC_LUT_RW_INDEX 0x6488
+#define AVIVO_DC_LUT_SEQ_COLOR 0x648c
+#define AVIVO_DC_LUT_PWL_DATA 0x6490
+#define AVIVO_DC_LUT_30_COLOR 0x6494
+#define AVIVO_DC_LUT_READ_PIPE_SELECT 0x6498
+#define AVIVO_DC_LUT_WRITE_EN_MASK 0x649c
+#define AVIVO_DC_LUT_AUTOFILL 0x64a0
+#define AVIVO_DC_LUTA_CONTROL 0x64c0
+#define AVIVO_DC_LUTA_BLACK_OFFSET_BLUE 0x64c4
+#define AVIVO_DC_LUTA_BLACK_OFFSET_GREEN 0x64c8
+#define AVIVO_DC_LUTA_BLACK_OFFSET_RED 0x64cc
+#define AVIVO_DC_LUTA_WHITE_OFFSET_BLUE 0x64d0
+#define AVIVO_DC_LUTA_WHITE_OFFSET_GREEN 0x64d4
+#define AVIVO_DC_LUTA_WHITE_OFFSET_RED 0x64d8
+#define AVIVO_DC_LUTB_CONTROL 0x6cc0
+#define AVIVO_DC_LUTB_BLACK_OFFSET_BLUE 0x6cc4
+#define AVIVO_DC_LUTB_BLACK_OFFSET_GREEN 0x6cc8
+#define AVIVO_DC_LUTB_BLACK_OFFSET_RED 0x6ccc
+#define AVIVO_DC_LUTB_WHITE_OFFSET_BLUE 0x6cd0
+#define AVIVO_DC_LUTB_WHITE_OFFSET_GREEN 0x6cd4
+#define AVIVO_DC_LUTB_WHITE_OFFSET_RED 0x6cd8
+
+enum ofdrm_model {
+ OFDRM_MODEL_UNKNOWN,
+ OFDRM_MODEL_MACH64, /* ATI Mach64 */
+ OFDRM_MODEL_RAGE128, /* ATI Rage128 */
+ OFDRM_MODEL_RAGE_M3A, /* ATI Rage Mobility M3 Head A */
+ OFDRM_MODEL_RAGE_M3B, /* ATI Rage Mobility M3 Head B */
+ OFDRM_MODEL_RADEON, /* ATI Radeon */
+ OFDRM_MODEL_GXT2000, /* IBM GXT2000 */
+ OFDRM_MODEL_AVIVO, /* ATI R5xx */
+ OFDRM_MODEL_QEMU, /* QEMU VGA */
+};
+
+/*
+ * Helpers for display nodes
+ */
+
+static int display_get_validated_int(struct drm_device *dev, const char *name, uint32_t value)
+{
+ if (value > INT_MAX) {
+ drm_err(dev, "invalid framebuffer %s of %u\n", name, value);
+ return -EINVAL;
+ }
+ return (int)value;
+}
+
+static int display_get_validated_int0(struct drm_device *dev, const char *name, uint32_t value)
+{
+ if (!value) {
+ drm_err(dev, "invalid framebuffer %s of %u\n", name, value);
+ return -EINVAL;
+ }
+ return display_get_validated_int(dev, name, value);
+}
+
+static const struct drm_format_info *display_get_validated_format(struct drm_device *dev,
+ u32 depth, bool big_endian)
+{
+ const struct drm_format_info *info;
+ u32 format;
+
+ switch (depth) {
+ case 8:
+ format = drm_mode_legacy_fb_format(8, 8);
+ break;
+ case 15:
+ case 16:
+ format = drm_mode_legacy_fb_format(16, depth);
+ break;
+ case 32:
+ format = drm_mode_legacy_fb_format(32, 24);
+ break;
+ default:
+ drm_err(dev, "unsupported framebuffer depth %u\n", depth);
+ return ERR_PTR(-EINVAL);
+ }
+
+ /*
+ * DRM formats assume little-endian byte order. Update the format
+ * if the scanout buffer uses big-endian ordering.
+ */
+ if (big_endian) {
+ switch (format) {
+ case DRM_FORMAT_XRGB8888:
+ format = DRM_FORMAT_BGRX8888;
+ break;
+ case DRM_FORMAT_ARGB8888:
+ format = DRM_FORMAT_BGRA8888;
+ break;
+ case DRM_FORMAT_RGB565:
+ format = DRM_FORMAT_RGB565 | DRM_FORMAT_BIG_ENDIAN;
+ break;
+ case DRM_FORMAT_XRGB1555:
+ format = DRM_FORMAT_XRGB1555 | DRM_FORMAT_BIG_ENDIAN;
+ break;
+ default:
+ break;
+ }
+ }
+
+ info = drm_format_info(format);
+ if (!info) {
+ drm_err(dev, "cannot find framebuffer format for depth %u\n", depth);
+ return ERR_PTR(-EINVAL);
+ }
+
+ return info;
+}
+
+static int display_read_u32_of(struct drm_device *dev, struct device_node *of_node,
+ const char *name, u32 *value)
+{
+ int ret = of_property_read_u32(of_node, name, value);
+
+ if (ret)
+ drm_err(dev, "cannot parse framebuffer %s: error %d\n", name, ret);
+ return ret;
+}
+
+static bool display_get_big_endian_of(struct drm_device *dev, struct device_node *of_node)
+{
+ bool big_endian;
+
+#ifdef __BIG_ENDIAN
+ big_endian = true;
+ if (of_get_property(of_node, "little-endian", NULL))
+ big_endian = false;
+#else
+ big_endian = false;
+ if (of_get_property(of_node, "big-endian", NULL))
+ big_endian = true;
+#endif
+
+ return big_endian;
+}
+
+static int display_get_width_of(struct drm_device *dev, struct device_node *of_node)
+{
+ u32 width;
+ int ret = display_read_u32_of(dev, of_node, "width", &width);
+
+ if (ret)
+ return ret;
+ return display_get_validated_int0(dev, "width", width);
+}
+
+static int display_get_height_of(struct drm_device *dev, struct device_node *of_node)
+{
+ u32 height;
+ int ret = display_read_u32_of(dev, of_node, "height", &height);
+
+ if (ret)
+ return ret;
+ return display_get_validated_int0(dev, "height", height);
+}
+
+static int display_get_depth_of(struct drm_device *dev, struct device_node *of_node)
+{
+ u32 depth;
+ int ret = display_read_u32_of(dev, of_node, "depth", &depth);
+
+ if (ret)
+ return ret;
+ return display_get_validated_int0(dev, "depth", depth);
+}
+
+static int display_get_linebytes_of(struct drm_device *dev, struct device_node *of_node)
+{
+ u32 linebytes;
+ int ret = display_read_u32_of(dev, of_node, "linebytes", &linebytes);
+
+ if (ret)
+ return ret;
+ return display_get_validated_int(dev, "linebytes", linebytes);
+}
+
+static u64 display_get_address_of(struct drm_device *dev, struct device_node *of_node)
+{
+ u32 address;
+ int ret;
+
+ /*
+ * Not all devices provide an address property, it's not
+ * a bug if this fails. The driver will try to find the
+ * framebuffer base address from the device's memory regions.
+ */
+ ret = of_property_read_u32(of_node, "address", &address);
+ if (ret)
+ return OF_BAD_ADDR;
+
+ return address;
+}
+
+static bool is_avivo(u32 vendor, u32 device)
+{
+ /* This will match most R5xx */
+ return (vendor == PCI_VENDOR_ID_ATI) &&
+ ((device >= PCI_VENDOR_ID_ATI_R520 && device < 0x7800) ||
+ (PCI_VENDOR_ID_ATI_R600 >= 0x9400));
+}
+
+static enum ofdrm_model display_get_model_of(struct drm_device *dev, struct device_node *of_node)
+{
+ enum ofdrm_model model = OFDRM_MODEL_UNKNOWN;
+
+ if (of_node_name_prefix(of_node, "ATY,Rage128")) {
+ model = OFDRM_MODEL_RAGE128;
+ } else if (of_node_name_prefix(of_node, "ATY,RageM3pA") ||
+ of_node_name_prefix(of_node, "ATY,RageM3p12A")) {
+ model = OFDRM_MODEL_RAGE_M3A;
+ } else if (of_node_name_prefix(of_node, "ATY,RageM3pB")) {
+ model = OFDRM_MODEL_RAGE_M3B;
+ } else if (of_node_name_prefix(of_node, "ATY,Rage6")) {
+ model = OFDRM_MODEL_RADEON;
+ } else if (of_node_name_prefix(of_node, "ATY,")) {
+ return OFDRM_MODEL_MACH64;
+ } else if (of_device_is_compatible(of_node, "pci1014,b7") ||
+ of_device_is_compatible(of_node, "pci1014,21c")) {
+ model = OFDRM_MODEL_GXT2000;
+ } else if (of_node_name_prefix(of_node, "vga,Display-")) {
+ struct device_node *of_parent;
+ const __be32 *vendor_p, *device_p;
+
+ /* Look for AVIVO initialized by SLOF */
+ of_parent = of_get_parent(of_node);
+ vendor_p = of_get_property(of_parent, "vendor-id", NULL);
+ device_p = of_get_property(of_parent, "device-id", NULL);
+ if (vendor_p && device_p) {
+ u32 vendor = be32_to_cpup(vendor_p);
+ u32 device = be32_to_cpup(device_p);
+
+ if (is_avivo(vendor, device))
+ model = OFDRM_MODEL_AVIVO;
+ }
+ of_node_put(of_parent);
+ } else if (of_device_is_compatible(of_node, "qemu,std-vga")) {
+ model = OFDRM_MODEL_QEMU;
+ }
+
+ return model;
+}
+
+/*
+ * Open Firmware display device
+ */
+
+struct ofdrm_device;
+
+struct ofdrm_device_funcs {
+ void __iomem *(*cmap_ioremap)(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_bas);
+ void (*cmap_write)(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b);
+};
+
+struct ofdrm_device {
+ struct drm_device dev;
+ struct platform_device *pdev;
+
+ const struct ofdrm_device_funcs *funcs;
+
+ /* firmware-buffer settings */
+ struct iosys_map screen_base;
+ struct drm_display_mode mode;
+ const struct drm_format_info *format;
+ unsigned int pitch;
+
+ /* colormap */
+ void __iomem *cmap_base;
+
+ /* modesetting */
+ uint32_t formats[8];
+ struct drm_plane primary_plane;
+ struct drm_crtc crtc;
+ struct drm_encoder encoder;
+ struct drm_connector connector;
+};
+
+static struct ofdrm_device *ofdrm_device_of_dev(struct drm_device *dev)
+{
+ return container_of(dev, struct ofdrm_device, dev);
+}
+
+/*
+ * Hardware
+ */
+
+#if defined(CONFIG_PCI)
+static struct pci_dev *display_get_pci_dev_of(struct drm_device *dev, struct device_node *of_node)
+{
+ const __be32 *vendor_p, *device_p;
+ u32 vendor, device;
+ struct pci_dev *pcidev;
+
+ vendor_p = of_get_property(of_node, "vendor-id", NULL);
+ if (!vendor_p)
+ return ERR_PTR(-ENODEV);
+ vendor = be32_to_cpup(vendor_p);
+
+ device_p = of_get_property(of_node, "device-id", NULL);
+ if (!device_p)
+ return ERR_PTR(-ENODEV);
+ device = be32_to_cpup(device_p);
+
+ pcidev = pci_get_device(vendor, device, NULL);
+ if (!pcidev)
+ return ERR_PTR(-ENODEV);
+
+ return pcidev;
+}
+
+static void ofdrm_pci_release(void *data)
+{
+ struct pci_dev *pcidev = data;
+
+ pci_disable_device(pcidev);
+}
+
+static int ofdrm_device_init_pci(struct ofdrm_device *odev)
+{
+ struct drm_device *dev = &odev->dev;
+ struct platform_device *pdev = to_platform_device(dev->dev);
+ struct device_node *of_node = pdev->dev.of_node;
+ struct pci_dev *pcidev;
+ int ret;
+
+ /*
+ * Never use pcim_ or other managed helpers on the returned PCI
+ * device. Otherwise, probing the native driver will fail for
+ * resource conflicts. PCI-device management has to be tied to
+ * the lifetime of the platform device until the native driver
+ * takes over.
+ */
+ pcidev = display_get_pci_dev_of(dev, of_node);
+ if (IS_ERR(pcidev))
+ return 0; /* no PCI device found; ignore the error */
+
+ ret = pci_enable_device(pcidev);
+ if (ret) {
+ drm_err(dev, "pci_enable_device(%s) failed: %d\n",
+ dev_name(&pcidev->dev), ret);
+ return ret;
+ }
+ ret = devm_add_action_or_reset(&pdev->dev, ofdrm_pci_release, pcidev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+#else
+static int ofdrm_device_init_pci(struct ofdrm_device *odev)
+{
+ return 0;
+}
+#endif
+
+/*
+ * OF display settings
+ */
+
+static struct resource *ofdrm_find_fb_resource(struct ofdrm_device *odev,
+ struct resource *fb_res)
+{
+ struct platform_device *pdev = to_platform_device(odev->dev.dev);
+ struct resource *res, *max_res = NULL;
+ u32 i;
+
+ for (i = 0; pdev->num_resources; ++i) {
+ res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+ if (!res)
+ break; /* all resources processed */
+ if (resource_size(res) < resource_size(fb_res))
+ continue; /* resource too small */
+ if (fb_res->start && resource_contains(res, fb_res))
+ return res; /* resource contains framebuffer */
+ if (!max_res || resource_size(res) > resource_size(max_res))
+ max_res = res; /* store largest resource as fallback */
+ }
+
+ return max_res;
+}
+
+/*
+ * Colormap / Palette
+ */
+
+static void __iomem *get_cmap_address_of(struct ofdrm_device *odev, struct device_node *of_node,
+ int bar_no, unsigned long offset, unsigned long size)
+{
+ struct drm_device *dev = &odev->dev;
+ const __be32 *addr_p;
+ u64 max_size, address;
+ unsigned int flags;
+ void __iomem *mem;
+
+ addr_p = of_get_pci_address(of_node, bar_no, &max_size, &flags);
+ if (!addr_p)
+ addr_p = of_get_address(of_node, bar_no, &max_size, &flags);
+ if (!addr_p)
+ return IOMEM_ERR_PTR(-ENODEV);
+
+ if ((flags & (IORESOURCE_IO | IORESOURCE_MEM)) == 0)
+ return IOMEM_ERR_PTR(-ENODEV);
+
+ if ((offset + size) >= max_size)
+ return IOMEM_ERR_PTR(-ENODEV);
+
+ address = of_translate_address(of_node, addr_p);
+ if (address == OF_BAD_ADDR)
+ return IOMEM_ERR_PTR(-ENODEV);
+
+ mem = devm_ioremap(dev->dev, address + offset, size);
+ if (!mem)
+ return IOMEM_ERR_PTR(-ENOMEM);
+
+ return mem;
+}
+
+static void __iomem *ofdrm_mach64_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ struct drm_device *dev = &odev->dev;
+ u64 address;
+ void __iomem *cmap_base;
+
+ address = fb_base & 0xff000000ul;
+ address += 0x7ff000;
+
+ cmap_base = devm_ioremap(dev->dev, address, 0x1000);
+ if (!cmap_base)
+ return IOMEM_ERR_PTR(-ENOMEM);
+
+ return cmap_base;
+}
+
+static void ofdrm_mach64_cmap_write(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b)
+{
+ void __iomem *addr = odev->cmap_base + 0xcc0;
+ void __iomem *data = odev->cmap_base + 0xcc0 + 1;
+
+ writeb(index, addr);
+ writeb(r, data);
+ writeb(g, data);
+ writeb(b, data);
+}
+
+static void __iomem *ofdrm_rage128_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ return get_cmap_address_of(odev, of_node, 2, 0, 0x1fff);
+}
+
+static void ofdrm_rage128_cmap_write(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b)
+{
+ void __iomem *addr = odev->cmap_base + 0xb0;
+ void __iomem *data = odev->cmap_base + 0xb4;
+ u32 color = (r << 16) | (g << 8) | b;
+
+ writeb(index, addr);
+ writel(color, data);
+}
+
+static void __iomem *ofdrm_rage_m3a_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ return get_cmap_address_of(odev, of_node, 2, 0, 0x1fff);
+}
+
+static void ofdrm_rage_m3a_cmap_write(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b)
+{
+ void __iomem *dac_ctl = odev->cmap_base + 0x58;
+ void __iomem *addr = odev->cmap_base + 0xb0;
+ void __iomem *data = odev->cmap_base + 0xb4;
+ u32 color = (r << 16) | (g << 8) | b;
+ u32 val;
+
+ /* Clear PALETTE_ACCESS_CNTL in DAC_CNTL */
+ val = readl(dac_ctl);
+ val &= ~0x20;
+ writel(val, dac_ctl);
+
+ /* Set color at palette index */
+ writeb(index, addr);
+ writel(color, data);
+}
+
+static void __iomem *ofdrm_rage_m3b_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ return get_cmap_address_of(odev, of_node, 2, 0, 0x1fff);
+}
+
+static void ofdrm_rage_m3b_cmap_write(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b)
+{
+ void __iomem *dac_ctl = odev->cmap_base + 0x58;
+ void __iomem *addr = odev->cmap_base + 0xb0;
+ void __iomem *data = odev->cmap_base + 0xb4;
+ u32 color = (r << 16) | (g << 8) | b;
+ u32 val;
+
+ /* Set PALETTE_ACCESS_CNTL in DAC_CNTL */
+ val = readl(dac_ctl);
+ val |= 0x20;
+ writel(val, dac_ctl);
+
+ /* Set color at palette index */
+ writeb(index, addr);
+ writel(color, data);
+}
+
+static void __iomem *ofdrm_radeon_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ return get_cmap_address_of(odev, of_node, 1, 0, 0x1fff);
+}
+
+static void __iomem *ofdrm_gxt2000_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ return get_cmap_address_of(odev, of_node, 0, 0x6000, 0x1000);
+}
+
+static void ofdrm_gxt2000_cmap_write(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b)
+{
+ void __iomem *data = ((unsigned int __iomem *)odev->cmap_base) + index;
+ u32 color = (r << 16) | (g << 8) | b;
+
+ writel(color, data);
+}
+
+static void __iomem *ofdrm_avivo_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ struct device_node *of_parent;
+ void __iomem *cmap_base;
+
+ of_parent = of_get_parent(of_node);
+ cmap_base = get_cmap_address_of(odev, of_parent, 0, 0, 0x10000);
+ of_node_put(of_parent);
+
+ return cmap_base;
+}
+
+static void ofdrm_avivo_cmap_write(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b)
+{
+ void __iomem *lutsel = odev->cmap_base + AVIVO_DC_LUT_RW_SELECT;
+ void __iomem *addr = odev->cmap_base + AVIVO_DC_LUT_RW_INDEX;
+ void __iomem *data = odev->cmap_base + AVIVO_DC_LUT_30_COLOR;
+ u32 color = (r << 22) | (g << 12) | (b << 2);
+
+ /* Write to both LUTs for now */
+
+ writel(1, lutsel);
+ writeb(index, addr);
+ writel(color, data);
+
+ writel(0, lutsel);
+ writeb(index, addr);
+ writel(color, data);
+}
+
+static void __iomem *ofdrm_qemu_cmap_ioremap(struct ofdrm_device *odev,
+ struct device_node *of_node,
+ u64 fb_base)
+{
+ static const __be32 io_of_addr[3] = {
+ cpu_to_be32(0x01000000),
+ cpu_to_be32(0x00),
+ cpu_to_be32(0x00),
+ };
+
+ struct drm_device *dev = &odev->dev;
+ u64 address;
+ void __iomem *cmap_base;
+
+ address = of_translate_address(of_node, io_of_addr);
+ if (address == OF_BAD_ADDR)
+ return IOMEM_ERR_PTR(-ENODEV);
+
+ cmap_base = devm_ioremap(dev->dev, address + 0x3c8, 2);
+ if (!cmap_base)
+ return IOMEM_ERR_PTR(-ENOMEM);
+
+ return cmap_base;
+}
+
+static void ofdrm_qemu_cmap_write(struct ofdrm_device *odev, unsigned char index,
+ unsigned char r, unsigned char g, unsigned char b)
+{
+ void __iomem *addr = odev->cmap_base;
+ void __iomem *data = odev->cmap_base + 1;
+
+ writeb(index, addr);
+ writeb(r, data);
+ writeb(g, data);
+ writeb(b, data);
+}
+
+static void ofdrm_device_set_gamma_linear(struct ofdrm_device *odev,
+ const struct drm_format_info *format)
+{
+ struct drm_device *dev = &odev->dev;
+ int i;
+
+ switch (format->format) {
+ case DRM_FORMAT_RGB565:
+ case DRM_FORMAT_RGB565 | DRM_FORMAT_BIG_ENDIAN:
+ /* Use better interpolation, to take 32 values from 0 to 255 */
+ for (i = 0; i < OFDRM_GAMMA_LUT_SIZE / 8; i++) {
+ unsigned char r = i * 8 + i / 4;
+ unsigned char g = i * 4 + i / 16;
+ unsigned char b = i * 8 + i / 4;
+
+ odev->funcs->cmap_write(odev, i, r, g, b);
+ }
+ /* Green has one more bit, so add padding with 0 for red and blue. */
+ for (i = OFDRM_GAMMA_LUT_SIZE / 8; i < OFDRM_GAMMA_LUT_SIZE / 4; i++) {
+ unsigned char r = 0;
+ unsigned char g = i * 4 + i / 16;
+ unsigned char b = 0;
+
+ odev->funcs->cmap_write(odev, i, r, g, b);
+ }
+ break;
+ case DRM_FORMAT_XRGB8888:
+ case DRM_FORMAT_BGRX8888:
+ for (i = 0; i < OFDRM_GAMMA_LUT_SIZE; i++)
+ odev->funcs->cmap_write(odev, i, i, i, i);
+ break;
+ default:
+ drm_warn_once(dev, "Unsupported format %p4cc for gamma correction\n",
+ &format->format);
+ break;
+ }
+}
+
+static void ofdrm_device_set_gamma(struct ofdrm_device *odev,
+ const struct drm_format_info *format,
+ struct drm_color_lut *lut)
+{
+ struct drm_device *dev = &odev->dev;
+ int i;
+
+ switch (format->format) {
+ case DRM_FORMAT_RGB565:
+ case DRM_FORMAT_RGB565 | DRM_FORMAT_BIG_ENDIAN:
+ /* Use better interpolation, to take 32 values from lut[0] to lut[255] */
+ for (i = 0; i < OFDRM_GAMMA_LUT_SIZE / 8; i++) {
+ unsigned char r = lut[i * 8 + i / 4].red >> 8;
+ unsigned char g = lut[i * 4 + i / 16].green >> 8;
+ unsigned char b = lut[i * 8 + i / 4].blue >> 8;
+
+ odev->funcs->cmap_write(odev, i, r, g, b);
+ }
+ /* Green has one more bit, so add padding with 0 for red and blue. */
+ for (i = OFDRM_GAMMA_LUT_SIZE / 8; i < OFDRM_GAMMA_LUT_SIZE / 4; i++) {
+ unsigned char r = 0;
+ unsigned char g = lut[i * 4 + i / 16].green >> 8;
+ unsigned char b = 0;
+
+ odev->funcs->cmap_write(odev, i, r, g, b);
+ }
+ break;
+ case DRM_FORMAT_XRGB8888:
+ case DRM_FORMAT_BGRX8888:
+ for (i = 0; i < OFDRM_GAMMA_LUT_SIZE; i++) {
+ unsigned char r = lut[i].red >> 8;
+ unsigned char g = lut[i].green >> 8;
+ unsigned char b = lut[i].blue >> 8;
+
+ odev->funcs->cmap_write(odev, i, r, g, b);
+ }
+ break;
+ default:
+ drm_warn_once(dev, "Unsupported format %p4cc for gamma correction\n",
+ &format->format);
+ break;
+ }
+}
+
+/*
+ * Modesetting
+ */
+
+struct ofdrm_crtc_state {
+ struct drm_crtc_state base;
+
+ /* Primary-plane format; required for color mgmt. */
+ const struct drm_format_info *format;
+};
+
+static struct ofdrm_crtc_state *to_ofdrm_crtc_state(struct drm_crtc_state *base)
+{
+ return container_of(base, struct ofdrm_crtc_state, base);
+}
+
+static void ofdrm_crtc_state_destroy(struct ofdrm_crtc_state *ofdrm_crtc_state)
+{
+ __drm_atomic_helper_crtc_destroy_state(&ofdrm_crtc_state->base);
+ kfree(ofdrm_crtc_state);
+}
+
+/*
+ * Support all formats of OF display and maybe more; in order
+ * of preference. The display's update function will do any
+ * conversion necessary.
+ *
+ * TODO: Add blit helpers for remaining formats and uncomment
+ * constants.
+ */
+static const uint32_t ofdrm_primary_plane_formats[] = {
+ DRM_FORMAT_XRGB8888,
+ DRM_FORMAT_RGB565,
+ //DRM_FORMAT_XRGB1555,
+ //DRM_FORMAT_C8,
+ /* Big-endian formats below */
+ DRM_FORMAT_BGRX8888,
+ DRM_FORMAT_RGB565 | DRM_FORMAT_BIG_ENDIAN,
+};
+
+static const uint64_t ofdrm_primary_plane_format_modifiers[] = {
+ DRM_FORMAT_MOD_LINEAR,
+ DRM_FORMAT_MOD_INVALID
+};
+
+static int ofdrm_primary_plane_helper_atomic_check(struct drm_plane *plane,
+ struct drm_atomic_state *new_state)
+{
+ struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(new_state, plane);
+ struct drm_framebuffer *new_fb = new_plane_state->fb;
+ struct drm_crtc *new_crtc = new_plane_state->crtc;
+ struct drm_crtc_state *new_crtc_state = NULL;
+ struct ofdrm_crtc_state *new_ofdrm_crtc_state;
+ int ret;
+
+ if (new_crtc)
+ new_crtc_state = drm_atomic_get_new_crtc_state(new_state, new_plane_state->crtc);
+
+ ret = drm_atomic_helper_check_plane_state(new_plane_state, new_crtc_state,
+ DRM_PLANE_NO_SCALING,
+ DRM_PLANE_NO_SCALING,
+ false, false);
+ if (ret)
+ return ret;
+ else if (!new_plane_state->visible)
+ return 0;
+
+ new_crtc_state = drm_atomic_get_new_crtc_state(new_state, new_plane_state->crtc);
+
+ new_ofdrm_crtc_state = to_ofdrm_crtc_state(new_crtc_state);
+ new_ofdrm_crtc_state->format = new_fb->format;
+
+ return 0;
+}
+
+static void ofdrm_primary_plane_helper_atomic_update(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_device *dev = plane->dev;
+ struct ofdrm_device *odev = ofdrm_device_of_dev(dev);
+ struct drm_plane_state *plane_state = drm_atomic_get_new_plane_state(state, plane);
+ struct drm_plane_state *old_plane_state = drm_atomic_get_old_plane_state(state, plane);
+ struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(plane_state);
+ struct drm_framebuffer *fb = plane_state->fb;
+ unsigned int dst_pitch = odev->pitch;
+ const struct drm_format_info *dst_format = odev->format;
+ struct drm_atomic_helper_damage_iter iter;
+ struct drm_rect damage;
+ int ret, idx;
+
+ ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE);
+ if (ret)
+ return;
+
+ if (!drm_dev_enter(dev, &idx))
+ goto out_drm_gem_fb_end_cpu_access;
+
+ drm_atomic_helper_damage_iter_init(&iter, old_plane_state, plane_state);
+ drm_atomic_for_each_plane_damage(&iter, &damage) {
+ struct iosys_map dst = odev->screen_base;
+ struct drm_rect dst_clip = plane_state->dst;
+
+ if (!drm_rect_intersect(&dst_clip, &damage))
+ continue;
+
+ iosys_map_incr(&dst, drm_fb_clip_offset(dst_pitch, dst_format, &dst_clip));
+ drm_fb_blit(&dst, &dst_pitch, dst_format->format, shadow_plane_state->data, fb,
+ &damage);
+ }
+
+ drm_dev_exit(idx);
+out_drm_gem_fb_end_cpu_access:
+ drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE);
+}
+
+static void ofdrm_primary_plane_helper_atomic_disable(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_device *dev = plane->dev;
+ struct ofdrm_device *odev = ofdrm_device_of_dev(dev);
+ struct iosys_map dst = odev->screen_base;
+ struct drm_plane_state *plane_state = drm_atomic_get_new_plane_state(state, plane);
+ void __iomem *dst_vmap = dst.vaddr_iomem; /* TODO: Use mapping abstraction */
+ unsigned int dst_pitch = odev->pitch;
+ const struct drm_format_info *dst_format = odev->format;
+ struct drm_rect dst_clip;
+ unsigned long lines, linepixels, i;
+ int idx;
+
+ drm_rect_init(&dst_clip,
+ plane_state->src_x >> 16, plane_state->src_y >> 16,
+ plane_state->src_w >> 16, plane_state->src_h >> 16);
+
+ lines = drm_rect_height(&dst_clip);
+ linepixels = drm_rect_width(&dst_clip);
+
+ if (!drm_dev_enter(dev, &idx))
+ return;
+
+ /* Clear buffer to black if disabled */
+ dst_vmap += drm_fb_clip_offset(dst_pitch, dst_format, &dst_clip);
+ for (i = 0; i < lines; ++i) {
+ memset_io(dst_vmap, 0, linepixels * dst_format->cpp[0]);
+ dst_vmap += dst_pitch;
+ }
+
+ drm_dev_exit(idx);
+}
+
+static const struct drm_plane_helper_funcs ofdrm_primary_plane_helper_funcs = {
+ DRM_GEM_SHADOW_PLANE_HELPER_FUNCS,
+ .atomic_check = ofdrm_primary_plane_helper_atomic_check,
+ .atomic_update = ofdrm_primary_plane_helper_atomic_update,
+ .atomic_disable = ofdrm_primary_plane_helper_atomic_disable,
+};
+
+static const struct drm_plane_funcs ofdrm_primary_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = drm_plane_cleanup,
+ DRM_GEM_SHADOW_PLANE_FUNCS,
+};
+
+static enum drm_mode_status ofdrm_crtc_helper_mode_valid(struct drm_crtc *crtc,
+ const struct drm_display_mode *mode)
+{
+ struct ofdrm_device *odev = ofdrm_device_of_dev(crtc->dev);
+
+ return drm_crtc_helper_mode_valid_fixed(crtc, mode, &odev->mode);
+}
+
+static int ofdrm_crtc_helper_atomic_check(struct drm_crtc *crtc,
+ struct drm_atomic_state *new_state)
+{
+ static const size_t gamma_lut_length = OFDRM_GAMMA_LUT_SIZE * sizeof(struct drm_color_lut);
+
+ struct drm_device *dev = crtc->dev;
+ struct drm_crtc_state *new_crtc_state = drm_atomic_get_new_crtc_state(new_state, crtc);
+ int ret;
+
+ if (!new_crtc_state->enable)
+ return 0;
+
+ ret = drm_atomic_helper_check_crtc_primary_plane(new_crtc_state);
+ if (ret)
+ return ret;
+
+ if (new_crtc_state->color_mgmt_changed) {
+ struct drm_property_blob *gamma_lut = new_crtc_state->gamma_lut;
+
+ if (gamma_lut && (gamma_lut->length != gamma_lut_length)) {
+ drm_dbg(dev, "Incorrect gamma_lut length %zu\n", gamma_lut->length);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static void ofdrm_crtc_helper_atomic_flush(struct drm_crtc *crtc, struct drm_atomic_state *state)
+{
+ struct ofdrm_device *odev = ofdrm_device_of_dev(crtc->dev);
+ struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
+ struct ofdrm_crtc_state *ofdrm_crtc_state = to_ofdrm_crtc_state(crtc_state);
+
+ if (crtc_state->enable && crtc_state->color_mgmt_changed) {
+ const struct drm_format_info *format = ofdrm_crtc_state->format;
+
+ if (crtc_state->gamma_lut)
+ ofdrm_device_set_gamma(odev, format, crtc_state->gamma_lut->data);
+ else
+ ofdrm_device_set_gamma_linear(odev, format);
+ }
+}
+
+/*
+ * The CRTC is always enabled. Screen updates are performed by
+ * the primary plane's atomic_update function. Disabling clears
+ * the screen in the primary plane's atomic_disable function.
+ */
+static const struct drm_crtc_helper_funcs ofdrm_crtc_helper_funcs = {
+ .mode_valid = ofdrm_crtc_helper_mode_valid,
+ .atomic_check = ofdrm_crtc_helper_atomic_check,
+ .atomic_flush = ofdrm_crtc_helper_atomic_flush,
+};
+
+static void ofdrm_crtc_reset(struct drm_crtc *crtc)
+{
+ struct ofdrm_crtc_state *ofdrm_crtc_state =
+ kzalloc(sizeof(*ofdrm_crtc_state), GFP_KERNEL);
+
+ if (crtc->state)
+ ofdrm_crtc_state_destroy(to_ofdrm_crtc_state(crtc->state));
+
+ if (ofdrm_crtc_state)
+ __drm_atomic_helper_crtc_reset(crtc, &ofdrm_crtc_state->base);
+ else
+ __drm_atomic_helper_crtc_reset(crtc, NULL);
+}
+
+static struct drm_crtc_state *ofdrm_crtc_atomic_duplicate_state(struct drm_crtc *crtc)
+{
+ struct drm_device *dev = crtc->dev;
+ struct drm_crtc_state *crtc_state = crtc->state;
+ struct ofdrm_crtc_state *new_ofdrm_crtc_state;
+ struct ofdrm_crtc_state *ofdrm_crtc_state;
+
+ if (drm_WARN_ON(dev, !crtc_state))
+ return NULL;
+
+ new_ofdrm_crtc_state = kzalloc(sizeof(*new_ofdrm_crtc_state), GFP_KERNEL);
+ if (!new_ofdrm_crtc_state)
+ return NULL;
+
+ ofdrm_crtc_state = to_ofdrm_crtc_state(crtc_state);
+
+ __drm_atomic_helper_crtc_duplicate_state(crtc, &new_ofdrm_crtc_state->base);
+ new_ofdrm_crtc_state->format = ofdrm_crtc_state->format;
+
+ return &new_ofdrm_crtc_state->base;
+}
+
+static void ofdrm_crtc_atomic_destroy_state(struct drm_crtc *crtc,
+ struct drm_crtc_state *crtc_state)
+{
+ ofdrm_crtc_state_destroy(to_ofdrm_crtc_state(crtc_state));
+}
+
+static const struct drm_crtc_funcs ofdrm_crtc_funcs = {
+ .reset = ofdrm_crtc_reset,
+ .destroy = drm_crtc_cleanup,
+ .set_config = drm_atomic_helper_set_config,
+ .page_flip = drm_atomic_helper_page_flip,
+ .atomic_duplicate_state = ofdrm_crtc_atomic_duplicate_state,
+ .atomic_destroy_state = ofdrm_crtc_atomic_destroy_state,
+};
+
+static int ofdrm_connector_helper_get_modes(struct drm_connector *connector)
+{
+ struct ofdrm_device *odev = ofdrm_device_of_dev(connector->dev);
+
+ return drm_connector_helper_get_modes_fixed(connector, &odev->mode);
+}
+
+static const struct drm_connector_helper_funcs ofdrm_connector_helper_funcs = {
+ .get_modes = ofdrm_connector_helper_get_modes,
+};
+
+static const struct drm_connector_funcs ofdrm_connector_funcs = {
+ .reset = drm_atomic_helper_connector_reset,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = drm_connector_cleanup,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static const struct drm_mode_config_funcs ofdrm_mode_config_funcs = {
+ .fb_create = drm_gem_fb_create_with_dirty,
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = drm_atomic_helper_commit,
+};
+
+/*
+ * Init / Cleanup
+ */
+
+static const struct ofdrm_device_funcs ofdrm_unknown_device_funcs = {
+};
+
+static const struct ofdrm_device_funcs ofdrm_mach64_device_funcs = {
+ .cmap_ioremap = ofdrm_mach64_cmap_ioremap,
+ .cmap_write = ofdrm_mach64_cmap_write,
+};
+
+static const struct ofdrm_device_funcs ofdrm_rage128_device_funcs = {
+ .cmap_ioremap = ofdrm_rage128_cmap_ioremap,
+ .cmap_write = ofdrm_rage128_cmap_write,
+};
+
+static const struct ofdrm_device_funcs ofdrm_rage_m3a_device_funcs = {
+ .cmap_ioremap = ofdrm_rage_m3a_cmap_ioremap,
+ .cmap_write = ofdrm_rage_m3a_cmap_write,
+};
+
+static const struct ofdrm_device_funcs ofdrm_rage_m3b_device_funcs = {
+ .cmap_ioremap = ofdrm_rage_m3b_cmap_ioremap,
+ .cmap_write = ofdrm_rage_m3b_cmap_write,
+};
+
+static const struct ofdrm_device_funcs ofdrm_radeon_device_funcs = {
+ .cmap_ioremap = ofdrm_radeon_cmap_ioremap,
+ .cmap_write = ofdrm_rage128_cmap_write, /* same as Rage128 */
+};
+
+static const struct ofdrm_device_funcs ofdrm_gxt2000_device_funcs = {
+ .cmap_ioremap = ofdrm_gxt2000_cmap_ioremap,
+ .cmap_write = ofdrm_gxt2000_cmap_write,
+};
+
+static const struct ofdrm_device_funcs ofdrm_avivo_device_funcs = {
+ .cmap_ioremap = ofdrm_avivo_cmap_ioremap,
+ .cmap_write = ofdrm_avivo_cmap_write,
+};
+
+static const struct ofdrm_device_funcs ofdrm_qemu_device_funcs = {
+ .cmap_ioremap = ofdrm_qemu_cmap_ioremap,
+ .cmap_write = ofdrm_qemu_cmap_write,
+};
+
+static struct drm_display_mode ofdrm_mode(unsigned int width, unsigned int height)
+{
+ /*
+ * Assume a monitor resolution of 96 dpi to
+ * get a somewhat reasonable screen size.
+ */
+ const struct drm_display_mode mode = {
+ DRM_MODE_INIT(60, width, height,
+ DRM_MODE_RES_MM(width, 96ul),
+ DRM_MODE_RES_MM(height, 96ul))
+ };
+
+ return mode;
+}
+
+static struct ofdrm_device *ofdrm_device_create(struct drm_driver *drv,
+ struct platform_device *pdev)
+{
+ struct device_node *of_node = pdev->dev.of_node;
+ struct ofdrm_device *odev;
+ struct drm_device *dev;
+ enum ofdrm_model model;
+ bool big_endian;
+ int width, height, depth, linebytes;
+ const struct drm_format_info *format;
+ u64 address;
+ resource_size_t fb_size, fb_base, fb_pgbase, fb_pgsize;
+ struct resource *res, *mem;
+ void __iomem *screen_base;
+ struct drm_plane *primary_plane;
+ struct drm_crtc *crtc;
+ struct drm_encoder *encoder;
+ struct drm_connector *connector;
+ unsigned long max_width, max_height;
+ size_t nformats;
+ int ret;
+
+ odev = devm_drm_dev_alloc(&pdev->dev, drv, struct ofdrm_device, dev);
+ if (IS_ERR(odev))
+ return ERR_CAST(odev);
+ dev = &odev->dev;
+ platform_set_drvdata(pdev, dev);
+
+ ret = ofdrm_device_init_pci(odev);
+ if (ret)
+ return ERR_PTR(ret);
+
+ /*
+ * OF display-node settings
+ */
+
+ model = display_get_model_of(dev, of_node);
+ drm_dbg(dev, "detected model %d\n", model);
+
+ switch (model) {
+ case OFDRM_MODEL_UNKNOWN:
+ odev->funcs = &ofdrm_unknown_device_funcs;
+ break;
+ case OFDRM_MODEL_MACH64:
+ odev->funcs = &ofdrm_mach64_device_funcs;
+ break;
+ case OFDRM_MODEL_RAGE128:
+ odev->funcs = &ofdrm_rage128_device_funcs;
+ break;
+ case OFDRM_MODEL_RAGE_M3A:
+ odev->funcs = &ofdrm_rage_m3a_device_funcs;
+ break;
+ case OFDRM_MODEL_RAGE_M3B:
+ odev->funcs = &ofdrm_rage_m3b_device_funcs;
+ break;
+ case OFDRM_MODEL_RADEON:
+ odev->funcs = &ofdrm_radeon_device_funcs;
+ break;
+ case OFDRM_MODEL_GXT2000:
+ odev->funcs = &ofdrm_gxt2000_device_funcs;
+ break;
+ case OFDRM_MODEL_AVIVO:
+ odev->funcs = &ofdrm_avivo_device_funcs;
+ break;
+ case OFDRM_MODEL_QEMU:
+ odev->funcs = &ofdrm_qemu_device_funcs;
+ break;
+ }
+
+ big_endian = display_get_big_endian_of(dev, of_node);
+
+ width = display_get_width_of(dev, of_node);
+ if (width < 0)
+ return ERR_PTR(width);
+ height = display_get_height_of(dev, of_node);
+ if (height < 0)
+ return ERR_PTR(height);
+ depth = display_get_depth_of(dev, of_node);
+ if (depth < 0)
+ return ERR_PTR(depth);
+ linebytes = display_get_linebytes_of(dev, of_node);
+ if (linebytes < 0)
+ return ERR_PTR(linebytes);
+
+ format = display_get_validated_format(dev, depth, big_endian);
+ if (IS_ERR(format))
+ return ERR_CAST(format);
+ if (!linebytes) {
+ linebytes = drm_format_info_min_pitch(format, 0, width);
+ if (drm_WARN_ON(dev, !linebytes))
+ return ERR_PTR(-EINVAL);
+ }
+
+ fb_size = linebytes * height;
+
+ /*
+ * Try to figure out the address of the framebuffer. Unfortunately, Open
+ * Firmware doesn't provide a standard way to do so. All we can do is a
+ * dodgy heuristic that happens to work in practice.
+ *
+ * On most machines, the "address" property contains what we need, though
+ * not on Matrox cards found in IBM machines. What appears to give good
+ * results is to go through the PCI ranges and pick one that encloses the
+ * "address" property. If none match, we pick the largest.
+ */
+ address = display_get_address_of(dev, of_node);
+ if (address != OF_BAD_ADDR) {
+ struct resource fb_res = DEFINE_RES_MEM(address, fb_size);
+
+ res = ofdrm_find_fb_resource(odev, &fb_res);
+ if (!res)
+ return ERR_PTR(-EINVAL);
+ if (resource_contains(res, &fb_res))
+ fb_base = address;
+ else
+ fb_base = res->start;
+ } else {
+ struct resource fb_res = DEFINE_RES_MEM(0u, fb_size);
+
+ res = ofdrm_find_fb_resource(odev, &fb_res);
+ if (!res)
+ return ERR_PTR(-EINVAL);
+ fb_base = res->start;
+ }
+
+ /*
+ * I/O resources
+ */
+
+ fb_pgbase = round_down(fb_base, PAGE_SIZE);
+ fb_pgsize = fb_base - fb_pgbase + round_up(fb_size, PAGE_SIZE);
+
+ ret = devm_aperture_acquire_from_firmware(dev, fb_pgbase, fb_pgsize);
+ if (ret) {
+ drm_err(dev, "could not acquire memory range %pr: error %d\n", &res, ret);
+ return ERR_PTR(ret);
+ }
+
+ mem = devm_request_mem_region(&pdev->dev, fb_pgbase, fb_pgsize, drv->name);
+ if (!mem) {
+ drm_warn(dev, "could not acquire memory region %pr\n", &res);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ screen_base = devm_ioremap(&pdev->dev, mem->start, resource_size(mem));
+ if (!screen_base)
+ return ERR_PTR(-ENOMEM);
+
+ if (odev->funcs->cmap_ioremap) {
+ void __iomem *cmap_base = odev->funcs->cmap_ioremap(odev, of_node, fb_base);
+
+ if (IS_ERR(cmap_base)) {
+ /* Don't fail; continue without colormap */
+ drm_warn(dev, "could not find colormap: error %ld\n", PTR_ERR(cmap_base));
+ } else {
+ odev->cmap_base = cmap_base;
+ }
+ }
+
+ /*
+ * Firmware framebuffer
+ */
+
+ iosys_map_set_vaddr_iomem(&odev->screen_base, screen_base);
+ odev->mode = ofdrm_mode(width, height);
+ odev->format = format;
+ odev->pitch = linebytes;
+
+ drm_dbg(dev, "display mode={" DRM_MODE_FMT "}\n", DRM_MODE_ARG(&odev->mode));
+ drm_dbg(dev, "framebuffer format=%p4cc, size=%dx%d, linebytes=%d byte\n",
+ &format->format, width, height, linebytes);
+
+ /*
+ * Mode-setting pipeline
+ */
+
+ ret = drmm_mode_config_init(dev);
+ if (ret)
+ return ERR_PTR(ret);
+
+ max_width = max_t(unsigned long, width, DRM_SHADOW_PLANE_MAX_WIDTH);
+ max_height = max_t(unsigned long, height, DRM_SHADOW_PLANE_MAX_HEIGHT);
+
+ dev->mode_config.min_width = width;
+ dev->mode_config.max_width = max_width;
+ dev->mode_config.min_height = height;
+ dev->mode_config.max_height = max_height;
+ dev->mode_config.funcs = &ofdrm_mode_config_funcs;
+ switch (depth) {
+ case 32:
+ dev->mode_config.preferred_depth = 24;
+ break;
+ default:
+ dev->mode_config.preferred_depth = depth;
+ break;
+ }
+ dev->mode_config.quirk_addfb_prefer_host_byte_order = true;
+
+ /* Primary plane */
+
+ nformats = drm_fb_build_fourcc_list(dev, &format->format, 1,
+ ofdrm_primary_plane_formats,
+ ARRAY_SIZE(ofdrm_primary_plane_formats),
+ odev->formats, ARRAY_SIZE(odev->formats));
+
+ primary_plane = &odev->primary_plane;
+ ret = drm_universal_plane_init(dev, primary_plane, 0, &ofdrm_primary_plane_funcs,
+ odev->formats, nformats,
+ ofdrm_primary_plane_format_modifiers,
+ DRM_PLANE_TYPE_PRIMARY, NULL);
+ if (ret)
+ return ERR_PTR(ret);
+ drm_plane_helper_add(primary_plane, &ofdrm_primary_plane_helper_funcs);
+ drm_plane_enable_fb_damage_clips(primary_plane);
+
+ /* CRTC */
+
+ crtc = &odev->crtc;
+ ret = drm_crtc_init_with_planes(dev, crtc, primary_plane, NULL,
+ &ofdrm_crtc_funcs, NULL);
+ if (ret)
+ return ERR_PTR(ret);
+ drm_crtc_helper_add(crtc, &ofdrm_crtc_helper_funcs);
+
+ if (odev->cmap_base) {
+ drm_mode_crtc_set_gamma_size(crtc, OFDRM_GAMMA_LUT_SIZE);
+ drm_crtc_enable_color_mgmt(crtc, 0, false, OFDRM_GAMMA_LUT_SIZE);
+ }
+
+ /* Encoder */
+
+ encoder = &odev->encoder;
+ ret = drm_simple_encoder_init(dev, encoder, DRM_MODE_ENCODER_NONE);
+ if (ret)
+ return ERR_PTR(ret);
+ encoder->possible_crtcs = drm_crtc_mask(crtc);
+
+ /* Connector */
+
+ connector = &odev->connector;
+ ret = drm_connector_init(dev, connector, &ofdrm_connector_funcs,
+ DRM_MODE_CONNECTOR_Unknown);
+ if (ret)
+ return ERR_PTR(ret);
+ drm_connector_helper_add(connector, &ofdrm_connector_helper_funcs);
+ drm_connector_set_panel_orientation_with_quirk(connector,
+ DRM_MODE_PANEL_ORIENTATION_UNKNOWN,
+ width, height);
+
+ ret = drm_connector_attach_encoder(connector, encoder);
+ if (ret)
+ return ERR_PTR(ret);
+
+ drm_mode_config_reset(dev);
+
+ return odev;
+}
+
+/*
+ * DRM driver
+ */
+
+DEFINE_DRM_GEM_FOPS(ofdrm_fops);
+
+static struct drm_driver ofdrm_driver = {
+ DRM_GEM_SHMEM_DRIVER_OPS,
+ .name = DRIVER_NAME,
+ .desc = DRIVER_DESC,
+ .date = DRIVER_DATE,
+ .major = DRIVER_MAJOR,
+ .minor = DRIVER_MINOR,
+ .driver_features = DRIVER_ATOMIC | DRIVER_GEM | DRIVER_MODESET,
+ .fops = &ofdrm_fops,
+};
+
+/*
+ * Platform driver
+ */
+
+static int ofdrm_probe(struct platform_device *pdev)
+{
+ struct ofdrm_device *odev;
+ struct drm_device *dev;
+ int ret;
+
+ odev = ofdrm_device_create(&ofdrm_driver, pdev);
+ if (IS_ERR(odev))
+ return PTR_ERR(odev);
+ dev = &odev->dev;
+
+ ret = drm_dev_register(dev, 0);
+ if (ret)
+ return ret;
+
+ /*
+ * FIXME: 24-bit color depth does not work reliably with a 32-bpp
+ * value. Force the bpp value of the scanout buffer's format.
+ */
+ drm_fbdev_generic_setup(dev, drm_format_info_bpp(odev->format, 0));
+
+ return 0;
+}
+
+static int ofdrm_remove(struct platform_device *pdev)
+{
+ struct drm_device *dev = platform_get_drvdata(pdev);
+
+ drm_dev_unplug(dev);
+
+ return 0;
+}
+
+static const struct of_device_id ofdrm_of_match_display[] = {
+ { .compatible = "display", },
+ { },
+};
+MODULE_DEVICE_TABLE(of, ofdrm_of_match_display);
+
+static struct platform_driver ofdrm_platform_driver = {
+ .driver = {
+ .name = "of-display",
+ .of_match_table = ofdrm_of_match_display,
+ },
+ .probe = ofdrm_probe,
+ .remove = ofdrm_remove,
+};
+
+module_platform_driver(ofdrm_platform_driver);
+
+MODULE_DESCRIPTION(DRIVER_DESC);
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/tiny/panel-mipi-dbi.c b/drivers/gpu/drm/tiny/panel-mipi-dbi.c
index 955a61d..03a7d56 100644
--- a/drivers/gpu/drm/tiny/panel-mipi-dbi.c
+++ b/drivers/gpu/drm/tiny/panel-mipi-dbi.c
@@ -16,7 +16,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/tiny/repaper.c b/drivers/gpu/drm/tiny/repaper.c
index e62f4d1..c2677d0 100644
--- a/drivers/gpu/drm/tiny/repaper.c
+++ b/drivers/gpu/drm/tiny/repaper.c
@@ -26,7 +26,7 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_atomic_helper.h>
diff --git a/drivers/gpu/drm/tiny/simpledrm.c b/drivers/gpu/drm/tiny/simpledrm.c
index 1848977..162eb44 100644
--- a/drivers/gpu/drm/tiny/simpledrm.c
+++ b/drivers/gpu/drm/tiny/simpledrm.c
@@ -11,10 +11,11 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_state_helper.h>
#include <drm/drm_connector.h>
+#include <drm/drm_crtc_helper.h>
#include <drm/drm_damage_helper.h>
#include <drm/drm_device.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
@@ -545,19 +546,6 @@ static enum drm_mode_status simpledrm_crtc_helper_mode_valid(struct drm_crtc *cr
return drm_crtc_helper_mode_valid_fixed(crtc, mode, &sdev->mode);
}
-static int simpledrm_crtc_helper_atomic_check(struct drm_crtc *crtc,
- struct drm_atomic_state *new_state)
-{
- struct drm_crtc_state *new_crtc_state = drm_atomic_get_new_crtc_state(new_state, crtc);
- int ret;
-
- ret = drm_atomic_helper_check_crtc_state(new_crtc_state, false);
- if (ret)
- return ret;
-
- return drm_atomic_add_affected_planes(new_state, crtc);
-}
-
/*
* The CRTC is always enabled. Screen updates are performed by
* the primary plane's atomic_update function. Disabling clears
@@ -565,7 +553,7 @@ static int simpledrm_crtc_helper_atomic_check(struct drm_crtc *crtc,
*/
static const struct drm_crtc_helper_funcs simpledrm_crtc_helper_funcs = {
.mode_valid = simpledrm_crtc_helper_mode_valid,
- .atomic_check = simpledrm_crtc_helper_atomic_check,
+ .atomic_check = drm_crtc_helper_atomic_check,
};
static const struct drm_crtc_funcs simpledrm_crtc_funcs = {
diff --git a/drivers/gpu/drm/tiny/st7586.c b/drivers/gpu/drm/tiny/st7586.c
index b6f620b..ce57fa9 100644
--- a/drivers/gpu/drm/tiny/st7586.c
+++ b/drivers/gpu/drm/tiny/st7586.c
@@ -16,7 +16,7 @@
#include <drm/drm_damage_helper.h>
#include <drm/drm_drv.h>
#include <drm/drm_fb_dma_helper.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_format_helper.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem_atomic_helper.h>
diff --git a/drivers/gpu/drm/tiny/st7735r.c b/drivers/gpu/drm/tiny/st7735r.c
index c36ba08..15d9cf2 100644
--- a/drivers/gpu/drm/tiny/st7735r.c
+++ b/drivers/gpu/drm/tiny/st7735r.c
@@ -18,7 +18,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c
index 7c8e8be..c3f4b33 100644
--- a/drivers/gpu/drm/ttm/ttm_bo.c
+++ b/drivers/gpu/drm/ttm/ttm_bo.c
@@ -51,9 +51,6 @@ static void ttm_bo_mem_space_debug(struct ttm_buffer_object *bo,
struct ttm_resource_manager *man;
int i, mem_type;
- drm_printf(&p, "No space for %p (%lu pages, %zuK, %zuM)\n",
- bo, bo->resource->num_pages, bo->base.size >> 10,
- bo->base.size >> 20);
for (i = 0; i < placement->num_placement; i++) {
mem_type = placement->placement[i].mem_type;
drm_printf(&p, " placement[%d]=0x%08X (%d)\n",
diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c
index fa04e62..ba3aa0a 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_util.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_util.c
@@ -173,7 +173,7 @@ int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
clear = src_iter->ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm));
if (!(clear && ttm && !(ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC)))
- ttm_move_memcpy(clear, dst_mem->num_pages, dst_iter, src_iter);
+ ttm_move_memcpy(clear, ttm->num_pages, dst_iter, src_iter);
if (!src_iter->ops->maps_tt)
ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, src_mem);
@@ -357,9 +357,9 @@ int ttm_bo_kmap(struct ttm_buffer_object *bo,
map->virtual = NULL;
map->bo = bo;
- if (num_pages > bo->resource->num_pages)
+ if (num_pages > PFN_UP(bo->resource->size))
return -EINVAL;
- if ((start_page + num_pages) > bo->resource->num_pages)
+ if ((start_page + num_pages) > PFN_UP(bo->resource->size))
return -EINVAL;
ret = ttm_mem_io_reserve(bo->bdev, bo->resource);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 3811931..5a3e4b89 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -217,7 +217,7 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf,
page_last = vma_pages(vma) + vma->vm_pgoff -
drm_vma_node_start(&bo->base.vma_node);
- if (unlikely(page_offset >= bo->resource->num_pages))
+ if (unlikely(page_offset >= PFN_UP(bo->base.size)))
return VM_FAULT_SIGBUS;
prot = ttm_io_prot(bo, bo->resource, prot);
@@ -412,7 +412,7 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr,
<< PAGE_SHIFT);
int ret;
- if (len < 1 || (offset + len) >> PAGE_SHIFT > bo->resource->num_pages)
+ if (len < 1 || (offset + len) > bo->base.size)
return -EIO;
ret = ttm_bo_reserve(bo, true, false, NULL);
diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c
index 21b6163..9f6764b 100644
--- a/drivers/gpu/drm/ttm/ttm_pool.c
+++ b/drivers/gpu/drm/ttm/ttm_pool.c
@@ -344,6 +344,28 @@ static unsigned int ttm_pool_page_order(struct ttm_pool *pool, struct page *p)
return p->private;
}
+/* Called when we got a page, either from a pool or newly allocated */
+static int ttm_pool_page_allocated(struct ttm_pool *pool, unsigned int order,
+ struct page *p, dma_addr_t **dma_addr,
+ unsigned long *num_pages,
+ struct page ***pages)
+{
+ unsigned int i;
+ int r;
+
+ if (*dma_addr) {
+ r = ttm_pool_map(pool, order, p, dma_addr);
+ if (r)
+ return r;
+ }
+
+ *num_pages -= 1 << order;
+ for (i = 1 << order; i; --i, ++(*pages), ++p)
+ **pages = p;
+
+ return 0;
+}
+
/**
* ttm_pool_alloc - Fill a ttm_tt object
*
@@ -385,17 +407,47 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
for (order = min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages));
num_pages;
order = min_t(unsigned int, order, __fls(num_pages))) {
- bool apply_caching = false;
struct ttm_pool_type *pt;
pt = ttm_pool_select_type(pool, tt->caching, order);
p = pt ? ttm_pool_type_take(pt) : NULL;
if (p) {
- apply_caching = true;
- } else {
- p = ttm_pool_alloc_page(pool, gfp_flags, order);
- if (p && PageHighMem(p))
- apply_caching = true;
+ r = ttm_pool_apply_caching(caching, pages,
+ tt->caching);
+ if (r)
+ goto error_free_page;
+
+ do {
+ r = ttm_pool_page_allocated(pool, order, p,
+ &dma_addr,
+ &num_pages,
+ &pages);
+ if (r)
+ goto error_free_page;
+
+ if (num_pages < (1 << order))
+ break;
+
+ p = ttm_pool_type_take(pt);
+ } while (p);
+ caching = pages;
+ }
+
+ while (num_pages >= (1 << order) &&
+ (p = ttm_pool_alloc_page(pool, gfp_flags, order))) {
+
+ if (PageHighMem(p)) {
+ r = ttm_pool_apply_caching(caching, pages,
+ tt->caching);
+ if (r)
+ goto error_free_page;
+ }
+ r = ttm_pool_page_allocated(pool, order, p, &dma_addr,
+ &num_pages, &pages);
+ if (r)
+ goto error_free_page;
+ if (PageHighMem(p))
+ caching = pages;
}
if (!p) {
@@ -406,24 +458,6 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
r = -ENOMEM;
goto error_free_all;
}
-
- if (apply_caching) {
- r = ttm_pool_apply_caching(caching, pages,
- tt->caching);
- if (r)
- goto error_free_page;
- caching = pages + (1 << order);
- }
-
- if (dma_addr) {
- r = ttm_pool_map(pool, order, p, &dma_addr);
- if (r)
- goto error_free_page;
- }
-
- num_pages -= 1 << order;
- for (i = 1 << order; i; --i)
- *(pages++) = p++;
}
r = ttm_pool_apply_caching(caching, pages, tt->caching);
diff --git a/drivers/gpu/drm/ttm/ttm_range_manager.c b/drivers/gpu/drm/ttm/ttm_range_manager.c
index 4cfef2b..0a8bc0b 100644
--- a/drivers/gpu/drm/ttm/ttm_range_manager.c
+++ b/drivers/gpu/drm/ttm/ttm_range_manager.c
@@ -83,7 +83,7 @@ static int ttm_range_man_alloc(struct ttm_resource_manager *man,
spin_lock(&rman->lock);
ret = drm_mm_insert_node_in_range(mm, &node->mm_nodes[0],
- node->base.num_pages,
+ PFN_UP(node->base.size),
bo->page_alignment, 0,
place->fpfn, lpfn, mode);
spin_unlock(&rman->lock);
@@ -229,7 +229,6 @@ int ttm_range_man_fini_nocheck(struct ttm_device *bdev,
return ret;
spin_lock(&rman->lock);
- drm_mm_clean(mm);
drm_mm_takedown(mm);
spin_unlock(&rman->lock);
diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c
index a729c32a..328391b 100644
--- a/drivers/gpu/drm/ttm/ttm_resource.c
+++ b/drivers/gpu/drm/ttm/ttm_resource.c
@@ -177,7 +177,7 @@ void ttm_resource_init(struct ttm_buffer_object *bo,
struct ttm_resource_manager *man;
res->start = 0;
- res->num_pages = PFN_UP(bo->base.size);
+ res->size = bo->base.size;
res->mem_type = place->mem_type;
res->placement = place->flags;
res->bus.addr = NULL;
@@ -192,7 +192,7 @@ void ttm_resource_init(struct ttm_buffer_object *bo,
list_add_tail(&res->lru, &bo->bdev->pinned);
else
list_add_tail(&res->lru, &man->lru[bo->priority]);
- man->usage += res->num_pages << PAGE_SHIFT;
+ man->usage += res->size;
spin_unlock(&bo->bdev->lru_lock);
}
EXPORT_SYMBOL(ttm_resource_init);
@@ -214,7 +214,7 @@ void ttm_resource_fini(struct ttm_resource_manager *man,
spin_lock(&bdev->lru_lock);
list_del_init(&res->lru);
- man->usage -= res->num_pages << PAGE_SHIFT;
+ man->usage -= res->size;
spin_unlock(&bdev->lru_lock);
}
EXPORT_SYMBOL(ttm_resource_fini);
@@ -665,17 +665,15 @@ ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io,
iosys_map_set_vaddr(&iter_io->dmap, mem->bus.addr);
iter_io->needs_unmap = false;
} else {
- size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
-
iter_io->needs_unmap = true;
memset(&iter_io->dmap, 0, sizeof(iter_io->dmap));
if (mem->bus.caching == ttm_write_combined)
iosys_map_set_vaddr_iomem(&iter_io->dmap,
ioremap_wc(mem->bus.offset,
- bus_size));
+ mem->size));
else if (mem->bus.caching == ttm_cached)
iosys_map_set_vaddr(&iter_io->dmap,
- memremap(mem->bus.offset, bus_size,
+ memremap(mem->bus.offset, mem->size,
MEMREMAP_WB |
MEMREMAP_WT |
MEMREMAP_WC));
@@ -684,7 +682,7 @@ ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io,
if (iosys_map_is_null(&iter_io->dmap))
iosys_map_set_vaddr_iomem(&iter_io->dmap,
ioremap(mem->bus.offset,
- bus_size));
+ mem->size));
if (iosys_map_is_null(&iter_io->dmap)) {
ret = -ENOMEM;
diff --git a/drivers/gpu/drm/tve200/tve200_drv.c b/drivers/gpu/drm/tve200/tve200_drv.c
index 04db72e..0d05c38 100644
--- a/drivers/gpu/drm/tve200/tve200_drv.c
+++ b/drivers/gpu/drm/tve200/tve200_drv.c
@@ -32,6 +32,7 @@
#include <linux/irq.h>
#include <linux/io.h>
#include <linux/module.h>
+#include <linux/of.h>
#include <linux/platform_device.h>
#include <linux/shmem_fs.h>
#include <linux/slab.h>
@@ -39,7 +40,7 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_dma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_module.h>
diff --git a/drivers/gpu/drm/udl/Makefile b/drivers/gpu/drm/udl/Makefile
index 24d61f6..3f6db17 100644
--- a/drivers/gpu/drm/udl/Makefile
+++ b/drivers/gpu/drm/udl/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
-udl-y := udl_drv.o udl_modeset.o udl_connector.o udl_main.o udl_transfer.o
+udl-y := udl_drv.o udl_modeset.o udl_main.o udl_transfer.o
obj-$(CONFIG_DRM_UDL) := udl.o
diff --git a/drivers/gpu/drm/udl/udl_connector.c b/drivers/gpu/drm/udl/udl_connector.c
deleted file mode 100644
index fade4c7..0000000
--- a/drivers/gpu/drm/udl/udl_connector.c
+++ /dev/null
@@ -1,139 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (C) 2012 Red Hat
- * based in parts on udlfb.c:
- * Copyright (C) 2009 Roberto De Ioris <roberto@unbit.it>
- * Copyright (C) 2009 Jaya Kumar <jayakumar.lkml@gmail.com>
- * Copyright (C) 2009 Bernie Thompson <bernie@plugable.com>
- */
-
-#include <drm/drm_atomic_state_helper.h>
-#include <drm/drm_edid.h>
-#include <drm/drm_crtc_helper.h>
-#include <drm/drm_probe_helper.h>
-
-#include "udl_connector.h"
-#include "udl_drv.h"
-
-static int udl_get_edid_block(void *data, u8 *buf, unsigned int block,
- size_t len)
-{
- int ret, i;
- u8 *read_buff;
- struct udl_device *udl = data;
- struct usb_device *udev = udl_to_usb_device(udl);
-
- read_buff = kmalloc(2, GFP_KERNEL);
- if (!read_buff)
- return -1;
-
- for (i = 0; i < len; i++) {
- int bval = (i + block * EDID_LENGTH) << 8;
- ret = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
- 0x02, (0x80 | (0x02 << 5)), bval,
- 0xA1, read_buff, 2, 1000);
- if (ret < 1) {
- DRM_ERROR("Read EDID byte %d failed err %x\n", i, ret);
- kfree(read_buff);
- return -1;
- }
- buf[i] = read_buff[1];
- }
-
- kfree(read_buff);
- return 0;
-}
-
-static int udl_get_modes(struct drm_connector *connector)
-{
- struct udl_drm_connector *udl_connector =
- container_of(connector,
- struct udl_drm_connector,
- connector);
-
- drm_connector_update_edid_property(connector, udl_connector->edid);
- if (udl_connector->edid)
- return drm_add_edid_modes(connector, udl_connector->edid);
- return 0;
-}
-
-static enum drm_mode_status udl_mode_valid(struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- struct udl_device *udl = to_udl(connector->dev);
- if (!udl->sku_pixel_limit)
- return 0;
-
- if (mode->vdisplay * mode->hdisplay > udl->sku_pixel_limit)
- return MODE_VIRTUAL_Y;
-
- return 0;
-}
-
-static enum drm_connector_status
-udl_detect(struct drm_connector *connector, bool force)
-{
- struct udl_device *udl = to_udl(connector->dev);
- struct udl_drm_connector *udl_connector =
- container_of(connector,
- struct udl_drm_connector,
- connector);
-
- /* cleanup previous edid */
- if (udl_connector->edid != NULL) {
- kfree(udl_connector->edid);
- udl_connector->edid = NULL;
- }
-
- udl_connector->edid = drm_do_get_edid(connector, udl_get_edid_block, udl);
- if (!udl_connector->edid)
- return connector_status_disconnected;
-
- return connector_status_connected;
-}
-
-static void udl_connector_destroy(struct drm_connector *connector)
-{
- struct udl_drm_connector *udl_connector =
- container_of(connector,
- struct udl_drm_connector,
- connector);
-
- drm_connector_cleanup(connector);
- kfree(udl_connector->edid);
- kfree(connector);
-}
-
-static const struct drm_connector_helper_funcs udl_connector_helper_funcs = {
- .get_modes = udl_get_modes,
- .mode_valid = udl_mode_valid,
-};
-
-static const struct drm_connector_funcs udl_connector_funcs = {
- .reset = drm_atomic_helper_connector_reset,
- .detect = udl_detect,
- .fill_modes = drm_helper_probe_single_connector_modes,
- .destroy = udl_connector_destroy,
- .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
-};
-
-struct drm_connector *udl_connector_init(struct drm_device *dev)
-{
- struct udl_drm_connector *udl_connector;
- struct drm_connector *connector;
-
- udl_connector = kzalloc(sizeof(struct udl_drm_connector), GFP_KERNEL);
- if (!udl_connector)
- return ERR_PTR(-ENOMEM);
-
- connector = &udl_connector->connector;
- drm_connector_init(dev, connector, &udl_connector_funcs,
- DRM_MODE_CONNECTOR_VGA);
- drm_connector_helper_add(connector, &udl_connector_helper_funcs);
-
- connector->polled = DRM_CONNECTOR_POLL_HPD |
- DRM_CONNECTOR_POLL_CONNECT | DRM_CONNECTOR_POLL_DISCONNECT;
-
- return connector;
-}
diff --git a/drivers/gpu/drm/udl/udl_connector.h b/drivers/gpu/drm/udl/udl_connector.h
deleted file mode 100644
index 7f2d392..0000000
--- a/drivers/gpu/drm/udl/udl_connector.h
+++ /dev/null
@@ -1,15 +0,0 @@
-#ifndef __UDL_CONNECTOR_H__
-#define __UDL_CONNECTOR_H__
-
-#include <drm/drm_crtc.h>
-
-struct edid;
-
-struct udl_drm_connector {
- struct drm_connector connector;
- /* last udl_detect edid */
- struct edid *edid;
-};
-
-
-#endif //__UDL_CONNECTOR_H__
diff --git a/drivers/gpu/drm/udl/udl_drv.c b/drivers/gpu/drm/udl/udl_drv.c
index 91effdce..e813521 100644
--- a/drivers/gpu/drm/udl/udl_drv.c
+++ b/drivers/gpu/drm/udl/udl_drv.c
@@ -7,7 +7,7 @@
#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_managed.h>
diff --git a/drivers/gpu/drm/udl/udl_drv.h b/drivers/gpu/drm/udl/udl_drv.h
index b4cc7cc..282ebd6 100644
--- a/drivers/gpu/drm/udl/udl_drv.h
+++ b/drivers/gpu/drm/udl/udl_drv.h
@@ -14,10 +14,13 @@
#include <linux/mm_types.h>
#include <linux/usb.h>
+#include <drm/drm_connector.h>
+#include <drm/drm_crtc.h>
#include <drm/drm_device.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem.h>
-#include <drm/drm_simple_kms_helper.h>
+#include <drm/drm_plane.h>
struct drm_mode_create_dumb;
@@ -46,21 +49,31 @@ struct urb_list {
size_t size;
};
+struct udl_connector {
+ struct drm_connector connector;
+ /* last udl_detect edid */
+ struct edid *edid;
+};
+
+static inline struct udl_connector *to_udl_connector(struct drm_connector *connector)
+{
+ return container_of(connector, struct udl_connector, connector);
+}
+
struct udl_device {
struct drm_device drm;
struct device *dev;
struct device *dmadev;
- struct drm_simple_display_pipe display_pipe;
+ struct drm_plane primary_plane;
+ struct drm_crtc crtc;
+ struct drm_encoder encoder;
struct mutex gem_lock;
int sku_pixel_limit;
struct urb_list urbs;
-
- char mode_buf[1024];
- uint32_t mode_buf_len;
};
#define to_udl(x) container_of(x, struct udl_device, drm)
@@ -89,23 +102,4 @@ int udl_render_hline(struct drm_device *dev, int log_bpp, struct urb **urb_ptr,
int udl_drop_usb(struct drm_device *dev);
int udl_select_std_channel(struct udl_device *udl);
-#define CMD_WRITE_RAW8 "\xAF\x60" /**< 8 bit raw write command. */
-#define CMD_WRITE_RL8 "\xAF\x61" /**< 8 bit run length command. */
-#define CMD_WRITE_COPY8 "\xAF\x62" /**< 8 bit copy command. */
-#define CMD_WRITE_RLX8 "\xAF\x63" /**< 8 bit extended run length command. */
-
-#define CMD_WRITE_RAW16 "\xAF\x68" /**< 16 bit raw write command. */
-#define CMD_WRITE_RL16 "\xAF\x69" /**< 16 bit run length command. */
-#define CMD_WRITE_COPY16 "\xAF\x6A" /**< 16 bit copy command. */
-#define CMD_WRITE_RLX16 "\xAF\x6B" /**< 16 bit extended run length command. */
-
-/* On/Off for driving the DisplayLink framebuffer to the display */
-#define UDL_REG_BLANK_MODE 0x1f
-
-#define UDL_BLANK_MODE_ON 0x00 /* hsync and vsync on, visible */
-#define UDL_BLANK_MODE_BLANKED 0x01 /* hsync and vsync on, blanked */
-#define UDL_BLANK_MODE_VSYNC_OFF 0x03 /* vsync off, blanked */
-#define UDL_BLANK_MODE_HSYNC_OFF 0x05 /* hsync off, blanked */
-#define UDL_BLANK_MODE_POWERDOWN 0x07 /* powered off; requires modeset */
-
#endif
diff --git a/drivers/gpu/drm/udl/udl_modeset.c b/drivers/gpu/drm/udl/udl_modeset.c
index ec6876f..4b79d44 100644
--- a/drivers/gpu/drm/udl/udl_modeset.c
+++ b/drivers/gpu/drm/udl/udl_modeset.c
@@ -8,70 +8,91 @@
* Copyright (C) 2009 Bernie Thompson <bernie@plugable.com>
*/
+#include <linux/bitfield.h>
+
+#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_damage_helper.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_edid.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_atomic_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_modeset_helper_vtables.h>
+#include <drm/drm_plane_helper.h>
+#include <drm/drm_probe_helper.h>
#include <drm/drm_vblank.h>
#include "udl_drv.h"
-
-#define UDL_COLOR_DEPTH_16BPP 0
+#include "udl_proto.h"
/*
- * All DisplayLink bulk operations start with 0xAF, followed by specific code
- * All operations are written to buffers which then later get sent to device
+ * All DisplayLink bulk operations start with 0xaf (UDL_MSG_BULK), followed by
+ * a specific command code. All operations are written to a command buffer, which
+ * the driver sends to the device.
*/
static char *udl_set_register(char *buf, u8 reg, u8 val)
{
- *buf++ = 0xAF;
- *buf++ = 0x20;
+ *buf++ = UDL_MSG_BULK;
+ *buf++ = UDL_CMD_WRITEREG;
*buf++ = reg;
*buf++ = val;
+
return buf;
}
static char *udl_vidreg_lock(char *buf)
{
- return udl_set_register(buf, 0xFF, 0x00);
+ return udl_set_register(buf, UDL_REG_VIDREG, UDL_VIDREG_LOCK);
}
static char *udl_vidreg_unlock(char *buf)
{
- return udl_set_register(buf, 0xFF, 0xFF);
+ return udl_set_register(buf, UDL_REG_VIDREG, UDL_VIDREG_UNLOCK);
}
static char *udl_set_blank_mode(char *buf, u8 mode)
{
- return udl_set_register(buf, UDL_REG_BLANK_MODE, mode);
+ return udl_set_register(buf, UDL_REG_BLANKMODE, mode);
}
static char *udl_set_color_depth(char *buf, u8 selection)
{
- return udl_set_register(buf, 0x00, selection);
+ return udl_set_register(buf, UDL_REG_COLORDEPTH, selection);
}
-static char *udl_set_base16bpp(char *wrptr, u32 base)
+static char *udl_set_base16bpp(char *buf, u32 base)
{
- /* the base pointer is 16 bits wide, 0x20 is hi byte. */
- wrptr = udl_set_register(wrptr, 0x20, base >> 16);
- wrptr = udl_set_register(wrptr, 0x21, base >> 8);
- return udl_set_register(wrptr, 0x22, base);
+ /* the base pointer is 24 bits wide, 0x20 is hi byte. */
+ u8 reg20 = FIELD_GET(UDL_BASE_ADDR2_MASK, base);
+ u8 reg21 = FIELD_GET(UDL_BASE_ADDR1_MASK, base);
+ u8 reg22 = FIELD_GET(UDL_BASE_ADDR0_MASK, base);
+
+ buf = udl_set_register(buf, UDL_REG_BASE16BPP_ADDR2, reg20);
+ buf = udl_set_register(buf, UDL_REG_BASE16BPP_ADDR1, reg21);
+ buf = udl_set_register(buf, UDL_REG_BASE16BPP_ADDR0, reg22);
+
+ return buf;
}
/*
* DisplayLink HW has separate 16bpp and 8bpp framebuffers.
* In 24bpp modes, the low 323 RGB bits go in the 8bpp framebuffer
*/
-static char *udl_set_base8bpp(char *wrptr, u32 base)
+static char *udl_set_base8bpp(char *buf, u32 base)
{
- wrptr = udl_set_register(wrptr, 0x26, base >> 16);
- wrptr = udl_set_register(wrptr, 0x27, base >> 8);
- return udl_set_register(wrptr, 0x28, base);
+ /* the base pointer is 24 bits wide, 0x26 is hi byte. */
+ u8 reg26 = FIELD_GET(UDL_BASE_ADDR2_MASK, base);
+ u8 reg27 = FIELD_GET(UDL_BASE_ADDR1_MASK, base);
+ u8 reg28 = FIELD_GET(UDL_BASE_ADDR0_MASK, base);
+
+ buf = udl_set_register(buf, UDL_REG_BASE8BPP_ADDR2, reg26);
+ buf = udl_set_register(buf, UDL_REG_BASE8BPP_ADDR1, reg27);
+ buf = udl_set_register(buf, UDL_REG_BASE8BPP_ADDR0, reg28);
+
+ return buf;
}
static char *udl_set_register_16(char *wrptr, u8 reg, u16 value)
@@ -122,84 +143,46 @@ static char *udl_set_register_lfsr16(char *wrptr, u8 reg, u16 value)
}
/*
- * This takes a standard fbdev screeninfo struct and all of its monitor mode
- * details and converts them into the DisplayLink equivalent register commands.
- ERR(vreg(dev, 0x00, (color_depth == 16) ? 0 : 1));
- ERR(vreg_lfsr16(dev, 0x01, xDisplayStart));
- ERR(vreg_lfsr16(dev, 0x03, xDisplayEnd));
- ERR(vreg_lfsr16(dev, 0x05, yDisplayStart));
- ERR(vreg_lfsr16(dev, 0x07, yDisplayEnd));
- ERR(vreg_lfsr16(dev, 0x09, xEndCount));
- ERR(vreg_lfsr16(dev, 0x0B, hSyncStart));
- ERR(vreg_lfsr16(dev, 0x0D, hSyncEnd));
- ERR(vreg_big_endian(dev, 0x0F, hPixels));
- ERR(vreg_lfsr16(dev, 0x11, yEndCount));
- ERR(vreg_lfsr16(dev, 0x13, vSyncStart));
- ERR(vreg_lfsr16(dev, 0x15, vSyncEnd));
- ERR(vreg_big_endian(dev, 0x17, vPixels));
- ERR(vreg_little_endian(dev, 0x1B, pixelClock5KHz));
-
- ERR(vreg(dev, 0x1F, 0));
-
- ERR(vbuf(dev, WRITE_VIDREG_UNLOCK, DSIZEOF(WRITE_VIDREG_UNLOCK)));
+ * Takes a DRM display mode and converts it into the DisplayLink
+ * equivalent register commands.
*/
-static char *udl_set_vid_cmds(char *wrptr, struct drm_display_mode *mode)
+static char *udl_set_display_mode(char *buf, struct drm_display_mode *mode)
{
- u16 xds, yds;
- u16 xde, yde;
- u16 yec;
+ u16 reg01 = mode->crtc_htotal - mode->crtc_hsync_start;
+ u16 reg03 = reg01 + mode->crtc_hdisplay;
+ u16 reg05 = mode->crtc_vtotal - mode->crtc_vsync_start;
+ u16 reg07 = reg05 + mode->crtc_vdisplay;
+ u16 reg09 = mode->crtc_htotal - 1;
+ u16 reg0b = 1; /* libdlo hardcodes hsync start to 1 */
+ u16 reg0d = mode->crtc_hsync_end - mode->crtc_hsync_start + 1;
+ u16 reg0f = mode->hdisplay;
+ u16 reg11 = mode->crtc_vtotal;
+ u16 reg13 = 0; /* libdlo hardcodes vsync start to 0 */
+ u16 reg15 = mode->crtc_vsync_end - mode->crtc_vsync_start;
+ u16 reg17 = mode->crtc_vdisplay;
+ u16 reg1b = mode->clock / 5;
- /* x display start */
- xds = mode->crtc_htotal - mode->crtc_hsync_start;
- wrptr = udl_set_register_lfsr16(wrptr, 0x01, xds);
- /* x display end */
- xde = xds + mode->crtc_hdisplay;
- wrptr = udl_set_register_lfsr16(wrptr, 0x03, xde);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_XDISPLAYSTART, reg01);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_XDISPLAYEND, reg03);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_YDISPLAYSTART, reg05);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_YDISPLAYEND, reg07);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_XENDCOUNT, reg09);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_HSYNCSTART, reg0b);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_HSYNCEND, reg0d);
+ buf = udl_set_register_16(buf, UDL_REG_HPIXELS, reg0f);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_YENDCOUNT, reg11);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_VSYNCSTART, reg13);
+ buf = udl_set_register_lfsr16(buf, UDL_REG_VSYNCEND, reg15);
+ buf = udl_set_register_16(buf, UDL_REG_VPIXELS, reg17);
+ buf = udl_set_register_16be(buf, UDL_REG_PIXELCLOCK5KHZ, reg1b);
- /* y display start */
- yds = mode->crtc_vtotal - mode->crtc_vsync_start;
- wrptr = udl_set_register_lfsr16(wrptr, 0x05, yds);
- /* y display end */
- yde = yds + mode->crtc_vdisplay;
- wrptr = udl_set_register_lfsr16(wrptr, 0x07, yde);
-
- /* x end count is active + blanking - 1 */
- wrptr = udl_set_register_lfsr16(wrptr, 0x09,
- mode->crtc_htotal - 1);
-
- /* libdlo hardcodes hsync start to 1 */
- wrptr = udl_set_register_lfsr16(wrptr, 0x0B, 1);
-
- /* hsync end is width of sync pulse + 1 */
- wrptr = udl_set_register_lfsr16(wrptr, 0x0D,
- mode->crtc_hsync_end - mode->crtc_hsync_start + 1);
-
- /* hpixels is active pixels */
- wrptr = udl_set_register_16(wrptr, 0x0F, mode->hdisplay);
-
- /* yendcount is vertical active + vertical blanking */
- yec = mode->crtc_vtotal;
- wrptr = udl_set_register_lfsr16(wrptr, 0x11, yec);
-
- /* libdlo hardcodes vsync start to 0 */
- wrptr = udl_set_register_lfsr16(wrptr, 0x13, 0);
-
- /* vsync end is width of vsync pulse */
- wrptr = udl_set_register_lfsr16(wrptr, 0x15, mode->crtc_vsync_end - mode->crtc_vsync_start);
-
- /* vpixels is active pixels */
- wrptr = udl_set_register_16(wrptr, 0x17, mode->crtc_vdisplay);
-
- wrptr = udl_set_register_16be(wrptr, 0x1B,
- mode->clock / 5);
-
- return wrptr;
+ return buf;
}
static char *udl_dummy_render(char *wrptr)
{
- *wrptr++ = 0xAF;
- *wrptr++ = 0x6A; /* copy */
+ *wrptr++ = UDL_MSG_BULK;
+ *wrptr++ = UDL_CMD_WRITECOPY16;
*wrptr++ = 0x00; /* from addr */
*wrptr++ = 0x00;
*wrptr++ = 0x00;
@@ -210,31 +193,6 @@ static char *udl_dummy_render(char *wrptr)
return wrptr;
}
-static int udl_crtc_write_mode_to_hw(struct drm_crtc *crtc)
-{
- struct drm_device *dev = crtc->dev;
- struct udl_device *udl = to_udl(dev);
- struct urb *urb;
- char *buf;
- int retval;
-
- if (udl->mode_buf_len == 0) {
- DRM_ERROR("No mode set\n");
- return -EINVAL;
- }
-
- urb = udl_get_urb(dev);
- if (!urb)
- return -ENOMEM;
-
- buf = (char *)urb->transfer_buffer;
-
- memcpy(buf, udl->mode_buf, udl->mode_buf_len);
- retval = udl_submit_urb(dev, urb, udl->mode_buf_len);
- DRM_DEBUG("write mode info %d\n", udl->mode_buf_len);
- return retval;
-}
-
static long udl_log_cpp(unsigned int cpp)
{
if (WARN_ON(!is_power_of_2(cpp)))
@@ -258,15 +216,9 @@ static int udl_handle_damage(struct drm_framebuffer *fb,
return ret;
log_bpp = ret;
- ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE);
- if (ret)
- return ret;
-
urb = udl_get_urb(dev);
- if (!urb) {
- ret = -ENOMEM;
- goto out_drm_gem_fb_end_cpu_access;
- }
+ if (!urb)
+ return -ENOMEM;
cmd = urb->transfer_buffer;
for (i = clip->y1; i < clip->y2; i++) {
@@ -278,145 +230,339 @@ static int udl_handle_damage(struct drm_framebuffer *fb,
&cmd, byte_offset, dev_byte_offset,
byte_width);
if (ret)
- goto out_drm_gem_fb_end_cpu_access;
+ return ret;
}
if (cmd > (char *)urb->transfer_buffer) {
/* Send partial buffer remaining before exiting */
int len;
if (cmd < (char *)urb->transfer_buffer + urb->transfer_buffer_length)
- *cmd++ = 0xAF;
+ *cmd++ = UDL_MSG_BULK;
len = cmd - (char *)urb->transfer_buffer;
ret = udl_submit_urb(dev, urb, len);
} else {
udl_urb_completion(urb);
}
- ret = 0;
-
-out_drm_gem_fb_end_cpu_access:
- drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE);
- return ret;
+ return 0;
}
/*
- * Simple display pipeline
+ * Primary plane
*/
-static const uint32_t udl_simple_display_pipe_formats[] = {
+static const uint32_t udl_primary_plane_formats[] = {
DRM_FORMAT_RGB565,
DRM_FORMAT_XRGB8888,
};
-static enum drm_mode_status
-udl_simple_display_pipe_mode_valid(struct drm_simple_display_pipe *pipe,
- const struct drm_display_mode *mode)
-{
- return MODE_OK;
-}
+static const uint64_t udl_primary_plane_fmtmods[] = {
+ DRM_FORMAT_MOD_LINEAR,
+ DRM_FORMAT_MOD_INVALID
+};
-static void
-udl_simple_display_pipe_enable(struct drm_simple_display_pipe *pipe,
- struct drm_crtc_state *crtc_state,
- struct drm_plane_state *plane_state)
+static void udl_primary_plane_helper_atomic_update(struct drm_plane *plane,
+ struct drm_atomic_state *state)
{
- struct drm_crtc *crtc = &pipe->crtc;
- struct drm_device *dev = crtc->dev;
- struct drm_framebuffer *fb = plane_state->fb;
- struct udl_device *udl = to_udl(dev);
- struct drm_display_mode *mode = &crtc_state->mode;
+ struct drm_device *dev = plane->dev;
+ struct drm_plane_state *plane_state = drm_atomic_get_new_plane_state(state, plane);
struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(plane_state);
- struct drm_rect clip = DRM_RECT_INIT(0, 0, fb->width, fb->height);
- char *buf;
- char *wrptr;
- int color_depth = UDL_COLOR_DEPTH_16BPP;
+ struct drm_framebuffer *fb = plane_state->fb;
+ struct drm_plane_state *old_plane_state = drm_atomic_get_old_plane_state(state, plane);
+ struct drm_atomic_helper_damage_iter iter;
+ struct drm_rect damage;
+ int ret, idx;
- buf = (char *)udl->mode_buf;
+ if (!fb)
+ return; /* no framebuffer; plane is disabled */
- /* This first section has to do with setting the base address on the
- * controller associated with the display. There are 2 base
- * pointers, currently, we only use the 16 bpp segment.
- */
- wrptr = udl_vidreg_lock(buf);
- wrptr = udl_set_color_depth(wrptr, color_depth);
- /* set base for 16bpp segment to 0 */
- wrptr = udl_set_base16bpp(wrptr, 0);
- /* set base for 8bpp segment to end of fb */
- wrptr = udl_set_base8bpp(wrptr, 2 * mode->vdisplay * mode->hdisplay);
+ ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE);
+ if (ret)
+ return;
- wrptr = udl_set_vid_cmds(wrptr, mode);
- wrptr = udl_set_blank_mode(wrptr, UDL_BLANK_MODE_ON);
- wrptr = udl_vidreg_unlock(wrptr);
+ if (!drm_dev_enter(dev, &idx))
+ goto out_drm_gem_fb_end_cpu_access;
- wrptr = udl_dummy_render(wrptr);
+ drm_atomic_helper_damage_iter_init(&iter, old_plane_state, plane_state);
+ drm_atomic_for_each_plane_damage(&iter, &damage) {
+ udl_handle_damage(fb, &shadow_plane_state->data[0], &damage);
+ }
- udl->mode_buf_len = wrptr - buf;
+ drm_dev_exit(idx);
- udl_handle_damage(fb, &shadow_plane_state->data[0], &clip);
-
- /* enable display */
- udl_crtc_write_mode_to_hw(crtc);
+out_drm_gem_fb_end_cpu_access:
+ drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE);
}
-static void
-udl_simple_display_pipe_disable(struct drm_simple_display_pipe *pipe)
+static const struct drm_plane_helper_funcs udl_primary_plane_helper_funcs = {
+ DRM_GEM_SHADOW_PLANE_HELPER_FUNCS,
+ .atomic_check = drm_plane_helper_atomic_check,
+ .atomic_update = udl_primary_plane_helper_atomic_update,
+};
+
+static const struct drm_plane_funcs udl_primary_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = drm_plane_cleanup,
+ DRM_GEM_SHADOW_PLANE_FUNCS,
+};
+
+/*
+ * CRTC
+ */
+
+static int udl_crtc_helper_atomic_check(struct drm_crtc *crtc, struct drm_atomic_state *state)
{
- struct drm_crtc *crtc = &pipe->crtc;
+ struct drm_crtc_state *new_crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
+
+ if (!new_crtc_state->enable)
+ return 0;
+
+ return drm_atomic_helper_check_crtc_primary_plane(new_crtc_state);
+}
+
+static void udl_crtc_helper_atomic_enable(struct drm_crtc *crtc, struct drm_atomic_state *state)
+{
struct drm_device *dev = crtc->dev;
+ struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
+ struct drm_display_mode *mode = &crtc_state->mode;
struct urb *urb;
char *buf;
+ int idx;
+
+ if (!drm_dev_enter(dev, &idx))
+ return;
urb = udl_get_urb(dev);
if (!urb)
- return;
+ goto out;
buf = (char *)urb->transfer_buffer;
buf = udl_vidreg_lock(buf);
- buf = udl_set_blank_mode(buf, UDL_BLANK_MODE_POWERDOWN);
+ buf = udl_set_color_depth(buf, UDL_COLORDEPTH_16BPP);
+ /* set base for 16bpp segment to 0 */
+ buf = udl_set_base16bpp(buf, 0);
+ /* set base for 8bpp segment to end of fb */
+ buf = udl_set_base8bpp(buf, 2 * mode->vdisplay * mode->hdisplay);
+ buf = udl_set_display_mode(buf, mode);
+ buf = udl_set_blank_mode(buf, UDL_BLANKMODE_ON);
buf = udl_vidreg_unlock(buf);
buf = udl_dummy_render(buf);
udl_submit_urb(dev, urb, buf - (char *)urb->transfer_buffer);
+
+out:
+ drm_dev_exit(idx);
}
-static void
-udl_simple_display_pipe_update(struct drm_simple_display_pipe *pipe,
- struct drm_plane_state *old_plane_state)
+static void udl_crtc_helper_atomic_disable(struct drm_crtc *crtc, struct drm_atomic_state *state)
{
- struct drm_plane_state *state = pipe->plane.state;
- struct drm_shadow_plane_state *shadow_plane_state = to_drm_shadow_plane_state(state);
- struct drm_framebuffer *fb = state->fb;
- struct drm_rect rect;
+ struct drm_device *dev = crtc->dev;
+ struct urb *urb;
+ char *buf;
+ int idx;
- if (!fb)
+ if (!drm_dev_enter(dev, &idx))
return;
- if (drm_atomic_helper_damage_merged(old_plane_state, state, &rect))
- udl_handle_damage(fb, &shadow_plane_state->data[0], &rect);
+ urb = udl_get_urb(dev);
+ if (!urb)
+ goto out;
+
+ buf = (char *)urb->transfer_buffer;
+ buf = udl_vidreg_lock(buf);
+ buf = udl_set_blank_mode(buf, UDL_BLANKMODE_POWERDOWN);
+ buf = udl_vidreg_unlock(buf);
+ buf = udl_dummy_render(buf);
+
+ udl_submit_urb(dev, urb, buf - (char *)urb->transfer_buffer);
+
+out:
+ drm_dev_exit(idx);
}
-static const struct drm_simple_display_pipe_funcs udl_simple_display_pipe_funcs = {
- .mode_valid = udl_simple_display_pipe_mode_valid,
- .enable = udl_simple_display_pipe_enable,
- .disable = udl_simple_display_pipe_disable,
- .update = udl_simple_display_pipe_update,
- DRM_GEM_SIMPLE_DISPLAY_PIPE_SHADOW_PLANE_FUNCS,
+static const struct drm_crtc_helper_funcs udl_crtc_helper_funcs = {
+ .atomic_check = udl_crtc_helper_atomic_check,
+ .atomic_enable = udl_crtc_helper_atomic_enable,
+ .atomic_disable = udl_crtc_helper_atomic_disable,
};
+static const struct drm_crtc_funcs udl_crtc_funcs = {
+ .reset = drm_atomic_helper_crtc_reset,
+ .destroy = drm_crtc_cleanup,
+ .set_config = drm_atomic_helper_set_config,
+ .page_flip = drm_atomic_helper_page_flip,
+ .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+};
+
+/*
+ * Encoder
+ */
+
+static const struct drm_encoder_funcs udl_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+/*
+ * Connector
+ */
+
+static int udl_connector_helper_get_modes(struct drm_connector *connector)
+{
+ struct udl_connector *udl_connector = to_udl_connector(connector);
+
+ drm_connector_update_edid_property(connector, udl_connector->edid);
+ if (udl_connector->edid)
+ return drm_add_edid_modes(connector, udl_connector->edid);
+
+ return 0;
+}
+
+static const struct drm_connector_helper_funcs udl_connector_helper_funcs = {
+ .get_modes = udl_connector_helper_get_modes,
+};
+
+static int udl_get_edid_block(void *data, u8 *buf, unsigned int block, size_t len)
+{
+ struct udl_device *udl = data;
+ struct drm_device *dev = &udl->drm;
+ struct usb_device *udev = udl_to_usb_device(udl);
+ u8 *read_buff;
+ int ret;
+ size_t i;
+
+ read_buff = kmalloc(2, GFP_KERNEL);
+ if (!read_buff)
+ return -ENOMEM;
+
+ for (i = 0; i < len; i++) {
+ int bval = (i + block * EDID_LENGTH) << 8;
+
+ ret = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
+ 0x02, (0x80 | (0x02 << 5)), bval,
+ 0xA1, read_buff, 2, USB_CTRL_GET_TIMEOUT);
+ if (ret < 0) {
+ drm_err(dev, "Read EDID byte %zu failed err %x\n", i, ret);
+ goto err_kfree;
+ } else if (ret < 1) {
+ ret = -EIO;
+ drm_err(dev, "Read EDID byte %zu failed\n", i);
+ goto err_kfree;
+ }
+
+ buf[i] = read_buff[1];
+ }
+
+ kfree(read_buff);
+
+ return 0;
+
+err_kfree:
+ kfree(read_buff);
+ return ret;
+}
+
+static enum drm_connector_status udl_connector_detect(struct drm_connector *connector, bool force)
+{
+ struct drm_device *dev = connector->dev;
+ struct udl_device *udl = to_udl(dev);
+ struct udl_connector *udl_connector = to_udl_connector(connector);
+ enum drm_connector_status status = connector_status_disconnected;
+ int idx;
+
+ /* cleanup previous EDID */
+ kfree(udl_connector->edid);
+ udl_connector->edid = NULL;
+
+ if (!drm_dev_enter(dev, &idx))
+ return connector_status_disconnected;
+
+ udl_connector->edid = drm_do_get_edid(connector, udl_get_edid_block, udl);
+ if (udl_connector->edid)
+ status = connector_status_connected;
+
+ drm_dev_exit(idx);
+
+ return status;
+}
+
+static void udl_connector_destroy(struct drm_connector *connector)
+{
+ struct udl_connector *udl_connector = to_udl_connector(connector);
+
+ drm_connector_cleanup(connector);
+ kfree(udl_connector->edid);
+ kfree(udl_connector);
+}
+
+static const struct drm_connector_funcs udl_connector_funcs = {
+ .reset = drm_atomic_helper_connector_reset,
+ .detect = udl_connector_detect,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = udl_connector_destroy,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+struct drm_connector *udl_connector_init(struct drm_device *dev)
+{
+ struct udl_connector *udl_connector;
+ struct drm_connector *connector;
+ int ret;
+
+ udl_connector = kzalloc(sizeof(*udl_connector), GFP_KERNEL);
+ if (!udl_connector)
+ return ERR_PTR(-ENOMEM);
+
+ connector = &udl_connector->connector;
+ ret = drm_connector_init(dev, connector, &udl_connector_funcs, DRM_MODE_CONNECTOR_VGA);
+ if (ret)
+ goto err_kfree;
+
+ drm_connector_helper_add(connector, &udl_connector_helper_funcs);
+
+ connector->polled = DRM_CONNECTOR_POLL_HPD |
+ DRM_CONNECTOR_POLL_CONNECT |
+ DRM_CONNECTOR_POLL_DISCONNECT;
+
+ return connector;
+
+err_kfree:
+ kfree(udl_connector);
+ return ERR_PTR(ret);
+}
+
/*
* Modesetting
*/
-static const struct drm_mode_config_funcs udl_mode_funcs = {
+static enum drm_mode_status udl_mode_config_mode_valid(struct drm_device *dev,
+ const struct drm_display_mode *mode)
+{
+ struct udl_device *udl = to_udl(dev);
+
+ if (udl->sku_pixel_limit) {
+ if (mode->vdisplay * mode->hdisplay > udl->sku_pixel_limit)
+ return MODE_MEM;
+ }
+
+ return MODE_OK;
+}
+
+static const struct drm_mode_config_funcs udl_mode_config_funcs = {
.fb_create = drm_gem_fb_create_with_dirty,
+ .mode_valid = udl_mode_config_mode_valid,
.atomic_check = drm_atomic_helper_check,
.atomic_commit = drm_atomic_helper_commit,
};
int udl_modeset_init(struct drm_device *dev)
{
- size_t format_count = ARRAY_SIZE(udl_simple_display_pipe_formats);
struct udl_device *udl = to_udl(dev);
+ struct drm_plane *primary_plane;
+ struct drm_crtc *crtc;
+ struct drm_encoder *encoder;
struct drm_connector *connector;
int ret;
@@ -426,28 +572,42 @@ int udl_modeset_init(struct drm_device *dev)
dev->mode_config.min_width = 640;
dev->mode_config.min_height = 480;
-
dev->mode_config.max_width = 2048;
dev->mode_config.max_height = 2048;
-
- dev->mode_config.prefer_shadow = 0;
dev->mode_config.preferred_depth = 16;
+ dev->mode_config.funcs = &udl_mode_config_funcs;
- dev->mode_config.funcs = &udl_mode_funcs;
+ primary_plane = &udl->primary_plane;
+ ret = drm_universal_plane_init(dev, primary_plane, 0,
+ &udl_primary_plane_funcs,
+ udl_primary_plane_formats,
+ ARRAY_SIZE(udl_primary_plane_formats),
+ udl_primary_plane_fmtmods,
+ DRM_PLANE_TYPE_PRIMARY, NULL);
+ if (ret)
+ return ret;
+ drm_plane_helper_add(primary_plane, &udl_primary_plane_helper_funcs);
+ drm_plane_enable_fb_damage_clips(primary_plane);
+
+ crtc = &udl->crtc;
+ ret = drm_crtc_init_with_planes(dev, crtc, primary_plane, NULL,
+ &udl_crtc_funcs, NULL);
+ if (ret)
+ return ret;
+ drm_crtc_helper_add(crtc, &udl_crtc_helper_funcs);
+
+ encoder = &udl->encoder;
+ ret = drm_encoder_init(dev, encoder, &udl_encoder_funcs, DRM_MODE_ENCODER_DAC, NULL);
+ if (ret)
+ return ret;
+ encoder->possible_crtcs = drm_crtc_mask(crtc);
connector = udl_connector_init(dev);
if (IS_ERR(connector))
return PTR_ERR(connector);
-
- format_count = ARRAY_SIZE(udl_simple_display_pipe_formats);
-
- ret = drm_simple_display_pipe_init(dev, &udl->display_pipe,
- &udl_simple_display_pipe_funcs,
- udl_simple_display_pipe_formats,
- format_count, NULL, connector);
+ ret = drm_connector_attach_encoder(connector, encoder);
if (ret)
return ret;
- drm_plane_enable_fb_damage_clips(&udl->display_pipe.plane);
drm_mode_config_reset(dev);
diff --git a/drivers/gpu/drm/udl/udl_proto.h b/drivers/gpu/drm/udl/udl_proto.h
new file mode 100644
index 0000000..c92d210
--- /dev/null
+++ b/drivers/gpu/drm/udl/udl_proto.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef UDL_PROTO_H
+#define UDL_PROTO_H
+
+#include <linux/bits.h>
+
+#define UDL_MSG_BULK 0xaf
+
+/* Register access */
+#define UDL_CMD_WRITEREG 0x20 /* See register constants below */
+
+/* Framebuffer access */
+#define UDL_CMD_WRITERAW8 0x60 /* 8 bit raw write command. */
+#define UDL_CMD_WRITERL8 0x61 /* 8 bit run length command. */
+#define UDL_CMD_WRITECOPY8 0x62 /* 8 bit copy command. */
+#define UDL_CMD_WRITERLX8 0x63 /* 8 bit extended run length command. */
+#define UDL_CMD_WRITERAW16 0x68 /* 16 bit raw write command. */
+#define UDL_CMD_WRITERL16 0x69 /* 16 bit run length command. */
+#define UDL_CMD_WRITECOPY16 0x6a /* 16 bit copy command. */
+#define UDL_CMD_WRITERLX16 0x6b /* 16 bit extended run length command. */
+
+/* Color depth */
+#define UDL_REG_COLORDEPTH 0x00
+#define UDL_COLORDEPTH_16BPP 0
+#define UDL_COLORDEPTH_24BPP 1
+
+/* Display-mode settings */
+#define UDL_REG_XDISPLAYSTART 0x01
+#define UDL_REG_XDISPLAYEND 0x03
+#define UDL_REG_YDISPLAYSTART 0x05
+#define UDL_REG_YDISPLAYEND 0x07
+#define UDL_REG_XENDCOUNT 0x09
+#define UDL_REG_HSYNCSTART 0x0b
+#define UDL_REG_HSYNCEND 0x0d
+#define UDL_REG_HPIXELS 0x0f
+#define UDL_REG_YENDCOUNT 0x11
+#define UDL_REG_VSYNCSTART 0x13
+#define UDL_REG_VSYNCEND 0x15
+#define UDL_REG_VPIXELS 0x17
+#define UDL_REG_PIXELCLOCK5KHZ 0x1b
+
+/* On/Off for driving the DisplayLink framebuffer to the display */
+#define UDL_REG_BLANKMODE 0x1f
+#define UDL_BLANKMODE_ON 0x00 /* hsync and vsync on, visible */
+#define UDL_BLANKMODE_BLANKED 0x01 /* hsync and vsync on, blanked */
+#define UDL_BLANKMODE_VSYNC_OFF 0x03 /* vsync off, blanked */
+#define UDL_BLANKMODE_HSYNC_OFF 0x05 /* hsync off, blanked */
+#define UDL_BLANKMODE_POWERDOWN 0x07 /* powered off; requires modeset */
+
+/* Framebuffer address */
+#define UDL_REG_BASE16BPP_ADDR2 0x20
+#define UDL_REG_BASE16BPP_ADDR1 0x21
+#define UDL_REG_BASE16BPP_ADDR0 0x22
+#define UDL_REG_BASE8BPP_ADDR2 0x26
+#define UDL_REG_BASE8BPP_ADDR1 0x27
+#define UDL_REG_BASE8BPP_ADDR0 0x28
+
+#define UDL_BASE_ADDR0_MASK GENMASK(7, 0)
+#define UDL_BASE_ADDR1_MASK GENMASK(15, 8)
+#define UDL_BASE_ADDR2_MASK GENMASK(23, 16)
+
+/* Lock/unlock video registers */
+#define UDL_REG_VIDREG 0xff
+#define UDL_VIDREG_LOCK 0x00
+#define UDL_VIDREG_UNLOCK 0xff
+
+#endif
diff --git a/drivers/gpu/drm/udl/udl_transfer.c b/drivers/gpu/drm/udl/udl_transfer.c
index b578446..5ff1037 100644
--- a/drivers/gpu/drm/udl/udl_transfer.c
+++ b/drivers/gpu/drm/udl/udl_transfer.c
@@ -10,6 +10,7 @@
#include <asm/unaligned.h>
#include "udl_drv.h"
+#include "udl_proto.h"
#define MAX_CMD_PIXELS 255
@@ -89,8 +90,8 @@ static void udl_compress_hline16(
const u8 *cmd_pixel_start, *cmd_pixel_end = NULL;
uint16_t pixel_val16;
- *cmd++ = 0xaf;
- *cmd++ = 0x6b;
+ *cmd++ = UDL_MSG_BULK;
+ *cmd++ = UDL_CMD_WRITERLX16;
*cmd++ = (uint8_t) ((dev_addr >> 16) & 0xFF);
*cmd++ = (uint8_t) ((dev_addr >> 8) & 0xFF);
*cmd++ = (uint8_t) ((dev_addr) & 0xFF);
@@ -152,7 +153,7 @@ static void udl_compress_hline16(
if (cmd_buffer_end <= MIN_RLX_CMD_BYTES + cmd) {
/* Fill leftover bytes with no-ops */
if (cmd_buffer_end > cmd)
- memset(cmd, 0xAF, cmd_buffer_end - cmd);
+ memset(cmd, UDL_MSG_BULK, cmd_buffer_end - cmd);
cmd = (uint8_t *) cmd_buffer_end;
}
diff --git a/drivers/gpu/drm/v3d/v3d_drv.c b/drivers/gpu/drm/v3d/v3d_drv.c
index e8c975b..478f1f0 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.c
+++ b/drivers/gpu/drm/v3d/v3d_drv.c
@@ -22,7 +22,6 @@
#include <linux/reset.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_managed.h>
#include <uapi/drm/v3d_drm.h>
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index b898044..96af1cb 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -10,6 +10,7 @@
#include <linux/sched/signal.h>
#include <linux/uaccess.h>
+#include <drm/drm_managed.h>
#include <drm/drm_syncobj.h>
#include <uapi/drm/v3d_drm.h>
@@ -1075,10 +1076,18 @@ v3d_gem_init(struct drm_device *dev)
spin_lock_init(&v3d->mm_lock);
spin_lock_init(&v3d->job_lock);
- mutex_init(&v3d->bo_lock);
- mutex_init(&v3d->reset_lock);
- mutex_init(&v3d->sched_lock);
- mutex_init(&v3d->cache_clean_lock);
+ ret = drmm_mutex_init(dev, &v3d->bo_lock);
+ if (ret)
+ return ret;
+ ret = drmm_mutex_init(dev, &v3d->reset_lock);
+ if (ret)
+ return ret;
+ ret = drmm_mutex_init(dev, &v3d->sched_lock);
+ if (ret)
+ return ret;
+ ret = drmm_mutex_init(dev, &v3d->cache_clean_lock);
+ if (ret)
+ return ret;
/* Note: We don't allocate address 0. Various bits of HW
* treat 0 as special, such as the occlusion query counters
diff --git a/drivers/gpu/drm/v3d/v3d_perfmon.c b/drivers/gpu/drm/v3d/v3d_perfmon.c
index 48aaaa9..e1be736 100644
--- a/drivers/gpu/drm/v3d/v3d_perfmon.c
+++ b/drivers/gpu/drm/v3d/v3d_perfmon.c
@@ -17,8 +17,10 @@ void v3d_perfmon_get(struct v3d_perfmon *perfmon)
void v3d_perfmon_put(struct v3d_perfmon *perfmon)
{
- if (perfmon && refcount_dec_and_test(&perfmon->refcnt))
+ if (perfmon && refcount_dec_and_test(&perfmon->refcnt)) {
+ mutex_destroy(&perfmon->lock);
kfree(perfmon);
+ }
}
void v3d_perfmon_start(struct v3d_dev *v3d, struct v3d_perfmon *perfmon)
@@ -113,6 +115,7 @@ void v3d_perfmon_close_file(struct v3d_file_priv *v3d_priv)
idr_for_each(&v3d_priv->perfmon.idr, v3d_perfmon_idr_del, NULL);
idr_destroy(&v3d_priv->perfmon.idr);
mutex_unlock(&v3d_priv->perfmon.lock);
+ mutex_destroy(&v3d_priv->perfmon.lock);
}
int v3d_perfmon_create_ioctl(struct drm_device *dev, void *data,
@@ -154,6 +157,7 @@ int v3d_perfmon_create_ioctl(struct drm_device *dev, void *data,
mutex_unlock(&v3d_priv->perfmon.lock);
if (ret < 0) {
+ mutex_destroy(&perfmon->lock);
kfree(perfmon);
return ret;
}
diff --git a/drivers/gpu/drm/vboxvideo/vbox_drv.c b/drivers/gpu/drm/vboxvideo/vbox_drv.c
index f4f2bd7..b450f44 100644
--- a/drivers/gpu/drm/vboxvideo/vbox_drv.c
+++ b/drivers/gpu/drm/vboxvideo/vbox_drv.c
@@ -14,7 +14,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_managed.h>
@@ -178,8 +178,6 @@ static const struct drm_driver driver = {
.driver_features =
DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC,
- .lastclose = drm_fb_helper_lastclose,
-
.fops = &vbox_fops,
.name = DRIVER_NAME,
.desc = DRIVER_DESC,
diff --git a/drivers/gpu/drm/vboxvideo/vbox_main.c b/drivers/gpu/drm/vboxvideo/vbox_main.c
index c9e8b3a..3b83e55 100644
--- a/drivers/gpu/drm/vboxvideo/vbox_main.c
+++ b/drivers/gpu/drm/vboxvideo/vbox_main.c
@@ -11,7 +11,6 @@
#include <linux/pci.h>
#include <linux/vbox_err.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_crtc_helper.h>
#include <drm/drm_damage_helper.h>
diff --git a/drivers/gpu/drm/vc4/vc4_bo.c b/drivers/gpu/drm/vc4/vc4_bo.c
index 231add8..43d9b3a 100644
--- a/drivers/gpu/drm/vc4/vc4_bo.c
+++ b/drivers/gpu/drm/vc4/vc4_bo.c
@@ -736,12 +736,12 @@ static int vc4_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct
struct vc4_bo *bo = to_vc4_bo(obj);
if (bo->validated_shader && (vma->vm_flags & VM_WRITE)) {
- DRM_DEBUG("mmaping of shader BOs for writing not allowed.\n");
+ DRM_DEBUG("mmapping of shader BOs for writing not allowed.\n");
return -EINVAL;
}
if (bo->madv != VC4_MADV_WILLNEED) {
- DRM_DEBUG("mmaping of %s BO not allowed\n",
+ DRM_DEBUG("mmapping of %s BO not allowed\n",
bo->madv == VC4_MADV_DONTNEED ?
"purgeable" : "purged");
return -EINVAL;
diff --git a/drivers/gpu/drm/vc4/vc4_drv.c b/drivers/gpu/drm/vc4/vc4_drv.c
index 8c329c0..5990d8f 100644
--- a/drivers/gpu/drm/vc4/vc4_drv.c
+++ b/drivers/gpu/drm/vc4/vc4_drv.c
@@ -33,7 +33,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_vblank.h>
#include <soc/bcm2835/raspberrypi-firmware.h>
diff --git a/drivers/gpu/drm/vc4/vc4_drv.h b/drivers/gpu/drm/vc4/vc4_drv.h
index 418a824..5152286 100644
--- a/drivers/gpu/drm/vc4/vc4_drv.h
+++ b/drivers/gpu/drm/vc4/vc4_drv.h
@@ -326,6 +326,8 @@ struct vc4_hvs {
struct clk *core_clk;
+ unsigned long max_core_rate;
+
/* Memory manager for CRTCs to allocate space in the display
* list. Units are dwords.
*/
@@ -337,6 +339,20 @@ struct vc4_hvs {
struct drm_mm_node mitchell_netravali_filter;
struct debugfs_regset32 regset;
+
+ /*
+ * Even if HDMI0 on the RPi4 can output modes requiring a pixel
+ * rate higher than 297MHz, it needs some adjustments in the
+ * config.txt file to be able to do so and thus won't always be
+ * available.
+ */
+ bool vc5_hdmi_enable_hdmi_20;
+
+ /*
+ * 4096x2160@60 requires a core overclock to work, so register
+ * whether that is sufficient.
+ */
+ bool vc5_hdmi_enable_4096by2160;
};
struct vc4_plane {
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c
index 470432c..12a00d6 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.c
@@ -124,9 +124,8 @@ static unsigned long long
vc4_hdmi_encoder_compute_mode_clock(const struct drm_display_mode *mode,
unsigned int bpc, enum vc4_hdmi_output_format fmt);
-static bool vc4_hdmi_supports_scrambling(struct drm_encoder *encoder)
+static bool vc4_hdmi_supports_scrambling(struct vc4_hdmi *vc4_hdmi)
{
- struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
struct drm_display_info *display = &vc4_hdmi->connector.display_info;
lockdep_assert_held(&vc4_hdmi->mutex);
@@ -319,9 +318,8 @@ static int reset_pipe(struct drm_crtc *crtc,
static int vc4_hdmi_reset_link(struct drm_connector *connector,
struct drm_modeset_acquire_ctx *ctx)
{
- struct drm_device *drm = connector->dev;
- struct vc4_hdmi *vc4_hdmi = connector_to_vc4_hdmi(connector);
- struct drm_encoder *encoder = &vc4_hdmi->encoder.base;
+ struct drm_device *drm;
+ struct vc4_hdmi *vc4_hdmi;
struct drm_connector_state *conn_state;
struct drm_crtc_state *crtc_state;
struct drm_crtc *crtc;
@@ -332,6 +330,7 @@ static int vc4_hdmi_reset_link(struct drm_connector *connector,
if (!connector)
return 0;
+ drm = connector->dev;
ret = drm_modeset_lock(&drm->mode_config.connection_mutex, ctx);
if (ret)
return ret;
@@ -349,9 +348,10 @@ static int vc4_hdmi_reset_link(struct drm_connector *connector,
if (!crtc_state->active)
return 0;
+ vc4_hdmi = connector_to_vc4_hdmi(connector);
mutex_lock(&vc4_hdmi->mutex);
- if (!vc4_hdmi_supports_scrambling(encoder)) {
+ if (!vc4_hdmi_supports_scrambling(vc4_hdmi)) {
mutex_unlock(&vc4_hdmi->mutex);
return 0;
}
@@ -471,6 +471,7 @@ static int vc4_hdmi_connector_detect_ctx(struct drm_connector *connector,
static int vc4_hdmi_connector_get_modes(struct drm_connector *connector)
{
struct vc4_hdmi *vc4_hdmi = connector_to_vc4_hdmi(connector);
+ struct vc4_dev *vc4 = to_vc4_dev(connector->dev);
int ret = 0;
struct edid *edid;
@@ -494,7 +495,7 @@ static int vc4_hdmi_connector_get_modes(struct drm_connector *connector)
ret = drm_add_edid_modes(connector, edid);
kfree(edid);
- if (vc4_hdmi->disable_4kp60) {
+ if (!vc4->hvs->vc5_hdmi_enable_hdmi_20) {
struct drm_device *drm = connector->dev;
const struct drm_display_mode *mode;
@@ -554,7 +555,7 @@ static void vc4_hdmi_connector_reset(struct drm_connector *connector)
new_state->base.max_bpc = 8;
new_state->base.max_requested_bpc = 8;
new_state->output_format = VC4_HDMI_OUTPUT_RGB;
- drm_atomic_helper_connector_tv_reset(connector);
+ drm_atomic_helper_connector_tv_margins_reset(connector);
}
static struct drm_connector_state *
@@ -878,7 +879,7 @@ static void vc4_hdmi_enable_scrambling(struct drm_encoder *encoder)
lockdep_assert_held(&vc4_hdmi->mutex);
- if (!vc4_hdmi_supports_scrambling(encoder))
+ if (!vc4_hdmi_supports_scrambling(vc4_hdmi))
return;
if (!vc4_hdmi_mode_needs_scrambling(mode,
@@ -1764,15 +1765,23 @@ vc4_hdmi_sink_supports_format_bpc(const struct vc4_hdmi *vc4_hdmi,
static enum drm_mode_status
vc4_hdmi_encoder_clock_valid(const struct vc4_hdmi *vc4_hdmi,
+ const struct drm_display_mode *mode,
unsigned long long clock)
{
const struct drm_connector *connector = &vc4_hdmi->connector;
const struct drm_display_info *info = &connector->display_info;
+ struct vc4_dev *vc4 = to_vc4_dev(connector->dev);
if (clock > vc4_hdmi->variant->max_pixel_clock)
return MODE_CLOCK_HIGH;
- if (vc4_hdmi->disable_4kp60 && clock > HDMI_14_MAX_TMDS_CLK)
+ if (!vc4->hvs->vc5_hdmi_enable_hdmi_20 && clock > HDMI_14_MAX_TMDS_CLK)
+ return MODE_CLOCK_HIGH;
+
+ /* 4096x2160@60 is not reliable without overclocking core */
+ if (!vc4->hvs->vc5_hdmi_enable_4096by2160 &&
+ mode->hdisplay > 3840 && mode->vdisplay >= 2160 &&
+ drm_mode_vrefresh(mode) >= 50)
return MODE_CLOCK_HIGH;
if (info->max_tmds_clock && clock > (info->max_tmds_clock * 1000))
@@ -1809,7 +1818,7 @@ vc4_hdmi_encoder_compute_clock(const struct vc4_hdmi *vc4_hdmi,
unsigned long long clock;
clock = vc4_hdmi_encoder_compute_mode_clock(mode, bpc, fmt);
- if (vc4_hdmi_encoder_clock_valid(vc4_hdmi, clock) != MODE_OK)
+ if (vc4_hdmi_encoder_clock_valid(vc4_hdmi, mode, clock) != MODE_OK)
return -EINVAL;
vc4_state->tmds_char_rate = clock;
@@ -1972,7 +1981,7 @@ vc4_hdmi_encoder_mode_valid(struct drm_encoder *encoder,
(mode->hsync_end % 2) || (mode->htotal % 2)))
return MODE_H_ILLEGAL;
- return vc4_hdmi_encoder_clock_valid(vc4_hdmi, mode->clock * 1000);
+ return vc4_hdmi_encoder_clock_valid(vc4_hdmi, mode, mode->clock * 1000);
}
static const struct drm_encoder_helper_funcs vc4_hdmi_encoder_helper_funcs = {
@@ -3481,14 +3490,6 @@ static int vc4_hdmi_bind(struct device *dev, struct device *master, void *data)
vc4_hdmi->disable_wifi_frequencies =
of_property_read_bool(dev->of_node, "wifi-2.4ghz-coexistence");
- if (variant->max_pixel_clock == 600000000) {
- struct vc4_dev *vc4 = to_vc4_dev(drm);
- long max_rate = clk_round_rate(vc4->hvs->core_clk, 550000000);
-
- if (max_rate < 550000000)
- vc4_hdmi->disable_4kp60 = true;
- }
-
ret = devm_pm_runtime_enable(dev);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.h b/drivers/gpu/drm/vc4/vc4_hdmi.h
index 1ad8e8c3..dc3ccd8 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.h
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.h
@@ -156,14 +156,6 @@ struct vc4_hdmi {
*/
bool disable_wifi_frequencies;
- /*
- * Even if HDMI0 on the RPi4 can output modes requiring a pixel
- * rate higher than 297MHz, it needs some adjustments in the
- * config.txt file to be able to do so and thus won't always be
- * available.
- */
- bool disable_4kp60;
-
struct cec_adapter *cec_adap;
struct cec_msg cec_rx_msg;
bool cec_tx_ok;
diff --git a/drivers/gpu/drm/vc4/vc4_hvs.c b/drivers/gpu/drm/vc4/vc4_hvs.c
index 4ac9f5a..c4453a5 100644
--- a/drivers/gpu/drm/vc4/vc4_hvs.c
+++ b/drivers/gpu/drm/vc4/vc4_hvs.c
@@ -28,6 +28,8 @@
#include <drm/drm_drv.h>
#include <drm/drm_vblank.h>
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
#include "vc4_drv.h"
#include "vc4_regs.h"
@@ -791,12 +793,36 @@ static int vc4_hvs_bind(struct device *dev, struct device *master, void *data)
hvs->regset.nregs = ARRAY_SIZE(hvs_regs);
if (vc4->is_vc5) {
+ struct rpi_firmware *firmware;
+ struct device_node *node;
+ unsigned int max_rate;
+
+ node = rpi_firmware_find_node();
+ if (!node)
+ return -EINVAL;
+
+ firmware = rpi_firmware_get(node);
+ of_node_put(node);
+ if (!firmware)
+ return -EPROBE_DEFER;
+
hvs->core_clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(hvs->core_clk)) {
dev_err(&pdev->dev, "Couldn't get core clock\n");
return PTR_ERR(hvs->core_clk);
}
+ max_rate = rpi_firmware_clk_get_max_rate(firmware,
+ RPI_FIRMWARE_CORE_CLK_ID);
+ rpi_firmware_put(firmware);
+ if (max_rate >= 550000000)
+ hvs->vc5_hdmi_enable_hdmi_20 = true;
+
+ if (max_rate >= 600000000)
+ hvs->vc5_hdmi_enable_4096by2160 = true;
+
+ hvs->max_core_rate = max_rate;
+
ret = clk_prepare_enable(hvs->core_clk);
if (ret) {
dev_err(&pdev->dev, "Couldn't enable the core clock\n");
diff --git a/drivers/gpu/drm/vc4/vc4_kms.c b/drivers/gpu/drm/vc4/vc4_kms.c
index 0a6347c..8fbeecd 100644
--- a/drivers/gpu/drm/vc4/vc4_kms.c
+++ b/drivers/gpu/drm/vc4/vc4_kms.c
@@ -396,8 +396,8 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state)
if (vc4->is_vc5) {
unsigned long state_rate = max(old_hvs_state->core_clock_rate,
new_hvs_state->core_clock_rate);
- unsigned long core_rate = max_t(unsigned long,
- 500000000, state_rate);
+ unsigned long core_rate = clamp_t(unsigned long, state_rate,
+ 500000000, hvs->max_core_rate);
drm_dbg(dev, "Raising the core clock at %lu Hz\n", core_rate);
@@ -431,14 +431,17 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state)
drm_atomic_helper_cleanup_planes(dev, state);
if (vc4->is_vc5) {
- drm_dbg(dev, "Running the core clock at %lu Hz\n",
- new_hvs_state->core_clock_rate);
+ unsigned long core_rate = min_t(unsigned long,
+ hvs->max_core_rate,
+ new_hvs_state->core_clock_rate);
+
+ drm_dbg(dev, "Running the core clock at %lu Hz\n", core_rate);
/*
* Request a clock rate based on the current HVS
* requirements.
*/
- WARN_ON(clk_set_min_rate(hvs->core_clk, new_hvs_state->core_clock_rate));
+ WARN_ON(clk_set_min_rate(hvs->core_clk, core_rate));
drm_dbg(dev, "Core clock actual rate: %lu Hz\n",
clk_get_rate(hvs->core_clk));
diff --git a/drivers/gpu/drm/vc4/vc4_vec.c b/drivers/gpu/drm/vc4/vc4_vec.c
index 0b33338..92c07e3 100644
--- a/drivers/gpu/drm/vc4/vc4_vec.c
+++ b/drivers/gpu/drm/vc4/vc4_vec.c
@@ -69,6 +69,7 @@
#define VEC_CONFIG0_STD_MASK GENMASK(1, 0)
#define VEC_CONFIG0_NTSC_STD 0
#define VEC_CONFIG0_PAL_BDGHI_STD 1
+#define VEC_CONFIG0_PAL_M_STD 2
#define VEC_CONFIG0_PAL_N_STD 3
#define VEC_SCHPH 0x108
@@ -255,10 +256,9 @@ static const struct vc4_vec_tv_mode vc4_vec_tv_modes[] = {
.config1 = VEC_CONFIG1_C_CVBS_CVBS,
},
[VC4_VEC_TV_MODE_PAL_M] = {
- .mode = &pal_mode,
- .config0 = VEC_CONFIG0_PAL_BDGHI_STD,
- .config1 = VEC_CONFIG1_C_CVBS_CVBS | VEC_CONFIG1_CUSTOM_FREQ,
- .custom_freq = 0x223b61d1,
+ .mode = &ntsc_mode,
+ .config0 = VEC_CONFIG0_PAL_M_STD,
+ .config1 = VEC_CONFIG1_C_CVBS_CVBS,
},
};
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 0035aff..ae97b98 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -35,6 +35,7 @@
#include <drm/drm_aperture.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include "virtgpu_drv.h"
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 9b98470..b7a64c7 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -35,7 +35,6 @@
#include <drm/drm_atomic.h>
#include <drm/drm_drv.h>
#include <drm/drm_encoder.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_gem.h>
diff --git a/drivers/gpu/drm/vkms/vkms_drv.c b/drivers/gpu/drm/vkms/vkms_drv.c
index 0ffe5f0..293dbca 100644
--- a/drivers/gpu/drm/vkms/vkms_drv.c
+++ b/drivers/gpu/drm/vkms/vkms_drv.c
@@ -17,7 +17,7 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_file.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_ioctl.h>
diff --git a/drivers/gpu/drm/vmwgfx/Kconfig b/drivers/gpu/drm/vmwgfx/Kconfig
index a4fabe2..faddae3 100644
--- a/drivers/gpu/drm/vmwgfx/Kconfig
+++ b/drivers/gpu/drm/vmwgfx/Kconfig
@@ -16,13 +16,6 @@
virtual hardware.
The compiled module will be called "vmwgfx.ko".
-config DRM_VMWGFX_FBCON
- depends on DRM_VMWGFX && DRM_FBDEV_EMULATION
- bool "Enable framebuffer console under vmwgfx by default"
- help
- Choose this option if you are shipping a new vmwgfx
- userspace driver that supports using the kernel driver.
-
config DRM_VMWGFX_MKSSTATS
bool "Enable mksGuestStats instrumentation of vmwgfx by default"
depends on DRM_VMWGFX
diff --git a/drivers/gpu/drm/vmwgfx/Makefile b/drivers/gpu/drm/vmwgfx/Makefile
index eee73b9a..2a644f0 100644
--- a/drivers/gpu/drm/vmwgfx/Makefile
+++ b/drivers/gpu/drm/vmwgfx/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-vmwgfx-y := vmwgfx_execbuf.o vmwgfx_gmr.o vmwgfx_hashtab.o vmwgfx_kms.o vmwgfx_drv.o \
+vmwgfx-y := vmwgfx_execbuf.o vmwgfx_gmr.o vmwgfx_kms.o vmwgfx_drv.o \
vmwgfx_ioctl.o vmwgfx_resource.o vmwgfx_ttm_buffer.o \
vmwgfx_cmd.o vmwgfx_irq.o vmwgfx_ldu.o vmwgfx_ttm_glue.o \
vmwgfx_overlay.o vmwgfx_gmrid_manager.o vmwgfx_fence.o \
@@ -12,6 +12,4 @@
vmwgfx_devcaps.o ttm_object.o vmwgfx_system_manager.o \
vmwgfx_gem.o
-vmwgfx-$(CONFIG_DRM_FBDEV_EMULATION) += vmwgfx_fb.o
-
obj-$(CONFIG_DRM_VMWGFX) := vmwgfx.o
diff --git a/drivers/gpu/drm/vmwgfx/ttm_object.c b/drivers/gpu/drm/vmwgfx/ttm_object.c
index 26a55fe..932b125 100644
--- a/drivers/gpu/drm/vmwgfx/ttm_object.c
+++ b/drivers/gpu/drm/vmwgfx/ttm_object.c
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 OR MIT */
/**************************************************************************
*
- * Copyright (c) 2009-2013 VMware, Inc., Palo Alto, CA., USA
+ * Copyright (c) 2009-2022 VMware, Inc., Palo Alto, CA., USA
* All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
@@ -44,16 +44,20 @@
#define pr_fmt(fmt) "[TTM] " fmt
+#include "ttm_object.h"
+#include "vmwgfx_drv.h"
+
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/slab.h>
#include <linux/atomic.h>
#include <linux/module.h>
-#include "ttm_object.h"
-#include "vmwgfx_drv.h"
+#include <linux/hashtable.h>
MODULE_IMPORT_NS(DMA_BUF);
+#define VMW_TTM_OBJECT_REF_HT_ORDER 10
+
/**
* struct ttm_object_file
*
@@ -74,16 +78,14 @@ struct ttm_object_file {
struct ttm_object_device *tdev;
spinlock_t lock;
struct list_head ref_list;
- struct vmwgfx_open_hash ref_hash;
+ DECLARE_HASHTABLE(ref_hash, VMW_TTM_OBJECT_REF_HT_ORDER);
struct kref refcount;
};
/*
* struct ttm_object_device
*
- * @object_lock: lock that protects the object_hash hash table.
- *
- * @object_hash: hash table for fast lookup of object global names.
+ * @object_lock: lock that protects idr.
*
* @object_count: Per device object count.
*
@@ -92,7 +94,6 @@ struct ttm_object_file {
struct ttm_object_device {
spinlock_t object_lock;
- struct vmwgfx_open_hash object_hash;
atomic_t object_count;
struct dma_buf_ops ops;
void (*dmabuf_release)(struct dma_buf *dma_buf);
@@ -138,6 +139,36 @@ ttm_object_file_ref(struct ttm_object_file *tfile)
return tfile;
}
+static int ttm_tfile_find_ref_rcu(struct ttm_object_file *tfile,
+ uint64_t key,
+ struct vmwgfx_hash_item **p_hash)
+{
+ struct vmwgfx_hash_item *hash;
+
+ hash_for_each_possible_rcu(tfile->ref_hash, hash, head, key) {
+ if (hash->key == key) {
+ *p_hash = hash;
+ return 0;
+ }
+ }
+ return -EINVAL;
+}
+
+static int ttm_tfile_find_ref(struct ttm_object_file *tfile,
+ uint64_t key,
+ struct vmwgfx_hash_item **p_hash)
+{
+ struct vmwgfx_hash_item *hash;
+
+ hash_for_each_possible(tfile->ref_hash, hash, head, key) {
+ if (hash->key == key) {
+ *p_hash = hash;
+ return 0;
+ }
+ }
+ return -EINVAL;
+}
+
static void ttm_object_file_destroy(struct kref *kref)
{
struct ttm_object_file *tfile =
@@ -240,37 +271,35 @@ void ttm_base_object_unref(struct ttm_base_object **p_base)
* Return: A pointer to the object if successful or NULL otherwise.
*/
struct ttm_base_object *
-ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint32_t key)
+ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint64_t key)
{
struct vmwgfx_hash_item *hash;
- struct vmwgfx_open_hash *ht = &tfile->ref_hash;
int ret;
rcu_read_lock();
- ret = vmwgfx_ht_find_item_rcu(ht, key, &hash);
+ ret = ttm_tfile_find_ref_rcu(tfile, key, &hash);
if (ret) {
rcu_read_unlock();
return NULL;
}
__release(RCU);
- return drm_hash_entry(hash, struct ttm_ref_object, hash)->obj;
+ return hlist_entry(hash, struct ttm_ref_object, hash)->obj;
}
EXPORT_SYMBOL(ttm_base_object_noref_lookup);
struct ttm_base_object *ttm_base_object_lookup(struct ttm_object_file *tfile,
- uint32_t key)
+ uint64_t key)
{
struct ttm_base_object *base = NULL;
struct vmwgfx_hash_item *hash;
- struct vmwgfx_open_hash *ht = &tfile->ref_hash;
int ret;
rcu_read_lock();
- ret = vmwgfx_ht_find_item_rcu(ht, key, &hash);
+ ret = ttm_tfile_find_ref_rcu(tfile, key, &hash);
if (likely(ret == 0)) {
- base = drm_hash_entry(hash, struct ttm_ref_object, hash)->obj;
+ base = hlist_entry(hash, struct ttm_ref_object, hash)->obj;
if (!kref_get_unless_zero(&base->refcount))
base = NULL;
}
@@ -280,7 +309,7 @@ struct ttm_base_object *ttm_base_object_lookup(struct ttm_object_file *tfile,
}
struct ttm_base_object *
-ttm_base_object_lookup_for_ref(struct ttm_object_device *tdev, uint32_t key)
+ttm_base_object_lookup_for_ref(struct ttm_object_device *tdev, uint64_t key)
{
struct ttm_base_object *base;
@@ -299,7 +328,6 @@ int ttm_ref_object_add(struct ttm_object_file *tfile,
bool *existed,
bool require_existed)
{
- struct vmwgfx_open_hash *ht = &tfile->ref_hash;
struct ttm_ref_object *ref;
struct vmwgfx_hash_item *hash;
int ret = -EINVAL;
@@ -312,10 +340,10 @@ int ttm_ref_object_add(struct ttm_object_file *tfile,
while (ret == -EINVAL) {
rcu_read_lock();
- ret = vmwgfx_ht_find_item_rcu(ht, base->handle, &hash);
+ ret = ttm_tfile_find_ref_rcu(tfile, base->handle, &hash);
if (ret == 0) {
- ref = drm_hash_entry(hash, struct ttm_ref_object, hash);
+ ref = hlist_entry(hash, struct ttm_ref_object, hash);
if (kref_get_unless_zero(&ref->kref)) {
rcu_read_unlock();
break;
@@ -337,21 +365,14 @@ int ttm_ref_object_add(struct ttm_object_file *tfile,
kref_init(&ref->kref);
spin_lock(&tfile->lock);
- ret = vmwgfx_ht_insert_item_rcu(ht, &ref->hash);
+ hash_add_rcu(tfile->ref_hash, &ref->hash.head, ref->hash.key);
+ ret = 0;
- if (likely(ret == 0)) {
- list_add_tail(&ref->head, &tfile->ref_list);
- kref_get(&base->refcount);
- spin_unlock(&tfile->lock);
- if (existed != NULL)
- *existed = false;
- break;
- }
-
+ list_add_tail(&ref->head, &tfile->ref_list);
+ kref_get(&base->refcount);
spin_unlock(&tfile->lock);
- BUG_ON(ret != -EINVAL);
-
- kfree(ref);
+ if (existed != NULL)
+ *existed = false;
}
return ret;
@@ -363,10 +384,8 @@ ttm_ref_object_release(struct kref *kref)
struct ttm_ref_object *ref =
container_of(kref, struct ttm_ref_object, kref);
struct ttm_object_file *tfile = ref->tfile;
- struct vmwgfx_open_hash *ht;
- ht = &tfile->ref_hash;
- (void)vmwgfx_ht_remove_item_rcu(ht, &ref->hash);
+ hash_del_rcu(&ref->hash.head);
list_del(&ref->head);
spin_unlock(&tfile->lock);
@@ -378,18 +397,17 @@ ttm_ref_object_release(struct kref *kref)
int ttm_ref_object_base_unref(struct ttm_object_file *tfile,
unsigned long key)
{
- struct vmwgfx_open_hash *ht = &tfile->ref_hash;
struct ttm_ref_object *ref;
struct vmwgfx_hash_item *hash;
int ret;
spin_lock(&tfile->lock);
- ret = vmwgfx_ht_find_item(ht, key, &hash);
+ ret = ttm_tfile_find_ref(tfile, key, &hash);
if (unlikely(ret != 0)) {
spin_unlock(&tfile->lock);
return -EINVAL;
}
- ref = drm_hash_entry(hash, struct ttm_ref_object, hash);
+ ref = hlist_entry(hash, struct ttm_ref_object, hash);
kref_put(&ref->kref, ttm_ref_object_release);
spin_unlock(&tfile->lock);
return 0;
@@ -416,16 +434,13 @@ void ttm_object_file_release(struct ttm_object_file **p_tfile)
}
spin_unlock(&tfile->lock);
- vmwgfx_ht_remove(&tfile->ref_hash);
ttm_object_file_unref(&tfile);
}
-struct ttm_object_file *ttm_object_file_init(struct ttm_object_device *tdev,
- unsigned int hash_order)
+struct ttm_object_file *ttm_object_file_init(struct ttm_object_device *tdev)
{
struct ttm_object_file *tfile = kmalloc(sizeof(*tfile), GFP_KERNEL);
- int ret;
if (unlikely(tfile == NULL))
return NULL;
@@ -435,34 +450,21 @@ struct ttm_object_file *ttm_object_file_init(struct ttm_object_device *tdev,
kref_init(&tfile->refcount);
INIT_LIST_HEAD(&tfile->ref_list);
- ret = vmwgfx_ht_create(&tfile->ref_hash, hash_order);
- if (ret)
- goto out_err;
+ hash_init(tfile->ref_hash);
return tfile;
-out_err:
- vmwgfx_ht_remove(&tfile->ref_hash);
-
- kfree(tfile);
-
- return NULL;
}
struct ttm_object_device *
-ttm_object_device_init(unsigned int hash_order,
- const struct dma_buf_ops *ops)
+ttm_object_device_init(const struct dma_buf_ops *ops)
{
struct ttm_object_device *tdev = kmalloc(sizeof(*tdev), GFP_KERNEL);
- int ret;
if (unlikely(tdev == NULL))
return NULL;
spin_lock_init(&tdev->object_lock);
atomic_set(&tdev->object_count, 0);
- ret = vmwgfx_ht_create(&tdev->object_hash, hash_order);
- if (ret != 0)
- goto out_no_object_hash;
/*
* Our base is at VMWGFX_NUM_MOB + 1 because we want to create
@@ -477,10 +479,6 @@ ttm_object_device_init(unsigned int hash_order,
tdev->dmabuf_release = tdev->ops.release;
tdev->ops.release = ttm_prime_dmabuf_release;
return tdev;
-
-out_no_object_hash:
- kfree(tdev);
- return NULL;
}
void ttm_object_device_release(struct ttm_object_device **p_tdev)
@@ -491,7 +489,6 @@ void ttm_object_device_release(struct ttm_object_device **p_tdev)
WARN_ON_ONCE(!idr_is_empty(&tdev->idr));
idr_destroy(&tdev->idr);
- vmwgfx_ht_remove(&tdev->object_hash);
kfree(tdev);
}
diff --git a/drivers/gpu/drm/vmwgfx/ttm_object.h b/drivers/gpu/drm/vmwgfx/ttm_object.h
index 1a2fa0f..f0ebbe3 100644
--- a/drivers/gpu/drm/vmwgfx/ttm_object.h
+++ b/drivers/gpu/drm/vmwgfx/ttm_object.h
@@ -1,6 +1,6 @@
/**************************************************************************
*
- * Copyright (c) 2006-2009 VMware, Inc., Palo Alto, CA., USA
+ * Copyright (c) 2006-2022 VMware, Inc., Palo Alto, CA., USA
* All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
@@ -42,8 +42,6 @@
#include <linux/list.h>
#include <linux/rcupdate.h>
-#include "vmwgfx_hashtab.h"
-
/**
* enum ttm_object_type
*
@@ -104,7 +102,7 @@ struct ttm_base_object {
struct ttm_object_file *tfile;
struct kref refcount;
void (*refcount_release) (struct ttm_base_object **base);
- u32 handle;
+ u64 handle;
enum ttm_object_type object_type;
u32 shareable;
};
@@ -164,7 +162,7 @@ extern int ttm_base_object_init(struct ttm_object_file *tfile,
*/
extern struct ttm_base_object *ttm_base_object_lookup(struct ttm_object_file
- *tfile, uint32_t key);
+ *tfile, uint64_t key);
/**
* ttm_base_object_lookup_for_ref
@@ -178,7 +176,7 @@ extern struct ttm_base_object *ttm_base_object_lookup(struct ttm_object_file
*/
extern struct ttm_base_object *
-ttm_base_object_lookup_for_ref(struct ttm_object_device *tdev, uint32_t key);
+ttm_base_object_lookup_for_ref(struct ttm_object_device *tdev, uint64_t key);
/**
* ttm_base_object_unref
@@ -237,14 +235,12 @@ extern int ttm_ref_object_base_unref(struct ttm_object_file *tfile,
* ttm_object_file_init - initialize a struct ttm_object file
*
* @tdev: A struct ttm_object device this file is initialized on.
- * @hash_order: Order of the hash table used to hold the reference objects.
*
* This is typically called by the file_ops::open function.
*/
extern struct ttm_object_file *ttm_object_file_init(struct ttm_object_device
- *tdev,
- unsigned int hash_order);
+ *tdev);
/**
* ttm_object_file_release - release data held by a ttm_object_file
@@ -262,7 +258,6 @@ extern void ttm_object_file_release(struct ttm_object_file **p_tfile);
/**
* ttm_object device init - initialize a struct ttm_object_device
*
- * @hash_order: Order of hash table used to hash the base objects.
* @ops: DMA buf ops for prime objects of this device.
*
* This function is typically called on device initialization to prepare
@@ -270,8 +265,7 @@ extern void ttm_object_file_release(struct ttm_object_file **p_tfile);
*/
extern struct ttm_object_device *
-ttm_object_device_init(unsigned int hash_order,
- const struct dma_buf_ops *ops);
+ttm_object_device_init(const struct dma_buf_ops *ops);
/**
* ttm_object_device_release - release data held by a ttm_object_device
@@ -314,7 +308,7 @@ extern int ttm_prime_handle_to_fd(struct ttm_object_file *tfile,
kfree_rcu(__obj, __prime.base.rhead)
struct ttm_base_object *
-ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint32_t key);
+ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint64_t key);
/**
* ttm_base_object_noref_release - release a base object pointer looked up
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c b/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c
index 09fe20e..c52c7bf 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c
@@ -483,8 +483,8 @@ int vmw_bo_cpu_blit(struct ttm_buffer_object *dst,
d.src_addr = NULL;
d.dst_pages = dst->ttm->pages;
d.src_pages = src->ttm->pages;
- d.dst_num_pages = dst->resource->num_pages;
- d.src_num_pages = src->resource->num_pages;
+ d.dst_num_pages = PFN_UP(dst->resource->size);
+ d.src_num_pages = PFN_UP(src->resource->size);
d.dst_prot = ttm_io_prot(dst, dst->resource, PAGE_KERNEL);
d.src_prot = ttm_io_prot(src, src->resource, PAGE_KERNEL);
d.diff = diff;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
index 822251a..321c551 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
@@ -194,7 +194,7 @@ int vmw_bo_pin_in_start_of_vram(struct vmw_private *dev_priv,
int ret = 0;
place = vmw_vram_placement.placement[0];
- place.lpfn = bo->resource->num_pages;
+ place.lpfn = PFN_UP(bo->resource->size);
placement.num_placement = 1;
placement.placement = &place;
placement.num_busy_placement = 1;
@@ -211,7 +211,7 @@ int vmw_bo_pin_in_start_of_vram(struct vmw_private *dev_priv,
* that situation.
*/
if (bo->resource->mem_type == TTM_PL_VRAM &&
- bo->resource->start < bo->resource->num_pages &&
+ bo->resource->start < PFN_UP(bo->resource->size) &&
bo->resource->start > 0 &&
buf->base.pin_count == 0) {
ctx.interruptible = false;
@@ -352,7 +352,7 @@ void *vmw_bo_map_and_cache(struct vmw_buffer_object *vbo)
if (virtual)
return virtual;
- ret = ttm_bo_kmap(bo, 0, bo->resource->num_pages, &vbo->map);
+ ret = ttm_bo_kmap(bo, 0, PFN_UP(bo->base.size), &vbo->map);
if (ret)
DRM_ERROR("Buffer object map failed: %d.\n", ret);
@@ -807,9 +807,23 @@ int vmw_dumb_create(struct drm_file *file_priv,
{
struct vmw_private *dev_priv = vmw_priv(dev);
struct vmw_buffer_object *vbo;
+ int cpp = DIV_ROUND_UP(args->bpp, 8);
int ret;
- args->pitch = args->width * ((args->bpp + 7) / 8);
+ switch (cpp) {
+ case 1: /* DRM_FORMAT_C8 */
+ case 2: /* DRM_FORMAT_RGB565 */
+ case 4: /* DRM_FORMAT_XRGB8888 */
+ break;
+ default:
+ /*
+ * Dumb buffers don't allow anything else.
+ * This is tested via IGT's dumb_buffers
+ */
+ return -EINVAL;
+ }
+
+ args->pitch = args->width * cpp;
args->size = ALIGN(args->pitch * args->height, PAGE_SIZE);
ret = vmw_gem_object_create_with_handle(dev_priv, file_priv,
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c
index 82ef58c..47bc0b4 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 OR MIT
/**************************************************************************
*
- * Copyright 2014-2015 VMware, Inc., Palo Alto, CA., USA
+ * Copyright 2014-2022 VMware, Inc., Palo Alto, CA., USA
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the
@@ -28,6 +28,8 @@
#include "vmwgfx_drv.h"
#include "vmwgfx_resource_priv.h"
+#include <linux/hashtable.h>
+
#define VMW_CMDBUF_RES_MAN_HT_ORDER 12
/**
@@ -59,7 +61,7 @@ struct vmw_cmdbuf_res {
* @resources and @list are protected by the cmdbuf mutex for now.
*/
struct vmw_cmdbuf_res_manager {
- struct vmwgfx_open_hash resources;
+ DECLARE_HASHTABLE(resources, VMW_CMDBUF_RES_MAN_HT_ORDER);
struct list_head list;
struct vmw_private *dev_priv;
};
@@ -82,14 +84,13 @@ vmw_cmdbuf_res_lookup(struct vmw_cmdbuf_res_manager *man,
u32 user_key)
{
struct vmwgfx_hash_item *hash;
- int ret;
unsigned long key = user_key | (res_type << 24);
- ret = vmwgfx_ht_find_item(&man->resources, key, &hash);
- if (unlikely(ret != 0))
- return ERR_PTR(ret);
-
- return drm_hash_entry(hash, struct vmw_cmdbuf_res, hash)->res;
+ hash_for_each_possible_rcu(man->resources, hash, head, key) {
+ if (hash->key == key)
+ return hlist_entry(hash, struct vmw_cmdbuf_res, hash)->res;
+ }
+ return ERR_PTR(-EINVAL);
}
/**
@@ -105,7 +106,7 @@ static void vmw_cmdbuf_res_free(struct vmw_cmdbuf_res_manager *man,
struct vmw_cmdbuf_res *entry)
{
list_del(&entry->head);
- WARN_ON(vmwgfx_ht_remove_item(&man->resources, &entry->hash));
+ hash_del_rcu(&entry->hash.head);
vmw_resource_unreference(&entry->res);
kfree(entry);
}
@@ -159,7 +160,6 @@ void vmw_cmdbuf_res_commit(struct list_head *list)
void vmw_cmdbuf_res_revert(struct list_head *list)
{
struct vmw_cmdbuf_res *entry, *next;
- int ret;
list_for_each_entry_safe(entry, next, list, head) {
switch (entry->state) {
@@ -167,8 +167,8 @@ void vmw_cmdbuf_res_revert(struct list_head *list)
vmw_cmdbuf_res_free(entry->man, entry);
break;
case VMW_CMDBUF_RES_DEL:
- ret = vmwgfx_ht_insert_item(&entry->man->resources, &entry->hash);
- BUG_ON(ret);
+ hash_add_rcu(entry->man->resources, &entry->hash.head,
+ entry->hash.key);
list_move_tail(&entry->head, &entry->man->list);
entry->state = VMW_CMDBUF_RES_COMMITTED;
break;
@@ -199,26 +199,20 @@ int vmw_cmdbuf_res_add(struct vmw_cmdbuf_res_manager *man,
struct list_head *list)
{
struct vmw_cmdbuf_res *cres;
- int ret;
cres = kzalloc(sizeof(*cres), GFP_KERNEL);
if (unlikely(!cres))
return -ENOMEM;
cres->hash.key = user_key | (res_type << 24);
- ret = vmwgfx_ht_insert_item(&man->resources, &cres->hash);
- if (unlikely(ret != 0)) {
- kfree(cres);
- goto out_invalid_key;
- }
+ hash_add_rcu(man->resources, &cres->hash.head, cres->hash.key);
cres->state = VMW_CMDBUF_RES_ADD;
cres->res = vmw_resource_reference(res);
cres->man = man;
list_add_tail(&cres->head, list);
-out_invalid_key:
- return ret;
+ return 0;
}
/**
@@ -243,24 +237,26 @@ int vmw_cmdbuf_res_remove(struct vmw_cmdbuf_res_manager *man,
struct list_head *list,
struct vmw_resource **res_p)
{
- struct vmw_cmdbuf_res *entry;
+ struct vmw_cmdbuf_res *entry = NULL;
struct vmwgfx_hash_item *hash;
- int ret;
+ unsigned long key = user_key | (res_type << 24);
- ret = vmwgfx_ht_find_item(&man->resources, user_key | (res_type << 24),
- &hash);
- if (likely(ret != 0))
+ hash_for_each_possible_rcu(man->resources, hash, head, key) {
+ if (hash->key == key) {
+ entry = hlist_entry(hash, struct vmw_cmdbuf_res, hash);
+ break;
+ }
+ }
+ if (unlikely(!entry))
return -EINVAL;
- entry = drm_hash_entry(hash, struct vmw_cmdbuf_res, hash);
-
switch (entry->state) {
case VMW_CMDBUF_RES_ADD:
vmw_cmdbuf_res_free(man, entry);
*res_p = NULL;
break;
case VMW_CMDBUF_RES_COMMITTED:
- (void) vmwgfx_ht_remove_item(&man->resources, &entry->hash);
+ hash_del_rcu(&entry->hash.head);
list_del(&entry->head);
entry->state = VMW_CMDBUF_RES_DEL;
list_add_tail(&entry->head, list);
@@ -287,7 +283,6 @@ struct vmw_cmdbuf_res_manager *
vmw_cmdbuf_res_man_create(struct vmw_private *dev_priv)
{
struct vmw_cmdbuf_res_manager *man;
- int ret;
man = kzalloc(sizeof(*man), GFP_KERNEL);
if (!man)
@@ -295,12 +290,8 @@ vmw_cmdbuf_res_man_create(struct vmw_private *dev_priv)
man->dev_priv = dev_priv;
INIT_LIST_HEAD(&man->list);
- ret = vmwgfx_ht_create(&man->resources, VMW_CMDBUF_RES_MAN_HT_ORDER);
- if (ret == 0)
- return man;
-
- kfree(man);
- return ERR_PTR(ret);
+ hash_init(man->resources);
+ return man;
}
/**
@@ -320,7 +311,6 @@ void vmw_cmdbuf_res_man_destroy(struct vmw_cmdbuf_res_manager *man)
list_for_each_entry_safe(entry, next, &man->list, head)
vmw_cmdbuf_res_free(man, entry);
- vmwgfx_ht_remove(&man->resources);
kfree(man);
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
index 79b30dc9..b78a103 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
@@ -33,6 +33,7 @@
#include <drm/ttm/ttm_placement.h>
#include "vmwgfx_drv.h"
+#include "vmwgfx_mksstat.h"
#include "vmwgfx_resource_priv.h"
#include "vmwgfx_so.h"
@@ -72,12 +73,24 @@ struct vmw_cotable_info {
bool);
};
+
+/*
+ * Getting the initial size right is difficult because it all depends
+ * on what the userspace is doing. The sizes will be aligned up to
+ * a PAGE_SIZE so we just want to make sure that for majority of apps
+ * the initial number of entries doesn't require an immediate resize.
+ * For all cotables except SVGACOTableDXElementLayoutEntry and
+ * SVGACOTableDXBlendStateEntry the initial number of entries fits
+ * within the PAGE_SIZE. For SVGACOTableDXElementLayoutEntry and
+ * SVGACOTableDXBlendStateEntry we want to reserve two pages,
+ * because that's what all apps will require initially.
+ */
static const struct vmw_cotable_info co_info[] = {
{1, sizeof(SVGACOTableDXRTViewEntry), &vmw_view_cotable_list_destroy},
{1, sizeof(SVGACOTableDXDSViewEntry), &vmw_view_cotable_list_destroy},
{1, sizeof(SVGACOTableDXSRViewEntry), &vmw_view_cotable_list_destroy},
- {1, sizeof(SVGACOTableDXElementLayoutEntry), NULL},
- {1, sizeof(SVGACOTableDXBlendStateEntry), NULL},
+ {PAGE_SIZE/sizeof(SVGACOTableDXElementLayoutEntry) + 1, sizeof(SVGACOTableDXElementLayoutEntry), NULL},
+ {PAGE_SIZE/sizeof(SVGACOTableDXBlendStateEntry) + 1, sizeof(SVGACOTableDXBlendStateEntry), NULL},
{1, sizeof(SVGACOTableDXDepthStencilEntry), NULL},
{1, sizeof(SVGACOTableDXRasterizerStateEntry), NULL},
{1, sizeof(SVGACOTableDXSamplerEntry), NULL},
@@ -395,9 +408,12 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size)
int ret;
size_t i;
+ MKS_STAT_TIME_DECL(MKSSTAT_KERN_COTABLE_RESIZE);
+ MKS_STAT_TIME_PUSH(MKSSTAT_KERN_COTABLE_RESIZE);
+
ret = vmw_cotable_readback(res);
if (ret)
- return ret;
+ goto out_done;
cur_size_read_back = vcotbl->size_read_back;
vcotbl->size_read_back = old_size_read_back;
@@ -411,7 +427,7 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size)
true, true, vmw_bo_bo_free, &buf);
if (ret) {
DRM_ERROR("Failed initializing new cotable MOB.\n");
- return ret;
+ goto out_done;
}
bo = &buf->base;
@@ -427,7 +443,7 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size)
* Do a page by page copy of COTables. This eliminates slow vmap()s.
* This should really be a TTM utility.
*/
- for (i = 0; i < old_bo->resource->num_pages; ++i) {
+ for (i = 0; i < PFN_UP(old_bo->resource->size); ++i) {
bool dummy;
ret = ttm_bo_kmap(old_bo, i, 1, &old_map);
@@ -485,6 +501,8 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size)
/* Release the pin acquired in vmw_bo_init */
ttm_bo_unpin(bo);
+ MKS_STAT_TIME_POP(MKSSTAT_KERN_COTABLE_RESIZE);
+
return 0;
out_map_new:
@@ -494,6 +512,9 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size)
ttm_bo_unreserve(bo);
vmw_bo_unreference(&buf);
+out_done:
+ MKS_STAT_TIME_POP(MKSSTAT_KERN_COTABLE_RESIZE);
+
return ret;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
index d7bd5eb..bd02cb0 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
@@ -25,13 +25,17 @@
*
**************************************************************************/
-#include <linux/dma-mapping.h>
-#include <linux/module.h>
-#include <linux/pci.h>
-#include <linux/cc_platform.h>
+
+#include "vmwgfx_drv.h"
+
+#include "vmwgfx_devcaps.h"
+#include "vmwgfx_mksstat.h"
+#include "vmwgfx_binding.h"
+#include "ttm_object.h"
#include <drm/drm_aperture.h>
#include <drm/drm_drv.h>
+#include <drm/drm_fbdev_generic.h>
#include <drm/drm_gem_ttm_helper.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_module.h>
@@ -41,17 +45,14 @@
#include <drm/ttm/ttm_placement.h>
#include <generated/utsrelease.h>
-#include "ttm_object.h"
-#include "vmwgfx_binding.h"
-#include "vmwgfx_devcaps.h"
-#include "vmwgfx_drv.h"
-#include "vmwgfx_mksstat.h"
+#include <linux/cc_platform.h>
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/version.h>
#define VMWGFX_DRIVER_DESC "Linux drm driver for VMware graphics devices"
-#define VMW_MIN_INITIAL_WIDTH 800
-#define VMW_MIN_INITIAL_HEIGHT 600
-
/*
* Fully encoded drm commands. Might move to vmw_drm.h
*/
@@ -262,7 +263,6 @@ static const struct pci_device_id vmw_pci_id_list[] = {
};
MODULE_DEVICE_TABLE(pci, vmw_pci_id_list);
-static int enable_fbdev = IS_ENABLED(CONFIG_DRM_VMWGFX_FBCON);
static int vmw_restrict_iommu;
static int vmw_force_coherent;
static int vmw_restrict_dma_mask;
@@ -272,8 +272,6 @@ static int vmw_probe(struct pci_dev *, const struct pci_device_id *);
static int vmwgfx_pm_notifier(struct notifier_block *nb, unsigned long val,
void *ptr);
-MODULE_PARM_DESC(enable_fbdev, "Enable vmwgfx fbdev");
-module_param_named(enable_fbdev, enable_fbdev, int, 0600);
MODULE_PARM_DESC(restrict_iommu, "Try to limit IOMMU usage for TTM pages");
module_param_named(restrict_iommu, vmw_restrict_iommu, int, 0600);
MODULE_PARM_DESC(force_coherent, "Force coherent TTM pages");
@@ -623,8 +621,8 @@ static void vmw_get_initial_size(struct vmw_private *dev_priv)
width = vmw_read(dev_priv, SVGA_REG_WIDTH);
height = vmw_read(dev_priv, SVGA_REG_HEIGHT);
- width = max_t(uint32_t, width, VMW_MIN_INITIAL_WIDTH);
- height = max_t(uint32_t, height, VMW_MIN_INITIAL_HEIGHT);
+ width = max_t(uint32_t, width, VMWGFX_MIN_INITIAL_WIDTH);
+ height = max_t(uint32_t, height, VMWGFX_MIN_INITIAL_HEIGHT);
if (width > dev_priv->fb_max_width ||
height > dev_priv->fb_max_height) {
@@ -633,8 +631,8 @@ static void vmw_get_initial_size(struct vmw_private *dev_priv)
* This is a host error and shouldn't occur.
*/
- width = VMW_MIN_INITIAL_WIDTH;
- height = VMW_MIN_INITIAL_HEIGHT;
+ width = VMWGFX_MIN_INITIAL_WIDTH;
+ height = VMWGFX_MIN_INITIAL_HEIGHT;
}
dev_priv->initial_width = width;
@@ -806,6 +804,43 @@ static int vmw_detect_version(struct vmw_private *dev)
return 0;
}
+static void vmw_write_driver_id(struct vmw_private *dev)
+{
+ if ((dev->capabilities2 & SVGA_CAP2_DX2) != 0) {
+ vmw_write(dev, SVGA_REG_GUEST_DRIVER_ID,
+ SVGA_REG_GUEST_DRIVER_ID_LINUX);
+
+ vmw_write(dev, SVGA_REG_GUEST_DRIVER_VERSION1,
+ LINUX_VERSION_MAJOR << 24 |
+ LINUX_VERSION_PATCHLEVEL << 16 |
+ LINUX_VERSION_SUBLEVEL);
+ vmw_write(dev, SVGA_REG_GUEST_DRIVER_VERSION2,
+ VMWGFX_DRIVER_MAJOR << 24 |
+ VMWGFX_DRIVER_MINOR << 16 |
+ VMWGFX_DRIVER_PATCHLEVEL);
+ vmw_write(dev, SVGA_REG_GUEST_DRIVER_VERSION3, 0);
+
+ vmw_write(dev, SVGA_REG_GUEST_DRIVER_ID,
+ SVGA_REG_GUEST_DRIVER_ID_SUBMIT);
+ }
+}
+
+static void vmw_sw_context_init(struct vmw_private *dev_priv)
+{
+ struct vmw_sw_context *sw_context = &dev_priv->ctx;
+
+ hash_init(sw_context->res_ht);
+}
+
+static void vmw_sw_context_fini(struct vmw_private *dev_priv)
+{
+ struct vmw_sw_context *sw_context = &dev_priv->ctx;
+
+ vfree(sw_context->cmd_bounce);
+ if (sw_context->staged_bindings)
+ vmw_binding_state_free(sw_context->staged_bindings);
+}
+
static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
{
int ret;
@@ -815,6 +850,8 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
dev_priv->drm.dev_private = dev_priv;
+ vmw_sw_context_init(dev_priv);
+
mutex_init(&dev_priv->cmdbuf_mutex);
mutex_init(&dev_priv->binding_mutex);
spin_lock_init(&dev_priv->resource_lock);
@@ -844,9 +881,6 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
dev_priv->assume_16bpp = !!vmw_assume_16bpp;
- dev_priv->enable_fb = enable_fbdev;
-
-
dev_priv->capabilities = vmw_read(dev_priv, SVGA_REG_CAPABILITIES);
vmw_print_bitmap(&dev_priv->drm, "Capabilities",
dev_priv->capabilities,
@@ -970,7 +1004,7 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
goto out_err0;
}
- dev_priv->tdev = ttm_object_device_init(12, &vmw_prime_dmabuf_ops);
+ dev_priv->tdev = ttm_object_device_init(&vmw_prime_dmabuf_ops);
if (unlikely(dev_priv->tdev == NULL)) {
drm_err(&dev_priv->drm,
@@ -1091,12 +1125,7 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
vmw_host_printf("vmwgfx: Module Version: %d.%d.%d (kernel: %s)",
VMWGFX_DRIVER_MAJOR, VMWGFX_DRIVER_MINOR,
VMWGFX_DRIVER_PATCHLEVEL, UTS_RELEASE);
-
- if (dev_priv->enable_fb) {
- vmw_fifo_resource_inc(dev_priv);
- vmw_svga_enable(dev_priv);
- vmw_fb_init(dev_priv);
- }
+ vmw_write_driver_id(dev_priv);
dev_priv->pm_nb.notifier_call = vmwgfx_pm_notifier;
register_pm_notifier(&dev_priv->pm_nb);
@@ -1143,15 +1172,10 @@ static void vmw_driver_unload(struct drm_device *dev)
unregister_pm_notifier(&dev_priv->pm_nb);
- if (dev_priv->ctx.res_ht_initialized)
- vmwgfx_ht_remove(&dev_priv->ctx.res_ht);
- vfree(dev_priv->ctx.cmd_bounce);
- if (dev_priv->enable_fb) {
- vmw_fb_off(dev_priv);
- vmw_fb_close(dev_priv);
- vmw_fifo_resource_dec(dev_priv);
- vmw_svga_disable(dev_priv);
- }
+ vmw_sw_context_fini(dev_priv);
+ vmw_fifo_resource_dec(dev_priv);
+
+ vmw_svga_disable(dev_priv);
vmw_kms_close(dev_priv);
vmw_overlay_close(dev_priv);
@@ -1173,8 +1197,6 @@ static void vmw_driver_unload(struct drm_device *dev)
vmw_irq_uninstall(&dev_priv->drm);
ttm_object_device_release(&dev_priv->tdev);
- if (dev_priv->ctx.staged_bindings)
- vmw_binding_state_free(dev_priv->ctx.staged_bindings);
for (i = vmw_res_context; i < vmw_res_max; ++i)
idr_destroy(&dev_priv->res_idr[i]);
@@ -1203,7 +1225,7 @@ static int vmw_driver_open(struct drm_device *dev, struct drm_file *file_priv)
if (unlikely(!vmw_fp))
return ret;
- vmw_fp->tfile = ttm_object_file_init(dev_priv->tdev, 10);
+ vmw_fp->tfile = ttm_object_file_init(dev_priv->tdev);
if (unlikely(vmw_fp->tfile == NULL))
goto out_no_tfile;
@@ -1291,8 +1313,6 @@ static void vmw_master_drop(struct drm_device *dev,
struct vmw_private *dev_priv = vmw_priv(dev);
vmw_kms_legacy_hotspot_clear(dev_priv);
- if (!dev_priv->enable_fb)
- vmw_svga_disable(dev_priv);
}
/**
@@ -1485,25 +1505,19 @@ static int vmw_pm_freeze(struct device *kdev)
DRM_ERROR("Failed to freeze modesetting.\n");
return ret;
}
- if (dev_priv->enable_fb)
- vmw_fb_off(dev_priv);
vmw_execbuf_release_pinned_bo(dev_priv);
vmw_resource_evict_all(dev_priv);
vmw_release_device_early(dev_priv);
while (ttm_device_swapout(&dev_priv->bdev, &ctx, GFP_KERNEL) > 0);
- if (dev_priv->enable_fb)
- vmw_fifo_resource_dec(dev_priv);
+ vmw_fifo_resource_dec(dev_priv);
if (atomic_read(&dev_priv->num_fifo_resources) != 0) {
DRM_ERROR("Can't hibernate while 3D resources are active.\n");
- if (dev_priv->enable_fb)
- vmw_fifo_resource_inc(dev_priv);
+ vmw_fifo_resource_inc(dev_priv);
WARN_ON(vmw_request_device_late(dev_priv));
dev_priv->suspend_locked = false;
if (dev_priv->suspend_state)
vmw_kms_resume(dev);
- if (dev_priv->enable_fb)
- vmw_fb_on(dev_priv);
return -EBUSY;
}
@@ -1523,24 +1537,19 @@ static int vmw_pm_restore(struct device *kdev)
vmw_detect_version(dev_priv);
- if (dev_priv->enable_fb)
- vmw_fifo_resource_inc(dev_priv);
+ vmw_fifo_resource_inc(dev_priv);
ret = vmw_request_device(dev_priv);
if (ret)
return ret;
- if (dev_priv->enable_fb)
- __vmw_svga_enable(dev_priv);
+ __vmw_svga_enable(dev_priv);
vmw_fence_fifo_up(dev_priv->fman);
dev_priv->suspend_locked = false;
if (dev_priv->suspend_state)
vmw_kms_resume(&dev_priv->drm);
- if (dev_priv->enable_fb)
- vmw_fb_on(dev_priv);
-
return 0;
}
@@ -1631,6 +1640,10 @@ static int vmw_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (ret)
goto out_unload;
+ vmw_fifo_resource_inc(vmw);
+ vmw_svga_enable(vmw);
+ drm_fbdev_generic_setup(&vmw->drm, 0);
+
vmw_debugfs_gem_init(vmw);
vmw_debugfs_resource_managers_init(vmw);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
index 09e2d73..b062b02 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
@@ -30,6 +30,7 @@
#include <linux/suspend.h>
#include <linux/sync_file.h>
+#include <linux/hashtable.h>
#include <drm/drm_auth.h>
#include <drm/drm_device.h>
@@ -42,7 +43,6 @@
#include "ttm_object.h"
#include "vmwgfx_fence.h"
-#include "vmwgfx_hashtab.h"
#include "vmwgfx_reg.h"
#include "vmwgfx_validation.h"
@@ -62,6 +62,9 @@
#define VMWGFX_MAX_DISPLAYS 16
#define VMWGFX_CMD_BOUNCE_INIT_SIZE 32768
+#define VMWGFX_MIN_INITIAL_WIDTH 1280
+#define VMWGFX_MIN_INITIAL_HEIGHT 800
+
#define VMWGFX_PCI_ID_SVGA2 0x0405
#define VMWGFX_PCI_ID_SVGA3 0x0406
@@ -93,6 +96,11 @@
#define VMW_RES_STREAM ttm_driver_type2
#define VMW_RES_FENCE ttm_driver_type3
#define VMW_RES_SHADER ttm_driver_type4
+#define VMW_RES_HT_ORDER 12
+
+#define VMW_CURSOR_SNOOP_FORMAT SVGA3D_A8R8G8B8
+#define VMW_CURSOR_SNOOP_WIDTH 64
+#define VMW_CURSOR_SNOOP_HEIGHT 64
#define MKSSTAT_CAPACITY_LOG2 5U
#define MKSSTAT_CAPACITY (1U << MKSSTAT_CAPACITY_LOG2)
@@ -102,6 +110,11 @@ struct vmw_fpriv {
bool gb_aware; /* user-space is guest-backed aware */
};
+struct vmwgfx_hash_item {
+ struct hlist_node head;
+ unsigned long key;
+};
+
/**
* struct vmw_buffer_object - TTM buffer object with vmwgfx additions
* @base: The TTM buffer object
@@ -425,8 +438,7 @@ struct vmw_ctx_validation_info;
* @ctx: The validation context
*/
struct vmw_sw_context{
- struct vmwgfx_open_hash res_ht;
- bool res_ht_initialized;
+ DECLARE_HASHTABLE(res_ht, VMW_RES_HT_ORDER);
bool kernel;
struct vmw_fpriv *fp;
struct drm_file *filp;
@@ -546,7 +558,6 @@ struct vmw_private {
* Framebuffer info.
*/
- void *fb_info;
enum vmw_display_unit_type active_display_unit;
struct vmw_legacy_display *ldu_priv;
struct vmw_overlay *overlay_priv;
@@ -605,8 +616,6 @@ struct vmw_private {
struct mutex cmdbuf_mutex;
struct mutex binding_mutex;
- bool enable_fb;
-
/**
* PM management.
*/
@@ -1184,35 +1193,6 @@ extern void vmw_generic_waiter_add(struct vmw_private *dev_priv, u32 flag,
extern void vmw_generic_waiter_remove(struct vmw_private *dev_priv,
u32 flag, int *waiter_count);
-
-/**
- * Kernel framebuffer - vmwgfx_fb.c
- */
-
-#ifdef CONFIG_DRM_FBDEV_EMULATION
-int vmw_fb_init(struct vmw_private *vmw_priv);
-int vmw_fb_close(struct vmw_private *dev_priv);
-int vmw_fb_off(struct vmw_private *vmw_priv);
-int vmw_fb_on(struct vmw_private *vmw_priv);
-#else
-static inline int vmw_fb_init(struct vmw_private *vmw_priv)
-{
- return 0;
-}
-static inline int vmw_fb_close(struct vmw_private *dev_priv)
-{
- return 0;
-}
-static inline int vmw_fb_off(struct vmw_private *vmw_priv)
-{
- return 0;
-}
-static inline int vmw_fb_on(struct vmw_private *vmw_priv)
-{
- return 0;
-}
-#endif
-
/**
* Kernel modesetting - vmwgfx_kms.c
*/
@@ -1232,9 +1212,6 @@ int vmw_kms_write_svga(struct vmw_private *vmw_priv,
bool vmw_kms_validate_mode_vram(struct vmw_private *dev_priv,
uint32_t pitch,
uint32_t height);
-u32 vmw_get_vblank_counter(struct drm_crtc *crtc);
-int vmw_enable_vblank(struct drm_crtc *crtc);
-void vmw_disable_vblank(struct drm_crtc *crtc);
int vmw_kms_present(struct vmw_private *dev_priv,
struct drm_file *file_priv,
struct vmw_framebuffer *vfb,
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c
index f085dbd..a5379f6 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 OR MIT
/**************************************************************************
*
- * Copyright 2009 - 2015 VMware, Inc., Palo Alto, CA., USA
+ * Copyright 2009 - 2022 VMware, Inc., Palo Alto, CA., USA
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the
@@ -25,6 +25,7 @@
*
**************************************************************************/
#include <linux/sync_file.h>
+#include <linux/hashtable.h>
#include "vmwgfx_drv.h"
#include "vmwgfx_reg.h"
@@ -34,7 +35,6 @@
#include "vmwgfx_binding.h"
#include "vmwgfx_mksstat.h"
-#define VMW_RES_HT_ORDER 12
/*
* Helper macro to get dx_ctx_node if available otherwise print an error
@@ -1047,7 +1047,7 @@ static int vmw_query_bo_switch_prepare(struct vmw_private *dev_priv,
if (unlikely(new_query_bo != sw_context->cur_query_bo)) {
- if (unlikely(new_query_bo->base.resource->num_pages > 4)) {
+ if (unlikely(PFN_UP(new_query_bo->base.resource->size) > 4)) {
VMW_DEBUG_USER("Query buffer too large.\n");
return -EINVAL;
}
@@ -3869,7 +3869,6 @@ int vmw_execbuf_fence_commands(struct drm_file *file_priv,
* @fence: Pointer to the fenc object.
* @fence_handle: User-space fence handle.
* @out_fence_fd: exported file descriptor for the fence. -1 if not used
- * @sync_file: Only used to clean up in case of an error in this function.
*
* This function copies fence information to user-space. If copying fails, the
* user-space struct drm_vmw_fence_rep::error member is hopefully left
@@ -4101,7 +4100,7 @@ int vmw_execbuf_process(struct drm_file *file_priv,
int ret;
int32_t out_fence_fd = -1;
struct sync_file *sync_file = NULL;
- DECLARE_VAL_CONTEXT(val_ctx, &sw_context->res_ht, 1);
+ DECLARE_VAL_CONTEXT(val_ctx, sw_context, 1);
if (flags & DRM_VMW_EXECBUF_FLAG_EXPORT_FENCE_FD) {
out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
@@ -4164,14 +4163,6 @@ int vmw_execbuf_process(struct drm_file *file_priv,
if (sw_context->staged_bindings)
vmw_binding_state_reset(sw_context->staged_bindings);
- if (!sw_context->res_ht_initialized) {
- ret = vmwgfx_ht_create(&sw_context->res_ht, VMW_RES_HT_ORDER);
- if (unlikely(ret != 0))
- goto out_unlock;
-
- sw_context->res_ht_initialized = true;
- }
-
INIT_LIST_HEAD(&sw_context->staged_cmd_res);
sw_context->ctx = &val_ctx;
ret = vmw_execbuf_tie_context(dev_priv, sw_context, dx_context_handle);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c b/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c
deleted file mode 100644
index 5b85b47..0000000
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c
+++ /dev/null
@@ -1,831 +0,0 @@
-/**************************************************************************
- *
- * Copyright © 2007 David Airlie
- * Copyright © 2009-2015 VMware, Inc., Palo Alto, CA., USA
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sub license, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial portions
- * of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
- * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
- * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
- * USE OR OTHER DEALINGS IN THE SOFTWARE.
- *
- **************************************************************************/
-
-#include <linux/fb.h>
-#include <linux/pci.h>
-
-#include <drm/drm_fourcc.h>
-#include <drm/ttm/ttm_placement.h>
-
-#include "vmwgfx_drv.h"
-#include "vmwgfx_kms.h"
-
-#define VMW_DIRTY_DELAY (HZ / 30)
-
-struct vmw_fb_par {
- struct vmw_private *vmw_priv;
-
- void *vmalloc;
-
- struct mutex bo_mutex;
- struct vmw_buffer_object *vmw_bo;
- unsigned bo_size;
- struct drm_framebuffer *set_fb;
- struct drm_display_mode *set_mode;
- u32 fb_x;
- u32 fb_y;
- bool bo_iowrite;
-
- u32 pseudo_palette[17];
-
- unsigned max_width;
- unsigned max_height;
-
- struct {
- spinlock_t lock;
- bool active;
- unsigned x1;
- unsigned y1;
- unsigned x2;
- unsigned y2;
- } dirty;
-
- struct drm_crtc *crtc;
- struct drm_connector *con;
- struct delayed_work local_work;
-};
-
-static int vmw_fb_setcolreg(unsigned regno, unsigned red, unsigned green,
- unsigned blue, unsigned transp,
- struct fb_info *info)
-{
- struct vmw_fb_par *par = info->par;
- u32 *pal = par->pseudo_palette;
-
- if (regno > 15) {
- DRM_ERROR("Bad regno %u.\n", regno);
- return 1;
- }
-
- switch (par->set_fb->format->depth) {
- case 24:
- case 32:
- pal[regno] = ((red & 0xff00) << 8) |
- (green & 0xff00) |
- ((blue & 0xff00) >> 8);
- break;
- default:
- DRM_ERROR("Bad depth %u, bpp %u.\n",
- par->set_fb->format->depth,
- par->set_fb->format->cpp[0] * 8);
- return 1;
- }
-
- return 0;
-}
-
-static int vmw_fb_check_var(struct fb_var_screeninfo *var,
- struct fb_info *info)
-{
- int depth = var->bits_per_pixel;
- struct vmw_fb_par *par = info->par;
- struct vmw_private *vmw_priv = par->vmw_priv;
-
- switch (var->bits_per_pixel) {
- case 32:
- depth = (var->transp.length > 0) ? 32 : 24;
- break;
- default:
- DRM_ERROR("Bad bpp %u.\n", var->bits_per_pixel);
- return -EINVAL;
- }
-
- switch (depth) {
- case 24:
- var->red.offset = 16;
- var->green.offset = 8;
- var->blue.offset = 0;
- var->red.length = 8;
- var->green.length = 8;
- var->blue.length = 8;
- var->transp.length = 0;
- var->transp.offset = 0;
- break;
- case 32:
- var->red.offset = 16;
- var->green.offset = 8;
- var->blue.offset = 0;
- var->red.length = 8;
- var->green.length = 8;
- var->blue.length = 8;
- var->transp.length = 8;
- var->transp.offset = 24;
- break;
- default:
- DRM_ERROR("Bad depth %u.\n", depth);
- return -EINVAL;
- }
-
- if ((var->xoffset + var->xres) > par->max_width ||
- (var->yoffset + var->yres) > par->max_height) {
- DRM_ERROR("Requested geom can not fit in framebuffer\n");
- return -EINVAL;
- }
-
- if (!vmw_kms_validate_mode_vram(vmw_priv,
- var->xres * var->bits_per_pixel/8,
- var->yoffset + var->yres)) {
- DRM_ERROR("Requested geom can not fit in framebuffer\n");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int vmw_fb_blank(int blank, struct fb_info *info)
-{
- return 0;
-}
-
-/**
- * vmw_fb_dirty_flush - flush dirty regions to the kms framebuffer
- *
- * @work: The struct work_struct associated with this task.
- *
- * This function flushes the dirty regions of the vmalloc framebuffer to the
- * kms framebuffer, and if the kms framebuffer is visible, also updated the
- * corresponding displays. Note that this function runs even if the kms
- * framebuffer is not bound to a crtc and thus not visible, but it's turned
- * off during hibernation using the par->dirty.active bool.
- */
-static void vmw_fb_dirty_flush(struct work_struct *work)
-{
- struct vmw_fb_par *par = container_of(work, struct vmw_fb_par,
- local_work.work);
- struct vmw_private *vmw_priv = par->vmw_priv;
- struct fb_info *info = vmw_priv->fb_info;
- unsigned long irq_flags;
- s32 dst_x1, dst_x2, dst_y1, dst_y2, w = 0, h = 0;
- u32 cpp, max_x, max_y;
- struct drm_clip_rect clip;
- struct drm_framebuffer *cur_fb;
- u8 *src_ptr, *dst_ptr;
- struct vmw_buffer_object *vbo = par->vmw_bo;
- void *virtual;
-
- if (!READ_ONCE(par->dirty.active))
- return;
-
- mutex_lock(&par->bo_mutex);
- cur_fb = par->set_fb;
- if (!cur_fb)
- goto out_unlock;
-
- (void) ttm_bo_reserve(&vbo->base, false, false, NULL);
- virtual = vmw_bo_map_and_cache(vbo);
- if (!virtual)
- goto out_unreserve;
-
- spin_lock_irqsave(&par->dirty.lock, irq_flags);
- if (!par->dirty.active) {
- spin_unlock_irqrestore(&par->dirty.lock, irq_flags);
- goto out_unreserve;
- }
-
- /*
- * Handle panning when copying from vmalloc to framebuffer.
- * Clip dirty area to framebuffer.
- */
- cpp = cur_fb->format->cpp[0];
- max_x = par->fb_x + cur_fb->width;
- max_y = par->fb_y + cur_fb->height;
-
- dst_x1 = par->dirty.x1 - par->fb_x;
- dst_y1 = par->dirty.y1 - par->fb_y;
- dst_x1 = max_t(s32, dst_x1, 0);
- dst_y1 = max_t(s32, dst_y1, 0);
-
- dst_x2 = par->dirty.x2 - par->fb_x;
- dst_y2 = par->dirty.y2 - par->fb_y;
- dst_x2 = min_t(s32, dst_x2, max_x);
- dst_y2 = min_t(s32, dst_y2, max_y);
- w = dst_x2 - dst_x1;
- h = dst_y2 - dst_y1;
- w = max_t(s32, 0, w);
- h = max_t(s32, 0, h);
-
- par->dirty.x1 = par->dirty.x2 = 0;
- par->dirty.y1 = par->dirty.y2 = 0;
- spin_unlock_irqrestore(&par->dirty.lock, irq_flags);
-
- if (w && h) {
- dst_ptr = (u8 *)virtual +
- (dst_y1 * par->set_fb->pitches[0] + dst_x1 * cpp);
- src_ptr = (u8 *)par->vmalloc +
- ((dst_y1 + par->fb_y) * info->fix.line_length +
- (dst_x1 + par->fb_x) * cpp);
-
- while (h-- > 0) {
- memcpy(dst_ptr, src_ptr, w*cpp);
- dst_ptr += par->set_fb->pitches[0];
- src_ptr += info->fix.line_length;
- }
-
- clip.x1 = dst_x1;
- clip.x2 = dst_x2;
- clip.y1 = dst_y1;
- clip.y2 = dst_y2;
- }
-
-out_unreserve:
- ttm_bo_unreserve(&vbo->base);
- if (w && h) {
- WARN_ON_ONCE(par->set_fb->funcs->dirty(cur_fb, NULL, 0, 0,
- &clip, 1));
- vmw_cmd_flush(vmw_priv, false);
- }
-out_unlock:
- mutex_unlock(&par->bo_mutex);
-}
-
-static void vmw_fb_dirty_mark(struct vmw_fb_par *par,
- unsigned x1, unsigned y1,
- unsigned width, unsigned height)
-{
- unsigned long flags;
- unsigned x2 = x1 + width;
- unsigned y2 = y1 + height;
-
- spin_lock_irqsave(&par->dirty.lock, flags);
- if (par->dirty.x1 == par->dirty.x2) {
- par->dirty.x1 = x1;
- par->dirty.y1 = y1;
- par->dirty.x2 = x2;
- par->dirty.y2 = y2;
- /* if we are active start the dirty work
- * we share the work with the defio system */
- if (par->dirty.active)
- schedule_delayed_work(&par->local_work,
- VMW_DIRTY_DELAY);
- } else {
- if (x1 < par->dirty.x1)
- par->dirty.x1 = x1;
- if (y1 < par->dirty.y1)
- par->dirty.y1 = y1;
- if (x2 > par->dirty.x2)
- par->dirty.x2 = x2;
- if (y2 > par->dirty.y2)
- par->dirty.y2 = y2;
- }
- spin_unlock_irqrestore(&par->dirty.lock, flags);
-}
-
-static int vmw_fb_pan_display(struct fb_var_screeninfo *var,
- struct fb_info *info)
-{
- struct vmw_fb_par *par = info->par;
-
- if ((var->xoffset + var->xres) > var->xres_virtual ||
- (var->yoffset + var->yres) > var->yres_virtual) {
- DRM_ERROR("Requested panning can not fit in framebuffer\n");
- return -EINVAL;
- }
-
- mutex_lock(&par->bo_mutex);
- par->fb_x = var->xoffset;
- par->fb_y = var->yoffset;
- if (par->set_fb)
- vmw_fb_dirty_mark(par, par->fb_x, par->fb_y, par->set_fb->width,
- par->set_fb->height);
- mutex_unlock(&par->bo_mutex);
-
- return 0;
-}
-
-static void vmw_deferred_io(struct fb_info *info, struct list_head *pagereflist)
-{
- struct vmw_fb_par *par = info->par;
- unsigned long start, end, min, max;
- unsigned long flags;
- struct fb_deferred_io_pageref *pageref;
- int y1, y2;
-
- min = ULONG_MAX;
- max = 0;
- list_for_each_entry(pageref, pagereflist, list) {
- start = pageref->offset;
- end = start + PAGE_SIZE - 1;
- min = min(min, start);
- max = max(max, end);
- }
-
- if (min < max) {
- y1 = min / info->fix.line_length;
- y2 = (max / info->fix.line_length) + 1;
-
- spin_lock_irqsave(&par->dirty.lock, flags);
- par->dirty.x1 = 0;
- par->dirty.y1 = y1;
- par->dirty.x2 = info->var.xres;
- par->dirty.y2 = y2;
- spin_unlock_irqrestore(&par->dirty.lock, flags);
-
- /*
- * Since we've already waited on this work once, try to
- * execute asap.
- */
- cancel_delayed_work(&par->local_work);
- schedule_delayed_work(&par->local_work, 0);
- }
-};
-
-static struct fb_deferred_io vmw_defio = {
- .delay = VMW_DIRTY_DELAY,
- .deferred_io = vmw_deferred_io,
-};
-
-/*
- * Draw code
- */
-
-static void vmw_fb_fillrect(struct fb_info *info, const struct fb_fillrect *rect)
-{
- cfb_fillrect(info, rect);
- vmw_fb_dirty_mark(info->par, rect->dx, rect->dy,
- rect->width, rect->height);
-}
-
-static void vmw_fb_copyarea(struct fb_info *info, const struct fb_copyarea *region)
-{
- cfb_copyarea(info, region);
- vmw_fb_dirty_mark(info->par, region->dx, region->dy,
- region->width, region->height);
-}
-
-static void vmw_fb_imageblit(struct fb_info *info, const struct fb_image *image)
-{
- cfb_imageblit(info, image);
- vmw_fb_dirty_mark(info->par, image->dx, image->dy,
- image->width, image->height);
-}
-
-/*
- * Bring up code
- */
-
-static int vmw_fb_create_bo(struct vmw_private *vmw_priv,
- size_t size, struct vmw_buffer_object **out)
-{
- struct vmw_buffer_object *vmw_bo;
- int ret;
-
- ret = vmw_bo_create(vmw_priv, size,
- &vmw_sys_placement,
- false, false,
- &vmw_bo_bo_free, &vmw_bo);
- if (unlikely(ret != 0))
- return ret;
-
- *out = vmw_bo;
-
- return ret;
-}
-
-static int vmw_fb_compute_depth(struct fb_var_screeninfo *var,
- int *depth)
-{
- switch (var->bits_per_pixel) {
- case 32:
- *depth = (var->transp.length > 0) ? 32 : 24;
- break;
- default:
- DRM_ERROR("Bad bpp %u.\n", var->bits_per_pixel);
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int vmwgfx_set_config_internal(struct drm_mode_set *set)
-{
- struct drm_crtc *crtc = set->crtc;
- struct drm_modeset_acquire_ctx ctx;
- int ret;
-
- drm_modeset_acquire_init(&ctx, 0);
-
-restart:
- ret = crtc->funcs->set_config(set, &ctx);
-
- if (ret == -EDEADLK) {
- drm_modeset_backoff(&ctx);
- goto restart;
- }
-
- drm_modeset_drop_locks(&ctx);
- drm_modeset_acquire_fini(&ctx);
-
- return ret;
-}
-
-static int vmw_fb_kms_detach(struct vmw_fb_par *par,
- bool detach_bo,
- bool unref_bo)
-{
- struct drm_framebuffer *cur_fb = par->set_fb;
- int ret;
-
- /* Detach the KMS framebuffer from crtcs */
- if (par->set_mode) {
- struct drm_mode_set set;
-
- set.crtc = par->crtc;
- set.x = 0;
- set.y = 0;
- set.mode = NULL;
- set.fb = NULL;
- set.num_connectors = 0;
- set.connectors = &par->con;
- ret = vmwgfx_set_config_internal(&set);
- if (ret) {
- DRM_ERROR("Could not unset a mode.\n");
- return ret;
- }
- drm_mode_destroy(&par->vmw_priv->drm, par->set_mode);
- par->set_mode = NULL;
- }
-
- if (cur_fb) {
- drm_framebuffer_put(cur_fb);
- par->set_fb = NULL;
- }
-
- if (par->vmw_bo && detach_bo && unref_bo)
- vmw_bo_unreference(&par->vmw_bo);
-
- return 0;
-}
-
-static int vmw_fb_kms_framebuffer(struct fb_info *info)
-{
- struct drm_mode_fb_cmd2 mode_cmd = {0};
- struct vmw_fb_par *par = info->par;
- struct fb_var_screeninfo *var = &info->var;
- struct drm_framebuffer *cur_fb;
- struct vmw_framebuffer *vfb;
- int ret = 0, depth;
- size_t new_bo_size;
-
- ret = vmw_fb_compute_depth(var, &depth);
- if (ret)
- return ret;
-
- mode_cmd.width = var->xres;
- mode_cmd.height = var->yres;
- mode_cmd.pitches[0] = ((var->bits_per_pixel + 7) / 8) * mode_cmd.width;
- mode_cmd.pixel_format =
- drm_mode_legacy_fb_format(var->bits_per_pixel, depth);
-
- cur_fb = par->set_fb;
- if (cur_fb && cur_fb->width == mode_cmd.width &&
- cur_fb->height == mode_cmd.height &&
- cur_fb->format->format == mode_cmd.pixel_format &&
- cur_fb->pitches[0] == mode_cmd.pitches[0])
- return 0;
-
- /* Need new buffer object ? */
- new_bo_size = (size_t) mode_cmd.pitches[0] * (size_t) mode_cmd.height;
- ret = vmw_fb_kms_detach(par,
- par->bo_size < new_bo_size ||
- par->bo_size > 2*new_bo_size,
- true);
- if (ret)
- return ret;
-
- if (!par->vmw_bo) {
- ret = vmw_fb_create_bo(par->vmw_priv, new_bo_size,
- &par->vmw_bo);
- if (ret) {
- DRM_ERROR("Failed creating a buffer object for "
- "fbdev.\n");
- return ret;
- }
- par->bo_size = new_bo_size;
- }
-
- vfb = vmw_kms_new_framebuffer(par->vmw_priv, par->vmw_bo, NULL,
- true, &mode_cmd);
- if (IS_ERR(vfb))
- return PTR_ERR(vfb);
-
- par->set_fb = &vfb->base;
-
- return 0;
-}
-
-static int vmw_fb_set_par(struct fb_info *info)
-{
- struct vmw_fb_par *par = info->par;
- struct vmw_private *vmw_priv = par->vmw_priv;
- struct drm_mode_set set;
- struct fb_var_screeninfo *var = &info->var;
- struct drm_display_mode new_mode = { DRM_MODE("fb_mode",
- DRM_MODE_TYPE_DRIVER,
- 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
- DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_PVSYNC)
- };
- struct drm_display_mode *mode;
- int ret;
-
- mode = drm_mode_duplicate(&vmw_priv->drm, &new_mode);
- if (!mode) {
- DRM_ERROR("Could not create new fb mode.\n");
- return -ENOMEM;
- }
-
- mode->hdisplay = var->xres;
- mode->vdisplay = var->yres;
- vmw_guess_mode_timing(mode);
-
- if (!vmw_kms_validate_mode_vram(vmw_priv,
- mode->hdisplay *
- DIV_ROUND_UP(var->bits_per_pixel, 8),
- mode->vdisplay)) {
- drm_mode_destroy(&vmw_priv->drm, mode);
- return -EINVAL;
- }
-
- mutex_lock(&par->bo_mutex);
- ret = vmw_fb_kms_framebuffer(info);
- if (ret)
- goto out_unlock;
-
- par->fb_x = var->xoffset;
- par->fb_y = var->yoffset;
-
- set.crtc = par->crtc;
- set.x = 0;
- set.y = 0;
- set.mode = mode;
- set.fb = par->set_fb;
- set.num_connectors = 1;
- set.connectors = &par->con;
-
- ret = vmwgfx_set_config_internal(&set);
- if (ret)
- goto out_unlock;
-
- vmw_fb_dirty_mark(par, par->fb_x, par->fb_y,
- par->set_fb->width, par->set_fb->height);
-
- /* If there already was stuff dirty we wont
- * schedule a new work, so lets do it now */
-
- schedule_delayed_work(&par->local_work, 0);
-
-out_unlock:
- if (par->set_mode)
- drm_mode_destroy(&vmw_priv->drm, par->set_mode);
- par->set_mode = mode;
-
- mutex_unlock(&par->bo_mutex);
-
- return ret;
-}
-
-
-static const struct fb_ops vmw_fb_ops = {
- .owner = THIS_MODULE,
- .fb_check_var = vmw_fb_check_var,
- .fb_set_par = vmw_fb_set_par,
- .fb_setcolreg = vmw_fb_setcolreg,
- .fb_fillrect = vmw_fb_fillrect,
- .fb_copyarea = vmw_fb_copyarea,
- .fb_imageblit = vmw_fb_imageblit,
- .fb_pan_display = vmw_fb_pan_display,
- .fb_blank = vmw_fb_blank,
- .fb_mmap = fb_deferred_io_mmap,
-};
-
-int vmw_fb_init(struct vmw_private *vmw_priv)
-{
- struct device *device = vmw_priv->drm.dev;
- struct vmw_fb_par *par;
- struct fb_info *info;
- unsigned fb_width, fb_height;
- unsigned int fb_bpp, fb_pitch, fb_size;
- struct drm_display_mode *init_mode;
- int ret;
-
- fb_bpp = 32;
-
- /* XXX As shouldn't these be as well. */
- fb_width = min(vmw_priv->fb_max_width, (unsigned)2048);
- fb_height = min(vmw_priv->fb_max_height, (unsigned)2048);
-
- fb_pitch = fb_width * fb_bpp / 8;
- fb_size = fb_pitch * fb_height;
-
- info = framebuffer_alloc(sizeof(*par), device);
- if (!info)
- return -ENOMEM;
-
- /*
- * Par
- */
- vmw_priv->fb_info = info;
- par = info->par;
- memset(par, 0, sizeof(*par));
- INIT_DELAYED_WORK(&par->local_work, &vmw_fb_dirty_flush);
- par->vmw_priv = vmw_priv;
- par->vmalloc = NULL;
- par->max_width = fb_width;
- par->max_height = fb_height;
-
- ret = vmw_kms_fbdev_init_data(vmw_priv, 0, par->max_width,
- par->max_height, &par->con,
- &par->crtc, &init_mode);
- if (ret)
- goto err_kms;
-
- info->var.xres = init_mode->hdisplay;
- info->var.yres = init_mode->vdisplay;
-
- /*
- * Create buffers and alloc memory
- */
- par->vmalloc = vzalloc(fb_size);
- if (unlikely(par->vmalloc == NULL)) {
- ret = -ENOMEM;
- goto err_free;
- }
-
- /*
- * Fixed and var
- */
- strcpy(info->fix.id, "svgadrmfb");
- info->fix.type = FB_TYPE_PACKED_PIXELS;
- info->fix.visual = FB_VISUAL_TRUECOLOR;
- info->fix.type_aux = 0;
- info->fix.xpanstep = 1; /* doing it in hw */
- info->fix.ypanstep = 1; /* doing it in hw */
- info->fix.ywrapstep = 0;
- info->fix.accel = FB_ACCEL_NONE;
- info->fix.line_length = fb_pitch;
-
- info->fix.smem_start = 0;
- info->fix.smem_len = fb_size;
-
- info->pseudo_palette = par->pseudo_palette;
- info->screen_base = (char __iomem *)par->vmalloc;
- info->screen_size = fb_size;
-
- info->fbops = &vmw_fb_ops;
-
- /* 24 depth per default */
- info->var.red.offset = 16;
- info->var.green.offset = 8;
- info->var.blue.offset = 0;
- info->var.red.length = 8;
- info->var.green.length = 8;
- info->var.blue.length = 8;
- info->var.transp.offset = 0;
- info->var.transp.length = 0;
-
- info->var.xres_virtual = fb_width;
- info->var.yres_virtual = fb_height;
- info->var.bits_per_pixel = fb_bpp;
- info->var.xoffset = 0;
- info->var.yoffset = 0;
- info->var.activate = FB_ACTIVATE_NOW;
- info->var.height = -1;
- info->var.width = -1;
-
- /* Use default scratch pixmap (info->pixmap.flags = FB_PIXMAP_SYSTEM) */
- info->apertures = alloc_apertures(1);
- if (!info->apertures) {
- ret = -ENOMEM;
- goto err_aper;
- }
- info->apertures->ranges[0].base = vmw_priv->vram_start;
- info->apertures->ranges[0].size = vmw_priv->vram_size;
-
- /*
- * Dirty & Deferred IO
- */
- par->dirty.x1 = par->dirty.x2 = 0;
- par->dirty.y1 = par->dirty.y2 = 0;
- par->dirty.active = true;
- spin_lock_init(&par->dirty.lock);
- mutex_init(&par->bo_mutex);
- info->fbdefio = &vmw_defio;
- fb_deferred_io_init(info);
-
- ret = register_framebuffer(info);
- if (unlikely(ret != 0))
- goto err_defio;
-
- vmw_fb_set_par(info);
-
- return 0;
-
-err_defio:
- fb_deferred_io_cleanup(info);
-err_aper:
-err_free:
- vfree(par->vmalloc);
-err_kms:
- framebuffer_release(info);
- vmw_priv->fb_info = NULL;
-
- return ret;
-}
-
-int vmw_fb_close(struct vmw_private *vmw_priv)
-{
- struct fb_info *info;
- struct vmw_fb_par *par;
-
- if (!vmw_priv->fb_info)
- return 0;
-
- info = vmw_priv->fb_info;
- par = info->par;
-
- /* ??? order */
- fb_deferred_io_cleanup(info);
- cancel_delayed_work_sync(&par->local_work);
- unregister_framebuffer(info);
-
- mutex_lock(&par->bo_mutex);
- (void) vmw_fb_kms_detach(par, true, true);
- mutex_unlock(&par->bo_mutex);
-
- vfree(par->vmalloc);
- framebuffer_release(info);
-
- return 0;
-}
-
-int vmw_fb_off(struct vmw_private *vmw_priv)
-{
- struct fb_info *info;
- struct vmw_fb_par *par;
- unsigned long flags;
-
- if (!vmw_priv->fb_info)
- return -EINVAL;
-
- info = vmw_priv->fb_info;
- par = info->par;
-
- spin_lock_irqsave(&par->dirty.lock, flags);
- par->dirty.active = false;
- spin_unlock_irqrestore(&par->dirty.lock, flags);
-
- flush_delayed_work(&info->deferred_work);
- flush_delayed_work(&par->local_work);
-
- return 0;
-}
-
-int vmw_fb_on(struct vmw_private *vmw_priv)
-{
- struct fb_info *info;
- struct vmw_fb_par *par;
- unsigned long flags;
-
- if (!vmw_priv->fb_info)
- return -EINVAL;
-
- info = vmw_priv->fb_info;
- par = info->par;
-
- spin_lock_irqsave(&par->dirty.lock, flags);
- par->dirty.active = true;
- spin_unlock_irqrestore(&par->dirty.lock, flags);
-
- /*
- * Need to reschedule a dirty update, because otherwise that's
- * only done in dirty_mark() if the previous coalesced
- * dirty region was empty.
- */
- schedule_delayed_work(&par->local_work, 0);
-
- return 0;
-}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c
index 60e3cc5..abd5e33 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c
@@ -71,7 +71,7 @@ static int vmw_gmrid_man_get_node(struct ttm_resource_manager *man,
spin_lock(&gman->lock);
if (gman->max_gmr_pages > 0) {
- gman->used_gmr_pages += (*res)->num_pages;
+ gman->used_gmr_pages += PFN_UP((*res)->size);
/*
* Because the graphics memory is a soft limit we can try to
* expand it instead of letting the userspace apps crash.
@@ -114,7 +114,7 @@ static int vmw_gmrid_man_get_node(struct ttm_resource_manager *man,
return 0;
nospace:
- gman->used_gmr_pages -= (*res)->num_pages;
+ gman->used_gmr_pages -= PFN_UP((*res)->size);
spin_unlock(&gman->lock);
ida_free(&gman->gmr_ida, id);
ttm_resource_fini(man, *res);
@@ -129,7 +129,7 @@ static void vmw_gmrid_man_put_node(struct ttm_resource_manager *man,
ida_free(&gman->gmr_ida, res->start);
spin_lock(&gman->lock);
- gman->used_gmr_pages -= res->num_pages;
+ gman->used_gmr_pages -= PFN_UP(res->size);
spin_unlock(&gman->lock);
ttm_resource_fini(man, res);
kfree(res);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.c b/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.c
deleted file mode 100644
index 06aebc1..0000000
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.c
+++ /dev/null
@@ -1,199 +0,0 @@
-/*
- * Copyright 2006 Tungsten Graphics, Inc., Bismarck, ND. USA.
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sub license, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial portions
- * of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
- * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
- * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
- * USE OR OTHER DEALINGS IN THE SOFTWARE.
- */
-
-/*
- * Simple open hash tab implementation.
- *
- * Authors:
- * Thomas Hellström <thomas-at-tungstengraphics-dot-com>
- */
-
-#include <linux/export.h>
-#include <linux/hash.h>
-#include <linux/mm.h>
-#include <linux/rculist.h>
-#include <linux/slab.h>
-#include <linux/vmalloc.h>
-
-#include <drm/drm_print.h>
-
-#include "vmwgfx_hashtab.h"
-
-int vmwgfx_ht_create(struct vmwgfx_open_hash *ht, unsigned int order)
-{
- unsigned int size = 1 << order;
-
- ht->order = order;
- ht->table = NULL;
- if (size <= PAGE_SIZE / sizeof(*ht->table))
- ht->table = kcalloc(size, sizeof(*ht->table), GFP_KERNEL);
- else
- ht->table = vzalloc(array_size(size, sizeof(*ht->table)));
- if (!ht->table) {
- DRM_ERROR("Out of memory for hash table\n");
- return -ENOMEM;
- }
- return 0;
-}
-
-void vmwgfx_ht_verbose_list(struct vmwgfx_open_hash *ht, unsigned long key)
-{
- struct vmwgfx_hash_item *entry;
- struct hlist_head *h_list;
- unsigned int hashed_key;
- int count = 0;
-
- hashed_key = hash_long(key, ht->order);
- DRM_DEBUG("Key is 0x%08lx, Hashed key is 0x%08x\n", key, hashed_key);
- h_list = &ht->table[hashed_key];
- hlist_for_each_entry(entry, h_list, head)
- DRM_DEBUG("count %d, key: 0x%08lx\n", count++, entry->key);
-}
-
-static struct hlist_node *vmwgfx_ht_find_key(struct vmwgfx_open_hash *ht, unsigned long key)
-{
- struct vmwgfx_hash_item *entry;
- struct hlist_head *h_list;
- unsigned int hashed_key;
-
- hashed_key = hash_long(key, ht->order);
- h_list = &ht->table[hashed_key];
- hlist_for_each_entry(entry, h_list, head) {
- if (entry->key == key)
- return &entry->head;
- if (entry->key > key)
- break;
- }
- return NULL;
-}
-
-static struct hlist_node *vmwgfx_ht_find_key_rcu(struct vmwgfx_open_hash *ht, unsigned long key)
-{
- struct vmwgfx_hash_item *entry;
- struct hlist_head *h_list;
- unsigned int hashed_key;
-
- hashed_key = hash_long(key, ht->order);
- h_list = &ht->table[hashed_key];
- hlist_for_each_entry_rcu(entry, h_list, head) {
- if (entry->key == key)
- return &entry->head;
- if (entry->key > key)
- break;
- }
- return NULL;
-}
-
-int vmwgfx_ht_insert_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item)
-{
- struct vmwgfx_hash_item *entry;
- struct hlist_head *h_list;
- struct hlist_node *parent;
- unsigned int hashed_key;
- unsigned long key = item->key;
-
- hashed_key = hash_long(key, ht->order);
- h_list = &ht->table[hashed_key];
- parent = NULL;
- hlist_for_each_entry(entry, h_list, head) {
- if (entry->key == key)
- return -EINVAL;
- if (entry->key > key)
- break;
- parent = &entry->head;
- }
- if (parent)
- hlist_add_behind_rcu(&item->head, parent);
- else
- hlist_add_head_rcu(&item->head, h_list);
- return 0;
-}
-
-/*
- * Just insert an item and return any "bits" bit key that hasn't been
- * used before.
- */
-int vmwgfx_ht_just_insert_please(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item,
- unsigned long seed, int bits, int shift,
- unsigned long add)
-{
- int ret;
- unsigned long mask = (1UL << bits) - 1;
- unsigned long first, unshifted_key;
-
- unshifted_key = hash_long(seed, bits);
- first = unshifted_key;
- do {
- item->key = (unshifted_key << shift) + add;
- ret = vmwgfx_ht_insert_item(ht, item);
- if (ret)
- unshifted_key = (unshifted_key + 1) & mask;
- } while (ret && (unshifted_key != first));
-
- if (ret) {
- DRM_ERROR("Available key bit space exhausted\n");
- return -EINVAL;
- }
- return 0;
-}
-
-int vmwgfx_ht_find_item(struct vmwgfx_open_hash *ht, unsigned long key,
- struct vmwgfx_hash_item **item)
-{
- struct hlist_node *list;
-
- list = vmwgfx_ht_find_key_rcu(ht, key);
- if (!list)
- return -EINVAL;
-
- *item = hlist_entry(list, struct vmwgfx_hash_item, head);
- return 0;
-}
-
-int vmwgfx_ht_remove_key(struct vmwgfx_open_hash *ht, unsigned long key)
-{
- struct hlist_node *list;
-
- list = vmwgfx_ht_find_key(ht, key);
- if (list) {
- hlist_del_init_rcu(list);
- return 0;
- }
- return -EINVAL;
-}
-
-int vmwgfx_ht_remove_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item)
-{
- hlist_del_init_rcu(&item->head);
- return 0;
-}
-
-void vmwgfx_ht_remove(struct vmwgfx_open_hash *ht)
-{
- if (ht->table) {
- kvfree(ht->table);
- ht->table = NULL;
- }
-}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.h b/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.h
deleted file mode 100644
index a9ce129..0000000
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_hashtab.h
+++ /dev/null
@@ -1,83 +0,0 @@
-/*
- * Copyright 2006 Tungsten Graphics, Inc., Bismack, ND. USA.
- * All Rights Reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sub license, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice (including the
- * next paragraph) shall be included in all copies or substantial portions
- * of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
- * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
- * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
- * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
- * USE OR OTHER DEALINGS IN THE SOFTWARE.
- */
-
-/*
- * Simple open hash tab implementation.
- *
- * Authors:
- * Thomas Hellström <thomas-at-tungstengraphics-dot-com>
- */
-
-/*
- * TODO: Replace this hashtable with Linux' generic implementation
- * from <linux/hashtable.h>.
- */
-
-#ifndef VMWGFX_HASHTAB_H
-#define VMWGFX_HASHTAB_H
-
-#include <linux/list.h>
-
-#define drm_hash_entry(_ptr, _type, _member) container_of(_ptr, _type, _member)
-
-struct vmwgfx_hash_item {
- struct hlist_node head;
- unsigned long key;
-};
-
-struct vmwgfx_open_hash {
- struct hlist_head *table;
- u8 order;
-};
-
-int vmwgfx_ht_create(struct vmwgfx_open_hash *ht, unsigned int order);
-int vmwgfx_ht_insert_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item);
-int vmwgfx_ht_just_insert_please(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item,
- unsigned long seed, int bits, int shift,
- unsigned long add);
-int vmwgfx_ht_find_item(struct vmwgfx_open_hash *ht, unsigned long key,
- struct vmwgfx_hash_item **item);
-
-void vmwgfx_ht_verbose_list(struct vmwgfx_open_hash *ht, unsigned long key);
-int vmwgfx_ht_remove_key(struct vmwgfx_open_hash *ht, unsigned long key);
-int vmwgfx_ht_remove_item(struct vmwgfx_open_hash *ht, struct vmwgfx_hash_item *item);
-void vmwgfx_ht_remove(struct vmwgfx_open_hash *ht);
-
-/*
- * RCU-safe interface
- *
- * The user of this API needs to make sure that two or more instances of the
- * hash table manipulation functions are never run simultaneously.
- * The lookup function vmwgfx_ht_find_item_rcu may, however, run simultaneously
- * with any of the manipulation functions as long as it's called from within
- * an RCU read-locked section.
- */
-#define vmwgfx_ht_insert_item_rcu vmwgfx_ht_insert_item
-#define vmwgfx_ht_just_insert_please_rcu vmwgfx_ht_just_insert_please
-#define vmwgfx_ht_remove_key_rcu vmwgfx_ht_remove_key
-#define vmwgfx_ht_remove_item_rcu vmwgfx_ht_remove_item
-#define vmwgfx_ht_find_item_rcu vmwgfx_ht_find_item
-
-#endif
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
index 214829c..257f090 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
@@ -25,15 +25,15 @@
*
**************************************************************************/
+#include "vmwgfx_kms.h"
+#include "vmw_surface_cache.h"
+
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_damage_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_rect.h>
#include <drm/drm_sysfs.h>
-#include <drm/drm_vblank.h>
-
-#include "vmwgfx_kms.h"
void vmw_du_cleanup(struct vmw_display_unit *du)
{
@@ -52,9 +52,9 @@ void vmw_du_cleanup(struct vmw_display_unit *du)
* Display Unit Cursor functions
*/
+static int vmw_du_cursor_plane_unmap_cm(struct vmw_plane_state *vps);
static void vmw_cursor_update_mob(struct vmw_private *dev_priv,
- struct ttm_buffer_object *bo,
- struct ttm_bo_kmap_obj *map,
+ struct vmw_plane_state *vps,
u32 *image, u32 width, u32 height,
u32 hotspotX, u32 hotspotY);
@@ -63,23 +63,23 @@ struct vmw_svga_fifo_cmd_define_cursor {
SVGAFifoCmdDefineAlphaCursor cursor;
};
-static void vmw_cursor_update_image(struct vmw_private *dev_priv,
- struct ttm_buffer_object *cm_bo,
- struct ttm_bo_kmap_obj *cm_map,
- u32 *image, u32 width, u32 height,
- u32 hotspotX, u32 hotspotY)
+/**
+ * vmw_send_define_cursor_cmd - queue a define cursor command
+ * @dev_priv: the private driver struct
+ * @image: buffer which holds the cursor image
+ * @width: width of the mouse cursor image
+ * @height: height of the mouse cursor image
+ * @hotspotX: the horizontal position of mouse hotspot
+ * @hotspotY: the vertical position of mouse hotspot
+ */
+static void vmw_send_define_cursor_cmd(struct vmw_private *dev_priv,
+ u32 *image, u32 width, u32 height,
+ u32 hotspotX, u32 hotspotY)
{
struct vmw_svga_fifo_cmd_define_cursor *cmd;
const u32 image_size = width * height * sizeof(*image);
const u32 cmd_size = sizeof(*cmd) + image_size;
- if (cm_bo != NULL) {
- vmw_cursor_update_mob(dev_priv, cm_bo, cm_map, image,
- width, height,
- hotspotX, hotspotY);
- return;
- }
-
/* Try to reserve fifocmd space and swallow any failures;
such reservations cannot be left unconsumed for long
under the risk of clogging other fifocmd users, so
@@ -87,7 +87,7 @@ static void vmw_cursor_update_image(struct vmw_private *dev_priv,
other fallible KMS-atomic resources at prepare_fb */
cmd = VMW_CMD_RESERVE(dev_priv, cmd_size);
- if (unlikely(cmd == NULL))
+ if (unlikely(!cmd))
return;
memset(cmd, 0, sizeof(*cmd));
@@ -105,11 +105,39 @@ static void vmw_cursor_update_image(struct vmw_private *dev_priv,
}
/**
+ * vmw_cursor_update_image - update the cursor image on the provided plane
+ * @dev_priv: the private driver struct
+ * @vps: the plane state of the cursor plane
+ * @image: buffer which holds the cursor image
+ * @width: width of the mouse cursor image
+ * @height: height of the mouse cursor image
+ * @hotspotX: the horizontal position of mouse hotspot
+ * @hotspotY: the vertical position of mouse hotspot
+ */
+static void vmw_cursor_update_image(struct vmw_private *dev_priv,
+ struct vmw_plane_state *vps,
+ u32 *image, u32 width, u32 height,
+ u32 hotspotX, u32 hotspotY)
+{
+ if (vps->cursor.bo)
+ vmw_cursor_update_mob(dev_priv, vps, image,
+ vps->base.crtc_w, vps->base.crtc_h,
+ hotspotX, hotspotY);
+
+ else
+ vmw_send_define_cursor_cmd(dev_priv, image, width, height,
+ hotspotX, hotspotY);
+}
+
+
+/**
* vmw_cursor_update_mob - Update cursor vis CursorMob mechanism
*
+ * Called from inside vmw_du_cursor_plane_atomic_update to actually
+ * make the cursor-image live.
+ *
* @dev_priv: device to work with
- * @bo: BO for the MOB
- * @map: kmap obj for the BO
+ * @vps: the plane state of the cursor plane
* @image: cursor source data to fill the MOB with
* @width: source data width
* @height: source data height
@@ -117,8 +145,7 @@ static void vmw_cursor_update_image(struct vmw_private *dev_priv,
* @hotspotY: cursor hotspot Y
*/
static void vmw_cursor_update_mob(struct vmw_private *dev_priv,
- struct ttm_buffer_object *bo,
- struct ttm_bo_kmap_obj *map,
+ struct vmw_plane_state *vps,
u32 *image, u32 width, u32 height,
u32 hotspotX, u32 hotspotY)
{
@@ -127,11 +154,11 @@ static void vmw_cursor_update_mob(struct vmw_private *dev_priv,
const u32 image_size = width * height * sizeof(*image);
bool dummy;
- BUG_ON(!image);
-
- header = (SVGAGBCursorHeader *)ttm_kmap_obj_virtual(map, &dummy);
+ header = ttm_kmap_obj_virtual(&vps->cursor.map, &dummy);
alpha_header = &header->header.alphaHeader;
+ memset(header, 0, sizeof(*header));
+
header->type = SVGA_ALPHA_CURSOR;
header->sizeInBytes = image_size;
@@ -141,102 +168,161 @@ static void vmw_cursor_update_mob(struct vmw_private *dev_priv,
alpha_header->height = height;
memcpy(header + 1, image, image_size);
-
- vmw_write(dev_priv, SVGA_REG_CURSOR_MOBID, bo->resource->start);
+ vmw_write(dev_priv, SVGA_REG_CURSOR_MOBID,
+ vps->cursor.bo->resource->start);
}
-void vmw_du_destroy_cursor_mob_array(struct vmw_cursor_plane *vcp)
-{
- size_t i;
- for (i = 0; i < ARRAY_SIZE(vcp->cursor_mob); i++) {
- if (vcp->cursor_mob[i] != NULL) {
- ttm_bo_unpin(vcp->cursor_mob[i]);
- ttm_bo_put(vcp->cursor_mob[i]);
- kfree(vcp->cursor_mob[i]);
- vcp->cursor_mob[i] = NULL;
+static u32 vmw_du_cursor_mob_size(u32 w, u32 h)
+{
+ return w * h * sizeof(u32) + sizeof(SVGAGBCursorHeader);
+}
+
+/**
+ * vmw_du_cursor_plane_acquire_image -- Acquire the image data
+ * @vps: cursor plane state
+ */
+static u32 *vmw_du_cursor_plane_acquire_image(struct vmw_plane_state *vps)
+{
+ bool dummy;
+ if (vps->surf) {
+ if (vps->surf_mapped)
+ return vmw_bo_map_and_cache(vps->surf->res.backup);
+ return vps->surf->snooper.image;
+ } else if (vps->bo)
+ return ttm_kmap_obj_virtual(&vps->bo->map, &dummy);
+ return NULL;
+}
+
+static bool vmw_du_cursor_plane_has_changed(struct vmw_plane_state *old_vps,
+ struct vmw_plane_state *new_vps)
+{
+ void *old_image;
+ void *new_image;
+ u32 size;
+ bool changed;
+
+ if (old_vps->base.crtc_w != new_vps->base.crtc_w ||
+ old_vps->base.crtc_h != new_vps->base.crtc_h)
+ return true;
+
+ if (old_vps->cursor.hotspot_x != new_vps->cursor.hotspot_x ||
+ old_vps->cursor.hotspot_y != new_vps->cursor.hotspot_y)
+ return true;
+
+ size = new_vps->base.crtc_w * new_vps->base.crtc_h * sizeof(u32);
+
+ old_image = vmw_du_cursor_plane_acquire_image(old_vps);
+ new_image = vmw_du_cursor_plane_acquire_image(new_vps);
+
+ changed = false;
+ if (old_image && new_image)
+ changed = memcmp(old_image, new_image, size) != 0;
+
+ return changed;
+}
+
+static void vmw_du_destroy_cursor_mob(struct ttm_buffer_object **bo)
+{
+ if (!(*bo))
+ return;
+
+ ttm_bo_unpin(*bo);
+ ttm_bo_put(*bo);
+ kfree(*bo);
+ *bo = NULL;
+}
+
+static void vmw_du_put_cursor_mob(struct vmw_cursor_plane *vcp,
+ struct vmw_plane_state *vps)
+{
+ u32 i;
+
+ if (!vps->cursor.bo)
+ return;
+
+ vmw_du_cursor_plane_unmap_cm(vps);
+
+ /* Look for a free slot to return this mob to the cache. */
+ for (i = 0; i < ARRAY_SIZE(vcp->cursor_mobs); i++) {
+ if (!vcp->cursor_mobs[i]) {
+ vcp->cursor_mobs[i] = vps->cursor.bo;
+ vps->cursor.bo = NULL;
+ return;
}
}
+
+ /* Cache is full: See if this mob is bigger than an existing mob. */
+ for (i = 0; i < ARRAY_SIZE(vcp->cursor_mobs); i++) {
+ if (vcp->cursor_mobs[i]->base.size <
+ vps->cursor.bo->base.size) {
+ vmw_du_destroy_cursor_mob(&vcp->cursor_mobs[i]);
+ vcp->cursor_mobs[i] = vps->cursor.bo;
+ vps->cursor.bo = NULL;
+ return;
+ }
+ }
+
+ /* Destroy it if it's not worth caching. */
+ vmw_du_destroy_cursor_mob(&vps->cursor.bo);
}
-#define CURSOR_MOB_SIZE(dimension) \
- ((dimension) * (dimension) * sizeof(u32) + sizeof(SVGAGBCursorHeader))
-
-int vmw_du_create_cursor_mob_array(struct vmw_cursor_plane *cursor)
+static int vmw_du_get_cursor_mob(struct vmw_cursor_plane *vcp,
+ struct vmw_plane_state *vps)
{
- struct vmw_private *dev_priv = cursor->base.dev->dev_private;
- uint32_t cursor_max_dim, mob_max_size;
- int ret = 0;
- size_t i;
+ struct vmw_private *dev_priv = vcp->base.dev->dev_private;
+ u32 size = vmw_du_cursor_mob_size(vps->base.crtc_w, vps->base.crtc_h);
+ u32 i;
+ u32 cursor_max_dim, mob_max_size;
+ int ret;
- if (!dev_priv->has_mob || (dev_priv->capabilities2 & SVGA_CAP2_CURSOR_MOB) == 0)
- return -ENOSYS;
+ if (!dev_priv->has_mob ||
+ (dev_priv->capabilities2 & SVGA_CAP2_CURSOR_MOB) == 0)
+ return -EINVAL;
mob_max_size = vmw_read(dev_priv, SVGA_REG_MOB_MAX_SIZE);
cursor_max_dim = vmw_read(dev_priv, SVGA_REG_CURSOR_MAX_DIMENSION);
- if (CURSOR_MOB_SIZE(cursor_max_dim) > mob_max_size)
- cursor_max_dim = 64; /* Mandatorily-supported cursor dimension */
+ if (size > mob_max_size || vps->base.crtc_w > cursor_max_dim ||
+ vps->base.crtc_h > cursor_max_dim)
+ return -EINVAL;
- for (i = 0; i < ARRAY_SIZE(cursor->cursor_mob); i++) {
- struct ttm_buffer_object **const bo = &cursor->cursor_mob[i];
-
- ret = vmw_bo_create_kernel(dev_priv,
- CURSOR_MOB_SIZE(cursor_max_dim),
- &vmw_mob_placement, bo);
-
- if (ret != 0)
- goto teardown;
-
- if ((*bo)->resource->mem_type != VMW_PL_MOB) {
- DRM_ERROR("Obtained buffer object is not a MOB.\n");
- ret = -ENOSYS;
- goto teardown;
- }
-
- /* Fence the mob creation so we are guarateed to have the mob */
- ret = ttm_bo_reserve(*bo, false, false, NULL);
-
- if (ret != 0)
- goto teardown;
-
- vmw_bo_fence_single(*bo, NULL);
-
- ttm_bo_unreserve(*bo);
-
- drm_info(&dev_priv->drm, "Using CursorMob mobid %lu, max dimension %u\n",
- (*bo)->resource->start, cursor_max_dim);
+ if (vps->cursor.bo) {
+ if (vps->cursor.bo->base.size >= size)
+ return 0;
+ vmw_du_put_cursor_mob(vcp, vps);
}
+ /* Look for an unused mob in the cache. */
+ for (i = 0; i < ARRAY_SIZE(vcp->cursor_mobs); i++) {
+ if (vcp->cursor_mobs[i] &&
+ vcp->cursor_mobs[i]->base.size >= size) {
+ vps->cursor.bo = vcp->cursor_mobs[i];
+ vcp->cursor_mobs[i] = NULL;
+ return 0;
+ }
+ }
+ /* Create a new mob if we can't find an existing one. */
+ ret = vmw_bo_create_kernel(dev_priv, size, &vmw_mob_placement,
+ &vps->cursor.bo);
+
+ if (ret != 0)
+ return ret;
+
+ /* Fence the mob creation so we are guarateed to have the mob */
+ ret = ttm_bo_reserve(vps->cursor.bo, false, false, NULL);
+ if (ret != 0)
+ goto teardown;
+
+ vmw_bo_fence_single(vps->cursor.bo, NULL);
+ ttm_bo_unreserve(vps->cursor.bo);
return 0;
teardown:
- vmw_du_destroy_cursor_mob_array(cursor);
-
+ vmw_du_destroy_cursor_mob(&vps->cursor.bo);
return ret;
}
-#undef CURSOR_MOB_SIZE
-
-static void vmw_cursor_update_bo(struct vmw_private *dev_priv,
- struct ttm_buffer_object *cm_bo,
- struct ttm_bo_kmap_obj *cm_map,
- struct vmw_buffer_object *bo,
- u32 width, u32 height,
- u32 hotspotX, u32 hotspotY)
-{
- void *virtual;
- bool dummy;
-
- virtual = ttm_kmap_obj_virtual(&bo->map, &dummy);
- if (virtual) {
- vmw_cursor_update_image(dev_priv, cm_bo, cm_map, virtual,
- width, height,
- hotspotX, hotspotY);
- atomic_dec(&bo->base_mapped_count);
- }
-}
-
static void vmw_cursor_update_position(struct vmw_private *dev_priv,
bool show, int x, int y)
@@ -266,7 +352,6 @@ static void vmw_cursor_update_position(struct vmw_private *dev_priv,
spin_unlock(&dev_priv->cursor_lock);
}
-
void vmw_kms_cursor_snoop(struct vmw_surface *srf,
struct ttm_object_file *tfile,
struct ttm_buffer_object *bo,
@@ -284,10 +369,13 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf,
SVGA3dCmdSurfaceDMA dma;
} *cmd;
int i, ret;
+ const struct SVGA3dSurfaceDesc *desc =
+ vmw_surface_get_desc(VMW_CURSOR_SNOOP_FORMAT);
+ const u32 image_pitch = VMW_CURSOR_SNOOP_WIDTH * desc->pitchBytesPerBlock;
cmd = container_of(header, struct vmw_dma_cmd, header);
- /* No snooper installed */
+ /* No snooper installed, nothing to copy */
if (!srf->snooper.image)
return;
@@ -308,7 +396,8 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf,
if (cmd->dma.guest.ptr.offset % PAGE_SIZE ||
box->x != 0 || box->y != 0 || box->z != 0 ||
box->srcx != 0 || box->srcy != 0 || box->srcz != 0 ||
- box->d != 1 || box_count != 1) {
+ box->d != 1 || box_count != 1 ||
+ box->w > VMW_CURSOR_SNOOP_WIDTH || box->h > VMW_CURSOR_SNOOP_HEIGHT) {
/* TODO handle none page aligned offsets */
/* TODO handle more dst & src != 0 */
/* TODO handle more then one copy */
@@ -322,7 +411,7 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf,
}
kmap_offset = cmd->dma.guest.ptr.offset >> PAGE_SHIFT;
- kmap_num = (64*64*4) >> PAGE_SHIFT;
+ kmap_num = (VMW_CURSOR_SNOOP_HEIGHT*image_pitch) >> PAGE_SHIFT;
ret = ttm_bo_reserve(bo, true, false, NULL);
if (unlikely(ret != 0)) {
@@ -336,14 +425,15 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf,
virtual = ttm_kmap_obj_virtual(&map, &dummy);
- if (box->w == 64 && cmd->dma.guest.pitch == 64*4) {
- memcpy(srf->snooper.image, virtual, 64*64*4);
+ if (box->w == VMW_CURSOR_SNOOP_WIDTH && cmd->dma.guest.pitch == image_pitch) {
+ memcpy(srf->snooper.image, virtual,
+ VMW_CURSOR_SNOOP_HEIGHT*image_pitch);
} else {
/* Image is unsigned pointer. */
for (i = 0; i < box->h; i++)
- memcpy(srf->snooper.image + i * 64,
+ memcpy(srf->snooper.image + i * image_pitch,
virtual + i * cmd->dma.guest.pitch,
- box->w * 4);
+ box->w * desc->pitchBytesPerBlock);
}
srf->snooper.age++;
@@ -387,15 +477,17 @@ void vmw_kms_cursor_post_execbuf(struct vmw_private *dev_priv)
list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
du = vmw_crtc_to_du(crtc);
if (!du->cursor_surface ||
- du->cursor_age == du->cursor_surface->snooper.age)
+ du->cursor_age == du->cursor_surface->snooper.age ||
+ !du->cursor_surface->snooper.image)
continue;
du->cursor_age = du->cursor_surface->snooper.age;
- vmw_cursor_update_image(dev_priv, NULL, NULL,
- du->cursor_surface->snooper.image,
- 64, 64,
- du->hotspot_x + du->core_hotspot_x,
- du->hotspot_y + du->core_hotspot_y);
+ vmw_send_define_cursor_cmd(dev_priv,
+ du->cursor_surface->snooper.image,
+ VMW_CURSOR_SNOOP_WIDTH,
+ VMW_CURSOR_SNOOP_HEIGHT,
+ du->hotspot_x + du->core_hotspot_x,
+ du->hotspot_y + du->core_hotspot_y);
}
mutex_unlock(&dev->mode_config.mutex);
@@ -404,8 +496,14 @@ void vmw_kms_cursor_post_execbuf(struct vmw_private *dev_priv)
void vmw_du_cursor_plane_destroy(struct drm_plane *plane)
{
+ struct vmw_cursor_plane *vcp = vmw_plane_to_vcp(plane);
+ u32 i;
+
vmw_cursor_update_position(plane->dev->dev_private, false, 0, 0);
- vmw_du_destroy_cursor_mob_array(vmw_plane_to_vcp(plane));
+
+ for (i = 0; i < ARRAY_SIZE(vcp->cursor_mobs); i++)
+ vmw_du_destroy_cursor_mob(&vcp->cursor_mobs[i]);
+
drm_plane_cleanup(plane);
}
@@ -463,6 +561,87 @@ vmw_du_plane_cleanup_fb(struct drm_plane *plane,
/**
+ * vmw_du_cursor_plane_map_cm - Maps the cursor mobs.
+ *
+ * @vps: plane_state
+ *
+ * Returns 0 on success
+ */
+
+static int
+vmw_du_cursor_plane_map_cm(struct vmw_plane_state *vps)
+{
+ int ret;
+ u32 size = vmw_du_cursor_mob_size(vps->base.crtc_w, vps->base.crtc_h);
+ struct ttm_buffer_object *bo = vps->cursor.bo;
+
+ if (!bo)
+ return -EINVAL;
+
+ if (bo->base.size < size)
+ return -EINVAL;
+
+ if (vps->cursor.mapped)
+ return 0;
+
+ ret = ttm_bo_reserve(bo, false, false, NULL);
+
+ if (unlikely(ret != 0))
+ return -ENOMEM;
+
+ ret = ttm_bo_kmap(bo, 0, PFN_UP(size), &vps->cursor.map);
+
+ /*
+ * We just want to try to get mob bind to finish
+ * so that the first write to SVGA_REG_CURSOR_MOBID
+ * is done with a buffer that the device has already
+ * seen
+ */
+ (void) ttm_bo_wait(bo, false, false);
+
+ ttm_bo_unreserve(bo);
+
+ if (unlikely(ret != 0))
+ return -ENOMEM;
+
+ vps->cursor.mapped = true;
+
+ return 0;
+}
+
+
+/**
+ * vmw_du_cursor_plane_unmap_cm - Unmaps the cursor mobs.
+ *
+ * @vps: state of the cursor plane
+ *
+ * Returns 0 on success
+ */
+
+static int
+vmw_du_cursor_plane_unmap_cm(struct vmw_plane_state *vps)
+{
+ int ret = 0;
+ struct ttm_buffer_object *bo = vps->cursor.bo;
+
+ if (!vps->cursor.mapped)
+ return 0;
+
+ if (!bo)
+ return 0;
+
+ ret = ttm_bo_reserve(bo, true, false, NULL);
+ if (likely(ret == 0)) {
+ ttm_bo_kunmap(&vps->cursor.map);
+ ttm_bo_unreserve(bo);
+ vps->cursor.mapped = false;
+ }
+
+ return ret;
+}
+
+
+/**
* vmw_du_cursor_plane_cleanup_fb - Unpins the plane surface
*
* @plane: cursor plane
@@ -476,10 +655,16 @@ void
vmw_du_cursor_plane_cleanup_fb(struct drm_plane *plane,
struct drm_plane_state *old_state)
{
+ struct vmw_cursor_plane *vcp = vmw_plane_to_vcp(plane);
struct vmw_plane_state *vps = vmw_plane_state_to_vps(old_state);
bool dummy;
- if (vps->bo != NULL && ttm_kmap_obj_virtual(&vps->bo->map, &dummy) != NULL) {
+ if (vps->surf_mapped) {
+ vmw_bo_unmap(vps->surf->res.backup);
+ vps->surf_mapped = false;
+ }
+
+ if (vps->bo && ttm_kmap_obj_virtual(&vps->bo->map, &dummy)) {
const int ret = ttm_bo_reserve(&vps->bo->base, true, false, NULL);
if (likely(ret == 0)) {
@@ -489,14 +674,8 @@ vmw_du_cursor_plane_cleanup_fb(struct drm_plane *plane,
}
}
- if (vps->cm_bo != NULL && ttm_kmap_obj_virtual(&vps->cm_map, &dummy) != NULL) {
- const int ret = ttm_bo_reserve(vps->cm_bo, true, false, NULL);
-
- if (likely(ret == 0)) {
- ttm_bo_kunmap(&vps->cm_map);
- ttm_bo_unreserve(vps->cm_bo);
- }
- }
+ vmw_du_cursor_plane_unmap_cm(vps);
+ vmw_du_put_cursor_mob(vcp, vps);
vmw_du_plane_unpin_surf(vps, false);
@@ -511,6 +690,7 @@ vmw_du_cursor_plane_cleanup_fb(struct drm_plane *plane,
}
}
+
/**
* vmw_du_cursor_plane_prepare_fb - Readies the cursor by referencing it
*
@@ -526,8 +706,6 @@ vmw_du_cursor_plane_prepare_fb(struct drm_plane *plane,
struct drm_framebuffer *fb = new_state->fb;
struct vmw_cursor_plane *vcp = vmw_plane_to_vcp(plane);
struct vmw_plane_state *vps = vmw_plane_state_to_vps(new_state);
- struct ttm_buffer_object *cm_bo = NULL;
- bool dummy;
int ret = 0;
if (vps->surf) {
@@ -550,13 +728,14 @@ vmw_du_cursor_plane_prepare_fb(struct drm_plane *plane,
}
}
- vps->cm_bo = NULL;
-
- if (vps->surf == NULL && vps->bo != NULL) {
+ if (!vps->surf && vps->bo) {
const u32 size = new_state->crtc_w * new_state->crtc_h * sizeof(u32);
- /* Not using vmw_bo_map_and_cache() helper here as we need to reserve
- the ttm_buffer_object first which wmw_bo_map_and_cache() omits. */
+ /*
+ * Not using vmw_bo_map_and_cache() helper here as we need to
+ * reserve the ttm_buffer_object first which
+ * vmw_bo_map_and_cache() omits.
+ */
ret = ttm_bo_reserve(&vps->bo->base, true, false, NULL);
if (unlikely(ret != 0))
@@ -571,69 +750,24 @@ vmw_du_cursor_plane_prepare_fb(struct drm_plane *plane,
if (unlikely(ret != 0))
return -ENOMEM;
+ } else if (vps->surf && !vps->bo && vps->surf->res.backup) {
+
+ WARN_ON(vps->surf->snooper.image);
+ ret = ttm_bo_reserve(&vps->surf->res.backup->base, true, false,
+ NULL);
+ if (unlikely(ret != 0))
+ return -ENOMEM;
+ vmw_bo_map_and_cache(vps->surf->res.backup);
+ ttm_bo_unreserve(&vps->surf->res.backup->base);
+ vps->surf_mapped = true;
}
if (vps->surf || vps->bo) {
- unsigned cursor_mob_idx = vps->cursor_mob_idx;
-
- /* Lazily set up cursor MOBs just once -- no reattempts. */
- if (cursor_mob_idx == 0 && vcp->cursor_mob[0] == NULL)
- if (vmw_du_create_cursor_mob_array(vcp) != 0)
- vps->cursor_mob_idx = cursor_mob_idx = -1U;
-
- if (cursor_mob_idx < ARRAY_SIZE(vcp->cursor_mob)) {
- const u32 size = sizeof(SVGAGBCursorHeader) +
- new_state->crtc_w * new_state->crtc_h * sizeof(u32);
-
- cm_bo = vcp->cursor_mob[cursor_mob_idx];
-
- if (cm_bo->resource->num_pages * PAGE_SIZE < size) {
- ret = -EINVAL;
- goto error_bo_unmap;
- }
-
- ret = ttm_bo_reserve(cm_bo, false, false, NULL);
-
- if (unlikely(ret != 0)) {
- ret = -ENOMEM;
- goto error_bo_unmap;
- }
-
- ret = ttm_bo_kmap(cm_bo, 0, PFN_UP(size), &vps->cm_map);
-
- /*
- * We just want to try to get mob bind to finish
- * so that the first write to SVGA_REG_CURSOR_MOBID
- * is done with a buffer that the device has already
- * seen
- */
- (void) ttm_bo_wait(cm_bo, false, false);
-
- ttm_bo_unreserve(cm_bo);
-
- if (unlikely(ret != 0)) {
- ret = -ENOMEM;
- goto error_bo_unmap;
- }
-
- vps->cursor_mob_idx = cursor_mob_idx ^ 1;
- vps->cm_bo = cm_bo;
- }
+ vmw_du_get_cursor_mob(vcp, vps);
+ vmw_du_cursor_plane_map_cm(vps);
}
return 0;
-
-error_bo_unmap:
- if (vps->bo != NULL && ttm_kmap_obj_virtual(&vps->bo->map, &dummy) != NULL) {
- const int ret = ttm_bo_reserve(&vps->bo->base, true, false, NULL);
- if (likely(ret == 0)) {
- atomic_dec(&vps->bo->base_mapped_count);
- ttm_bo_kunmap(&vps->bo->map);
- ttm_bo_unreserve(&vps->bo->base);
- }
- }
-
- return ret;
}
@@ -649,7 +783,9 @@ vmw_du_cursor_plane_atomic_update(struct drm_plane *plane,
struct vmw_private *dev_priv = vmw_priv(crtc->dev);
struct vmw_display_unit *du = vmw_crtc_to_du(crtc);
struct vmw_plane_state *vps = vmw_plane_state_to_vps(new_state);
+ struct vmw_plane_state *old_vps = vmw_plane_state_to_vps(old_state);
s32 hotspot_x, hotspot_y;
+ bool dummy;
hotspot_x = du->hotspot_x;
hotspot_y = du->hotspot_y;
@@ -662,25 +798,40 @@ vmw_du_cursor_plane_atomic_update(struct drm_plane *plane,
du->cursor_surface = vps->surf;
du->cursor_bo = vps->bo;
- if (vps->surf) {
- du->cursor_age = du->cursor_surface->snooper.age;
-
- vmw_cursor_update_image(dev_priv, vps->cm_bo, &vps->cm_map,
- vps->surf->snooper.image,
- new_state->crtc_w,
- new_state->crtc_h,
- hotspot_x, hotspot_y);
- } else if (vps->bo) {
- vmw_cursor_update_bo(dev_priv, vps->cm_bo, &vps->cm_map,
- vps->bo,
- new_state->crtc_w,
- new_state->crtc_h,
- hotspot_x, hotspot_y);
- } else {
+ if (!vps->surf && !vps->bo) {
vmw_cursor_update_position(dev_priv, false, 0, 0);
return;
}
+ vps->cursor.hotspot_x = hotspot_x;
+ vps->cursor.hotspot_y = hotspot_y;
+
+ if (vps->surf) {
+ du->cursor_age = du->cursor_surface->snooper.age;
+ }
+
+ if (!vmw_du_cursor_plane_has_changed(old_vps, vps)) {
+ /*
+ * If it hasn't changed, avoid making the device do extra
+ * work by keeping the old cursor active.
+ */
+ struct vmw_cursor_plane_state tmp = old_vps->cursor;
+ old_vps->cursor = vps->cursor;
+ vps->cursor = tmp;
+ } else {
+ void *image = vmw_du_cursor_plane_acquire_image(vps);
+ if (image)
+ vmw_cursor_update_image(dev_priv, vps, image,
+ new_state->crtc_w,
+ new_state->crtc_h,
+ hotspot_x, hotspot_y);
+ }
+
+ if (vps->bo) {
+ if (ttm_kmap_obj_virtual(&vps->bo->map, &dummy))
+ atomic_dec(&vps->bo->base_mapped_count);
+ }
+
du->cursor_x = new_state->crtc_x + du->set_gui_x;
du->cursor_y = new_state->crtc_y + du->set_gui_y;
@@ -778,12 +929,16 @@ int vmw_du_cursor_plane_atomic_check(struct drm_plane *plane,
return -EINVAL;
}
- if (!vmw_framebuffer_to_vfb(fb)->bo)
+ if (!vmw_framebuffer_to_vfb(fb)->bo) {
surface = vmw_framebuffer_to_vfbs(fb)->surface;
- if (surface && !surface->snooper.image) {
- DRM_ERROR("surface not suitable for cursor\n");
- return -EINVAL;
+ WARN_ON(!surface);
+
+ if (!surface ||
+ (!surface->snooper.image && !surface->res.backup)) {
+ DRM_ERROR("surface not suitable for cursor\n");
+ return -EINVAL;
+ }
}
return 0;
@@ -831,15 +986,6 @@ void vmw_du_crtc_atomic_begin(struct drm_crtc *crtc,
void vmw_du_crtc_atomic_flush(struct drm_crtc *crtc,
struct drm_atomic_state *state)
{
- struct drm_pending_vblank_event *event = crtc->state->event;
-
- if (event) {
- crtc->state->event = NULL;
-
- spin_lock_irq(&crtc->dev->event_lock);
- drm_crtc_send_vblank_event(crtc, event);
- spin_unlock_irq(&crtc->dev->event_lock);
- }
}
@@ -943,6 +1089,8 @@ vmw_du_plane_duplicate_state(struct drm_plane *plane)
vps->pinned = 0;
vps->cpp = 0;
+ memset(&vps->cursor, 0, sizeof(vps->cursor));
+
/* Each ref counted resource needs to be acquired again */
if (vps->surf)
(void) vmw_surface_reference(vps->surf);
@@ -997,7 +1145,6 @@ vmw_du_plane_destroy_state(struct drm_plane *plane,
{
struct vmw_plane_state *vps = vmw_plane_state_to_vps(state);
-
/* Should have been freed by cleanup_fb */
if (vps->surf)
vmw_surface_unreference(&vps->surf);
@@ -1664,7 +1811,7 @@ static struct drm_framebuffer *vmw_kms_fb_create(struct drm_device *dev,
if (IS_ERR(vfb)) {
ret = PTR_ERR(vfb);
goto err_out;
- }
+ }
err_out:
/* vmw_user_lookup_handle takes one ref so does new_fb */
@@ -2052,6 +2199,8 @@ int vmw_kms_init(struct vmw_private *dev_priv)
dev->mode_config.min_height = 1;
dev->mode_config.max_width = dev_priv->texture_max_width;
dev->mode_config.max_height = dev_priv->texture_max_height;
+ dev->mode_config.preferred_depth = dev_priv->assume_16bpp ? 16 : 32;
+ dev->mode_config.prefer_shadow_fbdev = !dev_priv->has_mob;
drm_mode_create_suggested_offset_properties(dev);
vmw_kms_create_hotplug_mode_update_property(dev_priv);
@@ -2093,7 +2242,6 @@ int vmw_kms_cursor_bypass_ioctl(struct drm_device *dev, void *data,
struct drm_crtc *crtc;
int ret = 0;
-
mutex_lock(&dev->mode_config.mutex);
if (arg->flags & DRM_VMW_CURSOR_BYPASS_ALL) {
@@ -2155,30 +2303,6 @@ bool vmw_kms_validate_mode_vram(struct vmw_private *dev_priv,
dev_priv->max_primary_mem : dev_priv->vram_size);
}
-
-/*
- * Function called by DRM code called with vbl_lock held.
- */
-u32 vmw_get_vblank_counter(struct drm_crtc *crtc)
-{
- return 0;
-}
-
-/*
- * Function called by DRM code called with vbl_lock held.
- */
-int vmw_enable_vblank(struct drm_crtc *crtc)
-{
- return -EINVAL;
-}
-
-/*
- * Function called by DRM code called with vbl_lock held.
- */
-void vmw_disable_vblank(struct drm_crtc *crtc)
-{
-}
-
/**
* vmw_du_update_layout - Update the display unit with topology from resolution
* plugin and generate DRM uevent
@@ -2207,7 +2331,7 @@ static int vmw_du_update_layout(struct vmw_private *dev_priv,
if (ret == -EDEADLK) {
drm_modeset_backoff(&ctx);
goto retry;
- }
+ }
goto out_fini;
}
}
@@ -2222,8 +2346,8 @@ static int vmw_du_update_layout(struct vmw_private *dev_priv,
du->gui_x = rects[du->unit].x1;
du->gui_y = rects[du->unit].y1;
} else {
- du->pref_width = 800;
- du->pref_height = 600;
+ du->pref_width = VMWGFX_MIN_INITIAL_WIDTH;
+ du->pref_height = VMWGFX_MIN_INITIAL_HEIGHT;
du->pref_active = false;
du->gui_x = 0;
du->gui_y = 0;
@@ -2250,13 +2374,13 @@ static int vmw_du_update_layout(struct vmw_private *dev_priv,
}
con->status = vmw_du_connector_detect(con, true);
}
-
- drm_sysfs_hotplug_event(dev);
out_fini:
drm_modeset_drop_locks(&ctx);
drm_modeset_acquire_fini(&ctx);
mutex_unlock(&dev->mode_config.mutex);
+ drm_sysfs_hotplug_event(dev);
+
return 0;
}
@@ -2536,10 +2660,9 @@ int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data,
int ret, i;
if (!arg->num_outputs) {
- struct drm_rect def_rect = {0, 0, 800, 600};
- VMW_DEBUG_KMS("Default layout x1 = %d y1 = %d x2 = %d y2 = %d\n",
- def_rect.x1, def_rect.y1,
- def_rect.x2, def_rect.y2);
+ struct drm_rect def_rect = {0, 0,
+ VMWGFX_MIN_INITIAL_WIDTH,
+ VMWGFX_MIN_INITIAL_HEIGHT};
vmw_du_update_layout(dev_priv, 1, &def_rect);
return 0;
}
@@ -2834,68 +2957,6 @@ int vmw_kms_update_proxy(struct vmw_resource *res,
return 0;
}
-int vmw_kms_fbdev_init_data(struct vmw_private *dev_priv,
- unsigned unit,
- u32 max_width,
- u32 max_height,
- struct drm_connector **p_con,
- struct drm_crtc **p_crtc,
- struct drm_display_mode **p_mode)
-{
- struct drm_connector *con;
- struct vmw_display_unit *du;
- struct drm_display_mode *mode;
- int i = 0;
- int ret = 0;
-
- mutex_lock(&dev_priv->drm.mode_config.mutex);
- list_for_each_entry(con, &dev_priv->drm.mode_config.connector_list,
- head) {
- if (i == unit)
- break;
-
- ++i;
- }
-
- if (&con->head == &dev_priv->drm.mode_config.connector_list) {
- DRM_ERROR("Could not find initial display unit.\n");
- ret = -EINVAL;
- goto out_unlock;
- }
-
- if (list_empty(&con->modes))
- (void) vmw_du_connector_fill_modes(con, max_width, max_height);
-
- if (list_empty(&con->modes)) {
- DRM_ERROR("Could not find initial display mode.\n");
- ret = -EINVAL;
- goto out_unlock;
- }
-
- du = vmw_connector_to_du(con);
- *p_con = con;
- *p_crtc = &du->crtc;
-
- list_for_each_entry(mode, &con->modes, head) {
- if (mode->type & DRM_MODE_TYPE_PREFERRED)
- break;
- }
-
- if (&mode->head == &con->modes) {
- WARN_ONCE(true, "Could not find initial preferred mode.\n");
- *p_mode = list_first_entry(&con->modes,
- struct drm_display_mode,
- head);
- } else {
- *p_mode = mode;
- }
-
- out_unlock:
- mutex_unlock(&dev_priv->drm.mode_config.mutex);
-
- return ret;
-}
-
/**
* vmw_kms_create_implicit_placement_property - Set up the implicit placement
* property.
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h
index 85f86fa..4d6e7b5 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h
@@ -272,6 +272,14 @@ struct vmw_crtc_state {
struct drm_crtc_state base;
};
+struct vmw_cursor_plane_state {
+ struct ttm_buffer_object *bo;
+ struct ttm_bo_kmap_obj map;
+ bool mapped;
+ s32 hotspot_x;
+ s32 hotspot_y;
+};
+
/**
* Derived class for plane state object
*
@@ -295,13 +303,8 @@ struct vmw_plane_state {
/* For CPU Blit */
unsigned int cpp;
- /* CursorMob flipping index; -1 if cursor mobs not used */
- unsigned int cursor_mob_idx;
- /* Currently-active CursorMob */
- struct ttm_buffer_object *cm_bo;
- /* CursorMob kmap_obj; expected valid at cursor_plane_atomic_update
- IFF currently-active CursorMob above is valid */
- struct ttm_bo_kmap_obj cm_map;
+ bool surf_mapped;
+ struct vmw_cursor_plane_state cursor;
};
@@ -338,11 +341,12 @@ struct vmw_connector_state {
* Derived class for cursor plane object
*
* @base DRM plane object
- * @cursor_mob array of two MOBs for CursorMob flipping
+ * @cursor.cursor_mobs Cursor mobs available for re-use
*/
struct vmw_cursor_plane {
struct drm_plane base;
- struct ttm_buffer_object *cursor_mob[2];
+
+ struct ttm_buffer_object *cursor_mobs[3];
};
/**
@@ -458,13 +462,6 @@ vmw_kms_new_framebuffer(struct vmw_private *dev_priv,
struct vmw_surface *surface,
bool only_2d,
const struct drm_mode_fb_cmd2 *mode_cmd);
-int vmw_kms_fbdev_init_data(struct vmw_private *dev_priv,
- unsigned unit,
- u32 max_width,
- u32 max_height,
- struct drm_connector **p_con,
- struct drm_crtc **p_crtc,
- struct drm_display_mode **p_mode);
void vmw_guess_mode_timing(struct drm_display_mode *mode);
void vmw_kms_update_implicit_fb(struct vmw_private *dev_priv);
void vmw_kms_create_implicit_placement_property(struct vmw_private *dev_priv);
@@ -472,8 +469,6 @@ void vmw_kms_create_implicit_placement_property(struct vmw_private *dev_priv);
/* Universal Plane Helpers */
void vmw_du_primary_plane_destroy(struct drm_plane *plane);
void vmw_du_cursor_plane_destroy(struct drm_plane *plane);
-int vmw_du_create_cursor_mob_array(struct vmw_cursor_plane *vcp);
-void vmw_du_destroy_cursor_mob_array(struct vmw_cursor_plane *vcp);
/* Atomic Helpers */
int vmw_du_primary_plane_atomic_check(struct drm_plane *plane,
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ldu.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ldu.c
index b8761f1..a56e5d0 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_ldu.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ldu.c
@@ -28,7 +28,6 @@
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_fourcc.h>
-#include <drm/drm_vblank.h>
#include "vmwgfx_kms.h"
@@ -235,9 +234,6 @@ static const struct drm_crtc_funcs vmw_legacy_crtc_funcs = {
.atomic_duplicate_state = vmw_du_crtc_duplicate_state,
.atomic_destroy_state = vmw_du_crtc_destroy_state,
.set_config = drm_atomic_helper_set_config,
- .get_vblank_counter = vmw_get_vblank_counter,
- .enable_vblank = vmw_enable_vblank,
- .disable_vblank = vmw_disable_vblank,
};
@@ -507,10 +503,6 @@ int vmw_kms_ldu_init_display(struct vmw_private *dev_priv)
dev_priv->ldu_priv->last_num_active = 0;
dev_priv->ldu_priv->fb = NULL;
- ret = drm_vblank_init(dev, num_display_units);
- if (ret != 0)
- goto err_free;
-
vmw_kms_create_implicit_placement_property(dev_priv);
for (i = 0; i < num_display_units; ++i) {
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_mksstat.h b/drivers/gpu/drm/vmwgfx/vmwgfx_mksstat.h
index 0509f55..ede74c7 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_mksstat.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_mksstat.h
@@ -29,6 +29,7 @@
#define _VMWGFX_MKSSTAT_H_
#include <asm/page.h>
+#include <linux/kconfig.h>
/* Reservation marker for mksstat pid's */
#define MKSSTAT_PID_RESERVED -1
@@ -41,6 +42,7 @@
typedef enum {
MKSSTAT_KERN_EXECBUF, /* vmw_execbuf_ioctl */
+ MKSSTAT_KERN_COTABLE_RESIZE,
MKSSTAT_KERN_COUNT /* Reserved entry; always last */
} mksstat_kern_stats_t;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_msg.c b/drivers/gpu/drm/vmwgfx/vmwgfx_msg.c
index 50fa3df..e76976a 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_msg.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_msg.c
@@ -85,7 +85,14 @@ struct rpc_channel {
u32 cookie_low;
};
-
+#if IS_ENABLED(CONFIG_DRM_VMWGFX_MKSSTATS)
+/* Kernel mksGuestStats counter names and desciptions; same order as enum mksstat_kern_stats_t */
+static const char* const mksstat_kern_name_desc[MKSSTAT_KERN_COUNT][2] =
+{
+ { "vmw_execbuf_ioctl", "vmw_execbuf_ioctl" },
+ { "vmw_cotable_resize", "vmw_cotable_resize" },
+};
+#endif
/**
* vmw_open_channel
@@ -695,12 +702,6 @@ static inline void hypervisor_ppn_remove(PPN64 pfn)
/* Header to the text description of mksGuestStat instance descriptor */
#define MKSSTAT_KERNEL_DESCRIPTION "vmwgfx"
-/* Kernel mksGuestStats counter names and desciptions; same order as enum mksstat_kern_stats_t */
-static const char* const mksstat_kern_name_desc[MKSSTAT_KERN_COUNT][2] =
-{
- { "vmw_execbuf_ioctl", "vmw_execbuf_ioctl" },
-};
-
/**
* mksstat_init_record: Initializes an MKSGuestStatCounter-based record
* for the respective mksGuestStat index.
@@ -786,6 +787,7 @@ static int mksstat_init_kern_id(struct page **ppage)
/* Set up all kernel-internal counters and corresponding structures */
pstrs_acc = pstrs;
pstrs_acc = mksstat_init_record_time(MKSSTAT_KERN_EXECBUF, pstat, pinfo, pstrs_acc);
+ pstrs_acc = mksstat_init_record_time(MKSSTAT_KERN_COTABLE_RESIZE, pstat, pinfo, pstrs_acc);
/* Add new counters above, in their order of appearance in mksstat_kern_stats_t */
@@ -1014,8 +1016,6 @@ int vmw_mksstat_add_ioctl(struct drm_device *dev, void *data,
struct vmw_private *const dev_priv = vmw_priv(dev);
- struct page *page;
- MKSGuestStatInstanceDescriptor *pdesc;
const size_t num_pages_stat = PFN_UP(arg->stat_len);
const size_t num_pages_info = PFN_UP(arg->info_len);
const size_t num_pages_strs = PFN_UP(arg->strs_len);
@@ -1023,10 +1023,13 @@ int vmw_mksstat_add_ioctl(struct drm_device *dev, void *data,
long nr_pinned_stat;
long nr_pinned_info;
long nr_pinned_strs;
- struct page *pages_stat[ARRAY_SIZE(pdesc->statPPNs)];
- struct page *pages_info[ARRAY_SIZE(pdesc->infoPPNs)];
- struct page *pages_strs[ARRAY_SIZE(pdesc->strsPPNs)];
+ MKSGuestStatInstanceDescriptor *pdesc;
+ struct page *page = NULL;
+ struct page **pages_stat = NULL;
+ struct page **pages_info = NULL;
+ struct page **pages_strs = NULL;
size_t i, slot;
+ int ret_err = -ENOMEM;
arg->id = -1;
@@ -1054,13 +1057,23 @@ int vmw_mksstat_add_ioctl(struct drm_device *dev, void *data,
BUG_ON(dev_priv->mksstat_user_pages[slot]);
+ /* Allocate statically-sized temp arrays for pages -- too big to keep in frame */
+ pages_stat = (struct page **)kmalloc_array(
+ ARRAY_SIZE(pdesc->statPPNs) +
+ ARRAY_SIZE(pdesc->infoPPNs) +
+ ARRAY_SIZE(pdesc->strsPPNs), sizeof(*pages_stat), GFP_KERNEL);
+
+ if (!pages_stat)
+ goto err_nomem;
+
+ pages_info = pages_stat + ARRAY_SIZE(pdesc->statPPNs);
+ pages_strs = pages_info + ARRAY_SIZE(pdesc->infoPPNs);
+
/* Allocate a page for the instance descriptor */
page = alloc_page(GFP_KERNEL | __GFP_ZERO);
- if (!page) {
- atomic_set(&dev_priv->mksstat_user_pids[slot], 0);
- return -ENOMEM;
- }
+ if (!page)
+ goto err_nomem;
/* Set up the instance descriptor */
pdesc = page_address(page);
@@ -1075,9 +1088,8 @@ int vmw_mksstat_add_ioctl(struct drm_device *dev, void *data,
ARRAY_SIZE(pdesc->description) - 1);
if (desc_len < 0) {
- atomic_set(&dev_priv->mksstat_user_pids[slot], 0);
- __free_page(page);
- return -EFAULT;
+ ret_err = -EFAULT;
+ goto err_nomem;
}
reset_ppn_array(pdesc->statPPNs, ARRAY_SIZE(pdesc->statPPNs));
@@ -1118,6 +1130,7 @@ int vmw_mksstat_add_ioctl(struct drm_device *dev, void *data,
DRM_DEV_INFO(dev->dev, "pid=%d arg.description='%.*s' id=%zu\n", current->pid, (int)desc_len, pdesc->description, slot);
+ kfree(pages_stat);
return 0;
err_pin_strs:
@@ -1132,9 +1145,13 @@ int vmw_mksstat_add_ioctl(struct drm_device *dev, void *data,
if (nr_pinned_stat > 0)
unpin_user_pages(pages_stat, nr_pinned_stat);
+err_nomem:
atomic_set(&dev_priv->mksstat_user_pids[slot], 0);
- __free_page(page);
- return -ENOMEM;
+ if (page)
+ __free_page(page);
+ kfree(pages_stat);
+
+ return ret_err;
}
/**
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c b/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c
index 7bc99b1..f41f041 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c
@@ -230,7 +230,7 @@ void vmw_bo_dirty_unmap(struct vmw_buffer_object *vbo,
int vmw_bo_dirty_add(struct vmw_buffer_object *vbo)
{
struct vmw_bo_dirty *dirty = vbo->dirty;
- pgoff_t num_pages = vbo->base.resource->num_pages;
+ pgoff_t num_pages = PFN_UP(vbo->base.resource->size);
size_t size;
int ret;
@@ -395,7 +395,7 @@ vm_fault_t vmw_bo_vm_mkwrite(struct vm_fault *vmf)
return ret;
page_offset = vmf->pgoff - drm_vma_node_start(&bo->base.vma_node);
- if (unlikely(page_offset >= bo->resource->num_pages)) {
+ if (unlikely(page_offset >= PFN_UP(bo->resource->size))) {
ret = VM_FAULT_SIGBUS;
goto out_unlock;
}
@@ -438,7 +438,7 @@ vm_fault_t vmw_bo_vm_fault(struct vm_fault *vmf)
page_offset = vmf->pgoff -
drm_vma_node_start(&bo->base.vma_node);
- if (page_offset >= bo->resource->num_pages ||
+ if (page_offset >= PFN_UP(bo->resource->size) ||
vmw_resources_clean(vbo, page_offset,
page_offset + PAGE_SIZE,
&allowed_prefault)) {
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
index 9c79873..e1f36a0 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
@@ -29,7 +29,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_damage_helper.h>
#include <drm/drm_fourcc.h>
-#include <drm/drm_vblank.h>
#include "vmwgfx_kms.h"
@@ -320,9 +319,6 @@ static const struct drm_crtc_funcs vmw_screen_object_crtc_funcs = {
.atomic_destroy_state = vmw_du_crtc_destroy_state,
.set_config = drm_atomic_helper_set_config,
.page_flip = drm_atomic_helper_page_flip,
- .get_vblank_counter = vmw_get_vblank_counter,
- .enable_vblank = vmw_enable_vblank,
- .disable_vblank = vmw_disable_vblank,
};
/*
@@ -730,7 +726,6 @@ vmw_sou_primary_plane_atomic_update(struct drm_plane *plane,
struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state, plane);
struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state, plane);
struct drm_crtc *crtc = new_state->crtc;
- struct drm_pending_vblank_event *event = NULL;
struct vmw_fence_obj *fence = NULL;
int ret;
@@ -754,24 +749,6 @@ vmw_sou_primary_plane_atomic_update(struct drm_plane *plane,
return;
}
- /* For error case vblank event is send from vmw_du_crtc_atomic_flush */
- event = crtc->state->event;
- if (event && fence) {
- struct drm_file *file_priv = event->base.file_priv;
-
- ret = vmw_event_fence_action_queue(file_priv,
- fence,
- &event->base,
- &event->event.vbl.tv_sec,
- &event->event.vbl.tv_usec,
- true);
-
- if (unlikely(ret != 0))
- DRM_ERROR("Failed to queue event on fence.\n");
- else
- crtc->state->event = NULL;
- }
-
if (fence)
vmw_fence_obj_unreference(&fence);
}
@@ -947,7 +924,7 @@ static int vmw_sou_init(struct vmw_private *dev_priv, unsigned unit)
int vmw_kms_sou_init_display(struct vmw_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
- int i, ret;
+ int i;
/* Screen objects won't work if GMR's aren't available */
if (!dev_priv->has_gmr)
@@ -957,12 +934,6 @@ int vmw_kms_sou_init_display(struct vmw_private *dev_priv)
return -ENOSYS;
}
- ret = -ENOMEM;
-
- ret = drm_vblank_init(dev, VMWGFX_NUM_DISPLAY_UNITS);
- if (unlikely(ret != 0))
- return ret;
-
for (i = 0; i < VMWGFX_NUM_DISPLAY_UNITS; ++i)
vmw_sou_init(dev_priv, i);
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
index 8650c3a..0090abe 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
@@ -29,7 +29,6 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_damage_helper.h>
#include <drm/drm_fourcc.h>
-#include <drm/drm_vblank.h>
#include "vmwgfx_kms.h"
#include "vmw_surface_cache.h"
@@ -925,9 +924,6 @@ static const struct drm_crtc_funcs vmw_stdu_crtc_funcs = {
.atomic_destroy_state = vmw_du_crtc_destroy_state,
.set_config = drm_atomic_helper_set_config,
.page_flip = drm_atomic_helper_page_flip,
- .get_vblank_counter = vmw_get_vblank_counter,
- .enable_vblank = vmw_enable_vblank,
- .disable_vblank = vmw_disable_vblank,
};
@@ -1591,7 +1587,6 @@ vmw_stdu_primary_plane_atomic_update(struct drm_plane *plane,
struct vmw_plane_state *vps = vmw_plane_state_to_vps(new_state);
struct drm_crtc *crtc = new_state->crtc;
struct vmw_screen_target_display_unit *stdu;
- struct drm_pending_vblank_event *event;
struct vmw_fence_obj *fence = NULL;
struct vmw_private *dev_priv;
int ret;
@@ -1640,23 +1635,6 @@ vmw_stdu_primary_plane_atomic_update(struct drm_plane *plane,
return;
}
- /* In case of error, vblank event is send in vmw_du_crtc_atomic_flush */
- event = crtc->state->event;
- if (event && fence) {
- struct drm_file *file_priv = event->base.file_priv;
-
- ret = vmw_event_fence_action_queue(file_priv,
- fence,
- &event->base,
- &event->event.vbl.tv_sec,
- &event->event.vbl.tv_usec,
- true);
- if (ret)
- DRM_ERROR("Failed to queue event on fence.\n");
- else
- crtc->state->event = NULL;
- }
-
if (fence)
vmw_fence_obj_unreference(&fence);
}
@@ -1883,10 +1861,6 @@ int vmw_kms_stdu_init_display(struct vmw_private *dev_priv)
if (!(dev_priv->capabilities & SVGA_CAP_GBOBJECTS))
return -ENOSYS;
- ret = drm_vblank_init(dev, VMWGFX_NUM_DISPLAY_UNITS);
- if (unlikely(ret != 0))
- return ret;
-
dev_priv->active_display_unit = vmw_du_screen_target;
for (i = 0; i < VMWGFX_NUM_DISPLAY_UNITS; ++i) {
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
index ace7ca1..3bc63ae 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
@@ -815,11 +815,15 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
res->backup_size = cur_bo_offset;
if (metadata->scanout &&
metadata->num_sizes == 1 &&
- metadata->sizes[0].width == 64 &&
- metadata->sizes[0].height == 64 &&
- metadata->format == SVGA3D_A8R8G8B8) {
-
- srf->snooper.image = kzalloc(64 * 64 * 4, GFP_KERNEL);
+ metadata->sizes[0].width == VMW_CURSOR_SNOOP_WIDTH &&
+ metadata->sizes[0].height == VMW_CURSOR_SNOOP_HEIGHT &&
+ metadata->format == VMW_CURSOR_SNOOP_FORMAT) {
+ const struct SVGA3dSurfaceDesc *desc =
+ vmw_surface_get_desc(VMW_CURSOR_SNOOP_FORMAT);
+ const u32 cursor_size_bytes = VMW_CURSOR_SNOOP_WIDTH *
+ VMW_CURSOR_SNOOP_HEIGHT *
+ desc->pitchBytesPerBlock;
+ srf->snooper.image = kzalloc(cursor_size_bytes, GFP_KERNEL);
if (!srf->snooper.image) {
DRM_ERROR("Failed to allocate cursor_image\n");
ret = -ENOMEM;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c
index f468910..f5c4a40 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 OR MIT
/**************************************************************************
*
- * Copyright © 2018 VMware, Inc., Palo Alto, CA., USA
+ * Copyright © 2018 - 2022 VMware, Inc., Palo Alto, CA., USA
* All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
@@ -180,11 +180,16 @@ vmw_validation_find_bo_dup(struct vmw_validation_context *ctx,
if (!ctx->merge_dups)
return NULL;
- if (ctx->ht) {
+ if (ctx->sw_context) {
struct vmwgfx_hash_item *hash;
+ unsigned long key = (unsigned long) vbo;
- if (!vmwgfx_ht_find_item(ctx->ht, (unsigned long) vbo, &hash))
- bo_node = container_of(hash, typeof(*bo_node), hash);
+ hash_for_each_possible_rcu(ctx->sw_context->res_ht, hash, head, key) {
+ if (hash->key == key) {
+ bo_node = container_of(hash, typeof(*bo_node), hash);
+ break;
+ }
+ }
} else {
struct vmw_validation_bo_node *entry;
@@ -217,11 +222,16 @@ vmw_validation_find_res_dup(struct vmw_validation_context *ctx,
if (!ctx->merge_dups)
return NULL;
- if (ctx->ht) {
+ if (ctx->sw_context) {
struct vmwgfx_hash_item *hash;
+ unsigned long key = (unsigned long) res;
- if (!vmwgfx_ht_find_item(ctx->ht, (unsigned long) res, &hash))
- res_node = container_of(hash, typeof(*res_node), hash);
+ hash_for_each_possible_rcu(ctx->sw_context->res_ht, hash, head, key) {
+ if (hash->key == key) {
+ res_node = container_of(hash, typeof(*res_node), hash);
+ break;
+ }
+ }
} else {
struct vmw_validation_res_node *entry;
@@ -269,20 +279,15 @@ int vmw_validation_add_bo(struct vmw_validation_context *ctx,
}
} else {
struct ttm_validate_buffer *val_buf;
- int ret;
bo_node = vmw_validation_mem_alloc(ctx, sizeof(*bo_node));
if (!bo_node)
return -ENOMEM;
- if (ctx->ht) {
+ if (ctx->sw_context) {
bo_node->hash.key = (unsigned long) vbo;
- ret = vmwgfx_ht_insert_item(ctx->ht, &bo_node->hash);
- if (ret) {
- DRM_ERROR("Failed to initialize a buffer "
- "validation entry.\n");
- return ret;
- }
+ hash_add_rcu(ctx->sw_context->res_ht, &bo_node->hash.head,
+ bo_node->hash.key);
}
val_buf = &bo_node->base;
val_buf->bo = ttm_bo_get_unless_zero(&vbo->base);
@@ -316,7 +321,6 @@ int vmw_validation_add_resource(struct vmw_validation_context *ctx,
bool *first_usage)
{
struct vmw_validation_res_node *node;
- int ret;
node = vmw_validation_find_res_dup(ctx, res);
if (node) {
@@ -330,14 +334,9 @@ int vmw_validation_add_resource(struct vmw_validation_context *ctx,
return -ENOMEM;
}
- if (ctx->ht) {
+ if (ctx->sw_context) {
node->hash.key = (unsigned long) res;
- ret = vmwgfx_ht_insert_item(ctx->ht, &node->hash);
- if (ret) {
- DRM_ERROR("Failed to initialize a resource validation "
- "entry.\n");
- return ret;
- }
+ hash_add_rcu(ctx->sw_context->res_ht, &node->hash.head, node->hash.key);
}
node->res = vmw_resource_reference_unless_doomed(res);
if (!node->res)
@@ -681,19 +680,19 @@ void vmw_validation_drop_ht(struct vmw_validation_context *ctx)
struct vmw_validation_bo_node *entry;
struct vmw_validation_res_node *val;
- if (!ctx->ht)
+ if (!ctx->sw_context)
return;
list_for_each_entry(entry, &ctx->bo_list, base.head)
- (void) vmwgfx_ht_remove_item(ctx->ht, &entry->hash);
+ hash_del_rcu(&entry->hash.head);
list_for_each_entry(val, &ctx->resource_list, head)
- (void) vmwgfx_ht_remove_item(ctx->ht, &val->hash);
+ hash_del_rcu(&val->hash.head);
list_for_each_entry(val, &ctx->resource_ctx_list, head)
- (void) vmwgfx_ht_remove_item(ctx->ht, &val->hash);
+ hash_del_rcu(&entry->hash.head);
- ctx->ht = NULL;
+ ctx->sw_context = NULL;
}
/**
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h
index f21df05..ab9ec22 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h
@@ -1,7 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 OR MIT */
/**************************************************************************
*
- * Copyright © 2018 VMware, Inc., Palo Alto, CA., USA
+ * Copyright © 2018 - 2022 VMware, Inc., Palo Alto, CA., USA
* All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
@@ -29,12 +29,11 @@
#define _VMWGFX_VALIDATION_H_
#include <linux/list.h>
+#include <linux/hashtable.h>
#include <linux/ww_mutex.h>
#include <drm/ttm/ttm_execbuf_util.h>
-#include "vmwgfx_hashtab.h"
-
#define VMW_RES_DIRTY_NONE 0
#define VMW_RES_DIRTY_SET BIT(0)
#define VMW_RES_DIRTY_CLEAR BIT(1)
@@ -59,7 +58,7 @@
* @total_mem: Amount of reserved memory.
*/
struct vmw_validation_context {
- struct vmwgfx_open_hash *ht;
+ struct vmw_sw_context *sw_context;
struct list_head resource_list;
struct list_head resource_ctx_list;
struct list_head bo_list;
@@ -82,16 +81,16 @@ struct vmw_fence_obj;
/**
* DECLARE_VAL_CONTEXT - Declare a validation context with initialization
* @_name: The name of the variable
- * @_ht: The hash table used to find dups or NULL if none
+ * @_sw_context: Contains the hash table used to find dups or NULL if none
* @_merge_dups: Whether to merge duplicate buffer object- or resource
* entries. If set to true, ideally a hash table pointer should be supplied
* as well unless the number of resources and buffer objects per validation
* is known to be very small
*/
#endif
-#define DECLARE_VAL_CONTEXT(_name, _ht, _merge_dups) \
+#define DECLARE_VAL_CONTEXT(_name, _sw_context, _merge_dups) \
struct vmw_validation_context _name = \
- { .ht = _ht, \
+ { .sw_context = _sw_context, \
.resource_list = LIST_HEAD_INIT((_name).resource_list), \
.resource_ctx_list = LIST_HEAD_INIT((_name).resource_ctx_list), \
.bo_list = LIST_HEAD_INIT((_name).bo_list), \
@@ -115,19 +114,6 @@ vmw_validation_has_bos(struct vmw_validation_context *ctx)
}
/**
- * vmw_validation_set_ht - Register a hash table for duplicate finding
- * @ctx: The validation context
- * @ht: Pointer to a hash table to use for duplicate finding
- * This function is intended to be used if the hash table wasn't
- * available at validation context declaration time
- */
-static inline void vmw_validation_set_ht(struct vmw_validation_context *ctx,
- struct vmwgfx_open_hash *ht)
-{
- ctx->ht = ht;
-}
-
-/**
* vmw_validation_bo_reserve - Reserve buffer objects registered with a
* validation context
* @ctx: The validation context
diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index e31554d..4c95ebc 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -12,7 +12,6 @@
#include <linux/scatterlist.h>
#include <linux/shmem_fs.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_gem.h>
#include <drm/drm_prime.h>
#include <drm/drm_probe_helper.h>
diff --git a/drivers/gpu/drm/xlnx/Makefile b/drivers/gpu/drm/xlnx/Makefile
index 51c24b7..ea1422a 100644
--- a/drivers/gpu/drm/xlnx/Makefile
+++ b/drivers/gpu/drm/xlnx/Makefile
@@ -1,2 +1,2 @@
-zynqmp-dpsub-y := zynqmp_disp.o zynqmp_dpsub.o zynqmp_dp.o
+zynqmp-dpsub-y := zynqmp_disp.o zynqmp_dpsub.o zynqmp_dp.o zynqmp_kms.o
obj-$(CONFIG_DRM_ZYNQMP_DPSUB) += zynqmp-dpsub.o
diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp.c b/drivers/gpu/drm/xlnx/zynqmp_disp.c
index bbb365f..3b87eeb 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_disp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_disp.c
@@ -9,29 +9,19 @@
* - Laurent Pinchart <laurent.pinchart@ideasonboard.com>
*/
-#include <drm/drm_atomic.h>
-#include <drm/drm_atomic_helper.h>
-#include <drm/drm_atomic_uapi.h>
-#include <drm/drm_blend.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_device.h>
#include <drm/drm_fb_dma_helper.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_framebuffer.h>
-#include <drm/drm_managed.h>
#include <drm/drm_plane.h>
-#include <drm/drm_vblank.h>
#include <linux/clk.h>
-#include <linux/delay.h>
#include <linux/dma/xilinx_dpdma.h>
#include <linux/dma-mapping.h>
#include <linux/dmaengine.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/platform_device.h>
-#include <linux/pm_runtime.h>
-#include <linux/spinlock.h>
+#include <linux/slab.h>
#include "zynqmp_disp.h"
#include "zynqmp_disp_regs.h"
@@ -72,46 +62,23 @@
#define ZYNQMP_DISP_AV_BUF_NUM_VID_GFX_BUFFERS 4
#define ZYNQMP_DISP_AV_BUF_NUM_BUFFERS 6
-#define ZYNQMP_DISP_NUM_LAYERS 2
#define ZYNQMP_DISP_MAX_NUM_SUB_PLANES 3
/**
* struct zynqmp_disp_format - Display subsystem format information
* @drm_fmt: DRM format (4CC)
* @buf_fmt: AV buffer format
- * @bus_fmt: Media bus formats (live formats)
* @swap: Flag to swap R & B for RGB formats, and U & V for YUV formats
* @sf: Scaling factors for color components
*/
struct zynqmp_disp_format {
u32 drm_fmt;
u32 buf_fmt;
- u32 bus_fmt;
bool swap;
const u32 *sf;
};
/**
- * enum zynqmp_disp_layer_id - Layer identifier
- * @ZYNQMP_DISP_LAYER_VID: Video layer
- * @ZYNQMP_DISP_LAYER_GFX: Graphics layer
- */
-enum zynqmp_disp_layer_id {
- ZYNQMP_DISP_LAYER_VID,
- ZYNQMP_DISP_LAYER_GFX
-};
-
-/**
- * enum zynqmp_disp_layer_mode - Layer mode
- * @ZYNQMP_DISP_LAYER_NONLIVE: non-live (memory) mode
- * @ZYNQMP_DISP_LAYER_LIVE: live (stream) mode
- */
-enum zynqmp_disp_layer_mode {
- ZYNQMP_DISP_LAYER_NONLIVE,
- ZYNQMP_DISP_LAYER_LIVE
-};
-
-/**
* struct zynqmp_disp_layer_dma - DMA channel for one data plane of a layer
* @chan: DMA channel
* @xt: Interleaved DMA descriptor template
@@ -136,8 +103,7 @@ struct zynqmp_disp_layer_info {
};
/**
- * struct zynqmp_disp_layer - Display layer (DRM plane)
- * @plane: DRM plane
+ * struct zynqmp_disp_layer - Display layer
* @id: Layer ID
* @disp: Back pointer to struct zynqmp_disp
* @info: Static layer information
@@ -147,8 +113,7 @@ struct zynqmp_disp_layer_info {
* @mode: Current operation mode
*/
struct zynqmp_disp_layer {
- struct drm_plane plane;
- enum zynqmp_disp_layer_id id;
+ enum zynqmp_dpsub_layer_id id;
struct zynqmp_disp *disp;
const struct zynqmp_disp_layer_info *info;
@@ -156,32 +121,22 @@ struct zynqmp_disp_layer {
const struct zynqmp_disp_format *disp_fmt;
const struct drm_format_info *drm_fmt;
- enum zynqmp_disp_layer_mode mode;
+ enum zynqmp_dpsub_layer_mode mode;
};
/**
* struct zynqmp_disp - Display controller
* @dev: Device structure
- * @drm: DRM core
* @dpsub: Display subsystem
- * @crtc: DRM CRTC
* @blend.base: Register I/O base address for the blender
* @avbuf.base: Register I/O base address for the audio/video buffer manager
* @audio.base: Registers I/O base address for the audio mixer
- * @audio.clk: Audio clock
- * @audio.clk_from_ps: True of the audio clock comes from PS, false from PL
* @layers: Layers (planes)
- * @event: Pending vblank event request
- * @pclk: Pixel clock
- * @pclk_from_ps: True of the video clock comes from PS, false from PL
*/
struct zynqmp_disp {
struct device *dev;
- struct drm_device *drm;
struct zynqmp_dpsub *dpsub;
- struct drm_crtc crtc;
-
struct {
void __iomem *base;
} blend;
@@ -190,16 +145,9 @@ struct zynqmp_disp {
} avbuf;
struct {
void __iomem *base;
- struct clk *clk;
- bool clk_from_ps;
} audio;
- struct zynqmp_disp_layer layers[ZYNQMP_DISP_NUM_LAYERS];
-
- struct drm_pending_vblank_event *event;
-
- struct clk *pclk;
- bool pclk_from_ps;
+ struct zynqmp_disp_layer layers[ZYNQMP_DPSUB_NUM_LAYERS];
};
/* -----------------------------------------------------------------------------
@@ -416,14 +364,9 @@ static void zynqmp_disp_avbuf_write(struct zynqmp_disp *disp, int reg, u32 val)
writel(val, disp->avbuf.base + reg);
}
-static bool zynqmp_disp_layer_is_gfx(const struct zynqmp_disp_layer *layer)
-{
- return layer->id == ZYNQMP_DISP_LAYER_GFX;
-}
-
static bool zynqmp_disp_layer_is_video(const struct zynqmp_disp_layer *layer)
{
- return layer->id == ZYNQMP_DISP_LAYER_VID;
+ return layer->id == ZYNQMP_DPSUB_LAYER_VID;
}
/**
@@ -566,27 +509,25 @@ static void zynqmp_disp_avbuf_disable_audio(struct zynqmp_disp *disp)
* zynqmp_disp_avbuf_enable_video - Enable a video layer
* @disp: Display controller
* @layer: The layer
- * @mode: Operating mode of layer
*
* Enable the video/graphics buffer for @layer.
*/
static void zynqmp_disp_avbuf_enable_video(struct zynqmp_disp *disp,
- struct zynqmp_disp_layer *layer,
- enum zynqmp_disp_layer_mode mode)
+ struct zynqmp_disp_layer *layer)
{
u32 val;
val = zynqmp_disp_avbuf_read(disp, ZYNQMP_DISP_AV_BUF_OUTPUT);
if (zynqmp_disp_layer_is_video(layer)) {
val &= ~ZYNQMP_DISP_AV_BUF_OUTPUT_VID1_MASK;
- if (mode == ZYNQMP_DISP_LAYER_NONLIVE)
+ if (layer->mode == ZYNQMP_DPSUB_LAYER_NONLIVE)
val |= ZYNQMP_DISP_AV_BUF_OUTPUT_VID1_MEM;
else
val |= ZYNQMP_DISP_AV_BUF_OUTPUT_VID1_LIVE;
} else {
val &= ~ZYNQMP_DISP_AV_BUF_OUTPUT_VID2_MASK;
val |= ZYNQMP_DISP_AV_BUF_OUTPUT_VID2_MEM;
- if (mode == ZYNQMP_DISP_LAYER_NONLIVE)
+ if (layer->mode == ZYNQMP_DPSUB_LAYER_NONLIVE)
val |= ZYNQMP_DISP_AV_BUF_OUTPUT_VID2_MEM;
else
val |= ZYNQMP_DISP_AV_BUF_OUTPUT_VID2_LIVE;
@@ -758,8 +699,8 @@ static void zynqmp_disp_blend_set_bg_color(struct zynqmp_disp *disp,
* @enable: True to enable global alpha blending
* @alpha: Global alpha value (ignored if @enabled is false)
*/
-static void zynqmp_disp_blend_set_global_alpha(struct zynqmp_disp *disp,
- bool enable, u32 alpha)
+void zynqmp_disp_blend_set_global_alpha(struct zynqmp_disp *disp,
+ bool enable, u32 alpha)
{
zynqmp_disp_blend_write(disp, ZYNQMP_DISP_V_BLEND_SET_GLOBAL_ALPHA,
ZYNQMP_DISP_V_BLEND_SET_GLOBAL_ALPHA_VALUE(alpha) |
@@ -902,80 +843,6 @@ static void zynqmp_disp_audio_disable(struct zynqmp_disp *disp)
ZYNQMP_DISP_AUD_SOFT_RESET_AUD_SRST);
}
-static void zynqmp_disp_audio_init(struct zynqmp_disp *disp)
-{
- /* Try the live PL audio clock. */
- disp->audio.clk = devm_clk_get(disp->dev, "dp_live_audio_aclk");
- if (!IS_ERR(disp->audio.clk)) {
- disp->audio.clk_from_ps = false;
- return;
- }
-
- /* If the live PL audio clock is not valid, fall back to PS clock. */
- disp->audio.clk = devm_clk_get(disp->dev, "dp_aud_clk");
- if (!IS_ERR(disp->audio.clk)) {
- disp->audio.clk_from_ps = true;
- return;
- }
-
- dev_err(disp->dev, "audio disabled due to missing clock\n");
-}
-
-/* -----------------------------------------------------------------------------
- * ZynqMP Display external functions for zynqmp_dp
- */
-
-/**
- * zynqmp_disp_handle_vblank - Handle the vblank event
- * @disp: Display controller
- *
- * This function handles the vblank interrupt, and sends an event to
- * CRTC object. This will be called by the DP vblank interrupt handler.
- */
-void zynqmp_disp_handle_vblank(struct zynqmp_disp *disp)
-{
- struct drm_crtc *crtc = &disp->crtc;
-
- drm_crtc_handle_vblank(crtc);
-}
-
-/**
- * zynqmp_disp_audio_enabled - If the audio is enabled
- * @disp: Display controller
- *
- * Return if the audio is enabled depending on the audio clock.
- *
- * Return: true if audio is enabled, or false.
- */
-bool zynqmp_disp_audio_enabled(struct zynqmp_disp *disp)
-{
- return !!disp->audio.clk;
-}
-
-/**
- * zynqmp_disp_get_audio_clk_rate - Get the current audio clock rate
- * @disp: Display controller
- *
- * Return: the current audio clock rate.
- */
-unsigned int zynqmp_disp_get_audio_clk_rate(struct zynqmp_disp *disp)
-{
- if (zynqmp_disp_audio_enabled(disp))
- return 0;
- return clk_get_rate(disp->audio.clk);
-}
-
-/**
- * zynqmp_disp_get_crtc_mask - Return the CRTC bit mask
- * @disp: Display controller
- *
- * Return: the crtc mask of the zyqnmp_disp CRTC.
- */
-uint32_t zynqmp_disp_get_crtc_mask(struct zynqmp_disp *disp)
-{
- return drm_crtc_mask(&disp->crtc);
-}
-
/* -----------------------------------------------------------------------------
* ZynqMP Display Layer & DRM Plane
*/
@@ -1006,19 +873,46 @@ zynqmp_disp_layer_find_format(struct zynqmp_disp_layer *layer,
}
/**
+ * zynqmp_disp_layer_drm_formats - Return the DRM formats supported by the layer
+ * @layer: The layer
+ * @num_formats: Pointer to the returned number of formats
+ *
+ * Return: A newly allocated u32 array that stores all the DRM formats
+ * supported by the layer. The number of formats in the array is returned
+ * through the num_formats argument.
+ */
+u32 *zynqmp_disp_layer_drm_formats(struct zynqmp_disp_layer *layer,
+ unsigned int *num_formats)
+{
+ unsigned int i;
+ u32 *formats;
+
+ formats = kcalloc(layer->info->num_formats, sizeof(*formats),
+ GFP_KERNEL);
+ if (!formats)
+ return NULL;
+
+ for (i = 0; i < layer->info->num_formats; ++i)
+ formats[i] = layer->info->formats[i].drm_fmt;
+
+ *num_formats = layer->info->num_formats;
+ return formats;
+}
+
+/**
* zynqmp_disp_layer_enable - Enable a layer
* @layer: The layer
+ * @mode: Operating mode of layer
*
* Enable the @layer in the audio/video buffer manager and the blender. DMA
* channels are started separately by zynqmp_disp_layer_update().
*/
-static void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer)
+void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer,
+ enum zynqmp_dpsub_layer_mode mode)
{
- zynqmp_disp_avbuf_enable_video(layer->disp, layer,
- ZYNQMP_DISP_LAYER_NONLIVE);
+ layer->mode = mode;
+ zynqmp_disp_avbuf_enable_video(layer->disp, layer);
zynqmp_disp_blend_layer_enable(layer->disp, layer);
-
- layer->mode = ZYNQMP_DISP_LAYER_NONLIVE;
}
/**
@@ -1028,12 +922,14 @@ static void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer)
* Disable the layer by stopping its DMA channels and disabling it in the
* audio/video buffer manager and the blender.
*/
-static void zynqmp_disp_layer_disable(struct zynqmp_disp_layer *layer)
+void zynqmp_disp_layer_disable(struct zynqmp_disp_layer *layer)
{
unsigned int i;
- for (i = 0; i < layer->drm_fmt->num_planes; i++)
- dmaengine_terminate_sync(layer->dmas[i].chan);
+ if (layer->disp->dpsub->dma_enabled) {
+ for (i = 0; i < layer->drm_fmt->num_planes; i++)
+ dmaengine_terminate_sync(layer->dmas[i].chan);
+ }
zynqmp_disp_avbuf_disable_video(layer->disp, layer);
zynqmp_disp_blend_layer_disable(layer->disp, layer);
@@ -1042,15 +938,13 @@ static void zynqmp_disp_layer_disable(struct zynqmp_disp_layer *layer)
/**
* zynqmp_disp_layer_set_format - Set the layer format
* @layer: The layer
- * @state: The plane state
+ * @info: The format info
*
- * Set the format for @layer based on @state->fb->format. The layer must be
- * disabled.
+ * Set the format for @layer to @info. The layer must be disabled.
*/
-static void zynqmp_disp_layer_set_format(struct zynqmp_disp_layer *layer,
- struct drm_plane_state *state)
+void zynqmp_disp_layer_set_format(struct zynqmp_disp_layer *layer,
+ const struct drm_format_info *info)
{
- const struct drm_format_info *info = state->fb->format;
unsigned int i;
layer->disp_fmt = zynqmp_disp_layer_find_format(layer, info->format);
@@ -1058,6 +952,9 @@ static void zynqmp_disp_layer_set_format(struct zynqmp_disp_layer *layer,
zynqmp_disp_avbuf_set_format(layer->disp, layer, layer->disp_fmt);
+ if (!layer->disp->dpsub->dma_enabled)
+ return;
+
/*
* Set pconfig for each DMA channel to indicate they're part of a
* video group.
@@ -1087,13 +984,16 @@ static void zynqmp_disp_layer_set_format(struct zynqmp_disp_layer *layer,
*
* Return: 0 on success, or the DMA descriptor failure error otherwise
*/
-static int zynqmp_disp_layer_update(struct zynqmp_disp_layer *layer,
- struct drm_plane_state *state)
+int zynqmp_disp_layer_update(struct zynqmp_disp_layer *layer,
+ struct drm_plane_state *state)
{
const struct drm_format_info *info = layer->drm_fmt;
unsigned int i;
- for (i = 0; i < layer->drm_fmt->num_planes; i++) {
+ if (!layer->disp->dpsub->dma_enabled)
+ return 0;
+
+ for (i = 0; i < info->num_planes; i++) {
unsigned int width = state->crtc_w / (i ? info->hsub : 1);
unsigned int height = state->crtc_h / (i ? info->vsub : 1);
struct zynqmp_disp_layer_dma *dma = &layer->dmas[i];
@@ -1128,143 +1028,6 @@ static int zynqmp_disp_layer_update(struct zynqmp_disp_layer *layer,
return 0;
}
-static inline struct zynqmp_disp_layer *plane_to_layer(struct drm_plane *plane)
-{
- return container_of(plane, struct zynqmp_disp_layer, plane);
-}
-
-static int
-zynqmp_disp_plane_atomic_check(struct drm_plane *plane,
- struct drm_atomic_state *state)
-{
- struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
- plane);
- struct drm_crtc_state *crtc_state;
-
- if (!new_plane_state->crtc)
- return 0;
-
- crtc_state = drm_atomic_get_crtc_state(state, new_plane_state->crtc);
- if (IS_ERR(crtc_state))
- return PTR_ERR(crtc_state);
-
- return drm_atomic_helper_check_plane_state(new_plane_state,
- crtc_state,
- DRM_PLANE_NO_SCALING,
- DRM_PLANE_NO_SCALING,
- false, false);
-}
-
-static void
-zynqmp_disp_plane_atomic_disable(struct drm_plane *plane,
- struct drm_atomic_state *state)
-{
- struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state,
- plane);
- struct zynqmp_disp_layer *layer = plane_to_layer(plane);
-
- if (!old_state->fb)
- return;
-
- zynqmp_disp_layer_disable(layer);
-
- if (zynqmp_disp_layer_is_gfx(layer))
- zynqmp_disp_blend_set_global_alpha(layer->disp, false,
- plane->state->alpha >> 8);
-}
-
-static void
-zynqmp_disp_plane_atomic_update(struct drm_plane *plane,
- struct drm_atomic_state *state)
-{
- struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state, plane);
- struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state, plane);
- struct zynqmp_disp_layer *layer = plane_to_layer(plane);
- bool format_changed = false;
-
- if (!old_state->fb ||
- old_state->fb->format->format != new_state->fb->format->format)
- format_changed = true;
-
- /*
- * If the format has changed (including going from a previously
- * disabled state to any format), reconfigure the format. Disable the
- * plane first if needed.
- */
- if (format_changed) {
- if (old_state->fb)
- zynqmp_disp_layer_disable(layer);
-
- zynqmp_disp_layer_set_format(layer, new_state);
- }
-
- zynqmp_disp_layer_update(layer, new_state);
-
- if (zynqmp_disp_layer_is_gfx(layer))
- zynqmp_disp_blend_set_global_alpha(layer->disp, true,
- plane->state->alpha >> 8);
-
- /* Enable or re-enable the plane is the format has changed. */
- if (format_changed)
- zynqmp_disp_layer_enable(layer);
-}
-
-static const struct drm_plane_helper_funcs zynqmp_disp_plane_helper_funcs = {
- .atomic_check = zynqmp_disp_plane_atomic_check,
- .atomic_update = zynqmp_disp_plane_atomic_update,
- .atomic_disable = zynqmp_disp_plane_atomic_disable,
-};
-
-static const struct drm_plane_funcs zynqmp_disp_plane_funcs = {
- .update_plane = drm_atomic_helper_update_plane,
- .disable_plane = drm_atomic_helper_disable_plane,
- .destroy = drm_plane_cleanup,
- .reset = drm_atomic_helper_plane_reset,
- .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
-};
-
-static int zynqmp_disp_create_planes(struct zynqmp_disp *disp)
-{
- unsigned int i, j;
- int ret;
-
- for (i = 0; i < ZYNQMP_DISP_NUM_LAYERS; i++) {
- struct zynqmp_disp_layer *layer = &disp->layers[i];
- enum drm_plane_type type;
- u32 *drm_formats;
-
- drm_formats = drmm_kcalloc(disp->drm, sizeof(*drm_formats),
- layer->info->num_formats,
- GFP_KERNEL);
- if (!drm_formats)
- return -ENOMEM;
-
- for (j = 0; j < layer->info->num_formats; ++j)
- drm_formats[j] = layer->info->formats[j].drm_fmt;
-
- /* Graphics layer is primary, and video layer is overlay. */
- type = zynqmp_disp_layer_is_video(layer)
- ? DRM_PLANE_TYPE_OVERLAY : DRM_PLANE_TYPE_PRIMARY;
- ret = drm_universal_plane_init(disp->drm, &layer->plane, 0,
- &zynqmp_disp_plane_funcs,
- drm_formats,
- layer->info->num_formats,
- NULL, type, NULL);
- if (ret)
- return ret;
-
- drm_plane_helper_add(&layer->plane,
- &zynqmp_disp_plane_helper_funcs);
-
- drm_plane_create_zpos_immutable_property(&layer->plane, i);
- if (zynqmp_disp_layer_is_gfx(layer))
- drm_plane_create_alpha_property(&layer->plane);
- }
-
- return 0;
-}
-
/**
* zynqmp_disp_layer_release_dma - Release DMA channels for a layer
* @disp: Display controller
@@ -1277,7 +1040,7 @@ static void zynqmp_disp_layer_release_dma(struct zynqmp_disp *disp,
{
unsigned int i;
- if (!layer->info)
+ if (!layer->info || !disp->dpsub->dma_enabled)
return;
for (i = 0; i < layer->info->num_channels; i++) {
@@ -1300,7 +1063,7 @@ static void zynqmp_disp_destroy_layers(struct zynqmp_disp *disp)
{
unsigned int i;
- for (i = 0; i < ZYNQMP_DISP_NUM_LAYERS; i++)
+ for (i = 0; i < ARRAY_SIZE(disp->layers); i++)
zynqmp_disp_layer_release_dma(disp, &disp->layers[i]);
}
@@ -1320,6 +1083,9 @@ static int zynqmp_disp_layer_request_dma(struct zynqmp_disp *disp,
unsigned int i;
int ret;
+ if (!disp->dpsub->dma_enabled)
+ return 0;
+
for (i = 0; i < layer->info->num_channels; i++) {
struct zynqmp_disp_layer_dma *dma = &layer->dmas[i];
char dma_channel_name[16];
@@ -1347,12 +1113,12 @@ static int zynqmp_disp_layer_request_dma(struct zynqmp_disp *disp,
static int zynqmp_disp_create_layers(struct zynqmp_disp *disp)
{
static const struct zynqmp_disp_layer_info layer_info[] = {
- [ZYNQMP_DISP_LAYER_VID] = {
+ [ZYNQMP_DPSUB_LAYER_VID] = {
.formats = avbuf_vid_fmts,
.num_formats = ARRAY_SIZE(avbuf_vid_fmts),
.num_channels = 3,
},
- [ZYNQMP_DISP_LAYER_GFX] = {
+ [ZYNQMP_DPSUB_LAYER_GFX] = {
.formats = avbuf_gfx_fmts,
.num_formats = ARRAY_SIZE(avbuf_gfx_fmts),
.num_channels = 1,
@@ -1362,7 +1128,7 @@ static int zynqmp_disp_create_layers(struct zynqmp_disp *disp)
unsigned int i;
int ret;
- for (i = 0; i < ZYNQMP_DISP_NUM_LAYERS; i++) {
+ for (i = 0; i < ARRAY_SIZE(disp->layers); i++) {
struct zynqmp_disp_layer *layer = &disp->layers[i];
layer->id = i;
@@ -1372,6 +1138,8 @@ static int zynqmp_disp_create_layers(struct zynqmp_disp *disp)
ret = zynqmp_disp_layer_request_dma(disp, layer);
if (ret)
goto err;
+
+ disp->dpsub->layers[i] = layer;
}
return 0;
@@ -1382,19 +1150,23 @@ static int zynqmp_disp_create_layers(struct zynqmp_disp *disp)
}
/* -----------------------------------------------------------------------------
- * ZynqMP Display & DRM CRTC
+ * ZynqMP Display
*/
/**
* zynqmp_disp_enable - Enable the display controller
* @disp: Display controller
*/
-static void zynqmp_disp_enable(struct zynqmp_disp *disp)
+void zynqmp_disp_enable(struct zynqmp_disp *disp)
{
+ zynqmp_disp_blend_set_output_format(disp, ZYNQMP_DPSUB_FORMAT_RGB);
+ zynqmp_disp_blend_set_bg_color(disp, 0, 0, 0);
+
zynqmp_disp_avbuf_enable(disp);
/* Choose clock source based on the DT clock handle. */
- zynqmp_disp_avbuf_set_clocks_sources(disp, disp->pclk_from_ps,
- disp->audio.clk_from_ps, true);
+ zynqmp_disp_avbuf_set_clocks_sources(disp, disp->dpsub->vid_clk_from_ps,
+ disp->dpsub->aud_clk_from_ps,
+ true);
zynqmp_disp_avbuf_enable_channels(disp);
zynqmp_disp_avbuf_enable_audio(disp);
@@ -1405,7 +1177,7 @@ static void zynqmp_disp_enable(struct zynqmp_disp *disp)
* zynqmp_disp_disable - Disable the display controller
* @disp: Display controller
*/
-static void zynqmp_disp_disable(struct zynqmp_disp *disp)
+void zynqmp_disp_disable(struct zynqmp_disp *disp)
{
zynqmp_disp_audio_disable(disp);
@@ -1414,27 +1186,27 @@ static void zynqmp_disp_disable(struct zynqmp_disp *disp)
zynqmp_disp_avbuf_disable(disp);
}
-static inline struct zynqmp_disp *crtc_to_disp(struct drm_crtc *crtc)
+/**
+ * zynqmp_disp_setup_clock - Configure the display controller pixel clock rate
+ * @disp: Display controller
+ * @mode_clock: The pixel clock rate, in Hz
+ *
+ * Return: 0 on success, or a negative error clock otherwise
+ */
+int zynqmp_disp_setup_clock(struct zynqmp_disp *disp,
+ unsigned long mode_clock)
{
- return container_of(crtc, struct zynqmp_disp, crtc);
-}
-
-static int zynqmp_disp_crtc_setup_clock(struct drm_crtc *crtc,
- struct drm_display_mode *adjusted_mode)
-{
- struct zynqmp_disp *disp = crtc_to_disp(crtc);
- unsigned long mode_clock = adjusted_mode->clock * 1000;
unsigned long rate;
long diff;
int ret;
- ret = clk_set_rate(disp->pclk, mode_clock);
+ ret = clk_set_rate(disp->dpsub->vid_clk, mode_clock);
if (ret) {
- dev_err(disp->dev, "failed to set a pixel clock\n");
+ dev_err(disp->dev, "failed to set the video clock\n");
return ret;
}
- rate = clk_get_rate(disp->pclk);
+ rate = clk_get_rate(disp->dpsub->vid_clk);
diff = rate - mode_clock;
if (abs(diff) > mode_clock / 20)
dev_info(disp->dev,
@@ -1448,245 +1220,63 @@ static int zynqmp_disp_crtc_setup_clock(struct drm_crtc *crtc,
return 0;
}
-static void
-zynqmp_disp_crtc_atomic_enable(struct drm_crtc *crtc,
- struct drm_atomic_state *state)
-{
- struct zynqmp_disp *disp = crtc_to_disp(crtc);
- struct drm_display_mode *adjusted_mode = &crtc->state->adjusted_mode;
- int ret, vrefresh;
-
- pm_runtime_get_sync(disp->dev);
-
- zynqmp_disp_crtc_setup_clock(crtc, adjusted_mode);
-
- ret = clk_prepare_enable(disp->pclk);
- if (ret) {
- dev_err(disp->dev, "failed to enable a pixel clock\n");
- pm_runtime_put_sync(disp->dev);
- return;
- }
-
- zynqmp_disp_blend_set_output_format(disp, ZYNQMP_DPSUB_FORMAT_RGB);
- zynqmp_disp_blend_set_bg_color(disp, 0, 0, 0);
-
- zynqmp_disp_enable(disp);
-
- /* Delay of 3 vblank intervals for timing gen to be stable */
- vrefresh = (adjusted_mode->clock * 1000) /
- (adjusted_mode->vtotal * adjusted_mode->htotal);
- msleep(3 * 1000 / vrefresh);
-}
-
-static void
-zynqmp_disp_crtc_atomic_disable(struct drm_crtc *crtc,
- struct drm_atomic_state *state)
-{
- struct zynqmp_disp *disp = crtc_to_disp(crtc);
- struct drm_plane_state *old_plane_state;
-
- /*
- * Disable the plane if active. The old plane state can be NULL in the
- * .shutdown() path if the plane is already disabled, skip
- * zynqmp_disp_plane_atomic_disable() in that case.
- */
- old_plane_state = drm_atomic_get_old_plane_state(state, crtc->primary);
- if (old_plane_state)
- zynqmp_disp_plane_atomic_disable(crtc->primary, state);
-
- zynqmp_disp_disable(disp);
-
- drm_crtc_vblank_off(&disp->crtc);
-
- spin_lock_irq(&crtc->dev->event_lock);
- if (crtc->state->event) {
- drm_crtc_send_vblank_event(crtc, crtc->state->event);
- crtc->state->event = NULL;
- }
- spin_unlock_irq(&crtc->dev->event_lock);
-
- clk_disable_unprepare(disp->pclk);
- pm_runtime_put_sync(disp->dev);
-}
-
-static int zynqmp_disp_crtc_atomic_check(struct drm_crtc *crtc,
- struct drm_atomic_state *state)
-{
- return drm_atomic_add_affected_planes(state, crtc);
-}
-
-static void
-zynqmp_disp_crtc_atomic_begin(struct drm_crtc *crtc,
- struct drm_atomic_state *state)
-{
- drm_crtc_vblank_on(crtc);
-}
-
-static void
-zynqmp_disp_crtc_atomic_flush(struct drm_crtc *crtc,
- struct drm_atomic_state *state)
-{
- if (crtc->state->event) {
- struct drm_pending_vblank_event *event;
-
- /* Consume the flip_done event from atomic helper. */
- event = crtc->state->event;
- crtc->state->event = NULL;
-
- event->pipe = drm_crtc_index(crtc);
-
- WARN_ON(drm_crtc_vblank_get(crtc) != 0);
-
- spin_lock_irq(&crtc->dev->event_lock);
- drm_crtc_arm_vblank_event(crtc, event);
- spin_unlock_irq(&crtc->dev->event_lock);
- }
-}
-
-static const struct drm_crtc_helper_funcs zynqmp_disp_crtc_helper_funcs = {
- .atomic_enable = zynqmp_disp_crtc_atomic_enable,
- .atomic_disable = zynqmp_disp_crtc_atomic_disable,
- .atomic_check = zynqmp_disp_crtc_atomic_check,
- .atomic_begin = zynqmp_disp_crtc_atomic_begin,
- .atomic_flush = zynqmp_disp_crtc_atomic_flush,
-};
-
-static int zynqmp_disp_crtc_enable_vblank(struct drm_crtc *crtc)
-{
- struct zynqmp_disp *disp = crtc_to_disp(crtc);
-
- zynqmp_dp_enable_vblank(disp->dpsub->dp);
-
- return 0;
-}
-
-static void zynqmp_disp_crtc_disable_vblank(struct drm_crtc *crtc)
-{
- struct zynqmp_disp *disp = crtc_to_disp(crtc);
-
- zynqmp_dp_disable_vblank(disp->dpsub->dp);
-}
-
-static const struct drm_crtc_funcs zynqmp_disp_crtc_funcs = {
- .destroy = drm_crtc_cleanup,
- .set_config = drm_atomic_helper_set_config,
- .page_flip = drm_atomic_helper_page_flip,
- .reset = drm_atomic_helper_crtc_reset,
- .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
- .enable_vblank = zynqmp_disp_crtc_enable_vblank,
- .disable_vblank = zynqmp_disp_crtc_disable_vblank,
-};
-
-static int zynqmp_disp_create_crtc(struct zynqmp_disp *disp)
-{
- struct drm_plane *plane = &disp->layers[ZYNQMP_DISP_LAYER_GFX].plane;
- int ret;
-
- ret = drm_crtc_init_with_planes(disp->drm, &disp->crtc, plane,
- NULL, &zynqmp_disp_crtc_funcs, NULL);
- if (ret < 0)
- return ret;
-
- drm_crtc_helper_add(&disp->crtc, &zynqmp_disp_crtc_helper_funcs);
-
- /* Start with vertical blanking interrupt reporting disabled. */
- drm_crtc_vblank_off(&disp->crtc);
-
- return 0;
-}
-
-static void zynqmp_disp_map_crtc_to_plane(struct zynqmp_disp *disp)
-{
- u32 possible_crtcs = drm_crtc_mask(&disp->crtc);
- unsigned int i;
-
- for (i = 0; i < ZYNQMP_DISP_NUM_LAYERS; i++)
- disp->layers[i].plane.possible_crtcs = possible_crtcs;
-}
-
/* -----------------------------------------------------------------------------
* Initialization & Cleanup
*/
-int zynqmp_disp_drm_init(struct zynqmp_dpsub *dpsub)
-{
- struct zynqmp_disp *disp = dpsub->disp;
- int ret;
-
- ret = zynqmp_disp_create_planes(disp);
- if (ret)
- return ret;
-
- ret = zynqmp_disp_create_crtc(disp);
- if (ret < 0)
- return ret;
-
- zynqmp_disp_map_crtc_to_plane(disp);
-
- return 0;
-}
-
-int zynqmp_disp_probe(struct zynqmp_dpsub *dpsub, struct drm_device *drm)
+int zynqmp_disp_probe(struct zynqmp_dpsub *dpsub)
{
struct platform_device *pdev = to_platform_device(dpsub->dev);
struct zynqmp_disp *disp;
- struct zynqmp_disp_layer *layer;
struct resource *res;
int ret;
- disp = drmm_kzalloc(drm, sizeof(*disp), GFP_KERNEL);
+ disp = kzalloc(sizeof(*disp), GFP_KERNEL);
if (!disp)
return -ENOMEM;
disp->dev = &pdev->dev;
disp->dpsub = dpsub;
- disp->drm = drm;
-
- dpsub->disp = disp;
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "blend");
disp->blend.base = devm_ioremap_resource(disp->dev, res);
- if (IS_ERR(disp->blend.base))
- return PTR_ERR(disp->blend.base);
+ if (IS_ERR(disp->blend.base)) {
+ ret = PTR_ERR(disp->blend.base);
+ goto error;
+ }
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "av_buf");
disp->avbuf.base = devm_ioremap_resource(disp->dev, res);
- if (IS_ERR(disp->avbuf.base))
- return PTR_ERR(disp->avbuf.base);
+ if (IS_ERR(disp->avbuf.base)) {
+ ret = PTR_ERR(disp->avbuf.base);
+ goto error;
+ }
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "aud");
disp->audio.base = devm_ioremap_resource(disp->dev, res);
- if (IS_ERR(disp->audio.base))
- return PTR_ERR(disp->audio.base);
-
- /* Try the live PL video clock */
- disp->pclk = devm_clk_get(disp->dev, "dp_live_video_in_clk");
- if (!IS_ERR(disp->pclk))
- disp->pclk_from_ps = false;
- else if (PTR_ERR(disp->pclk) == -EPROBE_DEFER)
- return PTR_ERR(disp->pclk);
-
- /* If the live PL video clock is not valid, fall back to PS clock */
- if (IS_ERR_OR_NULL(disp->pclk)) {
- disp->pclk = devm_clk_get(disp->dev, "dp_vtc_pixel_clk_in");
- if (IS_ERR(disp->pclk)) {
- dev_err(disp->dev, "failed to init any video clock\n");
- return PTR_ERR(disp->pclk);
- }
- disp->pclk_from_ps = true;
+ if (IS_ERR(disp->audio.base)) {
+ ret = PTR_ERR(disp->audio.base);
+ goto error;
}
- zynqmp_disp_audio_init(disp);
-
ret = zynqmp_disp_create_layers(disp);
if (ret)
- return ret;
+ goto error;
- layer = &disp->layers[ZYNQMP_DISP_LAYER_VID];
- dpsub->dma_align = 1 << layer->dmas[0].chan->device->copy_align;
+ if (disp->dpsub->dma_enabled) {
+ struct zynqmp_disp_layer *layer;
+
+ layer = &disp->layers[ZYNQMP_DPSUB_LAYER_VID];
+ dpsub->dma_align = 1 << layer->dmas[0].chan->device->copy_align;
+ }
+
+ dpsub->disp = disp;
return 0;
+
+error:
+ kfree(disp);
+ return ret;
}
void zynqmp_disp_remove(struct zynqmp_dpsub *dpsub)
diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp.h b/drivers/gpu/drm/xlnx/zynqmp_disp.h
index f402901..123cffa 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_disp.h
+++ b/drivers/gpu/drm/xlnx/zynqmp_disp.h
@@ -25,18 +25,52 @@
#define ZYNQMP_DISP_MAX_DMA_BIT 44
struct device;
-struct drm_device;
+struct drm_format_info;
+struct drm_plane_state;
struct platform_device;
struct zynqmp_disp;
+struct zynqmp_disp_layer;
struct zynqmp_dpsub;
-void zynqmp_disp_handle_vblank(struct zynqmp_disp *disp);
-bool zynqmp_disp_audio_enabled(struct zynqmp_disp *disp);
-unsigned int zynqmp_disp_get_audio_clk_rate(struct zynqmp_disp *disp);
-uint32_t zynqmp_disp_get_crtc_mask(struct zynqmp_disp *disp);
+/**
+ * enum zynqmp_dpsub_layer_id - Layer identifier
+ * @ZYNQMP_DPSUB_LAYER_VID: Video layer
+ * @ZYNQMP_DPSUB_LAYER_GFX: Graphics layer
+ */
+enum zynqmp_dpsub_layer_id {
+ ZYNQMP_DPSUB_LAYER_VID,
+ ZYNQMP_DPSUB_LAYER_GFX,
+};
-int zynqmp_disp_drm_init(struct zynqmp_dpsub *dpsub);
-int zynqmp_disp_probe(struct zynqmp_dpsub *dpsub, struct drm_device *drm);
+/**
+ * enum zynqmp_dpsub_layer_mode - Layer mode
+ * @ZYNQMP_DPSUB_LAYER_NONLIVE: non-live (memory) mode
+ * @ZYNQMP_DPSUB_LAYER_LIVE: live (stream) mode
+ */
+enum zynqmp_dpsub_layer_mode {
+ ZYNQMP_DPSUB_LAYER_NONLIVE,
+ ZYNQMP_DPSUB_LAYER_LIVE,
+};
+
+void zynqmp_disp_enable(struct zynqmp_disp *disp);
+void zynqmp_disp_disable(struct zynqmp_disp *disp);
+int zynqmp_disp_setup_clock(struct zynqmp_disp *disp,
+ unsigned long mode_clock);
+
+void zynqmp_disp_blend_set_global_alpha(struct zynqmp_disp *disp,
+ bool enable, u32 alpha);
+
+u32 *zynqmp_disp_layer_drm_formats(struct zynqmp_disp_layer *layer,
+ unsigned int *num_formats);
+void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer,
+ enum zynqmp_dpsub_layer_mode mode);
+void zynqmp_disp_layer_disable(struct zynqmp_disp_layer *layer);
+void zynqmp_disp_layer_set_format(struct zynqmp_disp_layer *layer,
+ const struct drm_format_info *info);
+int zynqmp_disp_layer_update(struct zynqmp_disp_layer *layer,
+ struct drm_plane_state *state);
+
+int zynqmp_disp_probe(struct zynqmp_dpsub *dpsub);
void zynqmp_disp_remove(struct zynqmp_dpsub *dpsub);
#endif /* _ZYNQMP_DISP_H_ */
diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
index d14612b..0a7b466 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
@@ -11,16 +11,12 @@
#include <drm/display/drm_dp_helper.h>
#include <drm/drm_atomic_helper.h>
-#include <drm/drm_connector.h>
#include <drm/drm_crtc.h>
#include <drm/drm_device.h>
#include <drm/drm_edid.h>
-#include <drm/drm_encoder.h>
-#include <drm/drm_managed.h>
+#include <drm/drm_fourcc.h>
#include <drm/drm_modes.h>
#include <drm/drm_of.h>
-#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include <linux/clk.h>
#include <linux/delay.h>
@@ -31,10 +27,12 @@
#include <linux/pm_runtime.h>
#include <linux/phy/phy.h>
#include <linux/reset.h>
+#include <linux/slab.h>
#include "zynqmp_disp.h"
#include "zynqmp_dp.h"
#include "zynqmp_dpsub.h"
+#include "zynqmp_kms.h"
static uint zynqmp_dp_aux_timeout_ms = 50;
module_param_named(aux_timeout_ms, zynqmp_dp_aux_timeout_ms, uint, 0444);
@@ -277,14 +275,13 @@ struct zynqmp_dp_config {
/**
* struct zynqmp_dp - Xilinx DisplayPort core
- * @encoder: the drm encoder structure
- * @connector: the drm connector structure
* @dev: device structure
* @dpsub: Display subsystem
- * @drm: DRM core
* @iomem: device I/O memory for register access
* @reset: reset controller
* @irq: irq
+ * @bridge: DRM bridge for the DP encoder
+ * @next_bridge: The downstream bridge
* @config: IP core configuration from DTS
* @aux: aux channel
* @phy: PHY handles for DP lanes
@@ -298,15 +295,15 @@ struct zynqmp_dp_config {
* @train_set: set of training data
*/
struct zynqmp_dp {
- struct drm_encoder encoder;
- struct drm_connector connector;
struct device *dev;
struct zynqmp_dpsub *dpsub;
- struct drm_device *drm;
void __iomem *iomem;
struct reset_control *reset;
int irq;
+ struct drm_bridge bridge;
+ struct drm_bridge *next_bridge;
+
struct zynqmp_dp_config config;
struct drm_dp_aux aux;
struct phy *phy[ZYNQMP_DP_MAX_LANES];
@@ -321,14 +318,9 @@ struct zynqmp_dp {
u8 train_set[ZYNQMP_DP_MAX_LANES];
};
-static inline struct zynqmp_dp *encoder_to_dp(struct drm_encoder *encoder)
+static inline struct zynqmp_dp *bridge_to_dp(struct drm_bridge *bridge)
{
- return container_of(encoder, struct zynqmp_dp, encoder);
-}
-
-static inline struct zynqmp_dp *connector_to_dp(struct drm_connector *connector)
-{
- return container_of(connector, struct zynqmp_dp, connector);
+ return container_of(bridge, struct zynqmp_dp, bridge);
}
static void zynqmp_dp_write(struct zynqmp_dp *dp, int offset, u32 val)
@@ -1064,7 +1056,7 @@ static int zynqmp_dp_aux_init(struct zynqmp_dp *dp)
dp->aux.name = "ZynqMP DP AUX";
dp->aux.dev = dp->dev;
- dp->aux.drm_dev = dp->drm;
+ dp->aux.drm_dev = dp->bridge.dev;
dp->aux.transfer = zynqmp_dp_aux_transfer;
return drm_dp_aux_register(&dp->aux);
@@ -1101,6 +1093,7 @@ static void zynqmp_dp_update_misc(struct zynqmp_dp *dp)
/**
* zynqmp_dp_set_format - Set the input format
* @dp: DisplayPort IP core structure
+ * @info: Display info
* @format: input format
* @bpc: bits per component
*
@@ -1109,10 +1102,10 @@ static void zynqmp_dp_update_misc(struct zynqmp_dp *dp)
* Return: 0 on success, or -EINVAL.
*/
static int zynqmp_dp_set_format(struct zynqmp_dp *dp,
+ const struct drm_display_info *info,
enum zynqmp_dpsub_format format,
unsigned int bpc)
{
- static const struct drm_display_info *display;
struct zynqmp_dp_config *config = &dp->config;
unsigned int num_colors;
@@ -1145,12 +1138,11 @@ static int zynqmp_dp_set_format(struct zynqmp_dp *dp,
return -EINVAL;
}
- display = &dp->connector.display_info;
- if (display->bpc && bpc > display->bpc) {
+ if (info && info->bpc && bpc > info->bpc) {
dev_warn(dp->dev,
"downgrading requested %ubpc to display limit %ubpc\n",
- bpc, display->bpc);
- bpc = display->bpc;
+ bpc, info->bpc);
+ bpc = info->bpc;
}
config->misc0 &= ~ZYNQMP_DP_MAIN_STREAM_MISC0_BPC_MASK;
@@ -1195,7 +1187,7 @@ static int zynqmp_dp_set_format(struct zynqmp_dp *dp,
*/
static void
zynqmp_dp_encoder_mode_set_transfer_unit(struct zynqmp_dp *dp,
- struct drm_display_mode *mode)
+ const struct drm_display_mode *mode)
{
u32 tu = ZYNQMP_DP_MSA_TRANSFER_UNIT_SIZE_TU_SIZE_DEF;
u32 bw, vid_kbytes, avg_bytes_per_tu, init_wait;
@@ -1255,12 +1247,12 @@ static void zynqmp_dp_encoder_mode_set_stream(struct zynqmp_dp *dp,
zynqmp_dp_write(dp, ZYNQMP_DP_MAIN_STREAM_VSTART,
mode->vtotal - mode->vsync_start);
- /* In synchronous mode, set the diviers */
+ /* In synchronous mode, set the dividers */
if (dp->config.misc0 & ZYNQMP_DP_MAIN_STREAM_MISC0_SYNC_LOCK) {
reg = drm_dp_bw_code_to_link_rate(dp->mode.bw_code);
zynqmp_dp_write(dp, ZYNQMP_DP_MAIN_STREAM_N_VID, reg);
zynqmp_dp_write(dp, ZYNQMP_DP_MAIN_STREAM_M_VID, mode->clock);
- rate = zynqmp_disp_get_audio_clk_rate(dp->dpsub->disp);
+ rate = zynqmp_dpsub_get_audio_clk_rate(dp->dpsub);
if (rate) {
dev_dbg(dp->dev, "Audio rate: %d\n", rate / 512);
zynqmp_dp_write(dp, ZYNQMP_DP_TX_N_AUD, reg);
@@ -1269,7 +1261,7 @@ static void zynqmp_dp_encoder_mode_set_stream(struct zynqmp_dp *dp,
}
/* Only 2 channel audio is supported now */
- if (zynqmp_disp_audio_enabled(dp->dpsub->disp))
+ if (zynqmp_dpsub_audio_enabled(dp->dpsub))
zynqmp_dp_write(dp, ZYNQMP_DP_TX_AUDIO_CHANNELS, 1);
zynqmp_dp_write(dp, ZYNQMP_DP_USER_PIX_WIDTH, 1);
@@ -1281,13 +1273,254 @@ static void zynqmp_dp_encoder_mode_set_stream(struct zynqmp_dp *dp,
}
/* -----------------------------------------------------------------------------
- * DRM Connector
+ * DISP Configuration
*/
-static enum drm_connector_status
-zynqmp_dp_connector_detect(struct drm_connector *connector, bool force)
+static void zynqmp_dp_disp_enable(struct zynqmp_dp *dp,
+ struct drm_bridge_state *old_bridge_state)
{
- struct zynqmp_dp *dp = connector_to_dp(connector);
+ enum zynqmp_dpsub_layer_id layer_id;
+ struct zynqmp_disp_layer *layer;
+ const struct drm_format_info *info;
+
+ if (dp->dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_VIDEO))
+ layer_id = ZYNQMP_DPSUB_LAYER_VID;
+ else if (dp->dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_GFX))
+ layer_id = ZYNQMP_DPSUB_LAYER_GFX;
+ else
+ return;
+
+ layer = dp->dpsub->layers[layer_id];
+
+ /* TODO: Make the format configurable. */
+ info = drm_format_info(DRM_FORMAT_YUV422);
+ zynqmp_disp_layer_set_format(layer, info);
+ zynqmp_disp_layer_enable(layer, ZYNQMP_DPSUB_LAYER_LIVE);
+
+ if (layer_id == ZYNQMP_DPSUB_LAYER_GFX)
+ zynqmp_disp_blend_set_global_alpha(dp->dpsub->disp, true, 255);
+ else
+ zynqmp_disp_blend_set_global_alpha(dp->dpsub->disp, false, 0);
+
+ zynqmp_disp_enable(dp->dpsub->disp);
+}
+
+static void zynqmp_dp_disp_disable(struct zynqmp_dp *dp,
+ struct drm_bridge_state *old_bridge_state)
+{
+ struct zynqmp_disp_layer *layer;
+
+ if (dp->dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_VIDEO))
+ layer = dp->dpsub->layers[ZYNQMP_DPSUB_LAYER_VID];
+ else if (dp->dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_GFX))
+ layer = dp->dpsub->layers[ZYNQMP_DPSUB_LAYER_GFX];
+ else
+ return;
+
+ zynqmp_disp_disable(dp->dpsub->disp);
+ zynqmp_disp_layer_disable(layer);
+}
+
+/* -----------------------------------------------------------------------------
+ * DRM Bridge
+ */
+
+static int zynqmp_dp_bridge_attach(struct drm_bridge *bridge,
+ enum drm_bridge_attach_flags flags)
+{
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
+ int ret;
+
+ /* Initialize and register the AUX adapter. */
+ ret = zynqmp_dp_aux_init(dp);
+ if (ret) {
+ dev_err(dp->dev, "failed to initialize DP aux\n");
+ return ret;
+ }
+
+ if (dp->next_bridge) {
+ ret = drm_bridge_attach(bridge->encoder, dp->next_bridge,
+ bridge, flags);
+ if (ret < 0)
+ goto error;
+ }
+
+ /* Now that initialisation is complete, enable interrupts. */
+ zynqmp_dp_write(dp, ZYNQMP_DP_INT_EN, ZYNQMP_DP_INT_ALL);
+
+ return 0;
+
+error:
+ zynqmp_dp_aux_cleanup(dp);
+ return ret;
+}
+
+static void zynqmp_dp_bridge_detach(struct drm_bridge *bridge)
+{
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
+
+ zynqmp_dp_aux_cleanup(dp);
+}
+
+static enum drm_mode_status
+zynqmp_dp_bridge_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *info,
+ const struct drm_display_mode *mode)
+{
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
+ int rate;
+
+ if (mode->clock > ZYNQMP_MAX_FREQ) {
+ dev_dbg(dp->dev, "filtered mode %s for high pixel rate\n",
+ mode->name);
+ drm_mode_debug_printmodeline(mode);
+ return MODE_CLOCK_HIGH;
+ }
+
+ /* Check with link rate and lane count */
+ rate = zynqmp_dp_max_rate(dp->link_config.max_rate,
+ dp->link_config.max_lanes, dp->config.bpp);
+ if (mode->clock > rate) {
+ dev_dbg(dp->dev, "filtered mode %s for high pixel rate\n",
+ mode->name);
+ drm_mode_debug_printmodeline(mode);
+ return MODE_CLOCK_HIGH;
+ }
+
+ return MODE_OK;
+}
+
+static void zynqmp_dp_bridge_atomic_enable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_bridge_state)
+{
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
+ struct drm_atomic_state *state = old_bridge_state->base.state;
+ const struct drm_crtc_state *crtc_state;
+ const struct drm_display_mode *adjusted_mode;
+ const struct drm_display_mode *mode;
+ struct drm_connector *connector;
+ struct drm_crtc *crtc;
+ unsigned int i;
+ int rate;
+ int ret;
+
+ pm_runtime_get_sync(dp->dev);
+
+ zynqmp_dp_disp_enable(dp, old_bridge_state);
+
+ /*
+ * Retrieve the CRTC mode and adjusted mode. This requires a little
+ * dance to go from the bridge to the encoder, to the connector and to
+ * the CRTC.
+ */
+ connector = drm_atomic_get_new_connector_for_encoder(state,
+ bridge->encoder);
+ crtc = drm_atomic_get_new_connector_state(state, connector)->crtc;
+ crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
+ adjusted_mode = &crtc_state->adjusted_mode;
+ mode = &crtc_state->mode;
+
+ zynqmp_dp_set_format(dp, &connector->display_info,
+ ZYNQMP_DPSUB_FORMAT_RGB, 8);
+
+ /* Check again as bpp or format might have been changed */
+ rate = zynqmp_dp_max_rate(dp->link_config.max_rate,
+ dp->link_config.max_lanes, dp->config.bpp);
+ if (mode->clock > rate) {
+ dev_err(dp->dev, "mode %s has too high pixel rate\n",
+ mode->name);
+ drm_mode_debug_printmodeline(mode);
+ }
+
+ /* Configure the mode */
+ ret = zynqmp_dp_mode_configure(dp, adjusted_mode->clock, 0);
+ if (ret < 0) {
+ pm_runtime_put_sync(dp->dev);
+ return;
+ }
+
+ zynqmp_dp_encoder_mode_set_transfer_unit(dp, adjusted_mode);
+ zynqmp_dp_encoder_mode_set_stream(dp, adjusted_mode);
+
+ /* Enable the encoder */
+ dp->enabled = true;
+ zynqmp_dp_update_misc(dp);
+ if (zynqmp_dpsub_audio_enabled(dp->dpsub))
+ zynqmp_dp_write(dp, ZYNQMP_DP_TX_AUDIO_CONTROL, 1);
+ zynqmp_dp_write(dp, ZYNQMP_DP_TX_PHY_POWER_DOWN, 0);
+ if (dp->status == connector_status_connected) {
+ for (i = 0; i < 3; i++) {
+ ret = drm_dp_dpcd_writeb(&dp->aux, DP_SET_POWER,
+ DP_SET_POWER_D0);
+ if (ret == 1)
+ break;
+ usleep_range(300, 500);
+ }
+ /* Some monitors take time to wake up properly */
+ msleep(zynqmp_dp_power_on_delay_ms);
+ }
+ if (ret != 1)
+ dev_dbg(dp->dev, "DP aux failed\n");
+ else
+ zynqmp_dp_train_loop(dp);
+ zynqmp_dp_write(dp, ZYNQMP_DP_SOFTWARE_RESET,
+ ZYNQMP_DP_SOFTWARE_RESET_ALL);
+ zynqmp_dp_write(dp, ZYNQMP_DP_MAIN_STREAM_ENABLE, 1);
+}
+
+static void zynqmp_dp_bridge_atomic_disable(struct drm_bridge *bridge,
+ struct drm_bridge_state *old_bridge_state)
+{
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
+
+ dp->enabled = false;
+ cancel_delayed_work(&dp->hpd_work);
+ zynqmp_dp_write(dp, ZYNQMP_DP_MAIN_STREAM_ENABLE, 0);
+ drm_dp_dpcd_writeb(&dp->aux, DP_SET_POWER, DP_SET_POWER_D3);
+ zynqmp_dp_write(dp, ZYNQMP_DP_TX_PHY_POWER_DOWN,
+ ZYNQMP_DP_TX_PHY_POWER_DOWN_ALL);
+ if (zynqmp_dpsub_audio_enabled(dp->dpsub))
+ zynqmp_dp_write(dp, ZYNQMP_DP_TX_AUDIO_CONTROL, 0);
+
+ zynqmp_dp_disp_disable(dp, old_bridge_state);
+
+ pm_runtime_put_sync(dp->dev);
+}
+
+#define ZYNQMP_DP_MIN_H_BACKPORCH 20
+
+static int zynqmp_dp_bridge_atomic_check(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
+ struct drm_display_mode *mode = &crtc_state->mode;
+ struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
+ int diff = mode->htotal - mode->hsync_end;
+
+ /*
+ * ZynqMP DP requires horizontal backporch to be greater than 12.
+ * This limitation may not be compatible with the sink device.
+ */
+ if (diff < ZYNQMP_DP_MIN_H_BACKPORCH) {
+ int vrefresh = (adjusted_mode->clock * 1000) /
+ (adjusted_mode->vtotal * adjusted_mode->htotal);
+
+ dev_dbg(dp->dev, "hbackporch adjusted: %d to %d",
+ diff, ZYNQMP_DP_MIN_H_BACKPORCH - diff);
+ diff = ZYNQMP_DP_MIN_H_BACKPORCH - diff;
+ adjusted_mode->htotal += diff;
+ adjusted_mode->clock = adjusted_mode->vtotal *
+ adjusted_mode->htotal * vrefresh / 1000;
+ }
+
+ return 0;
+}
+
+static enum drm_connector_status zynqmp_dp_bridge_detect(struct drm_bridge *bridge)
+{
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
struct zynqmp_dp_link_config *link_config = &dp->link_config;
u32 state, i;
int ret;
@@ -1327,192 +1560,26 @@ zynqmp_dp_connector_detect(struct drm_connector *connector, bool force)
return connector_status_disconnected;
}
-static int zynqmp_dp_connector_get_modes(struct drm_connector *connector)
+static struct edid *zynqmp_dp_bridge_get_edid(struct drm_bridge *bridge,
+ struct drm_connector *connector)
{
- struct zynqmp_dp *dp = connector_to_dp(connector);
- struct edid *edid;
- int ret;
+ struct zynqmp_dp *dp = bridge_to_dp(bridge);
- edid = drm_get_edid(connector, &dp->aux.ddc);
- if (!edid)
- return 0;
-
- drm_connector_update_edid_property(connector, edid);
- ret = drm_add_edid_modes(connector, edid);
- kfree(edid);
-
- return ret;
+ return drm_get_edid(connector, &dp->aux.ddc);
}
-static struct drm_encoder *
-zynqmp_dp_connector_best_encoder(struct drm_connector *connector)
-{
- struct zynqmp_dp *dp = connector_to_dp(connector);
-
- return &dp->encoder;
-}
-
-static int zynqmp_dp_connector_mode_valid(struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- struct zynqmp_dp *dp = connector_to_dp(connector);
- u8 max_lanes = dp->link_config.max_lanes;
- u8 bpp = dp->config.bpp;
- int max_rate = dp->link_config.max_rate;
- int rate;
-
- if (mode->clock > ZYNQMP_MAX_FREQ) {
- dev_dbg(dp->dev, "filtered the mode, %s,for high pixel rate\n",
- mode->name);
- drm_mode_debug_printmodeline(mode);
- return MODE_CLOCK_HIGH;
- }
-
- /* Check with link rate and lane count */
- rate = zynqmp_dp_max_rate(max_rate, max_lanes, bpp);
- if (mode->clock > rate) {
- dev_dbg(dp->dev, "filtered the mode, %s,for high pixel rate\n",
- mode->name);
- drm_mode_debug_printmodeline(mode);
- return MODE_CLOCK_HIGH;
- }
-
- return MODE_OK;
-}
-
-static const struct drm_connector_funcs zynqmp_dp_connector_funcs = {
- .detect = zynqmp_dp_connector_detect,
- .fill_modes = drm_helper_probe_single_connector_modes,
- .destroy = drm_connector_cleanup,
- .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
- .reset = drm_atomic_helper_connector_reset,
-};
-
-static const struct drm_connector_helper_funcs
-zynqmp_dp_connector_helper_funcs = {
- .get_modes = zynqmp_dp_connector_get_modes,
- .best_encoder = zynqmp_dp_connector_best_encoder,
- .mode_valid = zynqmp_dp_connector_mode_valid,
-};
-
-/* -----------------------------------------------------------------------------
- * DRM Encoder
- */
-
-static void zynqmp_dp_encoder_enable(struct drm_encoder *encoder)
-{
- struct zynqmp_dp *dp = encoder_to_dp(encoder);
- unsigned int i;
- int ret = 0;
-
- pm_runtime_get_sync(dp->dev);
- dp->enabled = true;
- zynqmp_dp_update_misc(dp);
- if (zynqmp_disp_audio_enabled(dp->dpsub->disp))
- zynqmp_dp_write(dp, ZYNQMP_DP_TX_AUDIO_CONTROL, 1);
- zynqmp_dp_write(dp, ZYNQMP_DP_TX_PHY_POWER_DOWN, 0);
- if (dp->status == connector_status_connected) {
- for (i = 0; i < 3; i++) {
- ret = drm_dp_dpcd_writeb(&dp->aux, DP_SET_POWER,
- DP_SET_POWER_D0);
- if (ret == 1)
- break;
- usleep_range(300, 500);
- }
- /* Some monitors take time to wake up properly */
- msleep(zynqmp_dp_power_on_delay_ms);
- }
- if (ret != 1)
- dev_dbg(dp->dev, "DP aux failed\n");
- else
- zynqmp_dp_train_loop(dp);
- zynqmp_dp_write(dp, ZYNQMP_DP_SOFTWARE_RESET,
- ZYNQMP_DP_SOFTWARE_RESET_ALL);
- zynqmp_dp_write(dp, ZYNQMP_DP_MAIN_STREAM_ENABLE, 1);
-}
-
-static void zynqmp_dp_encoder_disable(struct drm_encoder *encoder)
-{
- struct zynqmp_dp *dp = encoder_to_dp(encoder);
-
- dp->enabled = false;
- cancel_delayed_work(&dp->hpd_work);
- zynqmp_dp_write(dp, ZYNQMP_DP_MAIN_STREAM_ENABLE, 0);
- drm_dp_dpcd_writeb(&dp->aux, DP_SET_POWER, DP_SET_POWER_D3);
- zynqmp_dp_write(dp, ZYNQMP_DP_TX_PHY_POWER_DOWN,
- ZYNQMP_DP_TX_PHY_POWER_DOWN_ALL);
- if (zynqmp_disp_audio_enabled(dp->dpsub->disp))
- zynqmp_dp_write(dp, ZYNQMP_DP_TX_AUDIO_CONTROL, 0);
- pm_runtime_put_sync(dp->dev);
-}
-
-static void
-zynqmp_dp_encoder_atomic_mode_set(struct drm_encoder *encoder,
- struct drm_crtc_state *crtc_state,
- struct drm_connector_state *connector_state)
-{
- struct zynqmp_dp *dp = encoder_to_dp(encoder);
- struct drm_display_mode *mode = &crtc_state->mode;
- struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
- u8 max_lanes = dp->link_config.max_lanes;
- u8 bpp = dp->config.bpp;
- int rate, max_rate = dp->link_config.max_rate;
- int ret;
-
- zynqmp_dp_set_format(dp, ZYNQMP_DPSUB_FORMAT_RGB, 8);
-
- /* Check again as bpp or format might have been chagned */
- rate = zynqmp_dp_max_rate(max_rate, max_lanes, bpp);
- if (mode->clock > rate) {
- dev_err(dp->dev, "the mode, %s,has too high pixel rate\n",
- mode->name);
- drm_mode_debug_printmodeline(mode);
- }
-
- ret = zynqmp_dp_mode_configure(dp, adjusted_mode->clock, 0);
- if (ret < 0)
- return;
-
- zynqmp_dp_encoder_mode_set_transfer_unit(dp, adjusted_mode);
- zynqmp_dp_encoder_mode_set_stream(dp, adjusted_mode);
-}
-
-#define ZYNQMP_DP_MIN_H_BACKPORCH 20
-
-static int
-zynqmp_dp_encoder_atomic_check(struct drm_encoder *encoder,
- struct drm_crtc_state *crtc_state,
- struct drm_connector_state *conn_state)
-{
- struct drm_display_mode *mode = &crtc_state->mode;
- struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
- int diff = mode->htotal - mode->hsync_end;
-
- /*
- * ZynqMP DP requires horizontal backporch to be greater than 12.
- * This limitation may not be compatible with the sink device.
- */
- if (diff < ZYNQMP_DP_MIN_H_BACKPORCH) {
- int vrefresh = (adjusted_mode->clock * 1000) /
- (adjusted_mode->vtotal * adjusted_mode->htotal);
-
- dev_dbg(encoder->dev->dev, "hbackporch adjusted: %d to %d",
- diff, ZYNQMP_DP_MIN_H_BACKPORCH - diff);
- diff = ZYNQMP_DP_MIN_H_BACKPORCH - diff;
- adjusted_mode->htotal += diff;
- adjusted_mode->clock = adjusted_mode->vtotal *
- adjusted_mode->htotal * vrefresh / 1000;
- }
-
- return 0;
-}
-
-static const struct drm_encoder_helper_funcs zynqmp_dp_encoder_helper_funcs = {
- .enable = zynqmp_dp_encoder_enable,
- .disable = zynqmp_dp_encoder_disable,
- .atomic_mode_set = zynqmp_dp_encoder_atomic_mode_set,
- .atomic_check = zynqmp_dp_encoder_atomic_check,
+static const struct drm_bridge_funcs zynqmp_dp_bridge_funcs = {
+ .attach = zynqmp_dp_bridge_attach,
+ .detach = zynqmp_dp_bridge_detach,
+ .mode_valid = zynqmp_dp_bridge_mode_valid,
+ .atomic_enable = zynqmp_dp_bridge_atomic_enable,
+ .atomic_disable = zynqmp_dp_bridge_atomic_disable,
+ .atomic_duplicate_state = drm_atomic_helper_bridge_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
+ .atomic_reset = drm_atomic_helper_bridge_reset,
+ .atomic_check = zynqmp_dp_bridge_atomic_check,
+ .detect = zynqmp_dp_bridge_detect,
+ .get_edid = zynqmp_dp_bridge_get_edid,
};
/* -----------------------------------------------------------------------------
@@ -1543,12 +1610,12 @@ void zynqmp_dp_disable_vblank(struct zynqmp_dp *dp)
static void zynqmp_dp_hpd_work_func(struct work_struct *work)
{
- struct zynqmp_dp *dp;
+ struct zynqmp_dp *dp = container_of(work, struct zynqmp_dp,
+ hpd_work.work);
+ enum drm_connector_status status;
- dp = container_of(work, struct zynqmp_dp, hpd_work.work);
-
- if (dp->drm)
- drm_helper_hpd_irq_event(dp->drm);
+ status = zynqmp_dp_bridge_detect(&dp->bridge);
+ drm_bridge_hpd_notify(&dp->bridge, status);
}
static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
@@ -1570,7 +1637,7 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
if (status & ZYNQMP_DP_INT_VBLANK_START)
- zynqmp_disp_handle_vblank(dp->dpsub->disp);
+ zynqmp_dpsub_drm_handle_vblank(dp->dpsub);
if (status & ZYNQMP_DP_INT_HPD_EVENT)
schedule_delayed_work(&dp->hpd_work, 0);
@@ -1599,94 +1666,76 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
* Initialization & Cleanup
*/
-int zynqmp_dp_drm_init(struct zynqmp_dpsub *dpsub)
-{
- struct zynqmp_dp *dp = dpsub->dp;
- struct drm_encoder *encoder = &dp->encoder;
- struct drm_connector *connector = &dp->connector;
- int ret;
-
- dp->config.misc0 &= ~ZYNQMP_DP_MAIN_STREAM_MISC0_SYNC_LOCK;
- zynqmp_dp_set_format(dp, ZYNQMP_DPSUB_FORMAT_RGB, 8);
-
- /* Create the DRM encoder and connector. */
- encoder->possible_crtcs |= zynqmp_disp_get_crtc_mask(dpsub->disp);
- drm_simple_encoder_init(dp->drm, encoder, DRM_MODE_ENCODER_TMDS);
- drm_encoder_helper_add(encoder, &zynqmp_dp_encoder_helper_funcs);
-
- connector->polled = DRM_CONNECTOR_POLL_HPD;
- ret = drm_connector_init(encoder->dev, connector,
- &zynqmp_dp_connector_funcs,
- DRM_MODE_CONNECTOR_DisplayPort);
- if (ret) {
- dev_err(dp->dev, "failed to create the DRM connector\n");
- return ret;
- }
-
- drm_connector_helper_add(connector, &zynqmp_dp_connector_helper_funcs);
- drm_connector_register(connector);
- drm_connector_attach_encoder(connector, encoder);
-
- /* Initialize and register the AUX adapter. */
- ret = zynqmp_dp_aux_init(dp);
- if (ret) {
- dev_err(dp->dev, "failed to initialize DP aux\n");
- return ret;
- }
-
- /* Now that initialisation is complete, enable interrupts. */
- zynqmp_dp_write(dp, ZYNQMP_DP_INT_EN, ZYNQMP_DP_INT_ALL);
-
- return 0;
-}
-
-int zynqmp_dp_probe(struct zynqmp_dpsub *dpsub, struct drm_device *drm)
+int zynqmp_dp_probe(struct zynqmp_dpsub *dpsub)
{
struct platform_device *pdev = to_platform_device(dpsub->dev);
+ struct drm_bridge *bridge;
struct zynqmp_dp *dp;
struct resource *res;
int ret;
- dp = drmm_kzalloc(drm, sizeof(*dp), GFP_KERNEL);
+ dp = kzalloc(sizeof(*dp), GFP_KERNEL);
if (!dp)
return -ENOMEM;
dp->dev = &pdev->dev;
dp->dpsub = dpsub;
dp->status = connector_status_disconnected;
- dp->drm = drm;
INIT_DELAYED_WORK(&dp->hpd_work, zynqmp_dp_hpd_work_func);
- dpsub->dp = dp;
-
/* Acquire all resources (IOMEM, IRQ and PHYs). */
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dp");
dp->iomem = devm_ioremap_resource(dp->dev, res);
- if (IS_ERR(dp->iomem))
- return PTR_ERR(dp->iomem);
+ if (IS_ERR(dp->iomem)) {
+ ret = PTR_ERR(dp->iomem);
+ goto err_free;
+ }
dp->irq = platform_get_irq(pdev, 0);
- if (dp->irq < 0)
- return dp->irq;
+ if (dp->irq < 0) {
+ ret = dp->irq;
+ goto err_free;
+ }
dp->reset = devm_reset_control_get(dp->dev, NULL);
if (IS_ERR(dp->reset)) {
if (PTR_ERR(dp->reset) != -EPROBE_DEFER)
dev_err(dp->dev, "failed to get reset: %ld\n",
PTR_ERR(dp->reset));
- return PTR_ERR(dp->reset);
+ ret = PTR_ERR(dp->reset);
+ goto err_free;
}
ret = zynqmp_dp_reset(dp, false);
if (ret < 0)
- return ret;
+ goto err_free;
ret = zynqmp_dp_phy_probe(dp);
if (ret)
goto err_reset;
+ /* Initialize the bridge. */
+ bridge = &dp->bridge;
+ bridge->funcs = &zynqmp_dp_bridge_funcs;
+ bridge->ops = DRM_BRIDGE_OP_DETECT | DRM_BRIDGE_OP_EDID
+ | DRM_BRIDGE_OP_HPD;
+ bridge->type = DRM_MODE_CONNECTOR_DisplayPort;
+ dpsub->bridge = bridge;
+
+ /*
+ * Acquire the next bridge in the chain. Ignore errors caused by port@5
+ * not being connected for backward-compatibility with older DTs.
+ */
+ ret = drm_of_find_panel_or_bridge(dp->dev->of_node, 5, 0, NULL,
+ &dp->next_bridge);
+ if (ret < 0 && ret != -ENODEV)
+ goto err_reset;
+
/* Initialize the hardware. */
+ dp->config.misc0 &= ~ZYNQMP_DP_MAIN_STREAM_MISC0_SYNC_LOCK;
+ zynqmp_dp_set_format(dp, NULL, ZYNQMP_DPSUB_FORMAT_RGB, 8);
+
zynqmp_dp_write(dp, ZYNQMP_DP_TX_PHY_POWER_DOWN,
ZYNQMP_DP_TX_PHY_POWER_DOWN_ALL);
zynqmp_dp_set(dp, ZYNQMP_DP_PHY_RESET, ZYNQMP_DP_PHY_RESET_ALL_RESET);
@@ -1710,6 +1759,8 @@ int zynqmp_dp_probe(struct zynqmp_dpsub *dpsub, struct drm_device *drm)
if (ret < 0)
goto err_phy_exit;
+ dpsub->dp = dp;
+
dev_dbg(dp->dev, "ZynqMP DisplayPort Tx probed with %u lanes\n",
dp->num_lanes);
@@ -1719,7 +1770,8 @@ int zynqmp_dp_probe(struct zynqmp_dpsub *dpsub, struct drm_device *drm)
zynqmp_dp_phy_exit(dp);
err_reset:
zynqmp_dp_reset(dp, true);
-
+err_free:
+ kfree(dp);
return ret;
}
@@ -1731,7 +1783,6 @@ void zynqmp_dp_remove(struct zynqmp_dpsub *dpsub)
disable_irq(dp->irq);
cancel_delayed_work_sync(&dp->hpd_work);
- zynqmp_dp_aux_cleanup(dp);
zynqmp_dp_write(dp, ZYNQMP_DP_TRANSMITTER_ENABLE, 0);
zynqmp_dp_write(dp, ZYNQMP_DP_INT_DS, 0xffffffff);
diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.h b/drivers/gpu/drm/xlnx/zynqmp_dp.h
index 4507740..f077d7f 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dp.h
+++ b/drivers/gpu/drm/xlnx/zynqmp_dp.h
@@ -12,7 +12,6 @@
#ifndef _ZYNQMP_DP_H_
#define _ZYNQMP_DP_H_
-struct drm_device;
struct platform_device;
struct zynqmp_dp;
struct zynqmp_dpsub;
@@ -20,8 +19,7 @@ struct zynqmp_dpsub;
void zynqmp_dp_enable_vblank(struct zynqmp_dp *dp);
void zynqmp_dp_disable_vblank(struct zynqmp_dp *dp);
-int zynqmp_dp_drm_init(struct zynqmp_dpsub *dpsub);
-int zynqmp_dp_probe(struct zynqmp_dpsub *dpsub, struct drm_device *drm);
+int zynqmp_dp_probe(struct zynqmp_dpsub *dpsub);
void zynqmp_dp_remove(struct zynqmp_dpsub *dpsub);
#endif /* _ZYNQMP_DP_H_ */
diff --git a/drivers/gpu/drm/xlnx/zynqmp_dpsub.c b/drivers/gpu/drm/xlnx/zynqmp_dpsub.c
index 1de2d92..bab8624 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dpsub.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_dpsub.c
@@ -12,137 +12,21 @@
#include <linux/clk.h>
#include <linux/dma-mapping.h>
#include <linux/module.h>
+#include <linux/of_graph.h>
#include <linux/of_reserved_mem.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
+#include <linux/slab.h>
#include <drm/drm_atomic_helper.h>
-#include <drm/drm_device.h>
-#include <drm/drm_drv.h>
-#include <drm/drm_fb_helper.h>
-#include <drm/drm_fourcc.h>
-#include <drm/drm_gem_dma_helper.h>
-#include <drm/drm_gem_framebuffer_helper.h>
-#include <drm/drm_managed.h>
-#include <drm/drm_mode_config.h>
+#include <drm/drm_bridge.h>
+#include <drm/drm_modeset_helper.h>
#include <drm/drm_module.h>
-#include <drm/drm_probe_helper.h>
-#include <drm/drm_vblank.h>
#include "zynqmp_disp.h"
#include "zynqmp_dp.h"
#include "zynqmp_dpsub.h"
-
-/* -----------------------------------------------------------------------------
- * Dumb Buffer & Framebuffer Allocation
- */
-
-static int zynqmp_dpsub_dumb_create(struct drm_file *file_priv,
- struct drm_device *drm,
- struct drm_mode_create_dumb *args)
-{
- struct zynqmp_dpsub *dpsub = to_zynqmp_dpsub(drm);
- unsigned int pitch = DIV_ROUND_UP(args->width * args->bpp, 8);
-
- /* Enforce the alignment constraints of the DMA engine. */
- args->pitch = ALIGN(pitch, dpsub->dma_align);
-
- return drm_gem_dma_dumb_create_internal(file_priv, drm, args);
-}
-
-static struct drm_framebuffer *
-zynqmp_dpsub_fb_create(struct drm_device *drm, struct drm_file *file_priv,
- const struct drm_mode_fb_cmd2 *mode_cmd)
-{
- struct zynqmp_dpsub *dpsub = to_zynqmp_dpsub(drm);
- struct drm_mode_fb_cmd2 cmd = *mode_cmd;
- unsigned int i;
-
- /* Enforce the alignment constraints of the DMA engine. */
- for (i = 0; i < ARRAY_SIZE(cmd.pitches); ++i)
- cmd.pitches[i] = ALIGN(cmd.pitches[i], dpsub->dma_align);
-
- return drm_gem_fb_create(drm, file_priv, &cmd);
-}
-
-static const struct drm_mode_config_funcs zynqmp_dpsub_mode_config_funcs = {
- .fb_create = zynqmp_dpsub_fb_create,
- .atomic_check = drm_atomic_helper_check,
- .atomic_commit = drm_atomic_helper_commit,
-};
-
-/* -----------------------------------------------------------------------------
- * DRM/KMS Driver
- */
-
-DEFINE_DRM_GEM_DMA_FOPS(zynqmp_dpsub_drm_fops);
-
-static const struct drm_driver zynqmp_dpsub_drm_driver = {
- .driver_features = DRIVER_MODESET | DRIVER_GEM |
- DRIVER_ATOMIC,
-
- DRM_GEM_DMA_DRIVER_OPS_WITH_DUMB_CREATE(zynqmp_dpsub_dumb_create),
-
- .fops = &zynqmp_dpsub_drm_fops,
-
- .name = "zynqmp-dpsub",
- .desc = "Xilinx DisplayPort Subsystem Driver",
- .date = "20130509",
- .major = 1,
- .minor = 0,
-};
-
-static int zynqmp_dpsub_drm_init(struct zynqmp_dpsub *dpsub)
-{
- struct drm_device *drm = &dpsub->drm;
- int ret;
-
- /* Initialize mode config, vblank and the KMS poll helper. */
- ret = drmm_mode_config_init(drm);
- if (ret < 0)
- return ret;
-
- drm->mode_config.funcs = &zynqmp_dpsub_mode_config_funcs;
- drm->mode_config.min_width = 0;
- drm->mode_config.min_height = 0;
- drm->mode_config.max_width = ZYNQMP_DISP_MAX_WIDTH;
- drm->mode_config.max_height = ZYNQMP_DISP_MAX_HEIGHT;
-
- ret = drm_vblank_init(drm, 1);
- if (ret)
- return ret;
-
- drm_kms_helper_poll_init(drm);
-
- /*
- * Initialize the DISP and DP components. This will creates planes,
- * CRTC, encoder and connector. The DISP should be initialized first as
- * the DP encoder needs the CRTC.
- */
- ret = zynqmp_disp_drm_init(dpsub);
- if (ret)
- goto err_poll_fini;
-
- ret = zynqmp_dp_drm_init(dpsub);
- if (ret)
- goto err_poll_fini;
-
- /* Reset all components and register the DRM device. */
- drm_mode_config_reset(drm);
-
- ret = drm_dev_register(drm, 0);
- if (ret < 0)
- goto err_poll_fini;
-
- /* Initialize fbdev generic emulation. */
- drm_fbdev_generic_setup(drm, 24);
-
- return 0;
-
-err_poll_fini:
- drm_kms_helper_poll_fini(drm);
- return ret;
-}
+#include "zynqmp_kms.h"
/* -----------------------------------------------------------------------------
* Power Management
@@ -152,14 +36,20 @@ static int __maybe_unused zynqmp_dpsub_suspend(struct device *dev)
{
struct zynqmp_dpsub *dpsub = dev_get_drvdata(dev);
- return drm_mode_config_helper_suspend(&dpsub->drm);
+ if (!dpsub->drm)
+ return 0;
+
+ return drm_mode_config_helper_suspend(&dpsub->drm->dev);
}
static int __maybe_unused zynqmp_dpsub_resume(struct device *dev)
{
struct zynqmp_dpsub *dpsub = dev_get_drvdata(dev);
- return drm_mode_config_helper_resume(&dpsub->drm);
+ if (!dpsub->drm)
+ return 0;
+
+ return drm_mode_config_helper_resume(&dpsub->drm->dev);
}
static const struct dev_pm_ops zynqmp_dpsub_pm_ops = {
@@ -167,6 +57,36 @@ static const struct dev_pm_ops zynqmp_dpsub_pm_ops = {
};
/* -----------------------------------------------------------------------------
+ * DPSUB Configuration
+ */
+
+/**
+ * zynqmp_dpsub_audio_enabled - If the audio is enabled
+ * @dpsub: DisplayPort subsystem
+ *
+ * Return if the audio is enabled depending on the audio clock.
+ *
+ * Return: true if audio is enabled, or false.
+ */
+bool zynqmp_dpsub_audio_enabled(struct zynqmp_dpsub *dpsub)
+{
+ return !!dpsub->aud_clk;
+}
+
+/**
+ * zynqmp_dpsub_get_audio_clk_rate - Get the current audio clock rate
+ * @dpsub: DisplayPort subsystem
+ *
+ * Return: the current audio clock rate.
+ */
+unsigned int zynqmp_dpsub_get_audio_clk_rate(struct zynqmp_dpsub *dpsub)
+{
+ if (zynqmp_dpsub_audio_enabled(dpsub))
+ return 0;
+ return clk_get_rate(dpsub->aud_clk);
+}
+
+/* -----------------------------------------------------------------------------
* Probe & Remove
*/
@@ -184,19 +104,125 @@ static int zynqmp_dpsub_init_clocks(struct zynqmp_dpsub *dpsub)
return ret;
}
+ /*
+ * Try the live PL video clock, and fall back to the PS clock if the
+ * live PL video clock isn't valid.
+ */
+ dpsub->vid_clk = devm_clk_get(dpsub->dev, "dp_live_video_in_clk");
+ if (!IS_ERR(dpsub->vid_clk))
+ dpsub->vid_clk_from_ps = false;
+ else if (PTR_ERR(dpsub->vid_clk) == -EPROBE_DEFER)
+ return PTR_ERR(dpsub->vid_clk);
+
+ if (IS_ERR_OR_NULL(dpsub->vid_clk)) {
+ dpsub->vid_clk = devm_clk_get(dpsub->dev, "dp_vtc_pixel_clk_in");
+ if (IS_ERR(dpsub->vid_clk)) {
+ dev_err(dpsub->dev, "failed to init any video clock\n");
+ return PTR_ERR(dpsub->vid_clk);
+ }
+ dpsub->vid_clk_from_ps = true;
+ }
+
+ /*
+ * Try the live PL audio clock, and fall back to the PS clock if the
+ * live PL audio clock isn't valid. Missing audio clock disables audio
+ * but isn't an error.
+ */
+ dpsub->aud_clk = devm_clk_get(dpsub->dev, "dp_live_audio_aclk");
+ if (!IS_ERR(dpsub->aud_clk)) {
+ dpsub->aud_clk_from_ps = false;
+ return 0;
+ }
+
+ dpsub->aud_clk = devm_clk_get(dpsub->dev, "dp_aud_clk");
+ if (!IS_ERR(dpsub->aud_clk)) {
+ dpsub->aud_clk_from_ps = true;
+ return 0;
+ }
+
+ dev_info(dpsub->dev, "audio disabled due to missing clock\n");
return 0;
}
+static int zynqmp_dpsub_parse_dt(struct zynqmp_dpsub *dpsub)
+{
+ struct device_node *np;
+ unsigned int i;
+
+ /*
+ * For backward compatibility with old device trees that don't contain
+ * ports, consider that only the DP output port is connected if no
+ * ports child no exists.
+ */
+ np = of_get_child_by_name(dpsub->dev->of_node, "ports");
+ of_node_put(np);
+ if (!np) {
+ dev_warn(dpsub->dev, "missing ports, update DT bindings\n");
+ dpsub->connected_ports = BIT(ZYNQMP_DPSUB_PORT_OUT_DP);
+ dpsub->dma_enabled = true;
+ return 0;
+ }
+
+ /* Check which ports are connected. */
+ for (i = 0; i < ZYNQMP_DPSUB_NUM_PORTS; ++i) {
+ struct device_node *np;
+
+ np = of_graph_get_remote_node(dpsub->dev->of_node, i, -1);
+ if (np) {
+ dpsub->connected_ports |= BIT(i);
+ of_node_put(np);
+ }
+ }
+
+ /* Sanity checks. */
+ if ((dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_VIDEO)) &&
+ (dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_GFX))) {
+ dev_err(dpsub->dev, "only one live video input is supported\n");
+ return -EINVAL;
+ }
+
+ if ((dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_VIDEO)) ||
+ (dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_GFX))) {
+ if (dpsub->vid_clk_from_ps) {
+ dev_err(dpsub->dev,
+ "live video input requires PL clock\n");
+ return -EINVAL;
+ }
+ } else {
+ dpsub->dma_enabled = true;
+ }
+
+ if (dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_LIVE_AUDIO))
+ dev_warn(dpsub->dev, "live audio unsupported, ignoring\n");
+
+ if ((dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_OUT_VIDEO)) ||
+ (dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_OUT_AUDIO)))
+ dev_warn(dpsub->dev, "output to PL unsupported, ignoring\n");
+
+ if (!(dpsub->connected_ports & BIT(ZYNQMP_DPSUB_PORT_OUT_DP))) {
+ dev_err(dpsub->dev, "DP output port not connected\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void zynqmp_dpsub_release(struct zynqmp_dpsub *dpsub)
+{
+ kfree(dpsub->disp);
+ kfree(dpsub->dp);
+ kfree(dpsub);
+}
+
static int zynqmp_dpsub_probe(struct platform_device *pdev)
{
struct zynqmp_dpsub *dpsub;
int ret;
/* Allocate private data. */
- dpsub = devm_drm_dev_alloc(&pdev->dev, &zynqmp_dpsub_drm_driver,
- struct zynqmp_dpsub, drm);
- if (IS_ERR(dpsub))
- return PTR_ERR(dpsub);
+ dpsub = kzalloc(sizeof(*dpsub), GFP_KERNEL);
+ if (!dpsub)
+ return -ENOMEM;
dpsub->dev = &pdev->dev;
platform_set_drvdata(pdev, dpsub);
@@ -210,23 +236,31 @@ static int zynqmp_dpsub_probe(struct platform_device *pdev)
if (ret < 0)
goto err_mem;
+ ret = zynqmp_dpsub_parse_dt(dpsub);
+ if (ret < 0)
+ goto err_mem;
+
pm_runtime_enable(&pdev->dev);
/*
* DP should be probed first so that the zynqmp_disp can set the output
* format accordingly.
*/
- ret = zynqmp_dp_probe(dpsub, &dpsub->drm);
+ ret = zynqmp_dp_probe(dpsub);
if (ret)
goto err_pm;
- ret = zynqmp_disp_probe(dpsub, &dpsub->drm);
+ ret = zynqmp_disp_probe(dpsub);
if (ret)
goto err_dp;
- ret = zynqmp_dpsub_drm_init(dpsub);
- if (ret)
- goto err_disp;
+ if (dpsub->dma_enabled) {
+ ret = zynqmp_dpsub_drm_init(dpsub);
+ if (ret)
+ goto err_disp;
+ } else {
+ drm_bridge_add(dpsub->bridge);
+ }
dev_info(&pdev->dev, "ZynqMP DisplayPort Subsystem driver probed");
@@ -241,17 +275,19 @@ static int zynqmp_dpsub_probe(struct platform_device *pdev)
clk_disable_unprepare(dpsub->apb_clk);
err_mem:
of_reserved_mem_device_release(&pdev->dev);
+ if (!dpsub->drm)
+ zynqmp_dpsub_release(dpsub);
return ret;
}
static int zynqmp_dpsub_remove(struct platform_device *pdev)
{
struct zynqmp_dpsub *dpsub = platform_get_drvdata(pdev);
- struct drm_device *drm = &dpsub->drm;
- drm_dev_unregister(drm);
- drm_atomic_helper_shutdown(drm);
- drm_kms_helper_poll_fini(drm);
+ if (dpsub->drm)
+ zynqmp_dpsub_drm_cleanup(dpsub);
+ else
+ drm_bridge_remove(dpsub->bridge);
zynqmp_disp_remove(dpsub);
zynqmp_dp_remove(dpsub);
@@ -260,6 +296,9 @@ static int zynqmp_dpsub_remove(struct platform_device *pdev)
clk_disable_unprepare(dpsub->apb_clk);
of_reserved_mem_device_release(&pdev->dev);
+ if (!dpsub->drm)
+ zynqmp_dpsub_release(dpsub);
+
return 0;
}
@@ -267,7 +306,10 @@ static void zynqmp_dpsub_shutdown(struct platform_device *pdev)
{
struct zynqmp_dpsub *dpsub = platform_get_drvdata(pdev);
- drm_atomic_helper_shutdown(&dpsub->drm);
+ if (!dpsub->drm)
+ return;
+
+ drm_atomic_helper_shutdown(&dpsub->drm->dev);
}
static const struct of_device_id zynqmp_dpsub_of_match[] = {
diff --git a/drivers/gpu/drm/xlnx/zynqmp_dpsub.h b/drivers/gpu/drm/xlnx/zynqmp_dpsub.h
index c04026d..09ea018 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dpsub.h
+++ b/drivers/gpu/drm/xlnx/zynqmp_dpsub.h
@@ -14,9 +14,23 @@
struct clk;
struct device;
-struct drm_device;
+struct drm_bridge;
struct zynqmp_disp;
+struct zynqmp_disp_layer;
struct zynqmp_dp;
+struct zynqmp_dpsub_drm;
+
+#define ZYNQMP_DPSUB_NUM_LAYERS 2
+
+enum zynqmp_dpsub_port {
+ ZYNQMP_DPSUB_PORT_LIVE_VIDEO,
+ ZYNQMP_DPSUB_PORT_LIVE_GFX,
+ ZYNQMP_DPSUB_PORT_LIVE_AUDIO,
+ ZYNQMP_DPSUB_PORT_OUT_VIDEO,
+ ZYNQMP_DPSUB_PORT_OUT_AUDIO,
+ ZYNQMP_DPSUB_PORT_OUT_DP,
+ ZYNQMP_DPSUB_NUM_PORTS,
+};
enum zynqmp_dpsub_format {
ZYNQMP_DPSUB_FORMAT_RGB,
@@ -27,28 +41,46 @@ enum zynqmp_dpsub_format {
/**
* struct zynqmp_dpsub - ZynqMP DisplayPort Subsystem
- * @drm: The DRM/KMS device
* @dev: The physical device
* @apb_clk: The APB clock
+ * @vid_clk: Video clock
+ * @vid_clk_from_ps: True of the video clock comes from PS, false from PL
+ * @aud_clk: Audio clock
+ * @aud_clk_from_ps: True of the audio clock comes from PS, false from PL
+ * @connected_ports: Bitmask of connected ports in the device tree
+ * @dma_enabled: True if the DMA interface is enabled, false if the DPSUB is
+ * driven by the live input
+ * @drm: The DRM/KMS device data
+ * @bridge: The DP encoder bridge
* @disp: The display controller
* @dp: The DisplayPort controller
* @dma_align: DMA alignment constraint (must be a power of 2)
*/
struct zynqmp_dpsub {
- struct drm_device drm;
struct device *dev;
struct clk *apb_clk;
+ struct clk *vid_clk;
+ bool vid_clk_from_ps;
+ struct clk *aud_clk;
+ bool aud_clk_from_ps;
+
+ unsigned int connected_ports;
+ bool dma_enabled;
+
+ struct zynqmp_dpsub_drm *drm;
+ struct drm_bridge *bridge;
struct zynqmp_disp *disp;
+ struct zynqmp_disp_layer *layers[ZYNQMP_DPSUB_NUM_LAYERS];
struct zynqmp_dp *dp;
unsigned int dma_align;
};
-static inline struct zynqmp_dpsub *to_zynqmp_dpsub(struct drm_device *drm)
-{
- return container_of(drm, struct zynqmp_dpsub, drm);
-}
+bool zynqmp_dpsub_audio_enabled(struct zynqmp_dpsub *dpsub);
+unsigned int zynqmp_dpsub_get_audio_clk_rate(struct zynqmp_dpsub *dpsub);
+
+void zynqmp_dpsub_release(struct zynqmp_dpsub *dpsub);
#endif /* _ZYNQMP_DPSUB_H_ */
diff --git a/drivers/gpu/drm/xlnx/zynqmp_kms.c b/drivers/gpu/drm/xlnx/zynqmp_kms.c
new file mode 100644
index 0000000..776ef54
--- /dev/null
+++ b/drivers/gpu/drm/xlnx/zynqmp_kms.c
@@ -0,0 +1,534 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ZynqMP DisplayPort Subsystem - KMS API
+ *
+ * Copyright (C) 2017 - 2021 Xilinx, Inc.
+ *
+ * Authors:
+ * - Hyun Woo Kwon <hyun.kwon@xilinx.com>
+ * - Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_blend.h>
+#include <drm/drm_bridge.h>
+#include <drm/drm_bridge_connector.h>
+#include <drm/drm_connector.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_device.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_encoder.h>
+#include <drm/drm_fbdev_generic.h>
+#include <drm/drm_fourcc.h>
+#include <drm/drm_framebuffer.h>
+#include <drm/drm_gem_dma_helper.h>
+#include <drm/drm_gem_framebuffer_helper.h>
+#include <drm/drm_managed.h>
+#include <drm/drm_mode_config.h>
+#include <drm/drm_plane.h>
+#include <drm/drm_plane_helper.h>
+#include <drm/drm_probe_helper.h>
+#include <drm/drm_simple_kms_helper.h>
+#include <drm/drm_vblank.h>
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/pm_runtime.h>
+#include <linux/spinlock.h>
+
+#include "zynqmp_disp.h"
+#include "zynqmp_dp.h"
+#include "zynqmp_dpsub.h"
+#include "zynqmp_kms.h"
+
+static inline struct zynqmp_dpsub *to_zynqmp_dpsub(struct drm_device *drm)
+{
+ return container_of(drm, struct zynqmp_dpsub_drm, dev)->dpsub;
+}
+
+/* -----------------------------------------------------------------------------
+ * DRM Planes
+ */
+
+static int zynqmp_dpsub_plane_atomic_check(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
+ plane);
+ struct drm_crtc_state *crtc_state;
+
+ if (!new_plane_state->crtc)
+ return 0;
+
+ crtc_state = drm_atomic_get_crtc_state(state, new_plane_state->crtc);
+ if (IS_ERR(crtc_state))
+ return PTR_ERR(crtc_state);
+
+ return drm_atomic_helper_check_plane_state(new_plane_state,
+ crtc_state,
+ DRM_PLANE_NO_SCALING,
+ DRM_PLANE_NO_SCALING,
+ false, false);
+}
+
+static void zynqmp_dpsub_plane_atomic_disable(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state,
+ plane);
+ struct zynqmp_dpsub *dpsub = to_zynqmp_dpsub(plane->dev);
+ struct zynqmp_disp_layer *layer = dpsub->layers[plane->index];
+
+ if (!old_state->fb)
+ return;
+
+ zynqmp_disp_layer_disable(layer);
+
+ if (plane->index == ZYNQMP_DPSUB_LAYER_GFX)
+ zynqmp_disp_blend_set_global_alpha(dpsub->disp, false,
+ plane->state->alpha >> 8);
+}
+
+static void zynqmp_dpsub_plane_atomic_update(struct drm_plane *plane,
+ struct drm_atomic_state *state)
+{
+ struct drm_plane_state *old_state = drm_atomic_get_old_plane_state(state, plane);
+ struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state, plane);
+ struct zynqmp_dpsub *dpsub = to_zynqmp_dpsub(plane->dev);
+ struct zynqmp_disp_layer *layer = dpsub->layers[plane->index];
+ bool format_changed = false;
+
+ if (!old_state->fb ||
+ old_state->fb->format->format != new_state->fb->format->format)
+ format_changed = true;
+
+ /*
+ * If the format has changed (including going from a previously
+ * disabled state to any format), reconfigure the format. Disable the
+ * plane first if needed.
+ */
+ if (format_changed) {
+ if (old_state->fb)
+ zynqmp_disp_layer_disable(layer);
+
+ zynqmp_disp_layer_set_format(layer, new_state->fb->format);
+ }
+
+ zynqmp_disp_layer_update(layer, new_state);
+
+ if (plane->index == ZYNQMP_DPSUB_LAYER_GFX)
+ zynqmp_disp_blend_set_global_alpha(dpsub->disp, true,
+ plane->state->alpha >> 8);
+
+ /* Enable or re-enable the plane if the format has changed. */
+ if (format_changed)
+ zynqmp_disp_layer_enable(layer, ZYNQMP_DPSUB_LAYER_NONLIVE);
+}
+
+static const struct drm_plane_helper_funcs zynqmp_dpsub_plane_helper_funcs = {
+ .atomic_check = zynqmp_dpsub_plane_atomic_check,
+ .atomic_update = zynqmp_dpsub_plane_atomic_update,
+ .atomic_disable = zynqmp_dpsub_plane_atomic_disable,
+};
+
+static const struct drm_plane_funcs zynqmp_dpsub_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = drm_plane_cleanup,
+ .reset = drm_atomic_helper_plane_reset,
+ .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+};
+
+static int zynqmp_dpsub_create_planes(struct zynqmp_dpsub *dpsub)
+{
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < ARRAY_SIZE(dpsub->drm->planes); i++) {
+ struct zynqmp_disp_layer *layer = dpsub->layers[i];
+ struct drm_plane *plane = &dpsub->drm->planes[i];
+ enum drm_plane_type type;
+ unsigned int num_formats;
+ u32 *formats;
+
+ formats = zynqmp_disp_layer_drm_formats(layer, &num_formats);
+ if (!formats)
+ return -ENOMEM;
+
+ /* Graphics layer is primary, and video layer is overlay. */
+ type = i == ZYNQMP_DPSUB_LAYER_VID
+ ? DRM_PLANE_TYPE_OVERLAY : DRM_PLANE_TYPE_PRIMARY;
+ ret = drm_universal_plane_init(&dpsub->drm->dev, plane, 0,
+ &zynqmp_dpsub_plane_funcs,
+ formats, num_formats,
+ NULL, type, NULL);
+ kfree(formats);
+ if (ret)
+ return ret;
+
+ drm_plane_helper_add(plane, &zynqmp_dpsub_plane_helper_funcs);
+
+ drm_plane_create_zpos_immutable_property(plane, i);
+ if (i == ZYNQMP_DPSUB_LAYER_GFX)
+ drm_plane_create_alpha_property(plane);
+ }
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * DRM CRTC
+ */
+
+static inline struct zynqmp_dpsub *crtc_to_dpsub(struct drm_crtc *crtc)
+{
+ return container_of(crtc, struct zynqmp_dpsub_drm, crtc)->dpsub;
+}
+
+static void zynqmp_dpsub_crtc_atomic_enable(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ struct zynqmp_dpsub *dpsub = crtc_to_dpsub(crtc);
+ struct drm_display_mode *adjusted_mode = &crtc->state->adjusted_mode;
+ int ret, vrefresh;
+
+ pm_runtime_get_sync(dpsub->dev);
+
+ zynqmp_disp_setup_clock(dpsub->disp, adjusted_mode->clock * 1000);
+
+ ret = clk_prepare_enable(dpsub->vid_clk);
+ if (ret) {
+ dev_err(dpsub->dev, "failed to enable a pixel clock\n");
+ pm_runtime_put_sync(dpsub->dev);
+ return;
+ }
+
+ zynqmp_disp_enable(dpsub->disp);
+
+ /* Delay of 3 vblank intervals for timing gen to be stable */
+ vrefresh = (adjusted_mode->clock * 1000) /
+ (adjusted_mode->vtotal * adjusted_mode->htotal);
+ msleep(3 * 1000 / vrefresh);
+}
+
+static void zynqmp_dpsub_crtc_atomic_disable(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ struct zynqmp_dpsub *dpsub = crtc_to_dpsub(crtc);
+ struct drm_plane_state *old_plane_state;
+
+ /*
+ * Disable the plane if active. The old plane state can be NULL in the
+ * .shutdown() path if the plane is already disabled, skip
+ * zynqmp_disp_plane_atomic_disable() in that case.
+ */
+ old_plane_state = drm_atomic_get_old_plane_state(state, crtc->primary);
+ if (old_plane_state)
+ zynqmp_dpsub_plane_atomic_disable(crtc->primary, state);
+
+ zynqmp_disp_disable(dpsub->disp);
+
+ drm_crtc_vblank_off(crtc);
+
+ spin_lock_irq(&crtc->dev->event_lock);
+ if (crtc->state->event) {
+ drm_crtc_send_vblank_event(crtc, crtc->state->event);
+ crtc->state->event = NULL;
+ }
+ spin_unlock_irq(&crtc->dev->event_lock);
+
+ clk_disable_unprepare(dpsub->vid_clk);
+ pm_runtime_put_sync(dpsub->dev);
+}
+
+static int zynqmp_dpsub_crtc_atomic_check(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ return drm_atomic_add_affected_planes(state, crtc);
+}
+
+static void zynqmp_dpsub_crtc_atomic_begin(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ drm_crtc_vblank_on(crtc);
+}
+
+static void zynqmp_dpsub_crtc_atomic_flush(struct drm_crtc *crtc,
+ struct drm_atomic_state *state)
+{
+ if (crtc->state->event) {
+ struct drm_pending_vblank_event *event;
+
+ /* Consume the flip_done event from atomic helper. */
+ event = crtc->state->event;
+ crtc->state->event = NULL;
+
+ event->pipe = drm_crtc_index(crtc);
+
+ WARN_ON(drm_crtc_vblank_get(crtc) != 0);
+
+ spin_lock_irq(&crtc->dev->event_lock);
+ drm_crtc_arm_vblank_event(crtc, event);
+ spin_unlock_irq(&crtc->dev->event_lock);
+ }
+}
+
+static const struct drm_crtc_helper_funcs zynqmp_dpsub_crtc_helper_funcs = {
+ .atomic_enable = zynqmp_dpsub_crtc_atomic_enable,
+ .atomic_disable = zynqmp_dpsub_crtc_atomic_disable,
+ .atomic_check = zynqmp_dpsub_crtc_atomic_check,
+ .atomic_begin = zynqmp_dpsub_crtc_atomic_begin,
+ .atomic_flush = zynqmp_dpsub_crtc_atomic_flush,
+};
+
+static int zynqmp_dpsub_crtc_enable_vblank(struct drm_crtc *crtc)
+{
+ struct zynqmp_dpsub *dpsub = crtc_to_dpsub(crtc);
+
+ zynqmp_dp_enable_vblank(dpsub->dp);
+
+ return 0;
+}
+
+static void zynqmp_dpsub_crtc_disable_vblank(struct drm_crtc *crtc)
+{
+ struct zynqmp_dpsub *dpsub = crtc_to_dpsub(crtc);
+
+ zynqmp_dp_disable_vblank(dpsub->dp);
+}
+
+static const struct drm_crtc_funcs zynqmp_dpsub_crtc_funcs = {
+ .destroy = drm_crtc_cleanup,
+ .set_config = drm_atomic_helper_set_config,
+ .page_flip = drm_atomic_helper_page_flip,
+ .reset = drm_atomic_helper_crtc_reset,
+ .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .enable_vblank = zynqmp_dpsub_crtc_enable_vblank,
+ .disable_vblank = zynqmp_dpsub_crtc_disable_vblank,
+};
+
+static int zynqmp_dpsub_create_crtc(struct zynqmp_dpsub *dpsub)
+{
+ struct drm_plane *plane = &dpsub->drm->planes[ZYNQMP_DPSUB_LAYER_GFX];
+ struct drm_crtc *crtc = &dpsub->drm->crtc;
+ int ret;
+
+ ret = drm_crtc_init_with_planes(&dpsub->drm->dev, crtc, plane,
+ NULL, &zynqmp_dpsub_crtc_funcs, NULL);
+ if (ret < 0)
+ return ret;
+
+ drm_crtc_helper_add(crtc, &zynqmp_dpsub_crtc_helper_funcs);
+
+ /* Start with vertical blanking interrupt reporting disabled. */
+ drm_crtc_vblank_off(crtc);
+
+ return 0;
+}
+
+static void zynqmp_dpsub_map_crtc_to_plane(struct zynqmp_dpsub *dpsub)
+{
+ u32 possible_crtcs = drm_crtc_mask(&dpsub->drm->crtc);
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(dpsub->drm->planes); i++)
+ dpsub->drm->planes[i].possible_crtcs = possible_crtcs;
+}
+
+/**
+ * zynqmp_dpsub_drm_handle_vblank - Handle the vblank event
+ * @dpsub: DisplayPort subsystem
+ *
+ * This function handles the vblank interrupt, and sends an event to
+ * CRTC object. This will be called by the DP vblank interrupt handler.
+ */
+void zynqmp_dpsub_drm_handle_vblank(struct zynqmp_dpsub *dpsub)
+{
+ drm_crtc_handle_vblank(&dpsub->drm->crtc);
+}
+
+/* -----------------------------------------------------------------------------
+ * Dumb Buffer & Framebuffer Allocation
+ */
+
+static int zynqmp_dpsub_dumb_create(struct drm_file *file_priv,
+ struct drm_device *drm,
+ struct drm_mode_create_dumb *args)
+{
+ struct zynqmp_dpsub *dpsub = to_zynqmp_dpsub(drm);
+ unsigned int pitch = DIV_ROUND_UP(args->width * args->bpp, 8);
+
+ /* Enforce the alignment constraints of the DMA engine. */
+ args->pitch = ALIGN(pitch, dpsub->dma_align);
+
+ return drm_gem_dma_dumb_create_internal(file_priv, drm, args);
+}
+
+static struct drm_framebuffer *
+zynqmp_dpsub_fb_create(struct drm_device *drm, struct drm_file *file_priv,
+ const struct drm_mode_fb_cmd2 *mode_cmd)
+{
+ struct zynqmp_dpsub *dpsub = to_zynqmp_dpsub(drm);
+ struct drm_mode_fb_cmd2 cmd = *mode_cmd;
+ unsigned int i;
+
+ /* Enforce the alignment constraints of the DMA engine. */
+ for (i = 0; i < ARRAY_SIZE(cmd.pitches); ++i)
+ cmd.pitches[i] = ALIGN(cmd.pitches[i], dpsub->dma_align);
+
+ return drm_gem_fb_create(drm, file_priv, &cmd);
+}
+
+static const struct drm_mode_config_funcs zynqmp_dpsub_mode_config_funcs = {
+ .fb_create = zynqmp_dpsub_fb_create,
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = drm_atomic_helper_commit,
+};
+
+/* -----------------------------------------------------------------------------
+ * DRM/KMS Driver
+ */
+
+DEFINE_DRM_GEM_DMA_FOPS(zynqmp_dpsub_drm_fops);
+
+static const struct drm_driver zynqmp_dpsub_drm_driver = {
+ .driver_features = DRIVER_MODESET | DRIVER_GEM |
+ DRIVER_ATOMIC,
+
+ DRM_GEM_DMA_DRIVER_OPS_WITH_DUMB_CREATE(zynqmp_dpsub_dumb_create),
+
+ .fops = &zynqmp_dpsub_drm_fops,
+
+ .name = "zynqmp-dpsub",
+ .desc = "Xilinx DisplayPort Subsystem Driver",
+ .date = "20130509",
+ .major = 1,
+ .minor = 0,
+};
+
+static int zynqmp_dpsub_kms_init(struct zynqmp_dpsub *dpsub)
+{
+ struct drm_encoder *encoder = &dpsub->drm->encoder;
+ struct drm_connector *connector;
+ int ret;
+
+ /* Create the planes and the CRTC. */
+ ret = zynqmp_dpsub_create_planes(dpsub);
+ if (ret)
+ return ret;
+
+ ret = zynqmp_dpsub_create_crtc(dpsub);
+ if (ret < 0)
+ return ret;
+
+ zynqmp_dpsub_map_crtc_to_plane(dpsub);
+
+ /* Create the encoder and attach the bridge. */
+ encoder->possible_crtcs |= drm_crtc_mask(&dpsub->drm->crtc);
+ drm_simple_encoder_init(&dpsub->drm->dev, encoder, DRM_MODE_ENCODER_NONE);
+
+ ret = drm_bridge_attach(encoder, dpsub->bridge, NULL,
+ DRM_BRIDGE_ATTACH_NO_CONNECTOR);
+ if (ret) {
+ dev_err(dpsub->dev, "failed to attach bridge to encoder\n");
+ return ret;
+ }
+
+ /* Create the connector for the chain of bridges. */
+ connector = drm_bridge_connector_init(&dpsub->drm->dev, encoder);
+ if (IS_ERR(connector)) {
+ dev_err(dpsub->dev, "failed to created connector\n");
+ return PTR_ERR(connector);
+ }
+
+ ret = drm_connector_attach_encoder(connector, encoder);
+ if (ret < 0) {
+ dev_err(dpsub->dev, "failed to attach connector to encoder\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void zynqmp_dpsub_drm_release(struct drm_device *drm, void *res)
+{
+ struct zynqmp_dpsub_drm *dpdrm = res;
+
+ zynqmp_dpsub_release(dpdrm->dpsub);
+}
+
+int zynqmp_dpsub_drm_init(struct zynqmp_dpsub *dpsub)
+{
+ struct zynqmp_dpsub_drm *dpdrm;
+ struct drm_device *drm;
+ int ret;
+
+ /*
+ * Allocate the drm_device and immediately add a cleanup action to
+ * release the zynqmp_dpsub instance. If any of those operations fail,
+ * dpsub->drm will remain NULL, which tells the caller that it must
+ * cleanup manually.
+ */
+ dpdrm = devm_drm_dev_alloc(dpsub->dev, &zynqmp_dpsub_drm_driver,
+ struct zynqmp_dpsub_drm, dev);
+ if (IS_ERR(dpdrm))
+ return PTR_ERR(dpdrm);
+
+ dpdrm->dpsub = dpsub;
+ drm = &dpdrm->dev;
+
+ ret = drmm_add_action(drm, zynqmp_dpsub_drm_release, dpdrm);
+ if (ret < 0)
+ return ret;
+
+ dpsub->drm = dpdrm;
+
+ /* Initialize mode config, vblank and the KMS poll helper. */
+ ret = drmm_mode_config_init(drm);
+ if (ret < 0)
+ return ret;
+
+ drm->mode_config.funcs = &zynqmp_dpsub_mode_config_funcs;
+ drm->mode_config.min_width = 0;
+ drm->mode_config.min_height = 0;
+ drm->mode_config.max_width = ZYNQMP_DISP_MAX_WIDTH;
+ drm->mode_config.max_height = ZYNQMP_DISP_MAX_HEIGHT;
+
+ ret = drm_vblank_init(drm, 1);
+ if (ret)
+ return ret;
+
+ drm_kms_helper_poll_init(drm);
+
+ ret = zynqmp_dpsub_kms_init(dpsub);
+ if (ret < 0)
+ goto err_poll_fini;
+
+ /* Reset all components and register the DRM device. */
+ drm_mode_config_reset(drm);
+
+ ret = drm_dev_register(drm, 0);
+ if (ret < 0)
+ goto err_poll_fini;
+
+ /* Initialize fbdev generic emulation. */
+ drm_fbdev_generic_setup(drm, 24);
+
+ return 0;
+
+err_poll_fini:
+ drm_kms_helper_poll_fini(drm);
+ return ret;
+}
+
+void zynqmp_dpsub_drm_cleanup(struct zynqmp_dpsub *dpsub)
+{
+ struct drm_device *drm = &dpsub->drm->dev;
+
+ drm_dev_unregister(drm);
+ drm_atomic_helper_shutdown(drm);
+ drm_kms_helper_poll_fini(drm);
+}
diff --git a/drivers/gpu/drm/xlnx/zynqmp_kms.h b/drivers/gpu/drm/xlnx/zynqmp_kms.h
new file mode 100644
index 0000000..01be96b
--- /dev/null
+++ b/drivers/gpu/drm/xlnx/zynqmp_kms.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ZynqMP DisplayPort Subsystem - KMS API
+ *
+ * Copyright (C) 2017 - 2021 Xilinx, Inc.
+ *
+ * Authors:
+ * - Hyun Woo Kwon <hyun.kwon@xilinx.com>
+ * - Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+#ifndef _ZYNQMP_KMS_H_
+#define _ZYNQMP_KMS_H_
+
+#include <drm/drm_crtc.h>
+#include <drm/drm_device.h>
+#include <drm/drm_encoder.h>
+#include <drm/drm_plane.h>
+
+#include "zynqmp_dpsub.h"
+
+struct zynqmp_dpsub;
+
+/**
+ * struct zynqmp_dpsub - ZynqMP DisplayPort Subsystem DRM/KMS data
+ * @dpsub: Backpointer to the DisplayPort subsystem
+ * @drm: The DRM/KMS device
+ * @planes: The DRM planes
+ * @crtc: The DRM CRTC
+ * @encoder: The dummy DRM encoder
+ */
+struct zynqmp_dpsub_drm {
+ struct zynqmp_dpsub *dpsub;
+
+ struct drm_device dev;
+ struct drm_plane planes[ZYNQMP_DPSUB_NUM_LAYERS];
+ struct drm_crtc crtc;
+ struct drm_encoder encoder;
+};
+
+void zynqmp_dpsub_drm_handle_vblank(struct zynqmp_dpsub *dpsub);
+
+int zynqmp_dpsub_drm_init(struct zynqmp_dpsub *dpsub);
+void zynqmp_dpsub_drm_cleanup(struct zynqmp_dpsub *dpsub);
+
+#endif /* _ZYNQMP_KMS_H_ */
diff --git a/drivers/gpu/host1x/context.c b/drivers/gpu/host1x/context.c
index b08cf11..c8e7994 100644
--- a/drivers/gpu/host1x/context.c
+++ b/drivers/gpu/host1x/context.c
@@ -104,6 +104,7 @@ void host1x_memory_context_list_free(struct host1x_memory_context_list *cdl)
}
struct host1x_memory_context *host1x_memory_context_alloc(struct host1x *host1x,
+ struct device *dev,
struct pid *pid)
{
struct host1x_memory_context_list *cdl = &host1x->context_list;
@@ -118,6 +119,9 @@ struct host1x_memory_context *host1x_memory_context_alloc(struct host1x *host1x,
for (i = 0; i < cdl->len; i++) {
struct host1x_memory_context *cd = &cdl->devs[i];
+ if (cd->dev.iommu->iommu_dev != dev->iommu->iommu_dev)
+ continue;
+
if (cd->owner == pid) {
refcount_inc(&cd->ref);
mutex_unlock(&cdl->lock);
diff --git a/drivers/gpu/host1x/debug.c b/drivers/gpu/host1x/debug.c
index 34c2e36..6649b04 100644
--- a/drivers/gpu/host1x/debug.c
+++ b/drivers/gpu/host1x/debug.c
@@ -140,7 +140,7 @@ static void show_all(struct host1x *m, struct output *o, bool show_fifo)
}
}
-static int host1x_debug_show_all(struct seq_file *s, void *unused)
+static int host1x_debug_all_show(struct seq_file *s, void *unused)
{
struct output o = {
.fn = write_to_seqfile,
@@ -151,6 +151,7 @@ static int host1x_debug_show_all(struct seq_file *s, void *unused)
return 0;
}
+DEFINE_SHOW_ATTRIBUTE(host1x_debug_all);
static int host1x_debug_show(struct seq_file *s, void *unused)
{
@@ -163,30 +164,7 @@ static int host1x_debug_show(struct seq_file *s, void *unused)
return 0;
}
-
-static int host1x_debug_open_all(struct inode *inode, struct file *file)
-{
- return single_open(file, host1x_debug_show_all, inode->i_private);
-}
-
-static const struct file_operations host1x_debug_all_fops = {
- .open = host1x_debug_open_all,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
-
-static int host1x_debug_open(struct inode *inode, struct file *file)
-{
- return single_open(file, host1x_debug_show, inode->i_private);
-}
-
-static const struct file_operations host1x_debug_fops = {
- .open = host1x_debug_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
+DEFINE_SHOW_ATTRIBUTE(host1x_debug);
static void host1x_debugfs_init(struct host1x *host1x)
{
diff --git a/drivers/gpu/host1x/dev.c b/drivers/gpu/host1x/dev.c
index f60ea24..f31039a 100644
--- a/drivers/gpu/host1x/dev.c
+++ b/drivers/gpu/host1x/dev.c
@@ -225,6 +225,18 @@ static const struct host1x_sid_entry tegra234_sid_table[] = {
.offset = 0x34,
.limit = 0x34
},
+ {
+ /* NVDEC channel */
+ .base = 0x17c8,
+ .offset = 0x30,
+ .limit = 0x30,
+ },
+ {
+ /* NVDEC MMIO */
+ .base = 0x1698,
+ .offset = 0x34,
+ .limit = 0x34,
+ },
};
static const struct host1x_info host1x08_info = {
diff --git a/drivers/gpu/host1x/fence.c b/drivers/gpu/host1x/fence.c
index ecab7288..df428bc 100644
--- a/drivers/gpu/host1x/fence.c
+++ b/drivers/gpu/host1x/fence.c
@@ -99,7 +99,7 @@ static void host1x_syncpt_fence_release(struct dma_fence *f)
dma_fence_free(f);
}
-const struct dma_fence_ops host1x_syncpt_fence_ops = {
+static const struct dma_fence_ops host1x_syncpt_fence_ops = {
.get_driver_name = host1x_syncpt_fence_get_driver_name,
.get_timeline_name = host1x_syncpt_fence_get_timeline_name,
.enable_signaling = host1x_syncpt_fence_enable_signaling,
diff --git a/drivers/infiniband/core/umem_dmabuf.c b/drivers/infiniband/core/umem_dmabuf.c
index 04c04e6..43b26bc 100644
--- a/drivers/infiniband/core/umem_dmabuf.c
+++ b/drivers/infiniband/core/umem_dmabuf.c
@@ -26,7 +26,8 @@ int ib_umem_dmabuf_map_pages(struct ib_umem_dmabuf *umem_dmabuf)
if (umem_dmabuf->sgt)
goto wait_fence;
- sgt = dma_buf_map_attachment(umem_dmabuf->attach, DMA_BIDIRECTIONAL);
+ sgt = dma_buf_map_attachment_unlocked(umem_dmabuf->attach,
+ DMA_BIDIRECTIONAL);
if (IS_ERR(sgt))
return PTR_ERR(sgt);
@@ -102,8 +103,8 @@ void ib_umem_dmabuf_unmap_pages(struct ib_umem_dmabuf *umem_dmabuf)
umem_dmabuf->last_sg_trim = 0;
}
- dma_buf_unmap_attachment(umem_dmabuf->attach, umem_dmabuf->sgt,
- DMA_BIDIRECTIONAL);
+ dma_buf_unmap_attachment_unlocked(umem_dmabuf->attach, umem_dmabuf->sgt,
+ DMA_BIDIRECTIONAL);
umem_dmabuf->sgt = NULL;
}
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
index 8e55468..5f1175f 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
@@ -11,6 +11,7 @@
*/
#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
#include <linux/module.h>
#include <linux/refcount.h>
#include <linux/scatterlist.h>
@@ -101,7 +102,7 @@ static void *vb2_dc_vaddr(struct vb2_buffer *vb, void *buf_priv)
if (buf->db_attach) {
struct iosys_map map;
- if (!dma_buf_vmap(buf->db_attach->dmabuf, &map))
+ if (!dma_buf_vmap_unlocked(buf->db_attach->dmabuf, &map))
buf->vaddr = map.vaddr;
return buf->vaddr;
@@ -382,18 +383,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
{
struct vb2_dc_attachment *attach = db_attach->priv;
- /* stealing dmabuf mutex to serialize map/unmap operations */
- struct mutex *lock = &db_attach->dmabuf->lock;
struct sg_table *sgt;
- mutex_lock(lock);
-
sgt = &attach->sgt;
/* return previously mapped sg table */
- if (attach->dma_dir == dma_dir) {
- mutex_unlock(lock);
+ if (attach->dma_dir == dma_dir)
return sgt;
- }
/* release any previous cache */
if (attach->dma_dir != DMA_NONE) {
@@ -409,14 +404,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
DMA_ATTR_SKIP_CPU_SYNC)) {
pr_err("failed to map scatterlist\n");
- mutex_unlock(lock);
return ERR_PTR(-EIO);
}
attach->dma_dir = dma_dir;
- mutex_unlock(lock);
-
return sgt;
}
@@ -464,6 +456,8 @@ static int vb2_dc_dmabuf_ops_vmap(struct dma_buf *dbuf, struct iosys_map *map)
static int vb2_dc_dmabuf_ops_mmap(struct dma_buf *dbuf,
struct vm_area_struct *vma)
{
+ dma_resv_assert_held(dbuf->resv);
+
return vb2_dc_mmap(dbuf->priv, vma);
}
@@ -712,7 +706,7 @@ static int vb2_dc_map_dmabuf(void *mem_priv)
}
/* get the associated scatterlist for this buffer */
- sgt = dma_buf_map_attachment(buf->db_attach, buf->dma_dir);
+ sgt = dma_buf_map_attachment_unlocked(buf->db_attach, buf->dma_dir);
if (IS_ERR(sgt)) {
pr_err("Error getting dmabuf scatterlist\n");
return -EINVAL;
@@ -723,7 +717,8 @@ static int vb2_dc_map_dmabuf(void *mem_priv)
if (contig_size < buf->size) {
pr_err("contiguous chunk is too small %lu/%lu\n",
contig_size, buf->size);
- dma_buf_unmap_attachment(buf->db_attach, sgt, buf->dma_dir);
+ dma_buf_unmap_attachment_unlocked(buf->db_attach, sgt,
+ buf->dma_dir);
return -EFAULT;
}
@@ -751,10 +746,10 @@ static void vb2_dc_unmap_dmabuf(void *mem_priv)
}
if (buf->vaddr) {
- dma_buf_vunmap(buf->db_attach->dmabuf, &map);
+ dma_buf_vunmap_unlocked(buf->db_attach->dmabuf, &map);
buf->vaddr = NULL;
}
- dma_buf_unmap_attachment(buf->db_attach, sgt, buf->dma_dir);
+ dma_buf_unmap_attachment_unlocked(buf->db_attach, sgt, buf->dma_dir);
buf->dma_addr = 0;
buf->dma_sgt = NULL;
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
index 099693e..183037f 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
@@ -10,6 +10,7 @@
* the Free Software Foundation.
*/
+#include <linux/dma-resv.h>
#include <linux/module.h>
#include <linux/mm.h>
#include <linux/refcount.h>
@@ -311,7 +312,7 @@ static void *vb2_dma_sg_vaddr(struct vb2_buffer *vb, void *buf_priv)
if (!buf->vaddr) {
if (buf->db_attach) {
- ret = dma_buf_vmap(buf->db_attach->dmabuf, &map);
+ ret = dma_buf_vmap_unlocked(buf->db_attach->dmabuf, &map);
buf->vaddr = ret ? NULL : map.vaddr;
} else {
buf->vaddr = vm_map_ram(buf->pages, buf->num_pages, -1);
@@ -426,18 +427,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
{
struct vb2_dma_sg_attachment *attach = db_attach->priv;
- /* stealing dmabuf mutex to serialize map/unmap operations */
- struct mutex *lock = &db_attach->dmabuf->lock;
struct sg_table *sgt;
- mutex_lock(lock);
-
sgt = &attach->sgt;
/* return previously mapped sg table */
- if (attach->dma_dir == dma_dir) {
- mutex_unlock(lock);
+ if (attach->dma_dir == dma_dir)
return sgt;
- }
/* release any previous cache */
if (attach->dma_dir != DMA_NONE) {
@@ -448,14 +443,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
/* mapping to the client with new direction */
if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
pr_err("failed to map scatterlist\n");
- mutex_unlock(lock);
return ERR_PTR(-EIO);
}
attach->dma_dir = dma_dir;
- mutex_unlock(lock);
-
return sgt;
}
@@ -506,6 +498,8 @@ static int vb2_dma_sg_dmabuf_ops_vmap(struct dma_buf *dbuf,
static int vb2_dma_sg_dmabuf_ops_mmap(struct dma_buf *dbuf,
struct vm_area_struct *vma)
{
+ dma_resv_assert_held(dbuf->resv);
+
return vb2_dma_sg_mmap(dbuf->priv, vma);
}
@@ -567,7 +561,7 @@ static int vb2_dma_sg_map_dmabuf(void *mem_priv)
}
/* get the associated scatterlist for this buffer */
- sgt = dma_buf_map_attachment(buf->db_attach, buf->dma_dir);
+ sgt = dma_buf_map_attachment_unlocked(buf->db_attach, buf->dma_dir);
if (IS_ERR(sgt)) {
pr_err("Error getting dmabuf scatterlist\n");
return -EINVAL;
@@ -596,10 +590,10 @@ static void vb2_dma_sg_unmap_dmabuf(void *mem_priv)
}
if (buf->vaddr) {
- dma_buf_vunmap(buf->db_attach->dmabuf, &map);
+ dma_buf_vunmap_unlocked(buf->db_attach->dmabuf, &map);
buf->vaddr = NULL;
}
- dma_buf_unmap_attachment(buf->db_attach, sgt, buf->dma_dir);
+ dma_buf_unmap_attachment_unlocked(buf->db_attach, sgt, buf->dma_dir);
buf->dma_sgt = NULL;
}
diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
index 67d0b89..959b45b 100644
--- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
+++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
@@ -10,6 +10,7 @@
* the Free Software Foundation.
*/
+#include <linux/dma-resv.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/mm.h>
@@ -269,18 +270,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
{
struct vb2_vmalloc_attachment *attach = db_attach->priv;
- /* stealing dmabuf mutex to serialize map/unmap operations */
- struct mutex *lock = &db_attach->dmabuf->lock;
struct sg_table *sgt;
- mutex_lock(lock);
-
sgt = &attach->sgt;
/* return previously mapped sg table */
- if (attach->dma_dir == dma_dir) {
- mutex_unlock(lock);
+ if (attach->dma_dir == dma_dir)
return sgt;
- }
/* release any previous cache */
if (attach->dma_dir != DMA_NONE) {
@@ -291,14 +286,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
/* mapping to the client with new direction */
if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
pr_err("failed to map scatterlist\n");
- mutex_unlock(lock);
return ERR_PTR(-EIO);
}
attach->dma_dir = dma_dir;
- mutex_unlock(lock);
-
return sgt;
}
@@ -327,6 +319,8 @@ static int vb2_vmalloc_dmabuf_ops_vmap(struct dma_buf *dbuf,
static int vb2_vmalloc_dmabuf_ops_mmap(struct dma_buf *dbuf,
struct vm_area_struct *vma)
{
+ dma_resv_assert_held(dbuf->resv);
+
return vb2_vmalloc_mmap(dbuf->priv, vma);
}
@@ -378,7 +372,7 @@ static int vb2_vmalloc_map_dmabuf(void *mem_priv)
struct iosys_map map;
int ret;
- ret = dma_buf_vmap(buf->dbuf, &map);
+ ret = dma_buf_vmap_unlocked(buf->dbuf, &map);
if (ret)
return -EFAULT;
buf->vaddr = map.vaddr;
@@ -391,7 +385,7 @@ static void vb2_vmalloc_unmap_dmabuf(void *mem_priv)
struct vb2_vmalloc_buf *buf = mem_priv;
struct iosys_map map = IOSYS_MAP_INIT_VADDR(buf->vaddr);
- dma_buf_vunmap(buf->dbuf, &map);
+ dma_buf_vunmap_unlocked(buf->dbuf, &map);
buf->vaddr = NULL;
}
@@ -401,7 +395,7 @@ static void vb2_vmalloc_detach_dmabuf(void *mem_priv)
struct iosys_map map = IOSYS_MAP_INIT_VADDR(buf->vaddr);
if (buf->vaddr)
- dma_buf_vunmap(buf->dbuf, &map);
+ dma_buf_vunmap_unlocked(buf->dbuf, &map);
kfree(buf);
}
diff --git a/drivers/media/platform/nvidia/tegra-vde/dmabuf-cache.c b/drivers/media/platform/nvidia/tegra-vde/dmabuf-cache.c
index 69c3461..1c5b949 100644
--- a/drivers/media/platform/nvidia/tegra-vde/dmabuf-cache.c
+++ b/drivers/media/platform/nvidia/tegra-vde/dmabuf-cache.c
@@ -38,7 +38,7 @@ static void tegra_vde_release_entry(struct tegra_vde_cache_entry *entry)
if (entry->vde->domain)
tegra_vde_iommu_unmap(entry->vde, entry->iova);
- dma_buf_unmap_attachment(entry->a, entry->sgt, entry->dma_dir);
+ dma_buf_unmap_attachment_unlocked(entry->a, entry->sgt, entry->dma_dir);
dma_buf_detach(dmabuf, entry->a);
dma_buf_put(dmabuf);
@@ -102,7 +102,7 @@ int tegra_vde_dmabuf_cache_map(struct tegra_vde *vde,
goto err_unlock;
}
- sgt = dma_buf_map_attachment(attachment, dma_dir);
+ sgt = dma_buf_map_attachment_unlocked(attachment, dma_dir);
if (IS_ERR(sgt)) {
dev_err(dev, "Failed to get dmabufs sg_table\n");
err = PTR_ERR(sgt);
@@ -152,7 +152,7 @@ int tegra_vde_dmabuf_cache_map(struct tegra_vde *vde,
err_free:
kfree(entry);
err_unmap:
- dma_buf_unmap_attachment(attachment, sgt, dma_dir);
+ dma_buf_unmap_attachment_unlocked(attachment, sgt, dma_dir);
err_detach:
dma_buf_detach(dmabuf, attachment);
err_unlock:
diff --git a/drivers/memory/tegra/mc.c b/drivers/memory/tegra/mc.c
index 2f7a58a..5929075 100644
--- a/drivers/memory/tegra/mc.c
+++ b/drivers/memory/tegra/mc.c
@@ -107,6 +107,31 @@ int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev)
}
EXPORT_SYMBOL_GPL(tegra_mc_probe_device);
+int tegra_mc_get_carveout_info(struct tegra_mc *mc, unsigned int id,
+ phys_addr_t *base, u64 *size)
+{
+ u32 offset;
+
+ if (id < 1 || id >= mc->soc->num_carveouts)
+ return -EINVAL;
+
+ if (id < 6)
+ offset = 0xc0c + 0x50 * (id - 1);
+ else
+ offset = 0x2004 + 0x50 * (id - 6);
+
+ *base = mc_ch_readl(mc, MC_BROADCAST_CHANNEL, offset + 0x0);
+#ifdef CONFIG_PHYS_ADDR_T_64BIT
+ *base |= (phys_addr_t)mc_ch_readl(mc, MC_BROADCAST_CHANNEL, offset + 0x4) << 32;
+#endif
+
+ if (size)
+ *size = mc_ch_readl(mc, MC_BROADCAST_CHANNEL, offset + 0x8) << 17;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(tegra_mc_get_carveout_info);
+
static int tegra_mc_block_dma_common(struct tegra_mc *mc,
const struct tegra_mc_reset *rst)
{
diff --git a/drivers/memory/tegra/tegra234.c b/drivers/memory/tegra/tegra234.c
index 9bdaf8a..02dcc57 100644
--- a/drivers/memory/tegra/tegra234.c
+++ b/drivers/memory/tegra/tegra234.c
@@ -347,4 +347,9 @@ const struct tegra_mc_soc tegra234_mc_soc = {
.ops = &tegra186_mc_ops,
.ch_intmask = 0x0000ff00,
.global_intstatus_channel_shift = 8,
+ /*
+ * Additionally, there are lite carveouts but those are not currently
+ * supported.
+ */
+ .num_carveouts = 32,
};
diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c
index 7ff0b63..0f467a7 100644
--- a/drivers/misc/fastrpc.c
+++ b/drivers/misc/fastrpc.c
@@ -6,6 +6,7 @@
#include <linux/device.h>
#include <linux/dma-buf.h>
#include <linux/dma-mapping.h>
+#include <linux/dma-resv.h>
#include <linux/idr.h>
#include <linux/list.h>
#include <linux/miscdevice.h>
@@ -310,8 +311,8 @@ static void fastrpc_free_map(struct kref *ref)
return;
}
}
- dma_buf_unmap_attachment(map->attach, map->table,
- DMA_BIDIRECTIONAL);
+ dma_buf_unmap_attachment_unlocked(map->attach, map->table,
+ DMA_BIDIRECTIONAL);
dma_buf_detach(map->buf, map->attach);
dma_buf_put(map->buf);
}
@@ -682,6 +683,8 @@ static int fastrpc_mmap(struct dma_buf *dmabuf,
struct fastrpc_buf *buf = dmabuf->priv;
size_t size = vma->vm_end - vma->vm_start;
+ dma_resv_assert_held(dmabuf->resv);
+
return dma_mmap_coherent(buf->dev, vma, buf->virt,
FASTRPC_PHYS(buf->phys), size);
}
@@ -726,7 +729,7 @@ static int fastrpc_map_create(struct fastrpc_user *fl, int fd,
goto attach_err;
}
- map->table = dma_buf_map_attachment(map->attach, DMA_BIDIRECTIONAL);
+ map->table = dma_buf_map_attachment_unlocked(map->attach, DMA_BIDIRECTIONAL);
if (IS_ERR(map->table)) {
err = PTR_ERR(map->table);
goto map_err;
diff --git a/drivers/misc/mei/bus.c b/drivers/misc/mei/bus.c
index 46aa355..1fbe127 100644
--- a/drivers/misc/mei/bus.c
+++ b/drivers/misc/mei/bus.c
@@ -13,6 +13,7 @@
#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/interrupt.h>
+#include <linux/scatterlist.h>
#include <linux/mei_cl_bus.h>
#include "mei_dev.h"
@@ -100,9 +101,18 @@ ssize_t __mei_cl_send(struct mei_cl *cl, const u8 *buf, size_t length, u8 vtag,
cb->internal = !!(mode & MEI_CL_IO_TX_INTERNAL);
cb->blocking = !!(mode & MEI_CL_IO_TX_BLOCKING);
memcpy(cb->buf.data, buf, length);
+ /* hack we point data to header */
+ if (mode & MEI_CL_IO_SGL) {
+ cb->ext_hdr = (struct mei_ext_hdr *)cb->buf.data;
+ cb->buf.data = NULL;
+ cb->buf.size = 0;
+ }
rets = mei_cl_write(cl, cb);
+ if (mode & MEI_CL_IO_SGL && rets == 0)
+ rets = length;
+
out:
mutex_unlock(&bus->device_lock);
@@ -205,9 +215,16 @@ ssize_t __mei_cl_recv(struct mei_cl *cl, u8 *buf, size_t length, u8 *vtag,
goto free;
}
- r_length = min_t(size_t, length, cb->buf_idx);
- memcpy(buf, cb->buf.data, r_length);
+ /* for the GSC type - copy the extended header to the buffer */
+ if (cb->ext_hdr && cb->ext_hdr->type == MEI_EXT_HDR_GSC) {
+ r_length = min_t(size_t, length, cb->ext_hdr->length * sizeof(u32));
+ memcpy(buf, cb->ext_hdr, r_length);
+ } else {
+ r_length = min_t(size_t, length, cb->buf_idx);
+ memcpy(buf, cb->buf.data, r_length);
+ }
rets = r_length;
+
if (vtag)
*vtag = cb->vtag;
@@ -823,6 +840,131 @@ int mei_cldev_disable(struct mei_cl_device *cldev)
EXPORT_SYMBOL_GPL(mei_cldev_disable);
/**
+ * mei_cldev_send_gsc_command - sends a gsc command, by sending
+ * a gsl mei message to gsc and receiving reply from gsc
+ *
+ * @cldev: me client device
+ * @client_id: client id to send the command to
+ * @fence_id: fence id to send the command to
+ * @sg_in: scatter gather list containing addresses for rx message buffer
+ * @total_in_len: total length of data in 'in' sg, can be less than the sum of buffers sizes
+ * @sg_out: scatter gather list containing addresses for tx message buffer
+ *
+ * Return:
+ * * written size in bytes
+ * * < 0 on error
+ */
+ssize_t mei_cldev_send_gsc_command(struct mei_cl_device *cldev,
+ u8 client_id, u32 fence_id,
+ struct scatterlist *sg_in,
+ size_t total_in_len,
+ struct scatterlist *sg_out)
+{
+ struct mei_cl *cl;
+ struct mei_device *bus;
+ ssize_t ret = 0;
+
+ struct mei_ext_hdr_gsc_h2f *ext_hdr;
+ size_t buf_sz = sizeof(struct mei_ext_hdr_gsc_h2f);
+ int sg_out_nents, sg_in_nents;
+ int i;
+ struct scatterlist *sg;
+ struct mei_ext_hdr_gsc_f2h rx_msg;
+ unsigned int sg_len;
+
+ if (!cldev || !sg_in || !sg_out)
+ return -EINVAL;
+
+ cl = cldev->cl;
+ bus = cldev->bus;
+
+ dev_dbg(bus->dev, "client_id %u, fence_id %u\n", client_id, fence_id);
+
+ if (!bus->hbm_f_gsc_supported)
+ return -EOPNOTSUPP;
+
+ sg_out_nents = sg_nents(sg_out);
+ sg_in_nents = sg_nents(sg_in);
+ /* at least one entry in tx and rx sgls must be present */
+ if (sg_out_nents <= 0 || sg_in_nents <= 0)
+ return -EINVAL;
+
+ buf_sz += (sg_out_nents + sg_in_nents) * sizeof(struct mei_gsc_sgl);
+ ext_hdr = kzalloc(buf_sz, GFP_KERNEL);
+ if (!ext_hdr)
+ return -ENOMEM;
+
+ /* construct the GSC message */
+ ext_hdr->hdr.type = MEI_EXT_HDR_GSC;
+ ext_hdr->hdr.length = buf_sz / sizeof(u32); /* length is in dw */
+
+ ext_hdr->client_id = client_id;
+ ext_hdr->addr_type = GSC_ADDRESS_TYPE_PHYSICAL_SGL;
+ ext_hdr->fence_id = fence_id;
+ ext_hdr->input_address_count = sg_in_nents;
+ ext_hdr->output_address_count = sg_out_nents;
+ ext_hdr->reserved[0] = 0;
+ ext_hdr->reserved[1] = 0;
+
+ /* copy in-sgl to the message */
+ for (i = 0, sg = sg_in; i < sg_in_nents; i++, sg++) {
+ ext_hdr->sgl[i].low = lower_32_bits(sg_dma_address(sg));
+ ext_hdr->sgl[i].high = upper_32_bits(sg_dma_address(sg));
+ sg_len = min_t(unsigned int, sg_dma_len(sg), PAGE_SIZE);
+ ext_hdr->sgl[i].length = (sg_len <= total_in_len) ? sg_len : total_in_len;
+ total_in_len -= ext_hdr->sgl[i].length;
+ }
+
+ /* copy out-sgl to the message */
+ for (i = sg_in_nents, sg = sg_out; i < sg_in_nents + sg_out_nents; i++, sg++) {
+ ext_hdr->sgl[i].low = lower_32_bits(sg_dma_address(sg));
+ ext_hdr->sgl[i].high = upper_32_bits(sg_dma_address(sg));
+ sg_len = min_t(unsigned int, sg_dma_len(sg), PAGE_SIZE);
+ ext_hdr->sgl[i].length = sg_len;
+ }
+
+ /* send the message to GSC */
+ ret = __mei_cl_send(cl, (u8 *)ext_hdr, buf_sz, 0, MEI_CL_IO_SGL);
+ if (ret < 0) {
+ dev_err(bus->dev, "__mei_cl_send failed, returned %zd\n", ret);
+ goto end;
+ }
+ if (ret != buf_sz) {
+ dev_err(bus->dev, "__mei_cl_send returned %zd instead of expected %zd\n",
+ ret, buf_sz);
+ ret = -EIO;
+ goto end;
+ }
+
+ /* receive the reply from GSC, note that at this point sg_in should contain the reply */
+ ret = __mei_cl_recv(cl, (u8 *)&rx_msg, sizeof(rx_msg), NULL, MEI_CL_IO_SGL, 0);
+
+ if (ret != sizeof(rx_msg)) {
+ dev_err(bus->dev, "__mei_cl_recv returned %zd instead of expected %zd\n",
+ ret, sizeof(rx_msg));
+ if (ret >= 0)
+ ret = -EIO;
+ goto end;
+ }
+
+ /* check rx_msg.client_id and rx_msg.fence_id match the ones we send */
+ if (rx_msg.client_id != client_id || rx_msg.fence_id != fence_id) {
+ dev_err(bus->dev, "received client_id/fence_id %u/%u instead of %u/%u sent\n",
+ rx_msg.client_id, rx_msg.fence_id, client_id, fence_id);
+ ret = -EFAULT;
+ goto end;
+ }
+
+ dev_dbg(bus->dev, "gsc command: successfully written %u bytes\n", rx_msg.written);
+ ret = rx_msg.written;
+
+end:
+ kfree(ext_hdr);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(mei_cldev_send_gsc_command);
+
+/**
* mei_cl_device_find - find matching entry in the driver id table
*
* @cldev: me client device
diff --git a/drivers/misc/mei/client.c b/drivers/misc/mei/client.c
index 0b2fbe1..6c8b71a 100644
--- a/drivers/misc/mei/client.c
+++ b/drivers/misc/mei/client.c
@@ -322,6 +322,7 @@ void mei_io_cb_free(struct mei_cl_cb *cb)
list_del(&cb->list);
kfree(cb->buf.data);
+ kfree(cb->ext_hdr);
kfree(cb);
}
@@ -401,6 +402,7 @@ static struct mei_cl_cb *mei_io_cb_init(struct mei_cl *cl,
cb->buf_idx = 0;
cb->fop_type = type;
cb->vtag = 0;
+ cb->ext_hdr = NULL;
return cb;
}
@@ -1740,6 +1742,17 @@ static inline u8 mei_ext_hdr_set_vtag(void *ext, u8 vtag)
return vtag_hdr->hdr.length;
}
+static inline bool mei_ext_hdr_is_gsc(struct mei_ext_hdr *ext)
+{
+ return ext && ext->type == MEI_EXT_HDR_GSC;
+}
+
+static inline u8 mei_ext_hdr_set_gsc(struct mei_ext_hdr *ext, struct mei_ext_hdr *gsc_hdr)
+{
+ memcpy(ext, gsc_hdr, mei_ext_hdr_len(gsc_hdr));
+ return ext->length;
+}
+
/**
* mei_msg_hdr_init - allocate and initialize mei message header
*
@@ -1752,14 +1765,17 @@ static struct mei_msg_hdr *mei_msg_hdr_init(const struct mei_cl_cb *cb)
size_t hdr_len;
struct mei_ext_meta_hdr *meta;
struct mei_msg_hdr *mei_hdr;
- bool is_ext, is_vtag;
+ bool is_ext, is_hbm, is_gsc, is_vtag;
+ struct mei_ext_hdr *next_ext;
if (!cb)
return ERR_PTR(-EINVAL);
/* Extended header for vtag is attached only on the first fragment */
is_vtag = (cb->vtag && cb->buf_idx == 0);
- is_ext = is_vtag;
+ is_hbm = cb->cl->me_cl->client_id == 0;
+ is_gsc = ((!is_hbm) && cb->cl->dev->hbm_f_gsc_supported && mei_ext_hdr_is_gsc(cb->ext_hdr));
+ is_ext = is_vtag || is_gsc;
/* Compute extended header size */
hdr_len = sizeof(*mei_hdr);
@@ -1771,6 +1787,9 @@ static struct mei_msg_hdr *mei_msg_hdr_init(const struct mei_cl_cb *cb)
if (is_vtag)
hdr_len += sizeof(struct mei_ext_hdr_vtag);
+ if (is_gsc)
+ hdr_len += mei_ext_hdr_len(cb->ext_hdr);
+
setup_hdr:
mei_hdr = kzalloc(hdr_len, GFP_KERNEL);
if (!mei_hdr)
@@ -1785,10 +1804,20 @@ static struct mei_msg_hdr *mei_msg_hdr_init(const struct mei_cl_cb *cb)
goto out;
meta = (struct mei_ext_meta_hdr *)mei_hdr->extension;
+ meta->size = 0;
+ next_ext = (struct mei_ext_hdr *)meta->hdrs;
if (is_vtag) {
meta->count++;
- meta->size += mei_ext_hdr_set_vtag(meta->hdrs, cb->vtag);
+ meta->size += mei_ext_hdr_set_vtag(next_ext, cb->vtag);
+ next_ext = mei_ext_next(next_ext);
}
+
+ if (is_gsc) {
+ meta->count++;
+ meta->size += mei_ext_hdr_set_gsc(next_ext, cb->ext_hdr);
+ next_ext = mei_ext_next(next_ext);
+ }
+
out:
mei_hdr->length = hdr_len - sizeof(*mei_hdr);
return mei_hdr;
@@ -1812,14 +1841,14 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb,
struct mei_msg_hdr *mei_hdr = NULL;
size_t hdr_len;
size_t hbuf_len, dr_len;
- size_t buf_len;
+ size_t buf_len = 0;
size_t data_len;
int hbuf_slots;
u32 dr_slots;
u32 dma_len;
int rets;
bool first_chunk;
- const void *data;
+ const void *data = NULL;
if (WARN_ON(!cl || !cl->dev))
return -ENODEV;
@@ -1839,8 +1868,10 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb,
return 0;
}
- buf_len = buf->size - cb->buf_idx;
- data = buf->data + cb->buf_idx;
+ if (buf->data) {
+ buf_len = buf->size - cb->buf_idx;
+ data = buf->data + cb->buf_idx;
+ }
hbuf_slots = mei_hbuf_empty_slots(dev);
if (hbuf_slots < 0) {
rets = -EOVERFLOW;
@@ -1858,9 +1889,6 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb,
goto err;
}
- cl_dbg(dev, cl, "Extended Header %d vtag = %d\n",
- mei_hdr->extended, cb->vtag);
-
hdr_len = sizeof(*mei_hdr) + mei_hdr->length;
/**
@@ -1889,7 +1917,7 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb,
}
mei_hdr->length += data_len;
- if (mei_hdr->dma_ring)
+ if (mei_hdr->dma_ring && buf->data)
mei_dma_ring_write(dev, buf->data + cb->buf_idx, buf_len);
rets = mei_write_message(dev, mei_hdr, hdr_len, data, data_len);
@@ -1983,9 +2011,6 @@ ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb)
goto err;
}
- cl_dbg(dev, cl, "Extended Header %d vtag = %d\n",
- mei_hdr->extended, cb->vtag);
-
hdr_len = sizeof(*mei_hdr) + mei_hdr->length;
if (rets == 0) {
@@ -2030,7 +2055,7 @@ ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb)
mei_hdr->length += data_len;
- if (mei_hdr->dma_ring)
+ if (mei_hdr->dma_ring && buf->data)
mei_dma_ring_write(dev, buf->data, buf_len);
rets = mei_write_message(dev, mei_hdr, hdr_len, data, data_len);
diff --git a/drivers/misc/mei/hbm.c b/drivers/misc/mei/hbm.c
index de712cb..12a62a9 100644
--- a/drivers/misc/mei/hbm.c
+++ b/drivers/misc/mei/hbm.c
@@ -340,9 +340,13 @@ static int mei_hbm_capabilities_req(struct mei_device *dev)
req.hbm_cmd = MEI_HBM_CAPABILITIES_REQ_CMD;
if (dev->hbm_f_vt_supported)
req.capability_requested[0] |= HBM_CAP_VT;
+
if (dev->hbm_f_cd_supported)
req.capability_requested[0] |= HBM_CAP_CD;
+ if (dev->hbm_f_gsc_supported)
+ req.capability_requested[0] |= HBM_CAP_GSC;
+
ret = mei_hbm_write_message(dev, &mei_hdr, &req);
if (ret) {
dev_err(dev->dev,
@@ -1200,6 +1204,12 @@ static void mei_hbm_config_features(struct mei_device *dev)
dev->version.minor_version >= HBM_MINOR_VERSION_VT))
dev->hbm_f_vt_supported = 1;
+ /* GSC support */
+ if (dev->version.major_version > HBM_MAJOR_VERSION_GSC ||
+ (dev->version.major_version == HBM_MAJOR_VERSION_GSC &&
+ dev->version.minor_version >= HBM_MINOR_VERSION_GSC))
+ dev->hbm_f_gsc_supported = 1;
+
/* Capability message Support */
dev->hbm_f_cap_supported = 0;
if (dev->version.major_version > HBM_MAJOR_VERSION_CAP ||
@@ -1367,6 +1377,9 @@ int mei_hbm_dispatch(struct mei_device *dev, struct mei_msg_hdr *hdr)
if (!(capability_res->capability_granted[0] & HBM_CAP_CD))
dev->hbm_f_cd_supported = 0;
+ if (!(capability_res->capability_granted[0] & HBM_CAP_GSC))
+ dev->hbm_f_gsc_supported = 0;
+
if (dev->hbm_f_dr_supported) {
if (mei_dmam_ring_alloc(dev))
dev_info(dev->dev, "running w/o dma ring\n");
diff --git a/drivers/misc/mei/hw-me.c b/drivers/misc/mei/hw-me.c
index 9e2f781..da4ef0b 100644
--- a/drivers/misc/mei/hw-me.c
+++ b/drivers/misc/mei/hw-me.c
@@ -590,9 +590,14 @@ static int mei_me_hbuf_write(struct mei_device *dev,
u32 dw_cnt;
int empty_slots;
- if (WARN_ON(!hdr || !data || hdr_len & 0x3))
+ if (WARN_ON(!hdr || hdr_len & 0x3))
return -EINVAL;
+ if (!data && data_len) {
+ dev_err(dev->dev, "wrong parameters null data with data_len = %zu\n", data_len);
+ return -EINVAL;
+ }
+
dev_dbg(dev->dev, MEI_HDR_FMT, MEI_HDR_PRM((struct mei_msg_hdr *)hdr));
empty_slots = mei_hbuf_empty_slots(dev);
diff --git a/drivers/misc/mei/hw.h b/drivers/misc/mei/hw.h
index e7e020db..319418d 100644
--- a/drivers/misc/mei/hw.h
+++ b/drivers/misc/mei/hw.h
@@ -93,6 +93,12 @@
#define HBM_MAJOR_VERSION_VT 2
/*
+ * MEI version with GSC support
+ */
+#define HBM_MINOR_VERSION_GSC 2
+#define HBM_MAJOR_VERSION_GSC 2
+
+/*
* MEI version with capabilities message support
*/
#define HBM_MINOR_VERSION_CAP 2
@@ -229,18 +235,19 @@ enum mei_cl_disconnect_status {
*
* @MEI_EXT_HDR_NONE: sentinel
* @MEI_EXT_HDR_VTAG: vtag header
+ * @MEI_EXT_HDR_GSC: gsc header
*/
enum mei_ext_hdr_type {
MEI_EXT_HDR_NONE = 0,
MEI_EXT_HDR_VTAG = 1,
+ MEI_EXT_HDR_GSC = 2,
};
/**
* struct mei_ext_hdr - extend header descriptor (TLV)
* @type: enum mei_ext_hdr_type
* @length: length excluding descriptor
- * @ext_payload: payload of the specific extended header
- * @hdr: place holder for actual header
+ * @data: the extended header payload
*/
struct mei_ext_hdr {
u8 type;
@@ -279,12 +286,11 @@ struct mei_ext_hdr_vtag {
* Extended header iterator functions
*/
/**
- * mei_ext_hdr - extended header iterator begin
+ * mei_ext_begin - extended header iterator begin
*
* @meta: meta header of the extended header list
*
- * Return:
- * The first extended header
+ * Return: The first extended header
*/
static inline struct mei_ext_hdr *mei_ext_begin(struct mei_ext_meta_hdr *meta)
{
@@ -305,6 +311,60 @@ static inline bool mei_ext_last(struct mei_ext_meta_hdr *meta,
return (u8 *)ext >= (u8 *)meta + sizeof(*meta) + (meta->size * 4);
}
+struct mei_gsc_sgl {
+ u32 low;
+ u32 high;
+ u32 length;
+} __packed;
+
+#define GSC_HECI_MSG_KERNEL 0
+#define GSC_HECI_MSG_USER 1
+
+#define GSC_ADDRESS_TYPE_GTT 0
+#define GSC_ADDRESS_TYPE_PPGTT 1
+#define GSC_ADDRESS_TYPE_PHYSICAL_CONTINUOUS 2 /* max of 64K */
+#define GSC_ADDRESS_TYPE_PHYSICAL_SGL 3
+
+/**
+ * struct mei_ext_hdr_gsc_h2f - extended header: gsc host to firmware interface
+ *
+ * @hdr: extended header
+ * @client_id: GSC_HECI_MSG_KERNEL or GSC_HECI_MSG_USER
+ * @addr_type: GSC_ADDRESS_TYPE_{GTT, PPGTT, PHYSICAL_CONTINUOUS, PHYSICAL_SGL}
+ * @fence_id: synchronization marker
+ * @input_address_count: number of input sgl buffers
+ * @output_address_count: number of output sgl buffers
+ * @reserved: reserved
+ * @sgl: sg list
+ */
+struct mei_ext_hdr_gsc_h2f {
+ struct mei_ext_hdr hdr;
+ u8 client_id;
+ u8 addr_type;
+ u32 fence_id;
+ u8 input_address_count;
+ u8 output_address_count;
+ u8 reserved[2];
+ struct mei_gsc_sgl sgl[];
+} __packed;
+
+/**
+ * struct mei_ext_hdr_gsc_f2h - gsc firmware to host interface
+ *
+ * @hdr: extended header
+ * @client_id: GSC_HECI_MSG_KERNEL or GSC_HECI_MSG_USER
+ * @reserved: reserved
+ * @fence_id: synchronization marker
+ * @written: number of bytes written to firmware
+ */
+struct mei_ext_hdr_gsc_f2h {
+ struct mei_ext_hdr hdr;
+ u8 client_id;
+ u8 reserved;
+ u32 fence_id;
+ u32 written;
+} __packed;
+
/**
* mei_ext_next - following extended header on the TLV list
*
@@ -321,6 +381,21 @@ static inline struct mei_ext_hdr *mei_ext_next(struct mei_ext_hdr *ext)
}
/**
+ * mei_ext_hdr_len - get ext header length in bytes
+ *
+ * @ext: extend header
+ *
+ * Return: extend header length in bytes
+ */
+static inline u32 mei_ext_hdr_len(const struct mei_ext_hdr *ext)
+{
+ if (!ext)
+ return 0;
+
+ return ext->length * sizeof(u32);
+}
+
+/**
* struct mei_msg_hdr - MEI BUS Interface Section
*
* @me_addr: device address
@@ -682,6 +757,10 @@ struct hbm_dma_ring_ctrl {
/* virtual tag supported */
#define HBM_CAP_VT BIT(0)
+
+/* gsc extended header support */
+#define HBM_CAP_GSC BIT(1)
+
/* client dma supported */
#define HBM_CAP_CD BIT(2)
diff --git a/drivers/misc/mei/interrupt.c b/drivers/misc/mei/interrupt.c
index 07063221..0a0e984 100644
--- a/drivers/misc/mei/interrupt.c
+++ b/drivers/misc/mei/interrupt.c
@@ -98,9 +98,12 @@ static int mei_cl_irq_read_msg(struct mei_cl *cl,
struct mei_device *dev = cl->dev;
struct mei_cl_cb *cb;
+ struct mei_ext_hdr_vtag *vtag_hdr = NULL;
+ struct mei_ext_hdr_gsc_f2h *gsc_f2h = NULL;
+
size_t buf_sz;
u32 length;
- int ext_len;
+ u32 ext_len;
length = mei_hdr->length;
ext_len = 0;
@@ -122,18 +125,24 @@ static int mei_cl_irq_read_msg(struct mei_cl *cl,
}
if (mei_hdr->extended) {
- struct mei_ext_hdr *ext;
- struct mei_ext_hdr_vtag *vtag_hdr = NULL;
-
- ext = mei_ext_begin(meta);
+ struct mei_ext_hdr *ext = mei_ext_begin(meta);
do {
switch (ext->type) {
case MEI_EXT_HDR_VTAG:
vtag_hdr = (struct mei_ext_hdr_vtag *)ext;
break;
+ case MEI_EXT_HDR_GSC:
+ gsc_f2h = (struct mei_ext_hdr_gsc_f2h *)ext;
+ cb->ext_hdr = kzalloc(sizeof(*gsc_f2h), GFP_KERNEL);
+ if (!cb->ext_hdr) {
+ cb->status = -ENOMEM;
+ goto discard;
+ }
+ break;
case MEI_EXT_HDR_NONE:
fallthrough;
default:
+ cl_err(dev, cl, "unknown extended header\n");
cb->status = -EPROTO;
break;
}
@@ -141,12 +150,14 @@ static int mei_cl_irq_read_msg(struct mei_cl *cl,
ext = mei_ext_next(ext);
} while (!mei_ext_last(meta, ext));
- if (!vtag_hdr) {
- cl_dbg(dev, cl, "vtag not found in extended header.\n");
+ if (!vtag_hdr && !gsc_f2h) {
+ cl_dbg(dev, cl, "no vtag or gsc found in extended header.\n");
cb->status = -EPROTO;
goto discard;
}
+ }
+ if (vtag_hdr) {
cl_dbg(dev, cl, "vtag: %d\n", vtag_hdr->vtag);
if (cb->vtag && cb->vtag != vtag_hdr->vtag) {
cl_err(dev, cl, "mismatched tag: %d != %d\n",
@@ -157,6 +168,28 @@ static int mei_cl_irq_read_msg(struct mei_cl *cl,
cb->vtag = vtag_hdr->vtag;
}
+ if (gsc_f2h) {
+ u32 ext_hdr_len = mei_ext_hdr_len(&gsc_f2h->hdr);
+
+ if (!dev->hbm_f_gsc_supported) {
+ cl_err(dev, cl, "gsc extended header is not supported\n");
+ cb->status = -EPROTO;
+ goto discard;
+ }
+
+ if (length) {
+ cl_err(dev, cl, "no data allowed in cb with gsc\n");
+ cb->status = -EPROTO;
+ goto discard;
+ }
+ if (ext_hdr_len > sizeof(*gsc_f2h)) {
+ cl_err(dev, cl, "gsc extended header is too big %u\n", ext_hdr_len);
+ cb->status = -EPROTO;
+ goto discard;
+ }
+ memcpy(cb->ext_hdr, gsc_f2h, ext_hdr_len);
+ }
+
if (!mei_cl_is_connected(cl)) {
cl_dbg(dev, cl, "not connected\n");
cb->status = -ENODEV;
diff --git a/drivers/misc/mei/mei_dev.h b/drivers/misc/mei/mei_dev.h
index 6bb3e1b..8d80184 100644
--- a/drivers/misc/mei/mei_dev.h
+++ b/drivers/misc/mei/mei_dev.h
@@ -116,12 +116,16 @@ enum mei_cb_file_ops {
* @MEI_CL_IO_TX_INTERNAL: internal communication between driver and FW
*
* @MEI_CL_IO_RX_NONBLOCK: recv is non-blocking
+ *
+ * @MEI_CL_IO_SGL: send command with sgl list.
*/
enum mei_cl_io_mode {
MEI_CL_IO_TX_BLOCKING = BIT(0),
MEI_CL_IO_TX_INTERNAL = BIT(1),
MEI_CL_IO_RX_NONBLOCK = BIT(2),
+
+ MEI_CL_IO_SGL = BIT(3),
};
/*
@@ -206,6 +210,7 @@ struct mei_cl;
* @status: io status of the cb
* @internal: communication between driver and FW flag
* @blocking: transmission blocking mode
+ * @ext_hdr: extended header
*/
struct mei_cl_cb {
struct list_head list;
@@ -218,6 +223,7 @@ struct mei_cl_cb {
int status;
u32 internal:1;
u32 blocking:1;
+ struct mei_ext_hdr *ext_hdr;
};
/**
@@ -494,6 +500,7 @@ struct mei_dev_timeouts {
* @hbm_f_vt_supported : hbm feature vtag supported
* @hbm_f_cap_supported : hbm feature capabilities message supported
* @hbm_f_cd_supported : hbm feature client dma supported
+ * @hbm_f_gsc_supported : hbm feature gsc supported
*
* @fw_ver : FW versions
*
@@ -585,6 +592,7 @@ struct mei_device {
unsigned int hbm_f_vt_supported:1;
unsigned int hbm_f_cap_supported:1;
unsigned int hbm_f_cd_supported:1;
+ unsigned int hbm_f_gsc_supported:1;
struct mei_fw_version fw_ver[MEI_MAX_FW_VER_BLOCKS];
diff --git a/drivers/misc/mei/pxp/mei_pxp.c b/drivers/misc/mei/pxp/mei_pxp.c
index 5c39457..8dd09b1 100644
--- a/drivers/misc/mei/pxp/mei_pxp.c
+++ b/drivers/misc/mei/pxp/mei_pxp.c
@@ -77,10 +77,35 @@ mei_pxp_receive_message(struct device *dev, void *buffer, size_t size)
return byte;
}
+/**
+ * mei_pxp_gsc_command() - sends a gsc command, by sending
+ * a sgl mei message to gsc and receiving reply from gsc
+ *
+ * @dev: device corresponding to the mei_cl_device
+ * @client_id: client id to send the command to
+ * @fence_id: fence id to send the command to
+ * @sg_in: scatter gather list containing addresses for rx message buffer
+ * @total_in_len: total length of data in 'in' sg, can be less than the sum of buffers sizes
+ * @sg_out: scatter gather list containing addresses for tx message buffer
+ *
+ * Return: bytes sent on Success, <0 on Failure
+ */
+static ssize_t mei_pxp_gsc_command(struct device *dev, u8 client_id, u32 fence_id,
+ struct scatterlist *sg_in, size_t total_in_len,
+ struct scatterlist *sg_out)
+{
+ struct mei_cl_device *cldev;
+
+ cldev = to_mei_cl_device(dev);
+
+ return mei_cldev_send_gsc_command(cldev, client_id, fence_id, sg_in, total_in_len, sg_out);
+}
+
static const struct i915_pxp_component_ops mei_pxp_ops = {
.owner = THIS_MODULE,
.send = mei_pxp_send_message,
.recv = mei_pxp_receive_message,
+ .gsc_command = mei_pxp_gsc_command,
};
static int mei_component_master_bind(struct device *dev)
@@ -131,17 +156,24 @@ static int mei_pxp_component_match(struct device *dev, int subcomponent,
{
struct device *base = data;
+ if (!dev)
+ return 0;
+
if (!dev->driver || strcmp(dev->driver->name, "i915") ||
subcomponent != I915_COMPONENT_PXP)
return 0;
base = base->parent;
- if (!base)
+ if (!base) /* mei device */
return 0;
- base = base->parent;
- dev = dev->parent;
+ base = base->parent; /* pci device */
+ /* for dgfx */
+ if (base && dev == base)
+ return 1;
+ /* for pch */
+ dev = dev->parent;
return (base && dev && dev == base);
}
diff --git a/drivers/staging/sm750fb/Kconfig b/drivers/staging/sm750fb/Kconfig
index 8c0d8a8..acb6c08 100644
--- a/drivers/staging/sm750fb/Kconfig
+++ b/drivers/staging/sm750fb/Kconfig
@@ -6,6 +6,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Frame buffer driver for the Silicon Motion SM750 chip
with 2D accelearion and dual head support.
diff --git a/drivers/staging/sm750fb/sm750.c b/drivers/staging/sm750fb/sm750.c
index 168ae2e..effc7fc 100644
--- a/drivers/staging/sm750fb/sm750.c
+++ b/drivers/staging/sm750fb/sm750.c
@@ -1168,6 +1168,9 @@ static int __init lynxfb_init(void)
{
char *option;
+ if (fb_modesetting_disabled("sm750fb"))
+ return -ENODEV;
+
#ifdef MODULE
option = g_option;
#else
diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig
index 0587e21..6d2fde6 100644
--- a/drivers/video/Kconfig
+++ b/drivers/video/Kconfig
@@ -11,6 +11,10 @@
Support tracking and hand-over of aperture ownership. Required
by graphics drivers for firmware-provided framebuffers.
+config VIDEO_NOMODESET
+ bool
+ default n
+
if HAS_IOMEM
config HAVE_FB_ATMEL
diff --git a/drivers/video/Makefile b/drivers/video/Makefile
index 5bb6b45..a50eb52 100644
--- a/drivers/video/Makefile
+++ b/drivers/video/Makefile
@@ -2,6 +2,7 @@
obj-$(CONFIG_APERTURE_HELPERS) += aperture.o
obj-$(CONFIG_VGASTATE) += vgastate.o
+obj-$(CONFIG_VIDEO_NOMODESET) += nomodeset.o
obj-$(CONFIG_HDMI) += hdmi.o
obj-$(CONFIG_VT) += console/
diff --git a/drivers/video/fbdev/Kconfig b/drivers/video/fbdev/Kconfig
index cfc5527..71019b1 100644
--- a/drivers/video/fbdev/Kconfig
+++ b/drivers/video/fbdev/Kconfig
@@ -227,6 +227,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This enables support for Cirrus Logic GD542x/543x based boards on
Amiga: SD64, Piccolo, Picasso II/II+, Picasso IV, or EGS Spectrum.
@@ -245,6 +246,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for cards based on
the 3D Labs Permedia, Permedia 2 and Permedia 2V chips.
@@ -340,6 +342,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This enables support for the Integraphics CyberPro 20x0 and 5000
VGA chips used in the Rebel.com Netwinder and other machines.
@@ -455,6 +458,7 @@
config FB_OF
bool "Open Firmware frame buffer device support"
depends on (FB = y) && PPC && (!PPC_PSERIES || PCI)
+ depends on !DRM_OFDRM
select APERTURE_HELPERS
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
@@ -503,6 +507,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for the Chips & Technologies
65550 graphics chip in PowerBooks.
@@ -513,6 +518,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for the Asiliant 69030 chipset
@@ -521,6 +527,7 @@
depends on (FB = y) && PCI
select FB_CFB_IMAGEBLIT
select FB_MACMODES if PPC_PMAC
+ select VIDEO_NOMODESET
help
The IMS Twin Turbo is a PCI-based frame buffer card bundled with
many Macintosh and compatible computers.
@@ -584,6 +591,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select BITREVERSE
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for generic TGA and SFB+
graphic cards. These include DEC ZLXp-E1, -E2 and -E3 PCI cards,
@@ -776,6 +784,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This is the framebuffer device for the Sun XVR-500 and similar
graphics cards based upon the 3DLABS Wildcat chipset. The driver
@@ -789,6 +798,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This is the framebuffer device for the Sun XVR-2500 and similar
graphics cards based upon the 3DLABS Wildcat chipset. The driver
@@ -815,6 +825,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Say Y here if you have a PowerVR 2 card in your box. If you plan to
run linux on your Dreamcast, you will have to say Y here.
@@ -880,6 +891,7 @@
select FB_CFB_IMAGEBLIT
select BITREVERSE
select VGASTATE
+ select VIDEO_NOMODESET
help
This driver supports graphics boards with the nVidia chips, TNT
and newer. For very old chipsets, such as the RIVA128, then use
@@ -927,6 +939,7 @@
select FB_CFB_IMAGEBLIT
select BITREVERSE
select VGASTATE
+ select VIDEO_NOMODESET
help
This driver supports graphics boards with the nVidia Riva/Geforce
chips.
@@ -971,6 +984,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select VGASTATE
+ select VIDEO_NOMODESET
select FB_DDC
help
This driver supports graphics cards based on Intel740 chip.
@@ -983,6 +997,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select VGASTATE
+ select VIDEO_NOMODESET
help
This driver supports the on-board graphics built in to the Intel 810
and 815 chipsets. Say Y if you have and plan to use such a board.
@@ -1033,6 +1048,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This driver supports the LE80578 (Vermilion Range) chipset
@@ -1050,6 +1066,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select BOOT_VESA_SUPPORT if FB_INTEL = y
+ select VIDEO_NOMODESET
depends on !DRM_I915
help
This driver supports the on-board graphics built in to the Intel
@@ -1087,6 +1104,7 @@
select FB_CFB_IMAGEBLIT
select FB_TILEBLITTING
select FB_MACMODES if PPC_PMAC
+ select VIDEO_NOMODESET
help
Say Y here if you have a Matrox Millennium, Matrox Millennium II,
Matrox Mystique, Matrox Mystique 220, Matrox Productiva G100, Matrox
@@ -1207,6 +1225,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select FB_MACMODES if PPC
+ select VIDEO_NOMODESET
help
Choose this option if you want to use an ATI Radeon graphics card as
a framebuffer device. There are both PCI and AGP versions. You
@@ -1246,6 +1265,7 @@
select FB_CFB_IMAGEBLIT
select FB_BACKLIGHT if FB_ATY128_BACKLIGHT
select FB_MACMODES if PPC_PMAC
+ select VIDEO_NOMODESET
help
This driver supports graphics boards with the ATI Rage128 chips.
Say Y if you have such a graphics board and read
@@ -1270,6 +1290,7 @@
select FB_BACKLIGHT if FB_ATY_BACKLIGHT
select FB_MACMODES if PPC
select FB_ATY_CT if SPARC64 && PCI
+ select VIDEO_NOMODESET
help
This driver supports graphics boards with the ATI Mach64 chips.
Say Y if you have such a graphics board.
@@ -1320,6 +1341,7 @@
select FB_TILEBLITTING
select FB_SVGALIB
select VGASTATE
+ select VIDEO_NOMODESET
select FONT_8x16 if FRAMEBUFFER_CONSOLE
help
Driver for graphics boards with S3 Trio / S3 Virge chip.
@@ -1340,6 +1362,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select VGASTATE
+ select VIDEO_NOMODESET
help
This driver supports notebooks and computers with S3 Savage PCI/AGP
chips.
@@ -1378,6 +1401,7 @@
select FB_CFB_IMAGEBLIT
select BOOT_VESA_SUPPORT if FB_SIS = y
select FB_SIS_300 if !FB_SIS_315
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for the SiS 300, 315, 330
and 340 series as well as XGI V3XT, V5, V8, Z7 graphics chipsets.
@@ -1407,6 +1431,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select I2C_ALGOBIT
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for Graphics chips of VIA
UniChrome (Pro) Family (CLE266,PM800/CN400,P4M800CE/P4M800Pro/
@@ -1446,6 +1471,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select VGASTATE
+ select VIDEO_NOMODESET
help
This driver supports notebooks with NeoMagic PCI chips.
Say Y if you have such a graphics card.
@@ -1459,6 +1485,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Say Y here if you have a STG4000 / Kyro / PowerVR 3 based
graphics board.
@@ -1473,6 +1500,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_MODE_HELPERS
+ select VIDEO_NOMODESET
help
This driver supports graphics boards with the 3Dfx Banshee,
Voodoo3 or VSA-100 (aka Voodoo4/5) chips. Say Y if you have
@@ -1502,6 +1530,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Say Y here if you have a 3Dfx Voodoo Graphics (Voodoo1/sst1) or
Voodoo2 (cvg) based graphics card.
@@ -1523,6 +1552,7 @@
select FB_TILEBLITTING
select FB_SVGALIB
select VGASTATE
+ select VIDEO_NOMODESET
select FONT_8x16 if FRAMEBUFFER_CONSOLE
help
Driver for CastleRock integrated graphics core in the
@@ -1536,6 +1566,7 @@
select FB_CFB_IMAGEBLIT
select FB_DDC
select FB_MODE_HELPERS
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for Trident PCI/AGP chipsets.
Supported chipset families are TGUI 9440/96XX, 3DImage, Blade3D
@@ -1559,6 +1590,7 @@
select FB_TILEBLITTING
select FB_SVGALIB
select VGASTATE
+ select VIDEO_NOMODESET
select FONT_8x16 if FRAMEBUFFER_CONSOLE
help
Driver for PCI graphics boards with ARK 2000PV chip
@@ -1570,6 +1602,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for the 3DLabs Permedia3
chipset, used in Formac ProFormance III, 3DLabs Oxygen VX1 &
@@ -1582,6 +1615,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
This is the frame buffer device driver for the Fujitsu Carmine chip.
The driver provides two independent frame buffer devices.
@@ -1960,6 +1994,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Say Y here to enable support for the IBM GXT4000P/6000P and
GXT4500P/6500P display adaptor based on Raster Engine RC1000,
@@ -2100,6 +2135,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Frame buffer driver for Fujitsu Carmine/Coral-P(A)/Lime controllers.
@@ -2187,6 +2223,7 @@
select FB_CFB_IMAGEBLIT
select FB_DEFERRED_IO
select DMA_CMA if HAVE_DMA_CONTIGUOUS && CMA
+ select VIDEO_NOMODESET
help
This framebuffer driver supports Microsoft Hyper-V Synthetic Video.
@@ -2229,6 +2266,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Frame buffer driver for the Silicon Motion SM710, SM712, SM721
and SM722 chips.
diff --git a/drivers/video/fbdev/arkfb.c b/drivers/video/fbdev/arkfb.c
index 5f8fec9..60a96fd 100644
--- a/drivers/video/fbdev/arkfb.c
+++ b/drivers/video/fbdev/arkfb.c
@@ -1187,7 +1187,12 @@ static int __init arkfb_init(void)
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("arkfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("arkfb", &option))
return -ENODEV;
diff --git a/drivers/video/fbdev/asiliantfb.c b/drivers/video/fbdev/asiliantfb.c
index 3818437..8383468f 100644
--- a/drivers/video/fbdev/asiliantfb.c
+++ b/drivers/video/fbdev/asiliantfb.c
@@ -616,6 +616,9 @@ static struct pci_driver asiliantfb_driver = {
static int __init asiliantfb_init(void)
{
+ if (fb_modesetting_disabled("asiliantfb"))
+ return -ENODEV;
+
if (fb_get_options("asiliantfb", NULL))
return -ENODEV;
diff --git a/drivers/video/fbdev/aty/aty128fb.c b/drivers/video/fbdev/aty/aty128fb.c
index 57e398f..dd31b9d 100644
--- a/drivers/video/fbdev/aty/aty128fb.c
+++ b/drivers/video/fbdev/aty/aty128fb.c
@@ -2503,7 +2503,12 @@ static int aty128fb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("aty128fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("aty128fb", &option))
return -ENODEV;
aty128fb_setup(option);
diff --git a/drivers/video/fbdev/aty/atyfb_base.c b/drivers/video/fbdev/aty/atyfb_base.c
index b3463d1..0ccf5d4 100644
--- a/drivers/video/fbdev/aty/atyfb_base.c
+++ b/drivers/video/fbdev/aty/atyfb_base.c
@@ -3965,7 +3965,12 @@ static int __init atyfb_init(void)
int err1 = 1, err2 = 1;
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("atyfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("atyfb", &option))
return -ENODEV;
atyfb_setup(option);
diff --git a/drivers/video/fbdev/aty/radeon_base.c b/drivers/video/fbdev/aty/radeon_base.c
index 8b28c9b..6570642 100644
--- a/drivers/video/fbdev/aty/radeon_base.c
+++ b/drivers/video/fbdev/aty/radeon_base.c
@@ -2607,7 +2607,12 @@ static int __init radeonfb_init (void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("radeonfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("radeonfb", &option))
return -ENODEV;
radeonfb_setup(option);
diff --git a/drivers/video/fbdev/carminefb.c b/drivers/video/fbdev/carminefb.c
index 4651b48..4ae21db 100644
--- a/drivers/video/fbdev/carminefb.c
+++ b/drivers/video/fbdev/carminefb.c
@@ -773,6 +773,9 @@ static struct pci_driver carmine_pci_driver = {
static int __init carminefb_init(void)
{
+ if (fb_modesetting_disabled("carminefb"))
+ return -ENODEV;
+
if (!(fb_displays &
(CARMINE_USE_DISPLAY0 | CARMINE_USE_DISPLAY1))) {
printk(KERN_ERR "If you disable both displays than you don't "
diff --git a/drivers/video/fbdev/chipsfb.c b/drivers/video/fbdev/chipsfb.c
index f1c1c95..cc37ec3 100644
--- a/drivers/video/fbdev/chipsfb.c
+++ b/drivers/video/fbdev/chipsfb.c
@@ -506,6 +506,9 @@ static struct pci_driver chipsfb_driver = {
int __init chips_init(void)
{
+ if (fb_modesetting_disabled("chipsfb"))
+ return -ENODEV;
+
if (fb_get_options("chipsfb", NULL))
return -ENODEV;
diff --git a/drivers/video/fbdev/cirrusfb.c b/drivers/video/fbdev/cirrusfb.c
index b08bee4..ba45e21 100644
--- a/drivers/video/fbdev/cirrusfb.c
+++ b/drivers/video/fbdev/cirrusfb.c
@@ -2359,7 +2359,12 @@ static int __init cirrusfb_init(void)
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("cirrusfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("cirrusfb", &option))
return -ENODEV;
cirrusfb_setup(option);
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 1e70d8c..3a6c845 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -40,6 +40,7 @@
#include <asm/fb.h>
+#include <video/nomodeset.h>
#include <video/vga.h>
/*
@@ -1844,4 +1845,18 @@ int fb_new_modelist(struct fb_info *info)
return 0;
}
+#if defined(CONFIG_VIDEO_NOMODESET)
+bool fb_modesetting_disabled(const char *drvname)
+{
+ bool fwonly = video_firmware_drivers_only();
+
+ if (fwonly)
+ pr_warn("Driver %s not loading because of nomodeset parameter\n",
+ drvname);
+
+ return fwonly;
+}
+EXPORT_SYMBOL(fb_modesetting_disabled);
+#endif
+
MODULE_LICENSE("GPL");
diff --git a/drivers/video/fbdev/cyber2000fb.c b/drivers/video/fbdev/cyber2000fb.c
index 31ff1da..38c0a68 100644
--- a/drivers/video/fbdev/cyber2000fb.c
+++ b/drivers/video/fbdev/cyber2000fb.c
@@ -48,7 +48,6 @@
#include <linux/i2c.h>
#include <linux/i2c-algo-bit.h>
-
#ifdef __arm__
#include <asm/mach-types.h>
#endif
@@ -1878,7 +1877,12 @@ static int __init cyber2000fb_init(void)
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("CyberPro"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("cyber2000fb", &option))
return -ENODEV;
cyber2000fb_setup(option);
diff --git a/drivers/video/fbdev/geode/Kconfig b/drivers/video/fbdev/geode/Kconfig
index ac9c860..2f8f0fb 100644
--- a/drivers/video/fbdev/geode/Kconfig
+++ b/drivers/video/fbdev/geode/Kconfig
@@ -15,6 +15,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Framebuffer driver for the display controller integrated into the
AMD Geode LX processors.
@@ -30,6 +31,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Framebuffer driver for the display controller integrated into the
AMD Geode GX processors.
@@ -45,6 +47,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select VIDEO_NOMODESET
help
Framebuffer driver for the display controller integrated into the
AMD Geode GX1 processor.
diff --git a/drivers/video/fbdev/geode/gx1fb_core.c b/drivers/video/fbdev/geode/gx1fb_core.c
index 1514c65..9c94200 100644
--- a/drivers/video/fbdev/geode/gx1fb_core.c
+++ b/drivers/video/fbdev/geode/gx1fb_core.c
@@ -446,7 +446,12 @@ static int __init gx1fb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("gx1fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("gx1fb", &option))
return -ENODEV;
gx1fb_setup(option);
diff --git a/drivers/video/fbdev/geode/gxfb_core.c b/drivers/video/fbdev/geode/gxfb_core.c
index 2527bd8..8e05e76 100644
--- a/drivers/video/fbdev/geode/gxfb_core.c
+++ b/drivers/video/fbdev/geode/gxfb_core.c
@@ -511,7 +511,12 @@ static int __init gxfb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("gxfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("gxfb", &option))
return -ENODEV;
diff --git a/drivers/video/fbdev/geode/lxfb_core.c b/drivers/video/fbdev/geode/lxfb_core.c
index 9d26592..8130e9e 100644
--- a/drivers/video/fbdev/geode/lxfb_core.c
+++ b/drivers/video/fbdev/geode/lxfb_core.c
@@ -647,7 +647,12 @@ static int __init lxfb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("lxfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("lxfb", &option))
return -ENODEV;
diff --git a/drivers/video/fbdev/gxt4500.c b/drivers/video/fbdev/gxt4500.c
index 0dcef4b..5f42d3d 100644
--- a/drivers/video/fbdev/gxt4500.c
+++ b/drivers/video/fbdev/gxt4500.c
@@ -779,6 +779,9 @@ static struct pci_driver gxt4500_driver = {
static int gxt4500_init(void)
{
+ if (fb_modesetting_disabled("gxt4500"))
+ return -ENODEV;
+
#ifndef MODULE
if (fb_get_options("gxt4500", &mode_option))
return -ENODEV;
diff --git a/drivers/video/fbdev/hyperv_fb.c b/drivers/video/fbdev/hyperv_fb.c
index 3ce746a..fdbf02b 100644
--- a/drivers/video/fbdev/hyperv_fb.c
+++ b/drivers/video/fbdev/hyperv_fb.c
@@ -59,7 +59,6 @@
#include <linux/hyperv.h>
-
/* Hyper-V Synthetic Video Protocol definitions and structures */
#define MAX_VMBUS_PKT_SIZE 0x4000
@@ -1377,6 +1376,9 @@ static int __init hvfb_drv_init(void)
{
int ret;
+ if (fb_modesetting_disabled("hyper_fb"))
+ return -ENODEV;
+
ret = vmbus_driver_register(&hvfb_drv);
if (ret != 0)
return ret;
diff --git a/drivers/video/fbdev/i740fb.c b/drivers/video/fbdev/i740fb.c
index b795f65..3860b13 100644
--- a/drivers/video/fbdev/i740fb.c
+++ b/drivers/video/fbdev/i740fb.c
@@ -1285,7 +1285,12 @@ static int __init i740fb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("i740fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("i740fb", &option))
return -ENODEV;
i740fb_setup(option);
diff --git a/drivers/video/fbdev/i810/i810_main.c b/drivers/video/fbdev/i810/i810_main.c
index ff09f8c..85abb65 100644
--- a/drivers/video/fbdev/i810/i810_main.c
+++ b/drivers/video/fbdev/i810/i810_main.c
@@ -2143,6 +2143,9 @@ static int i810fb_init(void)
{
char *option = NULL;
+ if (fb_modesetting_disabled("i810fb"))
+ return -ENODEV;
+
if (fb_get_options("i810fb", &option))
return -ENODEV;
i810fb_setup(option);
@@ -2159,6 +2162,9 @@ static int i810fb_init(void)
static int i810fb_init(void)
{
+ if (fb_modesetting_disabled("i810fb"))
+ return -ENODEV;
+
hsync1 *= 1000;
hsync2 *= 1000;
diff --git a/drivers/video/fbdev/imsttfb.c b/drivers/video/fbdev/imsttfb.c
index d7edb9c..bea4564 100644
--- a/drivers/video/fbdev/imsttfb.c
+++ b/drivers/video/fbdev/imsttfb.c
@@ -1617,7 +1617,12 @@ static int __init imsttfb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("imsttfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("imsttfb", &option))
return -ENODEV;
diff --git a/drivers/video/fbdev/intelfb/intelfbdrv.c b/drivers/video/fbdev/intelfb/intelfbdrv.c
index d4a2891..0a9e506 100644
--- a/drivers/video/fbdev/intelfb/intelfbdrv.c
+++ b/drivers/video/fbdev/intelfb/intelfbdrv.c
@@ -389,6 +389,9 @@ static int __init intelfb_init(void)
if (idonly)
return -ENODEV;
+ if (fb_modesetting_disabled("intelfb"))
+ return -ENODEV;
+
#ifndef MODULE
if (fb_get_options("intelfb", &option))
return -ENODEV;
diff --git a/drivers/video/fbdev/kyro/fbdev.c b/drivers/video/fbdev/kyro/fbdev.c
index b4b9305..0596573 100644
--- a/drivers/video/fbdev/kyro/fbdev.c
+++ b/drivers/video/fbdev/kyro/fbdev.c
@@ -789,7 +789,12 @@ static int __init kyrofb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("kyrofb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("kyrofb", &option))
return -ENODEV;
kyrofb_setup(option);
diff --git a/drivers/video/fbdev/matrox/matroxfb_base.c b/drivers/video/fbdev/matrox/matroxfb_base.c
index 775d341..0d3cee7 100644
--- a/drivers/video/fbdev/matrox/matroxfb_base.c
+++ b/drivers/video/fbdev/matrox/matroxfb_base.c
@@ -2314,6 +2314,9 @@ static void __init matroxfb_init_params(void) {
static int __init matrox_init(void) {
int err;
+ if (fb_modesetting_disabled("matroxfb"))
+ return -ENODEV;
+
matroxfb_init_params();
err = pci_register_driver(&matroxfb_driver);
dev = -1; /* accept all new devices... */
diff --git a/drivers/video/fbdev/mb862xx/mb862xxfbdrv.c b/drivers/video/fbdev/mb862xx/mb862xxfbdrv.c
index 90c79e8..a236fc9 100644
--- a/drivers/video/fbdev/mb862xx/mb862xxfbdrv.c
+++ b/drivers/video/fbdev/mb862xx/mb862xxfbdrv.c
@@ -1181,6 +1181,9 @@ static int mb862xxfb_init(void)
{
int ret = -ENODEV;
+ if (fb_modesetting_disabled(DRV_NAME))
+ return -ENODEV;
+
#if defined(CONFIG_FB_MB862XX_LIME)
ret = platform_driver_register(&of_platform_mb862xxfb_driver);
#endif
diff --git a/drivers/video/fbdev/neofb.c b/drivers/video/fbdev/neofb.c
index 93a2d2d..39d8cde 100644
--- a/drivers/video/fbdev/neofb.c
+++ b/drivers/video/fbdev/neofb.c
@@ -2209,7 +2209,12 @@ static int __init neofb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("neofb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("neofb", &option))
return -ENODEV;
neofb_setup(option);
diff --git a/drivers/video/fbdev/nvidia/nvidia.c b/drivers/video/fbdev/nvidia/nvidia.c
index 329e2e8..1960916 100644
--- a/drivers/video/fbdev/nvidia/nvidia.c
+++ b/drivers/video/fbdev/nvidia/nvidia.c
@@ -1521,7 +1521,12 @@ static int nvidiafb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("nvidiafb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("nvidiafb", &option))
return -ENODEV;
nvidiafb_setup(option);
diff --git a/drivers/video/fbdev/pm2fb.c b/drivers/video/fbdev/pm2fb.c
index 7da715d..0823c9de8 100644
--- a/drivers/video/fbdev/pm2fb.c
+++ b/drivers/video/fbdev/pm2fb.c
@@ -1799,7 +1799,12 @@ static int __init pm2fb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("pm2fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("pm2fb", &option))
return -ENODEV;
pm2fb_setup(option);
diff --git a/drivers/video/fbdev/pm3fb.c b/drivers/video/fbdev/pm3fb.c
index ba69846..b46a471 100644
--- a/drivers/video/fbdev/pm3fb.c
+++ b/drivers/video/fbdev/pm3fb.c
@@ -1540,7 +1540,12 @@ static int __init pm3fb_init(void)
*/
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("pm3fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("pm3fb", &option))
return -ENODEV;
pm3fb_setup(option);
diff --git a/drivers/video/fbdev/pvr2fb.c b/drivers/video/fbdev/pvr2fb.c
index b73ad14..6888127 100644
--- a/drivers/video/fbdev/pvr2fb.c
+++ b/drivers/video/fbdev/pvr2fb.c
@@ -1082,7 +1082,12 @@ static int __init pvr2fb_init(void)
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("pvr2fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("pvr2fb", &option))
return -ENODEV;
pvr2fb_setup(option);
diff --git a/drivers/video/fbdev/riva/fbdev.c b/drivers/video/fbdev/riva/fbdev.c
index 0ea74e2..6442781 100644
--- a/drivers/video/fbdev/riva/fbdev.c
+++ b/drivers/video/fbdev/riva/fbdev.c
@@ -2165,7 +2165,12 @@ static int rivafb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("rivafb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("rivafb", &option))
return -ENODEV;
rivafb_setup(option);
diff --git a/drivers/video/fbdev/s3fb.c b/drivers/video/fbdev/s3fb.c
index 7713274..7d25748 100644
--- a/drivers/video/fbdev/s3fb.c
+++ b/drivers/video/fbdev/s3fb.c
@@ -1558,7 +1558,12 @@ static int __init s3fb_init(void)
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("s3fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("s3fb", &option))
return -ENODEV;
s3fb_setup(option);
diff --git a/drivers/video/fbdev/savage/savagefb_driver.c b/drivers/video/fbdev/savage/savagefb_driver.c
index b7818b6..4a27b68 100644
--- a/drivers/video/fbdev/savage/savagefb_driver.c
+++ b/drivers/video/fbdev/savage/savagefb_driver.c
@@ -2556,6 +2556,9 @@ static int __init savagefb_init(void)
DBG("savagefb_init");
+ if (fb_modesetting_disabled("savagefb"))
+ return -ENODEV;
+
if (fb_get_options("savagefb", &option))
return -ENODEV;
diff --git a/drivers/video/fbdev/sis/sis_main.c b/drivers/video/fbdev/sis/sis_main.c
index 1c197c3..cfba776 100644
--- a/drivers/video/fbdev/sis/sis_main.c
+++ b/drivers/video/fbdev/sis/sis_main.c
@@ -6588,7 +6588,12 @@ static int __init sisfb_init(void)
{
#ifndef MODULE
char *options = NULL;
+#endif
+ if (fb_modesetting_disabled("sisfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if(fb_get_options("sisfb", &options))
return -ENODEV;
diff --git a/drivers/video/fbdev/skeletonfb.c b/drivers/video/fbdev/skeletonfb.c
index a10f105..40c130a 100644
--- a/drivers/video/fbdev/skeletonfb.c
+++ b/drivers/video/fbdev/skeletonfb.c
@@ -987,7 +987,12 @@ static int __init xxxfb_init(void)
*/
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("xxxfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("xxxfb", &option))
return -ENODEV;
xxxfb_setup(option);
diff --git a/drivers/video/fbdev/sm712fb.c b/drivers/video/fbdev/sm712fb.c
index 3baf336..b528776 100644
--- a/drivers/video/fbdev/sm712fb.c
+++ b/drivers/video/fbdev/sm712fb.c
@@ -1756,6 +1756,9 @@ static int __init sm712fb_init(void)
{
char *option = NULL;
+ if (fb_modesetting_disabled("sm712fb"))
+ return -ENODEV;
+
if (fb_get_options("sm712fb", &option))
return -ENODEV;
if (option && *option)
diff --git a/drivers/video/fbdev/sstfb.c b/drivers/video/fbdev/sstfb.c
index a56b242..da296b2a 100644
--- a/drivers/video/fbdev/sstfb.c
+++ b/drivers/video/fbdev/sstfb.c
@@ -1503,6 +1503,9 @@ static int sstfb_init(void)
{
char *option = NULL;
+ if (fb_modesetting_disabled("sstfb"))
+ return -ENODEV;
+
if (fb_get_options("sstfb", &option))
return -ENODEV;
sstfb_setup(option);
diff --git a/drivers/video/fbdev/sunxvr2500.c b/drivers/video/fbdev/sunxvr2500.c
index f405952..2cab4b9 100644
--- a/drivers/video/fbdev/sunxvr2500.c
+++ b/drivers/video/fbdev/sunxvr2500.c
@@ -247,6 +247,9 @@ static struct pci_driver s3d_driver = {
static int __init s3d_init(void)
{
+ if (fb_modesetting_disabled("s3d"))
+ return -ENODEV;
+
if (fb_get_options("s3d", NULL))
return -ENODEV;
diff --git a/drivers/video/fbdev/sunxvr500.c b/drivers/video/fbdev/sunxvr500.c
index b0c8cf0..6ec358a 100644
--- a/drivers/video/fbdev/sunxvr500.c
+++ b/drivers/video/fbdev/sunxvr500.c
@@ -430,6 +430,9 @@ static struct pci_driver e3d_driver = {
static int __init e3d_init(void)
{
+ if (fb_modesetting_disabled("e3d"))
+ return -ENODEV;
+
if (fb_get_options("e3d", NULL))
return -ENODEV;
diff --git a/drivers/video/fbdev/tdfxfb.c b/drivers/video/fbdev/tdfxfb.c
index 592a913..d17e5e1 100644
--- a/drivers/video/fbdev/tdfxfb.c
+++ b/drivers/video/fbdev/tdfxfb.c
@@ -1632,7 +1632,12 @@ static int __init tdfxfb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("tdfxfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("tdfxfb", &option))
return -ENODEV;
diff --git a/drivers/video/fbdev/tgafb.c b/drivers/video/fbdev/tgafb.c
index 251dbd2..14d37c4 100644
--- a/drivers/video/fbdev/tgafb.c
+++ b/drivers/video/fbdev/tgafb.c
@@ -1597,7 +1597,12 @@ static int tgafb_init(void)
int status;
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("tgafb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("tgafb", &option))
return -ENODEV;
tgafb_setup(option);
diff --git a/drivers/video/fbdev/tridentfb.c b/drivers/video/fbdev/tridentfb.c
index 219ce72..6099b97 100644
--- a/drivers/video/fbdev/tridentfb.c
+++ b/drivers/video/fbdev/tridentfb.c
@@ -1811,7 +1811,12 @@ static int __init tridentfb_init(void)
{
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("tridentfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("tridentfb", &option))
return -ENODEV;
tridentfb_setup(option);
diff --git a/drivers/video/fbdev/vermilion/vermilion.c b/drivers/video/fbdev/vermilion/vermilion.c
index 82b36db..1465fb7 100644
--- a/drivers/video/fbdev/vermilion/vermilion.c
+++ b/drivers/video/fbdev/vermilion/vermilion.c
@@ -1057,7 +1057,12 @@ static int __init vmlfb_init(void)
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("vmlfb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options(MODULE_NAME, &option))
return -ENODEV;
#endif
diff --git a/drivers/video/fbdev/via/via-core.c b/drivers/video/fbdev/via/via-core.c
index 2ee8fca..32a6399 100644
--- a/drivers/video/fbdev/via/via-core.c
+++ b/drivers/video/fbdev/via/via-core.c
@@ -725,6 +725,9 @@ static int __init via_core_init(void)
{
int ret;
+ if (fb_modesetting_disabled("viafb"))
+ return -ENODEV;
+
ret = viafb_init();
if (ret)
return ret;
diff --git a/drivers/video/fbdev/vt8623fb.c b/drivers/video/fbdev/vt8623fb.c
index 49b9f148..034333e 100644
--- a/drivers/video/fbdev/vt8623fb.c
+++ b/drivers/video/fbdev/vt8623fb.c
@@ -920,7 +920,12 @@ static int __init vt8623fb_init(void)
#ifndef MODULE
char *option = NULL;
+#endif
+ if (fb_modesetting_disabled("vt8623fb"))
+ return -ENODEV;
+
+#ifndef MODULE
if (fb_get_options("vt8623fb", &option))
return -ENODEV;
diff --git a/drivers/gpu/drm/drm_nomodeset.c b/drivers/video/nomodeset.c
similarity index 62%
rename from drivers/gpu/drm/drm_nomodeset.c
rename to drivers/video/nomodeset.c
index f3978d5..13cc8b7 100644
--- a/drivers/gpu/drm/drm_nomodeset.c
+++ b/drivers/video/nomodeset.c
@@ -3,17 +3,19 @@
#include <linux/module.h>
#include <linux/types.h>
-static bool drm_nomodeset;
+#include <video/nomodeset.h>
-bool drm_firmware_drivers_only(void)
+static bool video_nomodeset;
+
+bool video_firmware_drivers_only(void)
{
- return drm_nomodeset;
+ return video_nomodeset;
}
-EXPORT_SYMBOL(drm_firmware_drivers_only);
+EXPORT_SYMBOL(video_firmware_drivers_only);
static int __init disable_modeset(char *str)
{
- drm_nomodeset = true;
+ video_nomodeset = true;
pr_warn("Booted with the nomodeset parameter. Only the system framebuffer will be available\n");
diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index 940e5e9..4440e62 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -600,7 +600,7 @@ dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct device *dev,
gntdev_dmabuf->u.imp.attach = attach;
- sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
+ sgt = dma_buf_map_attachment_unlocked(attach, DMA_BIDIRECTIONAL);
if (IS_ERR(sgt)) {
ret = ERR_CAST(sgt);
goto fail_detach;
@@ -658,7 +658,7 @@ dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct device *dev,
fail_end_access:
dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs, count);
fail_unmap:
- dma_buf_unmap_attachment(attach, sgt, DMA_BIDIRECTIONAL);
+ dma_buf_unmap_attachment_unlocked(attach, sgt, DMA_BIDIRECTIONAL);
fail_detach:
dma_buf_detach(dma_buf, attach);
fail_free_obj:
@@ -708,8 +708,8 @@ static int dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
attach = gntdev_dmabuf->u.imp.attach;
if (gntdev_dmabuf->u.imp.sgt)
- dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
- DMA_BIDIRECTIONAL);
+ dma_buf_unmap_attachment_unlocked(attach, gntdev_dmabuf->u.imp.sgt,
+ DMA_BIDIRECTIONAL);
dma_buf = attach->dmabuf;
dma_buf_detach(attach->dmabuf, attach);
dma_buf_put(dma_buf);
diff --git a/include/drm/drm_accel.h b/include/drm/drm_accel.h
new file mode 100644
index 0000000..65c0affb
--- /dev/null
+++ b/include/drm/drm_accel.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright 2022 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ *
+ */
+
+#ifndef DRM_ACCEL_H_
+#define DRM_ACCEL_H_
+
+#include <drm/drm_file.h>
+
+#define ACCEL_MAJOR 261
+#define ACCEL_MAX_MINORS 256
+
+/**
+ * DRM_ACCEL_FOPS - Default drm accelerators file operations
+ *
+ * This macro provides a shorthand for setting the accelerator file ops in the
+ * &file_operations structure. If all you need are the default ops, use
+ * DEFINE_DRM_ACCEL_FOPS instead.
+ */
+#define DRM_ACCEL_FOPS \
+ .open = accel_open,\
+ .release = drm_release,\
+ .unlocked_ioctl = drm_ioctl,\
+ .compat_ioctl = drm_compat_ioctl,\
+ .poll = drm_poll,\
+ .read = drm_read,\
+ .llseek = noop_llseek
+
+/**
+ * DEFINE_DRM_ACCEL_FOPS() - macro to generate file operations for accelerators drivers
+ * @name: name for the generated structure
+ *
+ * This macro autogenerates a suitable &struct file_operations for accelerators based
+ * drivers, which can be assigned to &drm_driver.fops. Note that this structure
+ * cannot be shared between drivers, because it contains a reference to the
+ * current module using THIS_MODULE.
+ *
+ * Note that the declaration is already marked as static - if you need a
+ * non-static version of this you're probably doing it wrong and will break the
+ * THIS_MODULE reference by accident.
+ */
+#define DEFINE_DRM_ACCEL_FOPS(name) \
+ static const struct file_operations name = {\
+ .owner = THIS_MODULE,\
+ DRM_ACCEL_FOPS,\
+ }
+
+#if IS_ENABLED(CONFIG_DRM_ACCEL)
+
+void accel_core_exit(void);
+int accel_core_init(void);
+void accel_minor_remove(int index);
+int accel_minor_alloc(void);
+void accel_minor_replace(struct drm_minor *minor, int index);
+void accel_set_device_instance_params(struct device *kdev, int index);
+int accel_open(struct inode *inode, struct file *filp);
+void accel_debugfs_init(struct drm_minor *minor, int minor_id);
+
+#else
+
+static inline void accel_core_exit(void)
+{
+}
+
+static inline int __init accel_core_init(void)
+{
+ /* Return 0 to allow drm_core_init to complete successfully */
+ return 0;
+}
+
+static inline void accel_minor_remove(int index)
+{
+}
+
+static inline int accel_minor_alloc(void)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void accel_minor_replace(struct drm_minor *minor, int index)
+{
+}
+
+static inline void accel_set_device_instance_params(struct device *kdev, int index)
+{
+}
+
+static inline void accel_debugfs_init(struct drm_minor *minor, int minor_id)
+{
+}
+
+#endif /* IS_ENABLED(CONFIG_DRM_ACCEL) */
+
+#endif /* DRM_ACCEL_H_ */
diff --git a/include/drm/drm_atomic_helper.h b/include/drm/drm_atomic_helper.h
index 06d8902..33f982c 100644
--- a/include/drm/drm_atomic_helper.h
+++ b/include/drm/drm_atomic_helper.h
@@ -58,10 +58,9 @@ int drm_atomic_helper_check_plane_state(struct drm_plane_state *plane_state,
int max_scale,
bool can_position,
bool can_update_disabled);
-int drm_atomic_helper_check_crtc_state(struct drm_crtc_state *crtc_state,
- bool can_disable_primary_plane);
int drm_atomic_helper_check_planes(struct drm_device *dev,
struct drm_atomic_state *state);
+int drm_atomic_helper_check_crtc_primary_plane(struct drm_crtc_state *crtc_state);
int drm_atomic_helper_check(struct drm_device *dev,
struct drm_atomic_state *state);
void drm_atomic_helper_commit_tail(struct drm_atomic_state *state);
diff --git a/include/drm/drm_atomic_state_helper.h b/include/drm/drm_atomic_state_helper.h
index 3f8f1d6..1927666 100644
--- a/include/drm/drm_atomic_state_helper.h
+++ b/include/drm/drm_atomic_state_helper.h
@@ -70,7 +70,7 @@ void __drm_atomic_helper_connector_state_reset(struct drm_connector_state *conn_
void __drm_atomic_helper_connector_reset(struct drm_connector *connector,
struct drm_connector_state *conn_state);
void drm_atomic_helper_connector_reset(struct drm_connector *connector);
-void drm_atomic_helper_connector_tv_reset(struct drm_connector *connector);
+void drm_atomic_helper_connector_tv_margins_reset(struct drm_connector *connector);
void
__drm_atomic_helper_connector_duplicate_state(struct drm_connector *connector,
struct drm_connector_state *state);
diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
index 56aee94..565cf9d 100644
--- a/include/drm/drm_connector.h
+++ b/include/drm/drm_connector.h
@@ -656,6 +656,12 @@ struct drm_display_info {
* @mso_pixel_overlap: eDP MSO segment pixel overlap, 0-8 pixels.
*/
u8 mso_pixel_overlap;
+
+ /**
+ * @max_dsc_bpp: Maximum DSC target bitrate, if it is set to 0 the
+ * monitor's default value is used instead.
+ */
+ u32 max_dsc_bpp;
};
int drm_display_info_set_bus_formats(struct drm_display_info *info,
@@ -692,7 +698,8 @@ struct drm_connector_tv_margins {
/**
* struct drm_tv_connector_state - TV connector related states
- * @subconnector: selected subconnector
+ * @select_subconnector: selected subconnector
+ * @subconnector: detected subconnector
* @margins: TV margins
* @mode: TV mode
* @brightness: brightness in percent
@@ -703,6 +710,7 @@ struct drm_connector_tv_margins {
* @hue: hue in percent
*/
struct drm_tv_connector_state {
+ enum drm_mode_subconnector select_subconnector;
enum drm_mode_subconnector subconnector;
struct drm_connector_tv_margins margins;
unsigned int mode;
@@ -1211,6 +1219,13 @@ struct drm_cmdline_mode {
bool bpp_specified;
/**
+ * @pixel_clock:
+ *
+ * Pixel Clock in kHz. Optional.
+ */
+ unsigned int pixel_clock;
+
+ /**
* @xres:
*
* Active resolution on the X axis, in pixels.
@@ -1548,12 +1563,20 @@ struct drm_connector {
struct drm_cmdline_mode cmdline_mode;
/** @force: a DRM_FORCE_<foo> state for forced mode sets */
enum drm_connector_force force;
+
/**
- * @override_edid: has the EDID been overwritten through debugfs for
- * testing? Do not modify outside of drm_edid_override_set() and
- * drm_edid_override_reset().
+ * @edid_override: Override EDID set via debugfs.
+ *
+ * Do not modify or access outside of the drm_edid_override_* family of
+ * functions.
*/
- bool override_edid;
+ const struct drm_edid *edid_override;
+
+ /**
+ * @edid_override_mutex: Protect access to edid_override.
+ */
+ struct mutex edid_override_mutex;
+
/** @epoch_counter: used to detect any other changes in connector, besides status */
u64 epoch_counter;
diff --git a/include/drm/drm_crtc_helper.h b/include/drm/drm_crtc_helper.h
index a6d520d..1840db2 100644
--- a/include/drm/drm_crtc_helper.h
+++ b/include/drm/drm_crtc_helper.h
@@ -50,6 +50,8 @@ bool drm_crtc_helper_set_mode(struct drm_crtc *crtc,
struct drm_display_mode *mode,
int x, int y,
struct drm_framebuffer *old_fb);
+int drm_crtc_helper_atomic_check(struct drm_crtc *crtc,
+ struct drm_atomic_state *state);
bool drm_helper_crtc_in_use(struct drm_crtc *crtc);
bool drm_helper_encoder_in_use(struct drm_encoder *encoder);
diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
index 9923c7a..933ce20 100644
--- a/include/drm/drm_device.h
+++ b/include/drm/drm_device.h
@@ -93,6 +93,9 @@ struct drm_device {
/** @render: Render node */
struct drm_minor *render;
+ /** @accel: Compute Acceleration node */
+ struct drm_minor *accel;
+
/**
* @registered:
*
diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
index f6159ac..d7c521e 100644
--- a/include/drm/drm_drv.h
+++ b/include/drm/drm_drv.h
@@ -30,6 +30,8 @@
#include <linux/list.h>
#include <linux/irqreturn.h>
+#include <video/nomodeset.h>
+
#include <drm/drm_device.h>
struct drm_file;
@@ -94,6 +96,14 @@ enum drm_driver_feature {
* synchronization of command submission.
*/
DRIVER_SYNCOBJ_TIMELINE = BIT(6),
+ /**
+ * @DRIVER_COMPUTE_ACCEL:
+ *
+ * Driver supports compute acceleration devices. This flag is mutually exclusive with
+ * @DRIVER_RENDER and @DRIVER_MODESET. Devices that support both graphics and compute
+ * acceleration should be handled by two drivers that are connected using auxiliary bus.
+ */
+ DRIVER_COMPUTE_ACCEL = BIT(7),
/* IMPORTANT: Below are all the legacy flags, add new ones above. */
@@ -602,6 +612,10 @@ static inline bool drm_drv_uses_atomic_modeset(struct drm_device *dev)
int drm_dev_set_unique(struct drm_device *dev, const char *name);
-extern bool drm_firmware_drivers_only(void);
+/* TODO: Inline drm_firmware_drivers_only() in all its callers. */
+static inline bool drm_firmware_drivers_only(void)
+{
+ return video_firmware_drivers_only();
+}
#endif
diff --git a/include/drm/drm_edid.h b/include/drm/drm_edid.h
index 1ed61e2..3729636 100644
--- a/include/drm/drm_edid.h
+++ b/include/drm/drm_edid.h
@@ -97,10 +97,13 @@ struct detailed_data_string {
#define DRM_EDID_RANGE_OFFSET_MIN_HFREQ (1 << 2) /* 1.4 */
#define DRM_EDID_RANGE_OFFSET_MAX_HFREQ (1 << 3) /* 1.4 */
-#define DRM_EDID_DEFAULT_GTF_SUPPORT_FLAG 0x00
-#define DRM_EDID_RANGE_LIMITS_ONLY_FLAG 0x01
-#define DRM_EDID_SECONDARY_GTF_SUPPORT_FLAG 0x02
-#define DRM_EDID_CVT_SUPPORT_FLAG 0x04
+#define DRM_EDID_DEFAULT_GTF_SUPPORT_FLAG 0x00 /* 1.3 */
+#define DRM_EDID_RANGE_LIMITS_ONLY_FLAG 0x01 /* 1.4 */
+#define DRM_EDID_SECONDARY_GTF_SUPPORT_FLAG 0x02 /* 1.3 */
+#define DRM_EDID_CVT_SUPPORT_FLAG 0x04 /* 1.4 */
+
+#define DRM_EDID_CVT_FLAGS_STANDARD_BLANKING (1 << 3)
+#define DRM_EDID_CVT_FLAGS_REDUCED_BLANKING (1 << 4)
struct detailed_data_monitor_range {
u8 min_vfreq;
@@ -206,7 +209,8 @@ struct detailed_timing {
#define DRM_EDID_DIGITAL_TYPE_DP (5 << 0) /* 1.4 */
#define DRM_EDID_DIGITAL_DFP_1_X (1 << 0) /* 1.3 */
-#define DRM_EDID_FEATURE_DEFAULT_GTF (1 << 0)
+#define DRM_EDID_FEATURE_DEFAULT_GTF (1 << 0) /* 1.2 */
+#define DRM_EDID_FEATURE_CONTINUOUS_FREQ (1 << 0) /* 1.4 */
#define DRM_EDID_FEATURE_PREFERRED_TIMING (1 << 1)
#define DRM_EDID_FEATURE_STANDARD_COLOR (1 << 2)
/* If analog */
@@ -384,15 +388,8 @@ int drm_av_sync_delay(struct drm_connector *connector,
const struct drm_display_mode *mode);
#ifdef CONFIG_DRM_LOAD_EDID_FIRMWARE
-struct edid *drm_load_edid_firmware(struct drm_connector *connector);
int __drm_set_edid_firmware_path(const char *path);
int __drm_get_edid_firmware_path(char *buf, size_t bufsize);
-#else
-static inline struct edid *
-drm_load_edid_firmware(struct drm_connector *connector)
-{
- return ERR_PTR(-ENOENT);
-}
#endif
bool drm_edid_are_equal(const struct edid *edid1, const struct edid *edid2);
@@ -573,7 +570,7 @@ struct edid *drm_get_edid_switcheroo(struct drm_connector *connector,
struct i2c_adapter *adapter);
struct edid *drm_edid_duplicate(const struct edid *edid);
int drm_add_edid_modes(struct drm_connector *connector, struct edid *edid);
-int drm_add_override_edid_modes(struct drm_connector *connector);
+int drm_edid_override_connector_update(struct drm_connector *connector);
u8 drm_match_cea_mode(const struct drm_display_mode *to_match);
bool drm_detect_hdmi_monitor(const struct edid *edid);
@@ -602,6 +599,7 @@ drm_display_mode_from_cea_vic(struct drm_device *dev,
const struct drm_edid *drm_edid_alloc(const void *edid, size_t size);
const struct drm_edid *drm_edid_dup(const struct drm_edid *drm_edid);
void drm_edid_free(const struct drm_edid *drm_edid);
+bool drm_edid_valid(const struct drm_edid *drm_edid);
const struct edid *drm_edid_raw(const struct drm_edid *drm_edid);
const struct drm_edid *drm_edid_read(struct drm_connector *connector);
const struct drm_edid *drm_edid_read_ddc(struct drm_connector *connector,
diff --git a/include/drm/drm_fb_helper.h b/include/drm/drm_fb_helper.h
index fddd0d1..b111dc7a 100644
--- a/include/drm/drm_fb_helper.h
+++ b/include/drm/drm_fb_helper.h
@@ -30,13 +30,12 @@
#ifndef DRM_FB_HELPER_H
#define DRM_FB_HELPER_H
+struct drm_clip_rect;
struct drm_fb_helper;
-#include <drm/drm_client.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_device.h>
#include <linux/fb.h>
-#include <linux/kgdb.h>
+
+#include <drm/drm_client.h>
enum mode_set_atomic {
LEAVE_ATOMIC_MODE_SET,
@@ -91,6 +90,20 @@ struct drm_fb_helper_funcs {
*/
int (*fb_probe)(struct drm_fb_helper *helper,
struct drm_fb_helper_surface_size *sizes);
+
+ /**
+ * @fb_dirty:
+ *
+ * Driver callback to update the framebuffer memory. If set, fbdev
+ * emulation will invoke this callback in regular intervals after
+ * the framebuffer has been written.
+ *
+ * This callback is optional.
+ *
+ * Returns:
+ * 0 on success, or an error code otherwise.
+ */
+ int (*fb_dirty)(struct drm_fb_helper *helper, struct drm_clip_rect *clip);
};
/**
@@ -98,7 +111,7 @@ struct drm_fb_helper_funcs {
* @fb: Scanout framebuffer object
* @dev: DRM device
* @funcs: driver callbacks for fb helper
- * @fbdev: emulated fbdev device info struct
+ * @info: emulated fbdev device info struct
* @pseudo_palette: fake palette of 16 colors
* @damage_clip: clip rectangle used with deferred_io to accumulate damage to
* the screen buffer
@@ -129,7 +142,7 @@ struct drm_fb_helper {
struct drm_framebuffer *fb;
struct drm_device *dev;
const struct drm_fb_helper_funcs *funcs;
- struct fb_info *fbdev;
+ struct fb_info *info;
u32 pseudo_palette[17];
struct drm_clip_rect damage_clip;
spinlock_t damage_lock;
@@ -186,6 +199,15 @@ struct drm_fb_helper {
* See also: @deferred_setup
*/
int preferred_bpp;
+
+ /**
+ * @hint_leak_smem_start:
+ *
+ * Hint to the fbdev emulation to store the framebuffer's physical
+ * address in struct &fb_info.fix.smem_start. If the hint is unset,
+ * the smem_start field should always be cleared to zero.
+ */
+ bool hint_leak_smem_start;
};
static inline struct drm_fb_helper *
@@ -224,8 +246,8 @@ int drm_fb_helper_check_var(struct fb_var_screeninfo *var,
int drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper);
-struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper);
-void drm_fb_helper_unregister_fbi(struct drm_fb_helper *fb_helper);
+struct fb_info *drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper);
+void drm_fb_helper_unregister_info(struct drm_fb_helper *fb_helper);
void drm_fb_helper_fill_info(struct fb_info *info,
struct drm_fb_helper *fb_helper,
struct drm_fb_helper_surface_size *sizes);
@@ -244,6 +266,11 @@ void drm_fb_helper_sys_copyarea(struct fb_info *info,
void drm_fb_helper_sys_imageblit(struct fb_info *info,
const struct fb_image *image);
+ssize_t drm_fb_helper_cfb_read(struct fb_info *info, char __user *buf,
+ size_t count, loff_t *ppos);
+ssize_t drm_fb_helper_cfb_write(struct fb_info *info, const char __user *buf,
+ size_t count, loff_t *ppos);
+
void drm_fb_helper_cfb_fillrect(struct fb_info *info,
const struct fb_fillrect *rect);
void drm_fb_helper_cfb_copyarea(struct fb_info *info,
@@ -267,9 +294,6 @@ int drm_fb_helper_debug_leave(struct fb_info *info);
void drm_fb_helper_lastclose(struct drm_device *dev);
void drm_fb_helper_output_poll_changed(struct drm_device *dev);
-
-void drm_fbdev_generic_setup(struct drm_device *dev,
- unsigned int preferred_bpp);
#else
static inline void drm_fb_helper_prepare(struct drm_device *dev,
struct drm_fb_helper *helper,
@@ -322,12 +346,12 @@ drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper)
}
static inline struct fb_info *
-drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
+drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper)
{
return NULL;
}
-static inline void drm_fb_helper_unregister_fbi(struct drm_fb_helper *fb_helper)
+static inline void drm_fb_helper_unregister_info(struct drm_fb_helper *fb_helper)
{
}
@@ -389,6 +413,18 @@ static inline void drm_fb_helper_sys_imageblit(struct fb_info *info,
{
}
+static inline ssize_t drm_fb_helper_cfb_read(struct fb_info *info, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ return -ENODEV;
+}
+
+static inline ssize_t drm_fb_helper_cfb_write(struct fb_info *info, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ return -ENODEV;
+}
+
static inline void drm_fb_helper_cfb_fillrect(struct fb_info *info,
const struct fb_fillrect *rect)
{
@@ -442,12 +478,6 @@ static inline void drm_fb_helper_lastclose(struct drm_device *dev)
static inline void drm_fb_helper_output_poll_changed(struct drm_device *dev)
{
}
-
-static inline void
-drm_fbdev_generic_setup(struct drm_device *dev, unsigned int preferred_bpp)
-{
-}
-
#endif
#endif
diff --git a/include/drm/drm_fbdev_generic.h b/include/drm/drm_fbdev_generic.h
new file mode 100644
index 0000000..7579934
--- /dev/null
+++ b/include/drm/drm_fbdev_generic.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: MIT */
+
+#ifndef DRM_FBDEV_GENERIC_H
+#define DRM_FBDEV_GENERIC_H
+
+struct drm_device;
+
+#ifdef CONFIG_DRM_FBDEV_EMULATION
+void drm_fbdev_generic_setup(struct drm_device *dev, unsigned int preferred_bpp);
+#else
+static inline void drm_fbdev_generic_setup(struct drm_device *dev, unsigned int preferred_bpp)
+{ }
+#endif
+
+#endif
diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h
index d780fd1..0d1f853 100644
--- a/include/drm/drm_file.h
+++ b/include/drm/drm_file.h
@@ -51,11 +51,15 @@ struct file;
/* Note that the order of this enum is ABI (it determines
* /dev/dri/renderD* numbers).
+ *
+ * Setting DRM_MINOR_ACCEL to 32 gives enough space for more drm minors to
+ * be implemented before we hit any future
*/
enum drm_minor_type {
DRM_MINOR_PRIMARY,
DRM_MINOR_CONTROL,
DRM_MINOR_RENDER,
+ DRM_MINOR_ACCEL = 32,
};
/**
@@ -70,7 +74,7 @@ enum drm_minor_type {
struct drm_minor {
/* private: */
int index; /* Minor device number */
- int type; /* Control or render */
+ int type; /* Control or render or accel */
struct device *kdev; /* Linux device */
struct drm_device *dev;
@@ -397,7 +401,22 @@ static inline bool drm_is_render_client(const struct drm_file *file_priv)
return file_priv->minor->type == DRM_MINOR_RENDER;
}
+/**
+ * drm_is_accel_client - is this an open file of the compute acceleration node
+ * @file_priv: DRM file
+ *
+ * Returns true if this is an open file of the compute acceleration node, i.e.
+ * &drm_file.minor of @file_priv is a accel minor.
+ *
+ * See also the :ref:`section on accel nodes <drm_accel_node>`.
+ */
+static inline bool drm_is_accel_client(const struct drm_file *file_priv)
+{
+ return file_priv->minor->type == DRM_MINOR_ACCEL;
+}
+
int drm_open(struct inode *inode, struct file *filp);
+int drm_open_helper(struct file *filp, struct drm_minor *minor);
ssize_t drm_read(struct file *filp, char __user *buffer,
size_t count, loff_t *offset);
int drm_release(struct inode *inode, struct file *filp);
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index bd42f25..a17c2f9 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -457,6 +457,9 @@ struct page **drm_gem_get_pages(struct drm_gem_object *obj);
void drm_gem_put_pages(struct drm_gem_object *obj, struct page **pages,
bool dirty, bool accessed);
+int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
+void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
+
int drm_gem_objects_lookup(struct drm_file *filp, void __user *bo_handles,
int count, struct drm_gem_object ***objs_out);
struct drm_gem_object *drm_gem_object_lookup(struct drm_file *filp, u32 handle);
diff --git a/include/drm/drm_gem_atomic_helper.h b/include/drm/drm_gem_atomic_helper.h
index 6e3319e..6970ccb 100644
--- a/include/drm/drm_gem_atomic_helper.h
+++ b/include/drm/drm_gem_atomic_helper.h
@@ -103,8 +103,8 @@ void drm_gem_destroy_shadow_plane_state(struct drm_plane *plane,
.atomic_duplicate_state = drm_gem_duplicate_shadow_plane_state, \
.atomic_destroy_state = drm_gem_destroy_shadow_plane_state
-int drm_gem_prepare_shadow_fb(struct drm_plane *plane, struct drm_plane_state *plane_state);
-void drm_gem_cleanup_shadow_fb(struct drm_plane *plane, struct drm_plane_state *plane_state);
+int drm_gem_begin_shadow_fb_access(struct drm_plane *plane, struct drm_plane_state *plane_state);
+void drm_gem_end_shadow_fb_access(struct drm_plane *plane, struct drm_plane_state *plane_state);
/**
* DRM_GEM_SHADOW_PLANE_HELPER_FUNCS -
@@ -115,13 +115,13 @@ void drm_gem_cleanup_shadow_fb(struct drm_plane *plane, struct drm_plane_state *
* functions.
*/
#define DRM_GEM_SHADOW_PLANE_HELPER_FUNCS \
- .prepare_fb = drm_gem_prepare_shadow_fb, \
- .cleanup_fb = drm_gem_cleanup_shadow_fb
+ .begin_fb_access = drm_gem_begin_shadow_fb_access, \
+ .end_fb_access = drm_gem_end_shadow_fb_access
-int drm_gem_simple_kms_prepare_shadow_fb(struct drm_simple_display_pipe *pipe,
- struct drm_plane_state *plane_state);
-void drm_gem_simple_kms_cleanup_shadow_fb(struct drm_simple_display_pipe *pipe,
- struct drm_plane_state *plane_state);
+int drm_gem_simple_kms_begin_shadow_fb_access(struct drm_simple_display_pipe *pipe,
+ struct drm_plane_state *plane_state);
+void drm_gem_simple_kms_end_shadow_fb_access(struct drm_simple_display_pipe *pipe,
+ struct drm_plane_state *plane_state);
void drm_gem_simple_kms_reset_shadow_plane(struct drm_simple_display_pipe *pipe);
struct drm_plane_state *
drm_gem_simple_kms_duplicate_shadow_plane_state(struct drm_simple_display_pipe *pipe);
@@ -137,8 +137,8 @@ void drm_gem_simple_kms_destroy_shadow_plane_state(struct drm_simple_display_pip
* functions.
*/
#define DRM_GEM_SIMPLE_DISPLAY_PIPE_SHADOW_PLANE_FUNCS \
- .prepare_fb = drm_gem_simple_kms_prepare_shadow_fb, \
- .cleanup_fb = drm_gem_simple_kms_cleanup_shadow_fb, \
+ .begin_fb_access = drm_gem_simple_kms_begin_shadow_fb_access, \
+ .end_fb_access = drm_gem_simple_kms_end_shadow_fb_access, \
.reset_plane = drm_gem_simple_kms_reset_shadow_plane, \
.duplicate_plane_state = drm_gem_simple_kms_duplicate_shadow_plane_state, \
.destroy_plane_state = drm_gem_simple_kms_destroy_shadow_plane_state
diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h
index 6b5e012..5362702 100644
--- a/include/drm/drm_mode_config.h
+++ b/include/drm/drm_mode_config.h
@@ -345,7 +345,6 @@ struct drm_mode_config_funcs {
* @max_width: maximum fb pixel width on this device
* @max_height: maximum fb pixel height on this device
* @funcs: core driver provided mode setting functions
- * @fb_base: base address of the framebuffer
* @poll_enabled: track polling support for this device
* @poll_running: track polling status for this device
* @delayed_event: track delayed poll uevent deliver for this device
@@ -542,7 +541,6 @@ struct drm_mode_config {
int min_width, min_height;
int max_width, max_height;
const struct drm_mode_config_funcs *funcs;
- resource_size_t fb_base;
/* output poll support */
bool poll_enabled;
diff --git a/include/drm/drm_modeset_helper_vtables.h b/include/drm/drm_modeset_helper_vtables.h
index fafa70ac..d9f2254a 100644
--- a/include/drm/drm_modeset_helper_vtables.h
+++ b/include/drm/drm_modeset_helper_vtables.h
@@ -1184,11 +1184,20 @@ struct drm_plane_helper_funcs {
* can call drm_gem_plane_helper_prepare_fb() from their @prepare_fb
* hook.
*
+ * The resources acquired in @prepare_fb persist after the end of
+ * the atomic commit. Resources that can be release at the commit's end
+ * should be acquired in @begin_fb_access and released in @end_fb_access.
+ * For example, a GEM buffer's pin operation belongs into @prepare_fb to
+ * keep the buffer pinned after the commit. But a vmap operation for
+ * shadow-plane helpers belongs into @begin_fb_access, so that atomic
+ * helpers remove the mapping at the end of the commit.
+ *
* The helpers will call @cleanup_fb with matching arguments for every
* successful call to this hook.
*
* This callback is used by the atomic modeset helpers and by the
- * transitional plane helpers, but it is optional.
+ * transitional plane helpers, but it is optional. See @begin_fb_access
+ * for preparing per-commit resources.
*
* RETURNS:
*
@@ -1212,6 +1221,36 @@ struct drm_plane_helper_funcs {
struct drm_plane_state *old_state);
/**
+ * @begin_fb_access:
+ *
+ * This hook prepares the plane for access during an atomic commit.
+ * In contrast to @prepare_fb, resources acquired in @begin_fb_access,
+ * are released at the end of the atomic commit in @end_fb_access.
+ *
+ * For example, with shadow-plane helpers, the GEM buffer's vmap
+ * operation belongs into @begin_fb_access, so that the buffer's
+ * memory will be unmapped at the end of the commit in @end_fb_access.
+ * But a GEM buffer's pin operation belongs into @prepare_fb
+ * to keep the buffer pinned after the commit.
+ *
+ * The callback is used by the atomic modeset helpers, but it is optional.
+ * See @end_fb_cleanup for undoing the effects of @begin_fb_access and
+ * @prepare_fb for acquiring resources until the next pageflip.
+ *
+ * Returns:
+ * 0 on success, or a negative errno code otherwise.
+ */
+ int (*begin_fb_access)(struct drm_plane *plane, struct drm_plane_state *new_plane_state);
+
+ /**
+ * @end_fb_access:
+ *
+ * This hook cleans up resources allocated by @begin_fb_access. It it called
+ * at the end of a commit for the new plane state.
+ */
+ void (*end_fb_access)(struct drm_plane *plane, struct drm_plane_state *new_plane_state);
+
+ /**
* @atomic_check:
*
* Drivers should check plane specific constraints in this hook.
diff --git a/include/drm/drm_simple_kms_helper.h b/include/drm/drm_simple_kms_helper.h
index 0b3647e6..2298fe3 100644
--- a/include/drm/drm_simple_kms_helper.h
+++ b/include/drm/drm_simple_kms_helper.h
@@ -136,6 +136,26 @@ struct drm_simple_display_pipe_funcs {
struct drm_plane_state *plane_state);
/**
+ * @begin_fb_access:
+ *
+ * Optional, called by &drm_plane_helper_funcs.begin_fb_access. Please read
+ * the documentation for the &drm_plane_helper_funcs.begin_fb_access hook for
+ * more details.
+ */
+ int (*begin_fb_access)(struct drm_simple_display_pipe *pipe,
+ struct drm_plane_state *new_plane_state);
+
+ /**
+ * @end_fb_access:
+ *
+ * Optional, called by &drm_plane_helper_funcs.end_fb_access. Please read
+ * the documentation for the &drm_plane_helper_funcs.end_fb_access hook for
+ * more details.
+ */
+ void (*end_fb_access)(struct drm_simple_display_pipe *pipe,
+ struct drm_plane_state *plane_state);
+
+ /**
* @enable_vblank:
*
* Optional, called by &drm_crtc_funcs.enable_vblank. Please read
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 2ae4fd6..ca857ec 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -41,6 +41,8 @@
*/
#define DRM_SCHED_FENCE_DONT_PIPELINE DMA_FENCE_FLAG_USER_BITS
+enum dma_resv_usage;
+struct dma_resv;
struct drm_gem_object;
struct drm_gpu_scheduler;
@@ -59,6 +61,12 @@ enum drm_sched_priority {
DRM_SCHED_PRIORITY_UNSET = -2
};
+/* Used to chose between FIFO and RR jobs scheduling */
+extern int drm_sched_policy;
+
+#define DRM_SCHED_POLICY_RR 0
+#define DRM_SCHED_POLICY_FIFO 1
+
/**
* struct drm_sched_entity - A wrapper around a job queue (typically
* attached to the DRM file_priv).
@@ -205,6 +213,21 @@ struct drm_sched_entity {
* drm_sched_entity_fini().
*/
struct completion entity_idle;
+
+ /**
+ * @oldest_job_waiting:
+ *
+ * Marks earliest job waiting in SW queue
+ */
+ ktime_t oldest_job_waiting;
+
+ /**
+ * @rb_tree_node:
+ *
+ * The node used to insert this entity into time based priority queue
+ */
+ struct rb_node rb_tree_node;
+
};
/**
@@ -214,6 +237,7 @@ struct drm_sched_entity {
* @sched: the scheduler to which this rq belongs to.
* @entities: list of the entities to be scheduled.
* @current_entity: the entity which is to be scheduled.
+ * @rb_tree_root: root of time based priory queue of entities for FIFO scheduling
*
* Run queue is a set of entities scheduling command submissions for
* one specific ring. It implements the scheduling policy that selects
@@ -224,6 +248,7 @@ struct drm_sched_rq {
struct drm_gpu_scheduler *sched;
struct list_head entities;
struct drm_sched_entity *current_entity;
+ struct rb_root_cached rb_tree_root;
};
/**
@@ -304,7 +329,7 @@ struct drm_sched_job {
*/
union {
struct dma_fence_cb finish_cb;
- struct work_struct work;
+ struct work_struct work;
};
uint64_t id;
@@ -323,6 +348,13 @@ struct drm_sched_job {
/** @last_dependency: tracks @dependencies as they signal */
unsigned long last_dependency;
+
+ /**
+ * @submit_ts:
+ *
+ * When the job was pushed into the entity queue.
+ */
+ ktime_t submit_ts;
};
static inline bool drm_sched_invalidate_job(struct drm_sched_job *s_job,
@@ -345,18 +377,17 @@ enum drm_gpu_sched_stat {
*/
struct drm_sched_backend_ops {
/**
- * @dependency:
+ * @prepare_job:
*
* Called when the scheduler is considering scheduling this job next, to
* get another struct dma_fence for this job to block on. Once it
* returns NULL, run_job() may be called.
*
- * If a driver exclusively uses drm_sched_job_add_dependency() and
- * drm_sched_job_add_implicit_dependencies() this can be ommitted and
- * left as NULL.
+ * Can be NULL if no additional preparation to the dependencies are
+ * necessary. Skipped when jobs are killed instead of run.
*/
- struct dma_fence *(*dependency)(struct drm_sched_job *sched_job,
- struct drm_sched_entity *s_entity);
+ struct dma_fence *(*prepare_job)(struct drm_sched_job *sched_job,
+ struct drm_sched_entity *s_entity);
/**
* @run_job: Called to execute the job once all of the dependencies
@@ -484,6 +515,9 @@ int drm_sched_job_init(struct drm_sched_job *job,
void drm_sched_job_arm(struct drm_sched_job *job);
int drm_sched_job_add_dependency(struct drm_sched_job *job,
struct dma_fence *fence);
+int drm_sched_job_add_resv_dependencies(struct drm_sched_job *job,
+ struct dma_resv *resv,
+ enum dma_resv_usage usage);
int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
struct drm_gem_object *obj,
bool write);
@@ -498,7 +532,6 @@ void drm_sched_wakeup(struct drm_gpu_scheduler *sched);
void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad);
void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery);
void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched);
-void drm_sched_resubmit_jobs_ext(struct drm_gpu_scheduler *sched, int max);
void drm_sched_increase_karma(struct drm_sched_job *bad);
void drm_sched_reset_karma(struct drm_sched_job *bad);
void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type);
@@ -512,6 +545,8 @@ void drm_sched_rq_add_entity(struct drm_sched_rq *rq,
void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,
struct drm_sched_entity *entity);
+void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts);
+
int drm_sched_entity_init(struct drm_sched_entity *entity,
enum drm_sched_priority priority,
struct drm_gpu_scheduler **sched_list,
diff --git a/include/drm/i915_pxp_tee_interface.h b/include/drm/i915_pxp_tee_interface.h
index af593ec..a702b6e 100644
--- a/include/drm/i915_pxp_tee_interface.h
+++ b/include/drm/i915_pxp_tee_interface.h
@@ -8,6 +8,7 @@
#include <linux/mutex.h>
#include <linux/device.h>
+struct scatterlist;
/**
* struct i915_pxp_component_ops - ops for PXP services.
@@ -23,6 +24,10 @@ struct i915_pxp_component_ops {
int (*send)(struct device *dev, const void *message, size_t size);
int (*recv)(struct device *dev, void *buffer, size_t size);
+ ssize_t (*gsc_command)(struct device *dev, u8 client_id, u32 fence_id,
+ struct scatterlist *sg_in, size_t total_in_len,
+ struct scatterlist *sg_out);
+
};
/**
diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h
index 5afc6d6..78a226e 100644
--- a/include/drm/ttm/ttm_resource.h
+++ b/include/drm/ttm/ttm_resource.h
@@ -197,7 +197,7 @@ struct ttm_bus_placement {
* struct ttm_resource
*
* @start: Start of the allocation.
- * @num_pages: Actual size of resource in pages.
+ * @size: Actual size of resource in bytes.
* @mem_type: Resource type of the allocation.
* @placement: Placement flags.
* @bus: Placement on io bus accessible to the CPU
@@ -208,7 +208,7 @@ struct ttm_bus_placement {
*/
struct ttm_resource {
unsigned long start;
- unsigned long num_pages;
+ size_t size;
uint32_t mem_type;
uint32_t placement;
struct ttm_bus_placement bus;
diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h
index 17a0310..b7d3f38 100644
--- a/include/drm/ttm/ttm_tt.h
+++ b/include/drm/ttm/ttm_tt.h
@@ -88,7 +88,7 @@ struct ttm_tt {
#define TTM_TT_FLAG_EXTERNAL (1 << 2)
#define TTM_TT_FLAG_EXTERNAL_MAPPABLE (1 << 3)
-#define TTM_TT_FLAG_PRIV_POPULATED (1 << 31)
+#define TTM_TT_FLAG_PRIV_POPULATED (1U << 31)
uint32_t page_flags;
/** @num_pages: Number of pages in the page array. */
uint32_t num_pages;
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index 7ed5320..c873145 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -398,15 +398,6 @@ struct dma_buf {
const struct dma_buf_ops *ops;
/**
- * @lock:
- *
- * Used internally to serialize list manipulation, attach/detach and
- * vmap/unmap. Note that in many cases this is superseeded by
- * dma_resv_lock() on @resv.
- */
- struct mutex lock;
-
- /**
* @vmapping_counter:
*
* Used internally to refcnt the vmaps returned by dma_buf_vmap().
@@ -707,10 +698,18 @@ int dma_buf_end_cpu_access(struct dma_buf *dma_buf,
int dma_buf_end_cpu_access_partial(struct dma_buf *dma_buf,
enum dma_data_direction dir,
unsigned int offset, unsigned int len);
+struct sg_table *
+dma_buf_map_attachment_unlocked(struct dma_buf_attachment *attach,
+ enum dma_data_direction direction);
+void dma_buf_unmap_attachment_unlocked(struct dma_buf_attachment *attach,
+ struct sg_table *sg_table,
+ enum dma_data_direction direction);
int dma_buf_mmap(struct dma_buf *, struct vm_area_struct *,
unsigned long);
int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map);
void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map);
+int dma_buf_vmap_unlocked(struct dma_buf *dmabuf, struct iosys_map *map);
+void dma_buf_vunmap_unlocked(struct dma_buf *dmabuf, struct iosys_map *map);
int dma_buf_get_flags(struct dma_buf *dmabuf, unsigned long *flags);
#endif /* __DMA_BUF_H__ */
diff --git a/include/linux/fb.h b/include/linux/fb.h
index bcb8658..96b9632 100644
--- a/include/linux/fb.h
+++ b/include/linux/fb.h
@@ -803,6 +803,15 @@ extern int fb_find_mode(struct fb_var_screeninfo *var,
const struct fb_videomode *default_mode,
unsigned int default_bpp);
+#if defined(CONFIG_VIDEO_NOMODESET)
+bool fb_modesetting_disabled(const char *drvname);
+#else
+static inline bool fb_modesetting_disabled(const char *drvname)
+{
+ return false;
+}
+#endif
+
/* Convenience logging macros */
#define fb_err(fb_info, fmt, ...) \
pr_err("fb%d: " fmt, (fb_info)->node, ##__VA_ARGS__)
diff --git a/include/linux/host1x.h b/include/linux/host1x.h
index cb2100d..dc55d9d 100644
--- a/include/linux/host1x.h
+++ b/include/linux/host1x.h
@@ -469,11 +469,13 @@ struct host1x_memory_context {
#ifdef CONFIG_IOMMU_API
struct host1x_memory_context *host1x_memory_context_alloc(struct host1x *host1x,
+ struct device *dev,
struct pid *pid);
void host1x_memory_context_get(struct host1x_memory_context *cd);
void host1x_memory_context_put(struct host1x_memory_context *cd);
#else
static inline struct host1x_memory_context *host1x_memory_context_alloc(struct host1x *host1x,
+ struct device *dev,
struct pid *pid)
{
return NULL;
diff --git a/include/linux/mei_cl_bus.h b/include/linux/mei_cl_bus.h
index df1fab4..fd6e062 100644
--- a/include/linux/mei_cl_bus.h
+++ b/include/linux/mei_cl_bus.h
@@ -11,6 +11,7 @@
struct mei_cl_device;
struct mei_device;
+struct scatterlist;
typedef void (*mei_cldev_cb_t)(struct mei_cl_device *cldev);
@@ -116,6 +117,11 @@ void mei_cldev_set_drvdata(struct mei_cl_device *cldev, void *data);
int mei_cldev_enable(struct mei_cl_device *cldev);
int mei_cldev_disable(struct mei_cl_device *cldev);
bool mei_cldev_enabled(const struct mei_cl_device *cldev);
+ssize_t mei_cldev_send_gsc_command(struct mei_cl_device *cldev,
+ u8 client_id, u32 fence_id,
+ struct scatterlist *sg_in,
+ size_t total_in_len,
+ struct scatterlist *sg_out);
void *mei_cldev_dma_map(struct mei_cl_device *cldev, u8 buffer_id, size_t size);
int mei_cldev_dma_unmap(struct mei_cl_device *cldev);
diff --git a/include/soc/bcm2835/raspberrypi-firmware.h b/include/soc/bcm2835/raspberrypi-firmware.h
index 811ea66..ab95559 100644
--- a/include/soc/bcm2835/raspberrypi-firmware.h
+++ b/include/soc/bcm2835/raspberrypi-firmware.h
@@ -136,12 +136,52 @@ enum rpi_firmware_property_tag {
RPI_FIRMWARE_GET_DMA_CHANNELS = 0x00060001,
};
+enum rpi_firmware_clk_id {
+ RPI_FIRMWARE_EMMC_CLK_ID = 1,
+ RPI_FIRMWARE_UART_CLK_ID,
+ RPI_FIRMWARE_ARM_CLK_ID,
+ RPI_FIRMWARE_CORE_CLK_ID,
+ RPI_FIRMWARE_V3D_CLK_ID,
+ RPI_FIRMWARE_H264_CLK_ID,
+ RPI_FIRMWARE_ISP_CLK_ID,
+ RPI_FIRMWARE_SDRAM_CLK_ID,
+ RPI_FIRMWARE_PIXEL_CLK_ID,
+ RPI_FIRMWARE_PWM_CLK_ID,
+ RPI_FIRMWARE_HEVC_CLK_ID,
+ RPI_FIRMWARE_EMMC2_CLK_ID,
+ RPI_FIRMWARE_M2MC_CLK_ID,
+ RPI_FIRMWARE_PIXEL_BVB_CLK_ID,
+ RPI_FIRMWARE_VEC_CLK_ID,
+ RPI_FIRMWARE_NUM_CLK_ID,
+};
+
+/**
+ * struct rpi_firmware_clk_rate_request - Firmware Request for a rate
+ * @id: ID of the clock being queried
+ * @rate: Rate in Hertz. Set by the firmware.
+ *
+ * Used by @RPI_FIRMWARE_GET_CLOCK_RATE, @RPI_FIRMWARE_GET_CLOCK_MEASURED,
+ * @RPI_FIRMWARE_GET_MAX_CLOCK_RATE and @RPI_FIRMWARE_GET_MIN_CLOCK_RATE.
+ */
+struct rpi_firmware_clk_rate_request {
+ __le32 id;
+ __le32 rate;
+} __packed;
+
+#define RPI_FIRMWARE_CLK_RATE_REQUEST(_id) \
+ { \
+ .id = _id, \
+ }
+
#if IS_ENABLED(CONFIG_RASPBERRYPI_FIRMWARE)
int rpi_firmware_property(struct rpi_firmware *fw,
u32 tag, void *data, size_t len);
int rpi_firmware_property_list(struct rpi_firmware *fw,
void *data, size_t tag_size);
void rpi_firmware_put(struct rpi_firmware *fw);
+unsigned int rpi_firmware_clk_get_max_rate(struct rpi_firmware *fw,
+ unsigned int id);
+struct device_node *rpi_firmware_find_node(void);
struct rpi_firmware *rpi_firmware_get(struct device_node *firmware_node);
struct rpi_firmware *devm_rpi_firmware_get(struct device *dev,
struct device_node *firmware_node);
@@ -159,6 +199,18 @@ static inline int rpi_firmware_property_list(struct rpi_firmware *fw,
}
static inline void rpi_firmware_put(struct rpi_firmware *fw) { }
+
+static inline unsigned int rpi_firmware_clk_get_max_rate(struct rpi_firmware *fw,
+ unsigned int id)
+{
+ return UINT_MAX;
+}
+
+static inline struct device_node *rpi_firmware_find_node(void)
+{
+ return NULL;
+}
+
static inline struct rpi_firmware *rpi_firmware_get(struct device_node *firmware_node)
{
return NULL;
diff --git a/include/soc/tegra/mc.h b/include/soc/tegra/mc.h
index 47ce6d4..51a2263 100644
--- a/include/soc/tegra/mc.h
+++ b/include/soc/tegra/mc.h
@@ -193,6 +193,8 @@ struct tegra_mc_soc {
unsigned int num_address_bits;
unsigned int atom_size;
+ unsigned int num_carveouts;
+
u16 client_id_mask;
u8 num_channels;
@@ -244,6 +246,8 @@ unsigned int tegra_mc_get_emem_device_count(struct tegra_mc *mc);
#ifdef CONFIG_TEGRA_MC
struct tegra_mc *devm_tegra_memory_controller_get(struct device *dev);
int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev);
+int tegra_mc_get_carveout_info(struct tegra_mc *mc, unsigned int id,
+ phys_addr_t *base, u64 *size);
#else
static inline struct tegra_mc *
devm_tegra_memory_controller_get(struct device *dev)
@@ -256,6 +260,13 @@ tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev)
{
return -ENODEV;
}
+
+static inline int
+tegra_mc_get_carveout_info(struct tegra_mc *mc, unsigned int id,
+ phys_addr_t *base, u64 *size)
+{
+ return -ENODEV;
+}
#endif
#endif /* __SOC_TEGRA_MC_H__ */
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 0d93ec1..4038abe8 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -144,6 +144,20 @@ extern "C" {
* content.
*/
#define AMDGPU_GEM_CREATE_DISCARDABLE (1 << 12)
+/* Flag that BO is shared coherently between multiple devices or CPU threads.
+ * May depend on GPU instructions to flush caches explicitly
+ *
+ * This influences the choice of MTYPE in the PTEs on GFXv9 and later GPUs and
+ * may override the MTYPE selected in AMDGPU_VA_OP_MAP.
+ */
+#define AMDGPU_GEM_CREATE_COHERENT (1 << 13)
+/* Flag that BO should not be cached by GPU. Coherent without having to flush
+ * GPU caches explicitly
+ *
+ * This influences the choice of MTYPE in the PTEs on GFXv9 and later GPUs and
+ * may override the MTYPE selected in AMDGPU_VA_OP_MAP.
+ */
+#define AMDGPU_GEM_CREATE_UNCACHED (1 << 14)
struct drm_amdgpu_gem_create_in {
/** the requested memory size */
diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index 868d690..bc056f2 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -744,6 +744,35 @@ extern "C" {
*/
#define DRM_FORMAT_MOD_VIVANTE_SPLIT_SUPER_TILED fourcc_mod_code(VIVANTE, 4)
+/*
+ * Vivante TS (tile-status) buffer modifiers. They can be combined with all of
+ * the color buffer tiling modifiers defined above. When TS is present it's a
+ * separate buffer containing the clear/compression status of each tile. The
+ * modifiers are defined as VIVANTE_MOD_TS_c_s, where c is the color buffer
+ * tile size in bytes covered by one entry in the status buffer and s is the
+ * number of status bits per entry.
+ * We reserve the top 8 bits of the Vivante modifier space for tile status
+ * clear/compression modifiers, as future cores might add some more TS layout
+ * variations.
+ */
+#define VIVANTE_MOD_TS_64_4 (1ULL << 48)
+#define VIVANTE_MOD_TS_64_2 (2ULL << 48)
+#define VIVANTE_MOD_TS_128_4 (3ULL << 48)
+#define VIVANTE_MOD_TS_256_4 (4ULL << 48)
+#define VIVANTE_MOD_TS_MASK (0xfULL << 48)
+
+/*
+ * Vivante compression modifiers. Those depend on a TS modifier being present
+ * as the TS bits get reinterpreted as compression tags instead of simple
+ * clear markers when compression is enabled.
+ */
+#define VIVANTE_MOD_COMP_DEC400 (1ULL << 52)
+#define VIVANTE_MOD_COMP_MASK (0xfULL << 52)
+
+/* Masking out the extension bits will yield the base modifier. */
+#define VIVANTE_MOD_EXT_MASK (VIVANTE_MOD_TS_MASK | \
+ VIVANTE_MOD_COMP_MASK)
+
/* NVIDIA frame buffer modifiers */
/*
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index fa95330..46beced 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -935,12 +935,31 @@ struct hdr_output_metadata {
};
};
+/**
+ * DRM_MODE_PAGE_FLIP_EVENT
+ *
+ * Request that the kernel sends back a vblank event (see
+ * struct drm_event_vblank) with the &DRM_EVENT_FLIP_COMPLETE type when the
+ * page-flip is done.
+ */
#define DRM_MODE_PAGE_FLIP_EVENT 0x01
+/**
+ * DRM_MODE_PAGE_FLIP_ASYNC
+ *
+ * Request that the page-flip is performed as soon as possible, ie. with no
+ * delay due to waiting for vblank. This may cause tearing to be visible on
+ * the screen.
+ */
#define DRM_MODE_PAGE_FLIP_ASYNC 0x02
#define DRM_MODE_PAGE_FLIP_TARGET_ABSOLUTE 0x4
#define DRM_MODE_PAGE_FLIP_TARGET_RELATIVE 0x8
#define DRM_MODE_PAGE_FLIP_TARGET (DRM_MODE_PAGE_FLIP_TARGET_ABSOLUTE | \
DRM_MODE_PAGE_FLIP_TARGET_RELATIVE)
+/**
+ * DRM_MODE_PAGE_FLIP_FLAGS
+ *
+ * Bitmask of flags suitable for &drm_mode_crtc_page_flip_target.flags.
+ */
#define DRM_MODE_PAGE_FLIP_FLAGS (DRM_MODE_PAGE_FLIP_EVENT | \
DRM_MODE_PAGE_FLIP_ASYNC | \
DRM_MODE_PAGE_FLIP_TARGET)
@@ -1034,11 +1053,53 @@ struct drm_mode_destroy_dumb {
__u32 handle;
};
-/* page-flip flags are valid, plus: */
+/**
+ * DRM_MODE_ATOMIC_TEST_ONLY
+ *
+ * Do not apply the atomic commit, instead check whether the hardware supports
+ * this configuration.
+ *
+ * See &drm_mode_config_funcs.atomic_check for more details on test-only
+ * commits.
+ */
#define DRM_MODE_ATOMIC_TEST_ONLY 0x0100
+/**
+ * DRM_MODE_ATOMIC_NONBLOCK
+ *
+ * Do not block while applying the atomic commit. The &DRM_IOCTL_MODE_ATOMIC
+ * IOCTL returns immediately instead of waiting for the changes to be applied
+ * in hardware. Note, the driver will still check that the update can be
+ * applied before retuning.
+ */
#define DRM_MODE_ATOMIC_NONBLOCK 0x0200
+/**
+ * DRM_MODE_ATOMIC_ALLOW_MODESET
+ *
+ * Allow the update to result in temporary or transient visible artifacts while
+ * the update is being applied. Applying the update may also take significantly
+ * more time than a page flip. All visual artifacts will disappear by the time
+ * the update is completed, as signalled through the vblank event's timestamp
+ * (see struct drm_event_vblank).
+ *
+ * This flag must be set when the KMS update might cause visible artifacts.
+ * Without this flag such KMS update will return a EINVAL error. What kind of
+ * update may cause visible artifacts depends on the driver and the hardware.
+ * User-space that needs to know beforehand if an update might cause visible
+ * artifacts can use &DRM_MODE_ATOMIC_TEST_ONLY without
+ * &DRM_MODE_ATOMIC_ALLOW_MODESET to see if it fails.
+ *
+ * To the best of the driver's knowledge, visual artifacts are guaranteed to
+ * not appear when this flag is not set. Some sinks might display visual
+ * artifacts outside of the driver's control.
+ */
#define DRM_MODE_ATOMIC_ALLOW_MODESET 0x0400
+/**
+ * DRM_MODE_ATOMIC_FLAGS
+ *
+ * Bitfield of flags accepted by the &DRM_IOCTL_MODE_ATOMIC IOCTL in
+ * &drm_mode_atomic.flags.
+ */
#define DRM_MODE_ATOMIC_FLAGS (\
DRM_MODE_PAGE_FLIP_EVENT |\
DRM_MODE_PAGE_FLIP_ASYNC |\
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index 520ad26..8df261c 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -645,6 +645,22 @@ typedef struct drm_i915_irq_wait {
*/
#define I915_SCHEDULER_CAP_STATIC_PRIORITY_MAP (1ul << 5)
+/*
+ * Query the status of HuC load.
+ *
+ * The query can fail in the following scenarios with the listed error codes:
+ * -ENODEV if HuC is not present on this platform,
+ * -EOPNOTSUPP if HuC firmware usage is disabled,
+ * -ENOPKG if HuC firmware fetch failed,
+ * -ENOEXEC if HuC firmware is invalid or mismatched,
+ * -ENOMEM if i915 failed to prepare the FW objects for transfer to the uC,
+ * -EIO if the FW transfer or the FW authentication failed.
+ *
+ * If the IOCTL is successful, the returned parameter will be set to one of the
+ * following values:
+ * * 0 if HuC firmware load is not complete,
+ * * 1 if HuC firmware is authenticated and running.
+ */
#define I915_PARAM_HUC_STATUS 42
/* Query whether DRM_I915_GEM_EXECBUFFER2 supports the ability to opt-out of
@@ -749,6 +765,12 @@ typedef struct drm_i915_irq_wait {
/* Query if the kernel supports the I915_USERPTR_PROBE flag. */
#define I915_PARAM_HAS_USERPTR_PROBE 56
+/*
+ * Frequency of the timestamps in OA reports. This used to be the same as the CS
+ * timestamp frequency, but differs on some platforms.
+ */
+#define I915_PARAM_OA_TIMESTAMP_FREQUENCY 57
+
/* Must be kept compact -- no holes and well documented */
/**
@@ -2650,6 +2672,10 @@ enum drm_i915_oa_format {
I915_OA_FORMAT_A12_B8_C8,
I915_OA_FORMAT_A32u40_A4u32_B8_C8,
+ /* DG2 */
+ I915_OAR_FORMAT_A32u40_A4u32_B8_C8,
+ I915_OA_FORMAT_A24u40_A14u32_B8_C8,
+
I915_OA_FORMAT_MAX /* non-ABI */
};
@@ -3493,27 +3519,13 @@ struct drm_i915_gem_create_ext {
*
* The (page-aligned) allocated size for the object will be returned.
*
- * DG2 64K min page size implications:
+ * On platforms like DG2/ATS the kernel will always use 64K or larger
+ * pages for I915_MEMORY_CLASS_DEVICE. The kernel also requires a
+ * minimum of 64K GTT alignment for such objects.
*
- * On discrete platforms, starting from DG2, we have to contend with GTT
- * page size restrictions when dealing with I915_MEMORY_CLASS_DEVICE
- * objects. Specifically the hardware only supports 64K or larger GTT
- * page sizes for such memory. The kernel will already ensure that all
- * I915_MEMORY_CLASS_DEVICE memory is allocated using 64K or larger page
- * sizes underneath.
- *
- * Note that the returned size here will always reflect any required
- * rounding up done by the kernel, i.e 4K will now become 64K on devices
- * such as DG2. The kernel will always select the largest minimum
- * page-size for the set of possible placements as the value to use when
- * rounding up the @size.
- *
- * Special DG2 GTT address alignment requirement:
- *
- * The GTT alignment will also need to be at least 2M for such objects.
- *
- * Note that due to how the hardware implements 64K GTT page support, we
- * have some further complications:
+ * NOTE: Previously the ABI here required a minimum GTT alignment of 2M
+ * on DG2/ATS, due to how the hardware implemented 64K GTT page support,
+ * where we had the following complications:
*
* 1) The entire PDE (which covers a 2MB virtual address range), must
* contain only 64K PTEs, i.e mixing 4K and 64K PTEs in the same
@@ -3522,12 +3534,10 @@ struct drm_i915_gem_create_ext {
* 2) We still need to support 4K PTEs for I915_MEMORY_CLASS_SYSTEM
* objects.
*
- * To keep things simple for userland, we mandate that any GTT mappings
- * must be aligned to and rounded up to 2MB. The kernel will internally
- * pad them out to the next 2MB boundary. As this only wastes virtual
- * address space and avoids userland having to copy any needlessly
- * complicated PDE sharing scheme (coloring) and only affects DG2, this
- * is deemed to be a good compromise.
+ * However on actual production HW this was completely changed to now
+ * allow setting a TLB hint at the PTE level (see PS64), which is a lot
+ * more flexible than the above. With this the 2M restriction was
+ * dropped where we now only require 64K.
*/
__u64 size;
diff --git a/include/uapi/drm/msm_drm.h b/include/uapi/drm/msm_drm.h
index 3c7b097..f54b48e 100644
--- a/include/uapi/drm/msm_drm.h
+++ b/include/uapi/drm/msm_drm.h
@@ -138,6 +138,7 @@ struct drm_msm_gem_new {
#define MSM_INFO_SET_NAME 0x02 /* set the debug name (by pointer) */
#define MSM_INFO_GET_NAME 0x03 /* get debug name, returned by pointer */
#define MSM_INFO_SET_IOVA 0x04 /* set the iova, passed by value */
+#define MSM_INFO_GET_FLAGS 0x05 /* get the MSM_BO_x flags */
struct drm_msm_gem_info {
__u32 handle; /* in */
diff --git a/include/video/nomodeset.h b/include/video/nomodeset.h
new file mode 100644
index 0000000..8f8688b
--- /dev/null
+++ b/include/video/nomodeset.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: MIT */
+
+#ifndef VIDEO_NOMODESET_H
+#define VIDEO_NOMODESET_H
+
+bool video_firmware_drivers_only(void);
+
+#endif
