Intel Stratix 10 Avalon -MM Hard IP for PCIe Design Example User Guide

RSS

UG-20054 | 2017.11.06

Quick Start Guide

Using Intel^® Quartus^® Prime Pro Edition, you can generate a simple DMA design example for the Avalon^® -MM Intel^® Stratix^® 10 Hard IP for PCI Express^® IP core. The generated design example reflects the parameters that you specify. It automatically creates the files necessary to simulate and compile in the Intel^® Quartus^® Prime Pro Edition software. You can download the compiled design to the Intel^® Stratix^® 10-GX Development Board. To download to custom hardware, update the Intel^® Quartus^® Prime Settings File (.qsf) with the correct pin assignments .

Figure 1. Development Steps for the Design Example

Design Components

Figure 2. Block Diagram for the Simple DMA for PCIe^® Design Example

Directory Structure

Figure 3. Directory Structure for the Generated Design Example

Generating the Design Example

Follow these steps to generate your design:

Figure 4. Procedure

In the Intel^® Quartus^® Prime Pro Edition software, create a new project (File > New Project Wizard).
Specify the Directory, Name, and Top-Level Entity.
For Project Type, accept the default value, Empty project. Click Next.
For Add Files click Next.
For Family, Device & Board Settings under Family, select Intel^® Stratix^® 10 and the Target Device for your design.
Click Finish.
In the IP Catalog locate and add the Intel^® Stratix^® 10 -MM Hard IP for PCI Express^® . Click Create.
In the New IP Variant dialog box, specify a name for your IP.
On the IP Settings tabs, specify the parameters for your IP variation.
On the Example Designs tab, make the following selections:
1. For Available Example Designs, select Simple DMA.
  
  Note:
  The Simple DMA design example is only available when you enable Enable high performance bursting Avalon^® -MM Slave interface (HPTXS) on the Avalon^® -MM Settings tab.
  
  (The DMA design example is only available when you turn on Enable Avalon^® -MM DMA on the Avalon^® -MM Settings tab.)
2. For Example Design Files, turn on the Simulation and Synthesis options.
3. If you have selected a x16 configuration, for Select simulation Root Ccomplex BFM, choose the appropriate BFM:
  - Intel FPGA BFM: for all configurations up to Gen3 x8. This bus functional model (BFM) supports x16 configurations by downtraining to x8.
  - Third-party BFM: for x16 configurations if you want to simulate all 16 lanes using a third-party BFM. Refer to AN-811: Using the Avery BFM for PCI Express Gen3x16 Simulation on Intel Stratix 10 Devices for information about simulating with the Avery BFM.
4. For Generated HDL Format, only Verilog is available in the current release.
5. For Target Development Kit, select the appropriate option.
  
  Note: If you select None, the generated design example targets the device specified. If you intend to test the design in hardware, make the appropriate pin assignments in the .qsf file.
Select Generate Example Design to create a design example that you can simulate and download to hardware. If you select one of the Intel^® Stratix^® 10 development boards, the device on that board overwrites the device previously selected in the Intel^® Quartus^® Prime project if the devices are different. When the prompt asks you to specify the directory for your example design, accept the default directory, <example_design>/pcie_s10_hip_ast_0_example_design
Click Finish. Save your .ip file when prompted.
The prompt, Recent changes have not been generated. Generate now?, allows you to create files for simulation and synthesis of the design example. Click No to simulate the testbench design example you have generated. The .sof file for the complete example design is what you download to a board to perform hardware verification.
Close your project.

Simulating the Design Example

Figure 5. Procedure

Change to the testbench simulation directory, pcie_example_design_tb.
Run the simulation script for the simulator of your choice. Refer to the table below.
Analyze the results.

Table 1. Steps to Run Simulation
Simulator	Working Directory	Instructions
ModelSim^®	`<example_design>`/pcie_example_design_tb/pcie_example_design_tb/sim/mentor/	do msim_setup.tcl ld_debug run -all A successful simulation ends with the following message, "Simulation stopped due to successful completion!"
VCS^®	<example_design>/pcie_example_design_tb/pcie_example_design_tb/sim/synopsys/vcs	`sh vcs_setup.sh USER_DEFINED_SIM_OPTIONS=""` A successful simulation ends with the following message, "Simulation stopped due to successful completion!"
NCSim^®	`<example_design>`/pcie_example_design_tb/pcie_example_design_tb/sim/cadence	`sh ncsim_setup.sh USER_DEFINED_SIM_OPTIONS="" USER_DEFINED_ELAB_OPTIONS ="-timescale\ 1ns/1ps"` A successful simulation ends with the following message, "Simulation stopped due to successful completion!"

The simple DMA testbench completes the following tasks:

Writes to the Endpoint memory using the DUT Endpoint Avalon^® -MM RX master interface.
Reads from Endpoint memory using the DUT Endpoint Avalon^® -MM RX master interface.
Verifies the data using the shmem_chk_ok task.
Writes to the Endpoint DMA controller, instructing the DMA controller to perform a MemRd request to the PCIe^® address space in host memory.
Writes to the Endpoint DMA controller, instructing the DMA controller to perform a MemWr request to PCIe^® address space in host memory. This MemWr uses the data from the previous MemRd.
Verifies the data using the shmem_chk_ok task.

The simulation reports, "Simulation stopped due to successful completion" if no errors occur.

Figure 6. Partial Transcript from Successful Simulation Testbench

Compiling the Design Example and Programming the Device

Navigate to <project_dir>/pcie_s10_hip_ast_0_example_design/ and open pcie_example_design.qpf.
On the Processing menu, select Start Compilation.
After successfully compiling your design, program the targeted device with the Programmer.

Installing the Linux Kernel Driver

Before you can test the design example in hardware, you must install the Linux kernel driver. You can use this driver to perform the following tests:

A PCIe^® link test that performs 100 writes and reads
Memory space DWORD¹ reads and writes
Configuration Space DWORD reads and writes

In addition, you can use the driver to change the value of the following parameters:

The BAR
The selects device by specifying the bus, function and device (BDF) numbers for the required device

Complete the following steps to install the kernel driver:

Navigate to ./software/kernel/Linux under the example design generation directory.
Change the permissions on the install, load, and unload files:
$ chmod 777 install load unload
Install the driver:
$ sudo ./install
Verify the driver installation:
$ lsmod | grep intel_fpga_pcie_drv

Expected result:
intel_fpga_pcie_drv 17792 0
Verify that Linux recognizes the PCIe^® design example:
$ lspci -d 1172:000 -v | grep intel_fpga_pcie_drv
Note: If you have changed the Vendor ID, substitute the new Vendor ID for Altera^® 's
Vendor ID in this command.

Expected result:
Kernel driver in use: intel_fpga_pcie_drv

¹ Throughout this user guide, the terms word, DWORD and QWORD have the same meaning that they have in the PCI Express Base Specification. A word is 16 bits, a DWORD is 32 bits, and a QWORD is 64 bits.

Running the Design Example Application

Navigate to ./software/user/example under the design example directory.
Compile the design example application:
$ make
Run the test:

$ ./intel_fpga_pcie_link_test

You can run the Intel^® FPGA IP PCIe^® link test in manual or automatic mode.
- In automatic mode, the application automatically selects the device. The test selects the Intel^® Stratix^® 10 PCIe^® device with the lowest BDF by matching the Vendor ID. The test also selects the lowest available BAR.
- In manual mode, the test queries you for the bus, device, and function number and BAR.
For the Intel^® Stratix^® 10-GX Development Kit, you can determine the BDF by typing the following command:
$ lspci -d 1172

Here are sample transcripts for automatic and manual modes:

Intel FPGA PCIe Link Test - Automatic Mode
Version 1.0
0: Automatically select a device
1: Manually select a device
***************************************************
>0
Opened a handle to BAR 0 of a device with BDF 0x100
***************************************************
0: Link test - 100 writes and reads
1: Write memory space
2: Read memory space
3. Write configuration space
4. Read configuration space
5. Change BAR
6. Change device
7. Enable SR-IOV
8. Quit program
***************************************************
> 0
Doing 100 writes and 100 reads . . 
Number of write errors:     0
Number of read errors:      0
Number of DWORD mismatches: 0

Intel FPGA PCIe Link Test - Manual Mode
Version 1.0
0: Automatically select a device
1: Manually select a device
***************************************************
> 1
Enter bus number:
> 1
Enter device number:
> 0
Enter function number:
> 0
BDF is 0x100
Enter BAR number (-1 for none):
> 4
Opened a handle to BAR 4 of a device with BDF 0x100

Design Example Description

Functional Description for the Simple DMA Design Example

The simple DMA design example simulation testbench includes the following components:

DUT: The Intel^® Stratix^® 10 Hard IP for PCI Express Endpoint with the Enable high performance bursting Avalon^® -MM slave interface (HPTXS) parameter turned on.
PCIE_DMA_CONTROLLER_256: A DMA controller that receives control signals from the DUT rxm_bar2 port. Drives the DUT high performance Avalon^® -MM slave interface (hptxs) using its Avalon^® -MM write_master interface.
MEM1: An on-chip RAM that connects to the DUT RXM_bar0, an Avalon^® -MM master interface.
MEM2: An on-chip RAM that connects to the PCIE_DMA_CONTROLLER_256 Avalon^® -MM read and write master interfaces.
A testbench driver that configures the Root Port, Endpoint, writes to Endpoint memory, and programs the simple DMA controller.
A testbench monitor that checks expected results.

Figure 7. Simple DMA Design Example

Serial Data Signals

This differential, serial interface is the physical link between a Root Port and an Endpoint.

The PCIe IP Core supports 1, 2, 4, 8, or 16 lanes. Each lane includes a TX and RX differential pair. Data is striped across all available lanes.

Table 2. 1-Bit Interface Signals . In the following table `<n>` is the number of lanes.
Signal	Direction	Description
`tx_out[<n>-1:0]`	Output	Transmit output. These signals are the serial outputs of lanes `<n>-1`–0.
`rx_in[<n>-1:0]`	Input	Receive input. These signals are the serial inputs of lanes `<n>-1`–0.

Refer to Pin-out Files for Intel Devices for pin-out tables for all Intel devices in .pdf, .txt, and .xls formats.

Transceiver channels are arranged in groups of six. For GX devices, the lowest six channels on the left side of the device are labeled GXB_L0, the next group is GXB_L1, and so on. Channels on the right side of the device are labeled GXB_R0, GXB_R1, and so on. Be sure to connect the Hard IP for PCI Express on the left side of the device to appropriate channels on the left side of the device, as specified in the Pin-out Files for Intel Devices.

Document Revision History for the Intel Stratix 10 Avalon -MM Hard IP for PCIe Design Example User Guide

Intel Stratix 10 Avalon -MM Hard IP for PCIe Design Example User Guide Revision History

Date	Software Version	Changes
November 2017	17.1	Made the following changes: Added compilation support. Changed design example to demonstrate the simple DMA design. Added Linux driver for hardware example. Added simulation support for NCSim. Revised Generating the Design topic to create a single .ip for PCIe^® instead of a complete system design. Generating the testbench creates a design example from the .ip . Added web link to information on using the PCIe^® Link Inspector.
May 2017	Quartus^®Prime Pro v17.1 Stratix 10 ES Editions	Initial release.