Harness the power of real-time data with gRPCs— a well-detailed guide on Pump.fun
Accessing real-time on-chain data is crucial for developers, researchers, and trading bots. While there are different ways to stream this data, gRPC is the fastest and most efficient, offering low latency and simpler code.
Pump.fun is a popular Solana platform where users can launch coins that are instantly tradable for under $2. With millions of meme coins launched and thousands more every minute, developers need real-time updates to stay ahead. gRPC provides these updates in milliseconds. In this article, you’ll learn how to stream Pump.fun events in real-time using gRPC.
Before Getting Started
To get started, we will need a few things.
Authentication: Your Shyft API Key, gRPC endpoint and gRPC token
You can get your own Shyft API Key(an auth parameter used by Shyft) from the Shyft website. You can also find your region-specific gRPC endpoint and access token on your Shyft Dashboard.
A server-side backend (like NodeJS) to receive gRPC data
As gRPC services are unsupported in web browsers, you would need a backend application to receive gRPC data. For this example, we have used NodeJS, but any other backend server-side environments such as C#, Go, Java, Kotlin, Python or PHP can also be used.
Introduction — Topics to be covered
In this article, we will look at streaming trading transactions on Pump.fun using gRPC. We will be covering the following topics,
- Cloning and installing dependencies
- Setting up a gRPC connection to the Solana network
- Subscribing to a streaming endpoint for trading transactions on Pump.fun
- Processing and deserializing the streaming data in real-time
Cloning & Installation
To get started, we clone the following and install the dependencies:
$ git clone https://github.com/Shyft-to/solana-defi.gitcd grpc-pump
npm installAll the code related to this article is available on our GitHub here. Please feel free to clone it and follow along.
Setting up a gRPC connection to the Solana network
Our very first step to streaming data from Pump.fun using gRPC is setting up our client service. It requires your gRPC URL and your X_TOKEN which looks like this
const client = new Client(
'Your Region specific Shyft gRPC URL',
'Shyft gRPC Access Token',
undefined,
);now we can go ahead and obtain our data from the blockchain.
Connecting to the Pump.fun Program ID
To track real-time events on Pump.fun, your gRPC client needs to “listen” specifically to the Pump.fun Smart Contract. In the Solana ecosystem, this is done by setting a filter for the Program ID. By filtering at the source, you reduce data noise and ensure your bot only processes relevant launches and trades.
6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P
The official Program ID for the Pump.fun bonding curve is 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P. When setting up your SubscribeRequest, you must include this address in your transaction filters.
Subscribing to a streaming endpoint for trading transactions on pumpfun
Our goal is to stream specific data from the Solana blockchain using gRpc. To accomplish this, we will use the handleStream() function, which takes in two parameters:
client: Client: This parameter is used to subscribe to on-chain events and establish a connection to the Solana blockchain.arg: SubscriberRequest: This parameter specifies the data that we want to fetch from the blockchain.
By using the handleStream() function in this way, we can efficiently and conveniently stream real-time on-chain data on the Solana blockchain.
interface SubscribeRequest {
accounts: { [key: string]: SubscribeRequestFilterAccounts };
slots: { [key: string]: SubscribeRequestFilterSlots };
transactions: { [key: string]: SubscribeRequestFilterTransactions };
transactionsStatus: { [key: string]: SubscribeRequestFilterTransactions };
blocks: { [key: string]: SubscribeRequestFilterBlocks };
blocksMeta: { [key: string]: SubscribeRequestFilterBlocksMeta };
entry: { [key: string]: SubscribeRequestFilterEntry };
commitment?: CommitmentLevel | undefined;
accountsDataSlice: SubscribeRequestAccountsDataSlice[];
ping?: SubscribeRequestPing | undefined;
}async function handleStream(client: Client, args: SubscribeRequest) {
// Subscribe for events
const stream = await client.subscribe();
// Create `error` / `end` handler
const streamClosed = new Promise<void>((resolve, reject) => {
stream.on("error", (error) => {
console.log("ERROR", error);
reject(error);
stream.end();
});
stream.on("end", () => {
resolve();
});
stream.on("close", () => {
resolve();
});
});
// Handle updates
stream.on("data", async (data) => {
try{
console.log(data);
}catch(error){
if(error){
console.log(error)
}
}
});
// Send subscribe request
await new Promise<void>((resolve, reject) => {
stream.write(args, (err: any) => {
if (err === null || err === undefined) {
resolve();
} else {
reject(err);
}
});
}).catch((reason) => {
console.error(reason);
throw reason;
});
await streamClosed;
}After defining the SubscriberRequest parameter for the handleStream() function, the next step is to specify the data that we want to fetch from the Solana blockchain. In this tutorial, we are focusing on fetching token transactions on pump.fun. To do this, we need to define a SubscriberRequest object that specifies the parameters for our data request. This object should include the following information:
- The type of data that we want to fetch (in this case, token transactions)
- The address of pump.fun
- Any additional filters or parameters that are needed to narrow down the data that we want to fetch
Here is an example of what the SubscriberRequest object for fetching token transactions on pump.fun might look like:
const req = {
accounts: {},
slots: {},
transactions: {
bondingCurve: {
vote: false,
failed: false,
signature: undefined,
accountInclude: ['6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P'], //Address 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P
accountExclude: [],
accountRequired: [],
},
},
transactionsStatus: {},
entry: {},
blocks: {},
blocksMeta: {},
accountsDataSlice: [],
ping: undefined,
commitment: CommitmentLevel.CONFIRMED, //for receiving confirmed txn updates
};The address 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P represents the on-chain address for the “pump.fun” platform on the Solana blockchain. However, this address alone is not sufficient to fetch the data that we need. To access the real-time on-chain data, we need to define the SubscribeCommand function. This function will be used to specify the parameters for our data request and subscribe to a streaming endpoint on the Solana blockchain. The SubscribeCommand function should take the SubscriberRequest object as a parameter and use it to establish a connection to the Solana blockchain and subscribe to the desired streaming endpoint.
async function subscribeCommand(client: Client, args: SubscribeRequest) {
while (true) {
try {
await handleStream(client, args);
} catch (error) {
console.error("Stream error, restarting in 1 second...", error);
await new Promise((resolve) => setTimeout(resolve, 1000));
}
}
}and call it
subscribeCommand(client, req);our output should look like this
{
filters: [ 'bondingCurve' ],
account: undefined,
slot: undefined,
transaction: {
transaction: {
signature: <Buffer 18 f2 41 ca 0d c3 2d 24 a3 4c 25 0b 0f d5 e0 bd fd 75 35 a1 ed a3 43 8a f8 fc 86 c0 01 08 70 32 f9 ac 53 3c 46 e8 e2 62 22 4c 79 ab 3f f5 4b d5 81 0c ... 14 more bytes>,
isVote: false,
transaction: [Object],
meta: [Object],
index: '315'
},
slot: '283417510'
},
block: undefined,
ping: undefined,
pong: undefined,
blockMeta: undefined,
entry: undefined
}Although this is the data we requested for, it is not the data we hope to get. We have come this close, we only need to deserialize our output to make it comprehensible.
In high-frequency trading, a 1-second disconnect can mean missing dozens of launches. To ensure 100% uptime, follow our guide on implementing gRPC Reconnect and Slot Replay logic.
Processing and Deserializing the streaming data in real-time
In order to convert the data received from the streaming endpoint into a more readable and usable format, a process known as deserialization is required. Here’s an example of how you could write a function to deserialize the data
import base58 from "bs58";
export function decodeTransact(data){
const output = base58.encode(Buffer.from(data,'base64'))
return output;
}The function decode any Buffered data to more usable format.
Next we would want to defined our output in a more formatted manner
export function tOutPut(data){
const dataTx = data.transaction.transaction
const signature = decodeTransact(dataTx.signature);
const message = dataTx.transaction?.message
const header = message.header;
const accountKeys = message.accountKeys.map((t)=>{
return decodeTransact(t)
})
const recentBlockhash = decodeTransact(message.recentBlockhash);
const instructions = message.instructions
const meta = dataTx?.meta
return {
signature,
message:{
header,
accountKeys,
recentBlockhash,
instructions
},
meta
}
}Then we rewrite our handleStream()function to fit into the updated codes
async function handleStream(client: Client, args: SubscribeRequest) {
// Subscribe for events
const stream = await client.subscribe();
// Create `error` / `end` handler
const streamClosed = new Promise<void>((resolve, reject) => {
stream.on("error", (error) => {
console.log("ERROR", error);
reject(error);
stream.end();
});
stream.on("end", () => {
resolve();
});
stream.on("close", () => {
resolve();
});
});
// Handle updates
stream.on("data", async (data) => {
try{
const result = await tOutPut(data);
console.log(result);
}catch(error){
if(error){
console.log(error)
}
}
});
// Send subscribe request
await new Promise<void>((resolve, reject) => {
stream.write(args, (err: any) => {
if (err === null || err === undefined) {
resolve();
} else {
reject(err);
}
});
}).catch((reason) => {
console.error(reason);
throw reason;
});
await streamClosed;
}Finally we make our call with the SubscribeCommand Function
subscribeCommand(client, req);our updated output should look like this
signature: '4aTaVifyJNwaxrqP1939p6Cwi7doEJ627eo7XV1dC6cHHDLVf5JJ8qSdsVzQbmFRd2vWDK7sDZRHWWvuWwsnaCKj',
message: {
header: {
numRequiredSignatures: 1,
numReadonlySignedAccounts: 0,
numReadonlyUnsignedAccounts: 8
},
accountKeys: [
'Dn2sbLk6cMYyRu9D5h1G66vUGFTE7BRgqho9X6c52wLE',
'CebN5WGQ4jvEPvsVU4EoHEpgzq1VV7AbicfhtW4xC9iM',
'Gf72ajaDxUPSgXbRCnYjAXpPQXofam6FmedepfYJNY4B',
'7GXjNDdNkSnsmaXmWUy4Woh6NAQgMCABBEPWtHyLBJ9J',
'CcCAgQcArc6jjF2zAaDUzQ1FKgisqDjubCNJvVmPHVMU',
'ZG98FUCjb8mJ824Gbs6RsgVmr1FhXb2oNiJHa2dwmPd',
'ComputeBudget111111111111111111111111111111',
'6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P',
'4wTV1YmiEkRvAtNtsSGPtUrqRYQMe5SKy2uB4Jjaxnjf',
'C1Fabt6rjQGcS68obCEH6zuywWfzpfvCCfKBJntCpump',
'11111111111111111111111111111111',
'ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL',
'TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA',
'Ce6TQqeHC9p8KetsN6JsjHK7UTZk7nasjjnr7XxXp9F1'
],
recentBlockhash: 'EGc4SgYd4xmex8ZS4LQcjtvzBW8aU3yafphxscM9hBTa',
instructions: [ [Object], [Object], [Object] ]
},
meta: {
err: undefined,
fee: '405000',
preBalances: [
'822302413', '272648835990597',
'1191944363', '2039280',
'2039280', '13219204698',
'1', '1141440',
'2500000', '1461600',
'1', '731913600',
'934087680', '0'
],
postBalances: [
'843092001', '272648836206849',
'1170319136', '2039280',
'2039280', '13219419085',
'1', '1141440',
'2500000', '1461600',
'1', '731913600',
'934087680', '0'
],
innerInstructions: [ [Object] ],
innerInstructionsNone: false,
logMessages: [
'Program ComputeBudget111111111111111111111111111111 invoke [1]',
'Program ComputeBudget111111111111111111111111111111 success',
'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P invoke [1]',
'Program log: Instruction: Sell',
'Program TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA invoke [2]',
'Program log: Instruction: Transfer',
'Program TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA consumed 4645 of 381291 compute units',
'Program TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA success',
'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P invoke [2]',
'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P consumed 2003 of 373161 compute units',
'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P success',
'Program data: vdt/007mYe6jgQqFQndU+0Owx6KwKFXj5LXaRYNPQ2h7er5FKWGLH4v5SQEAAAAAL/LStqYAAAAAvdV/G53p8Qe4ABBpTvcEEXUBRn8h6KfMWZcLzg+Qi3tazrtmAAAAAPCG0kEHAAAAuu/G0EirAwDw2q5FAAAAALpXtIS3rAIA',
'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P consumed 30418 of 399850 compute units',
'Program 6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P success',
'Program 11111111111111111111111111111111 invoke [1]',
'Program 11111111111111111111111111111111 success'
],
logMessagesNone: false,
preTokenBalances: [ [Object], [Object] ],
postTokenBalances: [ [Object], [Object] ],
rewards: [],
loadedWritableAddresses: [],
loadedReadonlyAddresses: [],
returnData: undefined,
returnDataNone: true,
computeUnitsConsumed: '30718'
}
}Get started with Shyft’s Yellowstone gRPC to access unmetered real-time data.
Conclusion
Streaming data on the Solana blockchain using dedicated nodes can indeed be a complex and resource-intensive task, requiring a larger and more intricate code base. However, with gRPC, developers can now conveniently stream on-chain data with ease, simplifying their code and allowing them to focus on building robust applications that tap into the power of the Solana blockchain. At Shyft, we are thrilled to introduce this innovative solution to the developer community. Our team is committed to providing the necessary tools and support to make this process seamless. We encourage developers to join our Discord Server or follow us on Twitter for updates and further assistance. With gRPCs, developers can take advantage of the low latency and reduced code base to build complex projects with ease. For further assistance, join our Discord Server or follow us on Twitter for updates.
All the code related to this article is available on our GitHub here. Please feel free to clone it and follow along. To know more about data streaming on Solana, checkout our other articles on streaming real-time data on Solana or Tracking new Pools on Raydium.
