forked from wownero/wownero
parent
5d850dde99
commit
e5214a2ca2
@ -0,0 +1,61 @@
|
|||||||
|
# The Current/Future Status of ZMQ in Monero
|
||||||
|
|
||||||
|
## ZMQ Pub/Sub
|
||||||
|
Client `ZMQ_SUB` sockets must "subscribe" to topics before it receives any data.
|
||||||
|
This allows filtering on the server side, so network traffic is reduced. Monero
|
||||||
|
allows for filtering on: (1) format, (2) context, and (3) event.
|
||||||
|
|
||||||
|
* **format** refers to the _wire_ format (i.e. JSON) used to send event
|
||||||
|
information.
|
||||||
|
* **context** allows for a reduction in fields for the event, so the
|
||||||
|
daemon doesn't waste cycles serializing fields that get ignored.
|
||||||
|
* **event** refers to status changes occurring within the daemon (i.e. new
|
||||||
|
block to main chain).
|
||||||
|
|
||||||
|
* Formats:
|
||||||
|
* `json`
|
||||||
|
* Contexts:
|
||||||
|
* `full` - the entire block or transaction is transmitted (the hash can be
|
||||||
|
computed remotely).
|
||||||
|
* `minimal` - the bare minimum for a remote client to react to an event is
|
||||||
|
sent.
|
||||||
|
* Events:
|
||||||
|
* `chain_main` - changes to the primary/main blockchain.
|
||||||
|
* `txpool_add` - new _publicly visible_ transactions in the mempool.
|
||||||
|
Includes previously unseen transactions in a block but _not_ the
|
||||||
|
`miner_tx`. Does not "re-publish" after a reorg. Includes `do_not_relay`
|
||||||
|
transactions.
|
||||||
|
|
||||||
|
The subscription topics are formatted as `format-context-event`, with prefix
|
||||||
|
matching supported by both Monero and ZMQ. The `format`, `context` and `event`
|
||||||
|
will _never_ have hyphens or colons in their name. For example, subscribing to
|
||||||
|
`json-minimal-chain_main` will send minimal information in JSON when changes
|
||||||
|
to the main/primary blockchain occur. Whereas, subscribing to `json-minimal`
|
||||||
|
will send minimal information in JSON on all available events supported by the
|
||||||
|
daemon.
|
||||||
|
|
||||||
|
The Monero daemon will ensure that events prefixed by `chain` will be sent in
|
||||||
|
"chain-order" - the `prev_id` (hash) field will _always_ refer to a previous
|
||||||
|
block. On rollbacks/reorgs, the event will reference an earlier block in the
|
||||||
|
chain instead of the last block. The Monero daemon also ensures that
|
||||||
|
`txpool_add` events are sent before `chain_*` events - the `chain_*` messages
|
||||||
|
will only serialize miner transactions since the other transactions were
|
||||||
|
previously published via `txpool_add`. This prevents transactions from being
|
||||||
|
serialized twice, even when the transaction was first observed in a block.
|
||||||
|
|
||||||
|
ZMQ Pub/Sub will drop messages if the network is congested, so the above rules
|
||||||
|
for send order are used for detecting lost messages. A missing gap in `height`
|
||||||
|
or `prev_id` for `chain_*` events indicates a lost pub message. Missing
|
||||||
|
`txpool_add` messages can only be detected at the next `chain_` message.
|
||||||
|
|
||||||
|
Since blockchain events can be dropped, clients will likely want to have a
|
||||||
|
timeout against `chain_main` events. The `GetLastBlockHeader` RPC is useful
|
||||||
|
for checking the current chain state. Dropped messages should be rare in most
|
||||||
|
conditions.
|
||||||
|
|
||||||
|
The Monero daemon will send a `txpool_add` pub exactly once for each
|
||||||
|
transaction, even after a reorg or restarts. Clients should use the
|
||||||
|
`GetTransactionPool` after a reorg to get all transactions that have been put
|
||||||
|
back into the tx pool or been invalidated due to a double-spend.
|
||||||
|
|
||||||
|
|
@ -0,0 +1,46 @@
|
|||||||
|
// Copyright (c) 2020, The Monero Project
|
||||||
|
//
|
||||||
|
// All rights reserved.
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without modification, are
|
||||||
|
// permitted provided that the following conditions are met:
|
||||||
|
//
|
||||||
|
// 1. Redistributions of source code must retain the above copyright notice, this list of
|
||||||
|
// conditions and the following disclaimer.
|
||||||
|
//
|
||||||
|
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
|
||||||
|
// of conditions and the following disclaimer in the documentation and/or other
|
||||||
|
// materials provided with the distribution.
|
||||||
|
//
|
||||||
|
// 3. Neither the name of the copyright holder nor the names of its contributors may be
|
||||||
|
// used to endorse or promote products derived from this software without specific
|
||||||
|
// prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
|
||||||
|
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||||
|
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
|
||||||
|
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
|
||||||
|
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
|
||||||
|
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include "crypto/hash.h"
|
||||||
|
#include "cryptonote_basic/cryptonote_basic.h"
|
||||||
|
|
||||||
|
namespace cryptonote
|
||||||
|
{
|
||||||
|
/*! Transactions are expensive to move or copy (lots of 32-byte internal
|
||||||
|
buffers). This allows `cryptonote::core` to do a single notification for
|
||||||
|
a vector of transactions, without having to move/copy duplicate or invalid
|
||||||
|
transactions. */
|
||||||
|
struct txpool_event
|
||||||
|
{
|
||||||
|
cryptonote::transaction tx;
|
||||||
|
crypto::hash hash;
|
||||||
|
bool res; //!< Listeners must ignore `tx` when this is false.
|
||||||
|
};
|
||||||
|
}
|
@ -0,0 +1,36 @@
|
|||||||
|
// Copyright (c) 2020, The Monero Project
|
||||||
|
//
|
||||||
|
// All rights reserved.
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without modification, are
|
||||||
|
// permitted provided that the following conditions are met:
|
||||||
|
//
|
||||||
|
// 1. Redistributions of source code must retain the above copyright notice, this list of
|
||||||
|
// conditions and the following disclaimer.
|
||||||
|
//
|
||||||
|
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
|
||||||
|
// of conditions and the following disclaimer in the documentation and/or other
|
||||||
|
// materials provided with the distribution.
|
||||||
|
//
|
||||||
|
// 3. Neither the name of the copyright holder nor the names of its contributors may be
|
||||||
|
// used to endorse or promote products derived from this software without specific
|
||||||
|
// prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
|
||||||
|
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||||
|
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
|
||||||
|
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
|
||||||
|
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
|
||||||
|
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
namespace cryptonote
|
||||||
|
{
|
||||||
|
struct block;
|
||||||
|
class transaction;
|
||||||
|
struct txpool_event;
|
||||||
|
}
|
@ -0,0 +1,37 @@
|
|||||||
|
// Copyright (c) 2019-2020, The Monero Project
|
||||||
|
//
|
||||||
|
// All rights reserved.
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without modification, are
|
||||||
|
// permitted provided that the following conditions are met:
|
||||||
|
//
|
||||||
|
// 1. Redistributions of source code must retain the above copyright notice, this list of
|
||||||
|
// conditions and the following disclaimer.
|
||||||
|
//
|
||||||
|
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
|
||||||
|
// of conditions and the following disclaimer in the documentation and/or other
|
||||||
|
// materials provided with the distribution.
|
||||||
|
//
|
||||||
|
// 3. Neither the name of the copyright holder nor the names of its contributors may be
|
||||||
|
// used to endorse or promote products derived from this software without specific
|
||||||
|
// prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
|
||||||
|
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||||
|
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
|
||||||
|
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
|
||||||
|
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
|
||||||
|
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
namespace cryptonote
|
||||||
|
{
|
||||||
|
namespace listener
|
||||||
|
{
|
||||||
|
class zmq_pub;
|
||||||
|
}
|
||||||
|
}
|
@ -0,0 +1,478 @@
|
|||||||
|
// Copyright (c) 2020, The Monero Project
|
||||||
|
//
|
||||||
|
// All rights reserved.
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without modification, are
|
||||||
|
// permitted provided that the following conditions are met:
|
||||||
|
//
|
||||||
|
// 1. Redistributions of source code must retain the above copyright notice, this list of
|
||||||
|
// conditions and the following disclaimer.
|
||||||
|
//
|
||||||
|
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
|
||||||
|
// of conditions and the following disclaimer in the documentation and/or other
|
||||||
|
// materials provided with the distribution.
|
||||||
|
//
|
||||||
|
// 3. Neither the name of the copyright holder nor the names of its contributors may be
|
||||||
|
// used to endorse or promote products derived from this software without specific
|
||||||
|
// prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
|
||||||
|
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||||
|
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
|
||||||
|
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
|
||||||
|
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
|
||||||
|
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
#include "zmq_pub.h"
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <boost/range/adaptor/filtered.hpp>
|
||||||
|
#include <boost/range/adaptor/transformed.hpp>
|
||||||
|
#include <boost/thread/locks.hpp>
|
||||||
|
#include <cassert>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstring>
|
||||||
|
#include <rapidjson/document.h>
|
||||||
|
#include <rapidjson/stringbuffer.h>
|
||||||
|
#include <rapidjson/writer.h>
|
||||||
|
#include <stdexcept>
|
||||||
|
#include <string>
|
||||||
|
#include <utility>
|
||||||
|
|
||||||
|
#include "common/expect.h"
|
||||||
|
#include "crypto/crypto.h"
|
||||||
|
#include "cryptonote_basic/cryptonote_format_utils.h"
|
||||||
|
#include "cryptonote_basic/events.h"
|
||||||
|
#include "misc_log_ex.h"
|
||||||
|
#include "serialization/json_object.h"
|
||||||
|
|
||||||
|
#undef MONERO_DEFAULT_LOG_CATEGORY
|
||||||
|
#define MONERO_DEFAULT_LOG_CATEGORY "net.zmq"
|
||||||
|
|
||||||
|
namespace
|
||||||
|
{
|
||||||
|
constexpr const char txpool_signal[] = "tx_signal";
|
||||||
|
|
||||||
|
using chain_writer = void(epee::byte_stream&, std::uint64_t, epee::span<const cryptonote::block>);
|
||||||
|
using txpool_writer = void(epee::byte_stream&, epee::span<const cryptonote::txpool_event>);
|
||||||
|
|
||||||
|
template<typename F>
|
||||||
|
struct context
|
||||||
|
{
|
||||||
|
char const* const name;
|
||||||
|
F* generate_pub;
|
||||||
|
};
|
||||||
|
|
||||||
|
template<typename T>
|
||||||
|
bool operator<(const context<T>& lhs, const context<T>& rhs) noexcept
|
||||||
|
{
|
||||||
|
return std::strcmp(lhs.name, rhs.name) < 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
template<typename T>
|
||||||
|
bool operator<(const context<T>& lhs, const boost::string_ref rhs) noexcept
|
||||||
|
{
|
||||||
|
return lhs.name < rhs;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct is_valid
|
||||||
|
{
|
||||||
|
bool operator()(const cryptonote::txpool_event& event) const noexcept
|
||||||
|
{
|
||||||
|
return event.res;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
template<typename T, std::size_t N>
|
||||||
|
void verify_sorted(const std::array<context<T>, N>& elems, const char* name)
|
||||||
|
{
|
||||||
|
auto unsorted = std::is_sorted_until(elems.begin(), elems.end());
|
||||||
|
if (unsorted != elems.end())
|
||||||
|
throw std::logic_error{name + std::string{" array is not properly sorted, see: "} + unsorted->name};
|
||||||
|
}
|
||||||
|
|
||||||
|
void write_header(epee::byte_stream& buf, const boost::string_ref name)
|
||||||
|
{
|
||||||
|
buf.write(name.data(), name.size());
|
||||||
|
buf.put(':');
|
||||||
|
}
|
||||||
|
|
||||||
|
//! \return `name:...` where `...` is JSON and `name` is directly copied (no quotes - not JSON).
|
||||||
|
template<typename T>
|
||||||
|
void json_pub(epee::byte_stream& buf, const T value)
|
||||||
|
{
|
||||||
|
rapidjson::Writer<epee::byte_stream> dest{buf};
|
||||||
|
using cryptonote::json::toJsonValue;
|
||||||
|
toJsonValue(dest, value);
|
||||||
|
}
|
||||||
|
|
||||||
|
//! Object for "minimal" block serialization
|
||||||
|
struct minimal_chain
|
||||||
|
{
|
||||||
|
const std::uint64_t height;
|
||||||
|
const epee::span<const cryptonote::block> blocks;
|
||||||
|
};
|
||||||
|
|
||||||
|
//! Object for "minimal" tx serialization
|
||||||
|
struct minimal_txpool
|
||||||
|
{
|
||||||
|
const cryptonote::transaction& tx;
|
||||||
|
};
|
||||||
|
|
||||||
|
void toJsonValue(rapidjson::Writer<epee::byte_stream>& dest, const minimal_chain self)
|
||||||
|
{
|
||||||
|
namespace adapt = boost::adaptors;
|
||||||
|
|
||||||
|
const auto to_block_id = [](const cryptonote::block& bl)
|
||||||
|
{
|
||||||
|
crypto::hash id;
|
||||||
|
if (!get_block_hash(bl, id))
|
||||||
|
MERROR("ZMQ/Pub failure: get_block_hash");
|
||||||
|
return id;
|
||||||
|
};
|
||||||
|
|
||||||
|
assert(!self.blocks.empty()); // checked in zmq_pub::send_chain_main
|
||||||
|
|
||||||
|
dest.StartObject();
|
||||||
|
INSERT_INTO_JSON_OBJECT(dest, first_height, self.height);
|
||||||
|
INSERT_INTO_JSON_OBJECT(dest, first_prev_id, self.blocks[0].prev_id);
|
||||||
|
INSERT_INTO_JSON_OBJECT(dest, ids, (self.blocks | adapt::transformed(to_block_id)));
|
||||||
|
dest.EndObject();
|
||||||
|
}
|
||||||
|
|
||||||
|
void toJsonValue(rapidjson::Writer<epee::byte_stream>& dest, const minimal_txpool self)
|
||||||
|
{
|
||||||
|
crypto::hash id{};
|
||||||
|
std::size_t blob_size = 0;
|
||||||
|
if (!get_transaction_hash(self.tx, id, blob_size))
|
||||||
|
{
|
||||||
|
MERROR("ZMQ/Pub failure: get_transaction_hash");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
dest.StartObject();
|
||||||
|
INSERT_INTO_JSON_OBJECT(dest, id, id);
|
||||||
|
INSERT_INTO_JSON_OBJECT(dest, blob_size, blob_size);
|
||||||
|
dest.EndObject();
|
||||||
|
}
|
||||||
|
|
||||||
|
void json_full_chain(epee::byte_stream& buf, const std::uint64_t height, const epee::span<const cryptonote::block> blocks)
|
||||||
|
{
|
||||||
|
json_pub(buf, blocks);
|
||||||
|
}
|
||||||
|
|
||||||
|
void json_minimal_chain(epee::byte_stream& buf, const std::uint64_t height, const epee::span<const cryptonote::block> blocks)
|
||||||
|
{
|
||||||
|
json_pub(buf, minimal_chain{height, blocks});
|
||||||
|
}
|
||||||
|
|
||||||
|
// boost::adaptors are in place "views" - no copy/move takes place
|
||||||
|
// moving transactions (via sort, etc.), is expensive!
|
||||||
|
|
||||||
|
void json_full_txpool(epee::byte_stream& buf, epee::span<const cryptonote::txpool_event> txes)
|
||||||
|
{
|
||||||
|
namespace adapt = boost::adaptors;
|
||||||
|
const auto to_full_tx = [](const cryptonote::txpool_event& event)
|
||||||
|
{
|
||||||
|
return event.tx;
|
||||||
|
};
|
||||||
|
json_pub(buf, (txes | adapt::filtered(is_valid{}) | adapt::transformed(to_full_tx)));
|
||||||
|
}
|
||||||
|
|
||||||
|
void json_minimal_txpool(epee::byte_stream& buf, epee::span<const cryptonote::txpool_event> txes)
|
||||||
|
{
|
||||||
|
namespace adapt = boost::adaptors;
|
||||||
|
const auto to_minimal_tx = [](const cryptonote::txpool_event& event)
|
||||||
|
{
|
||||||
|
return minimal_txpool{event.tx};
|
||||||
|
};
|
||||||
|
json_pub(buf, (txes | adapt::filtered(is_valid{}) | adapt::transformed(to_minimal_tx)));
|
||||||
|
}
|
||||||
|
|
||||||
|
constexpr const std::array<context<chain_writer>, 2> chain_contexts =
|
||||||
|
{{
|
||||||
|
{u8"json-full-chain_main", json_full_chain},
|
||||||
|
{u8"json-minimal-chain_main", json_minimal_chain}
|
||||||
|
}};
|
||||||
|
|
||||||
|
constexpr const std::array<context<txpool_writer>, 2> txpool_contexts =
|
||||||
|
{{
|
||||||
|
{u8"json-full-txpool_add", json_full_txpool},
|
||||||
|
{u8"json-minimal-txpool_add", json_minimal_txpool}
|
||||||
|
}};
|
||||||
|
|
||||||
|
template<typename T, std::size_t N>
|
||||||
|
epee::span<const context<T>> get_range(const std::array<context<T>, N>& contexts, const boost::string_ref value)
|
||||||
|
{
|
||||||
|
const auto not_prefix = [](const boost::string_ref lhs, const context<T>& rhs)
|
||||||
|
{
|
||||||
|
return !(boost::string_ref{rhs.name}.starts_with(lhs));
|
||||||
|
};
|
||||||
|
|
||||||
|
const auto lower = std::lower_bound(contexts.begin(), contexts.end(), value);
|
||||||
|
const auto upper = std::upper_bound(lower, contexts.end(), value, not_prefix);
|
||||||
|
return {lower, std::size_t(upper - lower)};
|
||||||
|
}
|
||||||
|
|
||||||
|
template<std::size_t N, typename T>
|
||||||
|
void add_subscriptions(std::array<std::size_t, N>& subs, const epee::span<const context<T>> range, context<T> const* const first)
|
||||||
|
{
|
||||||
|
assert(range.size() <= N);
|
||||||
|
assert(range.begin() - first <= N - range.size());
|
||||||
|
|
||||||
|
for (const auto& ctx : range)
|
||||||
|
{
|
||||||
|
const std::size_t i = std::addressof(ctx) - first;
|
||||||
|
subs[i] = std::min(std::numeric_limits<std::size_t>::max() - 1, subs[i]) + 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
template<std::size_t N, typename T>
|
||||||
|
void remove_subscriptions(std::array<std::size_t, N>& subs, const epee::span<const context<T>> range, context<T> const* const first)
|
||||||
|
{
|
||||||
|
assert(range.size() <= N);
|
||||||
|
assert(range.begin() - first <= N - range.size());
|
||||||
|
|
||||||
|
for (const auto& ctx : range)
|
||||||
|
{
|
||||||
|
const std::size_t i = std::addressof(ctx) - first;
|
||||||
|
subs[i] = std::max(std::size_t(1), subs[i]) - 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
template<std::size_t N, typename T, typename... U>
|
||||||
|
std::array<epee::byte_slice, N> make_pubs(const std::array<std::size_t, N>& subs, const std::array<context<T>, N>& contexts, U&&... args)
|
||||||
|
{
|
||||||
|
epee::byte_stream buf{};
|
||||||
|
|
||||||
|
std::size_t last_offset = 0;
|
||||||
|
std::array<std::size_t, N> offsets{{}};
|
||||||
|
for (std::size_t i = 0; i < N; ++i)
|
||||||
|
{
|
||||||
|
if (subs[i])
|
||||||
|
{
|
||||||
|
write_header(buf, contexts[i].name);
|
||||||
|
contexts[i].generate_pub(buf, std::forward<U>(args)...);
|
||||||
|
offsets[i] = buf.size() - last_offset;
|
||||||
|
last_offset = buf.size();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
epee::byte_slice bytes{std::move(buf)};
|
||||||
|
std::array<epee::byte_slice, N> out;
|
||||||
|
for (std::size_t i = 0; i < N; ++i)
|
||||||
|
out[i] = bytes.take_slice(offsets[i]);
|
||||||
|
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
template<std::size_t N>
|
||||||
|
std::size_t send_messages(void* const socket, std::array<epee::byte_slice, N>& messages)
|
||||||
|
{
|
||||||
|
std::size_t count = 0;
|
||||||
|
for (epee::byte_slice& message : messages)
|
||||||
|
{
|
||||||
|
if (!message.empty())
|
||||||
|
{
|
||||||
|
const expect<void> sent = net::zmq::send(std::move(message), socket, ZMQ_DONTWAIT);
|
||||||
|
if (!sent)
|
||||||
|
MERROR("Failed to send ZMQ/Pub message: " << sent.error().message());
|
||||||
|
else
|
||||||
|
++count;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
expect<bool> relay_block_pub(void* const relay, void* const pub) noexcept
|
||||||
|
{
|
||||||
|
zmq_msg_t msg;
|
||||||
|
zmq_msg_init(std::addressof(msg));
|
||||||
|
MONERO_CHECK(net::zmq::retry_op(zmq_msg_recv, std::addressof(msg), relay, ZMQ_DONTWAIT));
|
||||||
|
|
||||||
|
const boost::string_ref payload{
|
||||||
|
reinterpret_cast<const char*>(zmq_msg_data(std::addressof(msg))),
|
||||||
|
zmq_msg_size(std::addressof(msg))
|
||||||
|
};
|
||||||
|
|
||||||
|
if (payload == txpool_signal)
|
||||||
|
{
|
||||||
|
zmq_msg_close(std::addressof(msg));
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// forward block messages (serialized on P2P thread for now)
|
||||||
|
const expect<void> sent = net::zmq::retry_op(zmq_msg_send, std::addressof(msg), pub, ZMQ_DONTWAIT);
|
||||||
|
if (!sent)
|
||||||
|
{
|
||||||
|
zmq_msg_close(std::addressof(msg));
|
||||||
|
return sent.error();
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
} // anonymous
|
||||||
|
|
||||||
|
namespace cryptonote { namespace listener
|
||||||
|
{
|
||||||
|
|
||||||
|
zmq_pub::zmq_pub(void* context)
|
||||||
|
: relay_(),
|
||||||
|
chain_subs_{{0}},
|
||||||
|
txpool_subs_{{0}},
|
||||||
|
sync_()
|
||||||
|
{
|
||||||
|
if (!context)
|
||||||
|
throw std::logic_error{"ZMQ context cannot be NULL"};
|
||||||
|
|
||||||
|
verify_sorted(chain_contexts, "chain_contexts");
|
||||||
|
verify_sorted(txpool_contexts, "txpool_contexts");
|
||||||
|
|
||||||
|
relay_.reset(zmq_socket(context, ZMQ_PAIR));
|
||||||
|
if (!relay_)
|
||||||
|
MONERO_ZMQ_THROW("Failed to create relay socket");
|
||||||
|
if (zmq_connect(relay_.get(), relay_endpoint()) != 0)
|
||||||
|
MONERO_ZMQ_THROW("Failed to connect relay socket");
|
||||||
|
}
|
||||||
|
|
||||||
|
zmq_pub::~zmq_pub()
|
||||||
|
{}
|
||||||
|
|
||||||
|
bool zmq_pub::sub_request(boost::string_ref message)
|
||||||
|
{
|
||||||
|
if (!message.empty())
|
||||||
|
{
|
||||||
|
const char tag = message[0];
|
||||||
|
message.remove_prefix(1);
|
||||||
|
|
||||||
|
const auto chain_range = get_range(chain_contexts, message);
|
||||||
|
const auto txpool_range = get_range(txpool_contexts, message);
|
||||||
|
|
||||||
|
if (!chain_range.empty() || !txpool_range.empty())
|
||||||
|
{
|
||||||
|
MDEBUG("Client " << (tag ? "subscribed" : "unsubscribed") << " to " <<
|
||||||
|
chain_range.size() << " chain topic(s) and " << txpool_range.size() << " txpool topic(s)");
|
||||||
|
|
||||||
|
const boost::lock_guard<boost::mutex> lock{sync_};
|
||||||
|
switch (tag)
|
||||||
|
{
|
||||||
|
case 0:
|
||||||
|
remove_subscriptions(chain_subs_, chain_range, chain_contexts.begin());
|
||||||
|
remove_subscriptions(txpool_subs_, txpool_range, txpool_contexts.begin());
|
||||||
|
return true;
|
||||||
|
case 1:
|
||||||
|
add_subscriptions(chain_subs_, chain_range, chain_contexts.begin());
|
||||||
|
add_subscriptions(txpool_subs_, txpool_range, txpool_contexts.begin());
|
||||||
|
return true;
|
||||||
|
default:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
MERROR("Invalid ZMQ/Sub message");
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
bool zmq_pub::relay_to_pub(void* const relay, void* const pub)
|
||||||
|
{
|
||||||
|
const expect<bool> relayed = relay_block_pub(relay, pub);
|
||||||
|
if (!relayed)
|
||||||
|
{
|
||||||
|
MERROR("Error relaying ZMQ/Pub: " << relayed.error().message());
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!*relayed)
|
||||||
|
{
|
||||||
|
std::array<std::size_t, 2> subs;
|
||||||
|
std::vector<cryptonote::txpool_event> events;
|
||||||
|
{
|
||||||
|
const boost::lock_guard<boost::mutex> lock{sync_};
|
||||||
|
if (txes_.empty())
|
||||||
|
return false;
|
||||||
|
|
||||||
|
subs = txpool_subs_;
|
||||||
|
events = std::move(txes_.front());
|
||||||
|
txes_.pop_front();
|
||||||
|
}
|
||||||
|
auto messages = make_pubs(subs, txpool_contexts, epee::to_span(events));
|
||||||
|
send_messages(pub, messages);
|
||||||
|
MDEBUG("Sent txpool ZMQ/Pub");
|
||||||
|
}
|
||||||
|
else
|
||||||
|
MDEBUG("Sent chain_main ZMQ/Pub");
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::size_t zmq_pub::send_chain_main(const std::uint64_t height, const epee::span<const cryptonote::block> blocks)
|
||||||
|
{
|
||||||
|
if (blocks.empty())
|
||||||
|
return 0;
|
||||||
|
|
||||||
|
/* Block format only sends one block at a time - multiple block notifications
|
||||||
|
are less common and only occur on rollbacks. */
|
||||||
|
|
||||||
|
boost::unique_lock<boost::mutex> guard{sync_};
|
||||||
|
|
||||||
|
const auto subs_copy = chain_subs_;
|
||||||
|
guard.unlock();
|
||||||
|
|
||||||
|
for (const std::size_t sub : subs_copy)
|
||||||
|
{
|
||||||
|
if (sub)
|
||||||
|
{
|
||||||
|
/* cryptonote_core/blockchain.cpp cannot "give" us the block like core
|
||||||
|
does for txpool events. Since copying the block is expensive anyway,
|
||||||
|
serialization is done right here on the p2p thread (for now). */
|
||||||
|
|
||||||
|
auto messages = make_pubs(subs_copy, chain_contexts, height, blocks);
|
||||||
|
guard.lock();
|
||||||
|
return send_messages(relay_.get(), messages);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::size_t zmq_pub::send_txpool_add(std::vector<txpool_event> txes)
|
||||||
|
{
|
||||||
|
if (txes.empty())
|
||||||
|
return 0;
|
||||||
|
|
||||||
|
const boost::lock_guard<boost::mutex> lock{sync_};
|
||||||
|
for (const std::size_t sub : txpool_subs_)
|
||||||
|
{
|
||||||
|
if (sub)
|
||||||
|
{
|
||||||
|
const expect<void> sent = net::zmq::retry_op(zmq_send_const, relay_.get(), txpool_signal, sizeof(txpool_signal) - 1, ZMQ_DONTWAIT);
|
||||||
|
if (sent)
|
||||||
|
txes_.emplace_back(std::move(txes));
|
||||||
|
else
|
||||||
|
MERROR("ZMQ/Pub failure, relay queue error: " << sent.error().message());
|
||||||
|
return bool(sent);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
void zmq_pub::chain_main::operator()(const std::uint64_t height, epee::span<const cryptonote::block> blocks) const
|
||||||
|
{
|
||||||
|
const std::shared_ptr<zmq_pub> self = self_.lock();
|
||||||
|
if (self)
|
||||||
|
self->send_chain_main(height, blocks);
|
||||||
|
else
|
||||||
|
MERROR("Unable to send ZMQ/Pub - ZMQ server destroyed");
|
||||||
|
}
|
||||||
|
|
||||||
|
void zmq_pub::txpool_add::operator()(std::vector<cryptonote::txpool_event> txes) const
|
||||||
|
{
|
||||||
|
const std::shared_ptr<zmq_pub> self = self_.lock();
|
||||||
|
if (self)
|
||||||
|
self->send_txpool_add(std::move(txes));
|
||||||
|
else
|
||||||
|
MERROR("Unable to send ZMQ/Pub - ZMQ server destroyed");
|
||||||
|
}
|
||||||
|
|
||||||
|
}}
|
@ -0,0 +1,110 @@
|
|||||||
|
// Copyright (c) 2020, The Monero Project
|
||||||
|
//
|
||||||
|
// All rights reserved.
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without modification, are
|
||||||
|
// permitted provided that the following conditions are met:
|
||||||
|
//
|
||||||
|
// 1. Redistributions of source code must retain the above copyright notice, this list of
|
||||||
|
// conditions and the following disclaimer.
|
||||||
|
//
|
||||||
|
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
|
||||||
|
// of conditions and the following disclaimer in the documentation and/or other
|
||||||
|
// materials provided with the distribution.
|
||||||
|
//
|
||||||
|
// 3. Neither the name of the copyright holder nor the names of its contributors may be
|
||||||
|
// used to endorse or promote products derived from this software without specific
|
||||||
|
// prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
|
||||||
|
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||||
|
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
|
||||||
|
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
|
||||||
|
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
|
||||||
|
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <array>
|
||||||
|
#include <boost/thread/mutex.hpp>
|
||||||
|
#include <boost/utility/string_ref.hpp>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <deque>
|
||||||
|
#include <memory>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#include "cryptonote_basic/fwd.h"
|
||||||
|
#include "net/zmq.h"
|
||||||
|
#include "span.h"
|
||||||
|
|
||||||
|
namespace cryptonote { namespace listener
|
||||||
|
{
|
||||||
|
/*! \brief Sends ZMQ PUB messages on cryptonote events
|
||||||
|
|
||||||
|
Clients must ensure that all transaction(s) are notified before any blocks
|
||||||
|
they are contained in, and must ensure that each block is notified in chain
|
||||||
|
order. An external lock **must** be held by clients during the entire
|
||||||
|
txpool check and notification sequence and (a possibly second) lock is held
|
||||||
|
during the entire block check and notification sequence. Otherwise, events
|
||||||
|
could be sent in a different order than processed. */
|
||||||
|
class zmq_pub
|
||||||
|
{
|
||||||
|
/* Each socket has its own internal queue. So we can only use one socket, else
|
||||||
|
the messages being published are not guaranteed to be in the same order
|
||||||
|
pushed. */
|
||||||
|
|
||||||
|
net::zmq::socket relay_;
|
||||||
|
std::deque<std::vector<txpool_event>> txes_;
|
||||||
|
std::array<std::size_t, 2> chain_subs_;
|
||||||
|
std::array<std::size_t, 2> txpool_subs_;
|
||||||
|
boost::mutex sync_; //!< Synchronizes counts in `*_subs_` arrays.
|
||||||
|
|
||||||
|
public:
|
||||||
|
//! \return Name of ZMQ_PAIR endpoint for pub notifications
|
||||||
|
static constexpr const char* relay_endpoint() noexcept { return "inproc://pub_relay"; }
|
||||||
|
|
||||||
|
explicit zmq_pub(void* context);
|
||||||
|
|
||||||
|
zmq_pub(const zmq_pub&) = delete;
|
||||||
|
zmq_pub(zmq_pub&&) = delete;
|
||||||
|
|
||||||
|
~zmq_pub();
|
||||||
|
|
||||||
|
zmq_pub& operator=(const zmq_pub&) = delete;
|
||||||
|
zmq_pub& operator=(zmq_pub&&) = delete;
|
||||||
|
|
||||||
|
//! Process a client subscription request (from XPUB sockets). Thread-safe.
|
||||||
|
bool sub_request(const boost::string_ref message);
|
||||||
|
|
||||||
|
/*! Forward ZMQ messages sent to `relay` via `send_chain_main` or
|
||||||
|
`send_txpool_add` to `pub`. Used by `ZmqServer`. */
|
||||||
|
bool relay_to_pub(void* relay, void* pub);
|
||||||
|
|
||||||
|
/*! Send a `ZMQ_PUB` notification for a change to the main chain.
|
||||||
|
Thread-safe.
|
||||||
|
\return Number of ZMQ messages sent to relay. */
|
||||||
|
std::size_t send_chain_main(std::uint64_t height, epee::span<const cryptonote::block> blocks);
|
||||||
|
|
||||||
|
/*! Send a `ZMQ_PUB` notification for new tx(es) being added to the local
|
||||||
|
pool. Thread-safe.
|
||||||
|
\return Number of ZMQ messages sent to relay. */
|
||||||
|
std::size_t send_txpool_add(std::vector<cryptonote::txpool_event> txes);
|
||||||
|
|
||||||
|
//! Callable for `send_chain_main` with weak ownership to `zmq_pub` object.
|
||||||
|
struct chain_main
|
||||||
|
{
|
||||||
|
std::weak_ptr<zmq_pub> self_;
|
||||||
|
void operator()(std::uint64_t height, epee::span<const cryptonote::block> blocks) const;
|
||||||
|
};
|
||||||
|
|
||||||
|
//! Callable for `send_txpool_add` with weak ownership to `zmq_pub` object.
|
||||||
|
struct txpool_add
|
||||||
|
{
|
||||||
|
std::weak_ptr<zmq_pub> self_;
|
||||||
|
void operator()(std::vector<cryptonote::txpool_event> txes) const;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}}
|
@ -0,0 +1,42 @@
|
|||||||
|
// Copyright (c) 2020, The Monero Project
|
||||||
|
//
|
||||||
|
// All rights reserved.
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without modification, are
|
||||||
|
// permitted provided that the following conditions are met:
|
||||||
|
//
|
||||||
|
// 1. Redistributions of source code must retain the above copyright notice, this list of
|
||||||
|
// conditions and the following disclaimer.
|
||||||
|
//
|
||||||
|
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
|
||||||
|
// of conditions and the following disclaimer in the documentation and/or other
|
||||||
|
// materials provided with the distribution.
|
||||||
|
//
|
||||||
|
// 3. Neither the name of the copyright holder nor the names of its contributors may be
|
||||||
|
// used to endorse or promote products derived from this software without specific
|
||||||
|
// prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
|
||||||
|
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||||
|
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
|
||||||
|
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
|
||||||
|
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
|
||||||
|
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
namespace test
|
||||||
|
{
|
||||||
|
cryptonote::transaction make_miner_transaction(cryptonote::account_public_address const& to);
|
||||||
|
|
||||||
|
cryptonote::transaction
|
||||||
|
make_transaction(
|
||||||
|
cryptonote::account_keys const& from,
|
||||||
|
std::vector<cryptonote::transaction> const& sources,
|
||||||
|
std::vector<cryptonote::account_public_address> const& destinations,
|
||||||
|
bool rct,
|
||||||
|
bool bulletproof);
|
||||||
|
}
|
Loading…
Reference in new issue