Execute following command on influx cli:
show tag keys from sensor;Your point should look like:
{
"measurement": "sensor",
"time": 1559630455,
"tags": {
"test_tag": "test"
},
"fields": {
"some_field": 1
}
}i'm new in InfluxDB, the problem consists in anycodings_apache-kafka that i have to insert python dicts in anycodings_apache-kafka InfluxDB via Nifi and i have tried different anycodings_apache-kafka ways and i allways got the same error anycodings_apache-kafka telling that tag is missing:,And i got the same error using tags and anycodings_apache-kafka fields simultaneously:,This will list all tagged columns from anycodings_json your measurement. Make sure you are anycodings_json passing all those while trying to insert anycodings_json new point.,If the record(point as per InfluxDB anycodings_json terminology) that you are trying to anycodings_json insert into measurement 'sensor' do not anycodings_json have value or has null value for any anycodings_json tags, you will get this error.
org.influxdb.InfluxDBException: {
"error": "unable to parse '[ {
"measurement": "sensor",
"time": 1559560006,
"tags": [
"sensor",
"id",
"date",
"info",
"aleatory_number",
"aleatory_number_square_root"
],
"fields": {
"id": 8,
"date": 1559559961002,
"info": "info sensor8",
"aleatory_number": 1778687859,
"aleatory_number_square_root": 42174.492
}
}
]
': missing tag value"
}Another example:
{
"error": "unable to parse
'[{
"measurement": "sensor",
"time": 1559630455,
"tags": {
"test_tag": "test"
},
"fields": {}
}]
': missing tag value "
}Another:
org.influxdb.InfluxDBException: {
"error": "unable to parse
'[{
"measurement": "sensor",
"time": 1559631341,
"tags": {},
"fields": {}
}]
': missing tag value "}And i got the same error using tags and anycodings_apache-kafka fields simultaneously:
org.influxdb.InfluxDBException: {
"error": "unable to parse
'[{
"measurement": "sensor",
"time": 1559720142,
"tags": {
"test_tag": "test"
},
"fields": {
"id": 3,
"date": 1559718332366,
"info": "info sensor3",
"aleatory_number": 141969819,
"aleatory_number_square_root": 11915.108
}
}]
': missing tag value"}Execute following command on influx cli:
show tag keys from sensor;Your point should look like:
{
"measurement": "sensor",
"time": 1559630455,
"tags": {
"test_tag": "test"
},
"fields": {
"some_field": 1
}
}The name of the field in the Record Schema is used as the key of the Tag. The value of the field is used as the value of the Tag. Any data type is converted into a String type and used as the Tag value see also handling complex types.,The value is determined from the field in the Record Schema. If the field is not found in the schema then is used the value of Measurement property. Any data type is converted into a String type and used as the value.,The value is determined from the field in the Record Schema. If the field is not found in the schema or field has not defined value the timestamp is not specified for the Data Point. The precision for the supplied time value is determined from the property Timestamp precision.,Choice - for the value is used the compatible type from Choice definition
$ cp~/Downloads/nifi - influx - database - nar - 1.1.nar $NIFI_HOME / lib. / scripts / nifi - restart.sh
{
"type": "record",
"name": "twitter_schema",
"namespace": "io.bonitoo.nifi",
"doc:": "AVRO scheme for Tweets",
"fields": [{
"name": "id",
"type": "long"
},
{
"name": "text",
"type": "string"
},
{
"name": "lang",
"type": "string"
},
{
"name": "keyword",
"type": "string"
},
{
"name": "retweet_count",
"type": "int"
},
{
"name": "tweet_id",
"type": "string"
},
{
"name": "followers_count",
"type": "int"
},
{
"name": "screen_name",
"type": "string"
},
{
"name": "friends_count",
"type": "int"
},
{
"name": "favourites_count",
"type": "int"
},
{
"name": "user_verified",
"type": "boolean"
},
{
"name": "timestamp",
"type": {
"type": "long",
"logicalType": "timestamp-millis"
}
}
]
}SHOW TAG KEYS ON twitter_demo FROM tweets
name: tweets
tagKey
-- -- --
keyword
lang
user_verifiedSHOW FIELD KEYS ON twitter_demo name: tweets fieldKey fieldType -- -- -- -- -- -- -- -- - favourites_count integer followers_count integer friends_count integer retweet_count integer screen_name string text string tweet_id string
select * from tweets
name: tweets
time favourites_count followers_count friends_count keyword lang retweet_count screen_name text
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - -- -- -- -- -- -- - -- -- -- - -- -- -- -- -- -- -- -- - -- -- -- -- -- -- -- - -- --
1550133996000000000 1651 304 699 truth en 0 TheeSeanH...
1550133997000000000 10 12 66 en 0 black_vadik...
1550133998000000000 0 22 41 BITMEX nl 0 100 btcP...
1550133998000000000 24078 1025 4894 en 0 SolarCoinNews...
1550133999000000000 12406 474 761 en 0 Airdrop_BOMBER...
...Created 05-31-2018 01:03 PM , Created 05-31-2018 04:32 AM
This is done by adding a new key/value pair to the nifi,properties configuration file:
nifi.nar.library.directory.custom-lib=/<path to custom lib dir>/custom-libThis is done by adding a new key/value pair to the nifi,properties configuration file:
nifi.nar.library.directory.custom-lib=/<path to custom lib dir>/custom-libIncoming sensor values from MiNiFi will be transformed to InfluxDB format and stored in InfluxDB ,Machine #1 (NiFi): Incoming sensor values from MiNiFi will be transformed to InfluxDB format and stored in InfluxDB If incoming sensor values exceed a fixed threshold, publish a warning message on Slack ,You will get an error about missing an Input Port, because our NiFi canvas has no Input Ports that would take care of data sent to it via Remote Process Groups. We will take care of this in a moment. ,Try to connect the GenerateFlowFile output to the Remote Process Group. You will get an error about missing an Input Port, because our NiFi canvas has no Input Ports that would take care of data sent to it via Remote Process Groups. We will take care of this in a moment.
docker run--name nifi\
-
p 8080: 8080\ -
p 8081: 8081\ -
p 10000: 10000\ -
d\ -
e NIFI_WEB_HTTP_PORT = '8080'\
apache / nifi: 1.9 .2nifi.minifi.notifier.ingestors.pull.http.hostname=<<YOUR_NIFI_VM_IP>>
nifi.minifi.notifier.ingestors.pull.http.port=10080
nifi.minifi.notifier.ingestors.pull.http.path=/c2/confignifi.minifi.notifier.ingestors.pull.http.query = class = sensor - devicenifi.minifi.notifier.ingestors.pull.http.period.ms = 30000