blob: b02e88bd0b0d92c9e723ff9190f94759fce48db9 [file] [log] [blame]
  1. Protocol Buffers - Google's data interchange format
  2. Copyright 2008 Google Inc.
  3. This directory contains the Java Protocol Buffers runtime library.
  4. Installation - With Maven
  5. =========================
  6. The Protocol Buffers build is managed using Maven. If you would
  7. rather build without Maven, see below.
  8. 1) Install Apache Maven if you don't have it:
  10. 2) Build the C++ code, or obtain a binary distribution of protoc. If
  11. you install a binary distribution, make sure that it is the same
  12. version as this package. If in doubt, run:
  13. $ protoc --version
  14. You will need to place the protoc executable in ../src. (If you
  15. built it yourself, it should already be there.)
  16. 3) Run the tests:
  17. $ mvn test
  18. If some tests fail, this library may not work correctly on your
  19. system. Continue at your own risk.
  20. 4) Install the library into your Maven repository:
  21. $ mvn install
  22. 5) If you do not use Maven to manage your own build, you can build a
  23. .jar file to use:
  24. $ mvn package
  25. The .jar will be placed in the "target" directory.
  26. Installation - 'Lite' Version - With Maven
  27. ==========================================
  28. Building the 'lite' version of the Java Protocol Buffers library is
  29. the same as building the full version, except that all commands are
  30. run using the 'lite' profile. (see
  32. E.g. to install the lite version of the jar, you would run:
  33. $ mvn install -P lite
  34. The resulting artifact has the 'lite' classifier. To reference it
  35. for dependency resolution, you would specify it as:
  36. <dependency>
  37. <groupId></groupId>
  38. <artifactId>protobuf-java</artifactId>
  39. <version>${version}</version>
  40. <classifier>lite</classifier>
  41. </dependency>
  42. Installation - Without Maven
  43. ============================
  44. If you would rather not install Maven to build the library, you may
  45. follow these instructions instead. Note that these instructions skip
  46. running unit tests.
  47. 1) Build the C++ code, or obtain a binary distribution of protoc. If
  48. you install a binary distribution, make sure that it is the same
  49. version as this package. If in doubt, run:
  50. $ protoc --version
  51. If you built the C++ code without installing, the compiler binary
  52. should be located in ../src.
  53. 2) Invoke protoc to build
  54. $ protoc --java_out=src/main/java -I../src \
  55. ../src/google/protobuf/descriptor.proto
  56. 3) Compile the code in src/main/java using whatever means you prefer.
  57. 4) Install the classes wherever you prefer.
  58. Micro version
  59. ============================
  60. The runtime and generated code for MICRO_RUNTIME is smaller
  61. because it does not include support for the descriptor and
  62. reflection, and enums are generated as integer constants in
  63. the parent message or the file's outer class, with no
  64. protection against invalid values set to enum fields. Also,
  65. not currently supported are packed repeated elements or
  66. extensions.
  67. To create a jar file for the runtime and run tests invoke
  68. "mvn package -P micro" from the <protobuf-root>/java
  69. directory. The generated jar file is
  70. <protobuf-root>java/target/protobuf-java-2.6.0-micro.jar.
  71. If you wish to compile the MICRO_RUNTIME your self, place
  72. the 7 files below, in <root>/com/google/protobuf and
  73. create a jar file for use with your code and the generated
  74. code:
  81. If you wish to change on the code generator it is located
  82. in /src/google/protobuf/compiler/javamicro.
  83. To generate code for the MICRO_RUNTIME invoke protoc with
  84. --javamicro_out command line parameter. javamicro_out takes
  85. a series of optional sub-parameters separated by commas
  86. and a final parameter, with a colon separator, which defines
  87. the source directory. Sub-parameters begin with a name
  88. followed by an equal and if that sub-parameter has multiple
  89. parameters they are seperated by "|". The command line options
  90. are:
  91. opt -> speed or space
  92. java_use_vector -> true or false
  93. java_package -> <file-name>|<package-name>
  94. java_outer_classname -> <file-name>|<package-name>
  95. java_multiple_files -> true or false
  96. opt={speed,space} (default: space)
  97. This changes the code generation to optimize for speed or
  98. space. When opt=speed this changes the code generation
  99. for strings so that multiple conversions to Utf8 are
  100. eliminated.
  101. java_use_vector={true,false} (default: false)
  102. This specifies the collection class for repeated elements.
  103. If false, repeated elements use java.util.ArrayList<> and
  104. the code must be compiled with Java 1.5 or above. If true,
  105. repeated elements use java.util.Vector and the code can
  106. be compiled with Java 1.3 or above. The 'source'
  107. parameter of 'javac' may be used to control the version
  108. of the source: "javac -source 1.3". You can also change
  109. the <source> xml element for the maven-compiler-plugin.
  110. Below is for 1.5 sources:
  111. <plugin>
  112. <artifactId>maven-compiler-plugin</artifactId>
  113. <configuration>
  114. <source>1.5</source>
  115. <target>1.5</target>
  116. </configuration>
  117. </plugin>
  118. And below would be for 1.3 sources (note when changing
  119. to 1.3 you must also set java_use_vector=true):
  120. <plugin>
  121. <artifactId>maven-compiler-plugin</artifactId>
  122. <configuration>
  123. <source>1.3</source>
  124. <target>1.5</target>
  125. </configuration>
  126. </plugin>
  127. java_package=<file-name>|<package-name> (no default)
  128. This allows overriding the 'java_package' option value
  129. for the given file from the command line. Use multiple
  130. java_package options to override the option for multiple
  131. files. The final Java package for each file is the value
  132. of this command line option if present, or the value of
  133. the same option defined in the file if present, or the
  134. proto package if present, or the default Java package.
  135. java_outer_classname=<file-name>|<outer-classname> (no default)
  136. This allows overriding the 'java_outer_classname' option
  137. for the given file from the command line. Use multiple
  138. java_outer_classname options to override the option for
  139. multiple files. The final Java outer class name for each
  140. file is the value of this command line option if present,
  141. or the value of the same option defined in the file if
  142. present, or the file name converted to CamelCase. This
  143. outer class will nest all classes and integer constants
  144. generated from file-scope messages and enums.
  145. java_multiple_files={true,false} (no default)
  146. This allows overriding the 'java_multiple_files' option
  147. in all source files and their imported files from the
  148. command line. The final value of this option for each
  149. file is the value defined in this command line option, or
  150. the value of the same option defined in the file if
  151. present, or false. This specifies whether to generate
  152. package-level classes for the file-scope messages in the
  153. same Java package as the outer class (instead of nested
  154. classes in the outer class). File-scope enum constants
  155. are still generated as integer constants in the outer
  156. class. This affects the fully qualified references in the
  157. Java code. NOTE: because the command line option
  158. overrides the value for all files and their imported
  159. files, using this option inconsistently may result in
  160. incorrect references to the imported messages and enum
  161. constants.
  162. IMPORTANT: change of javamicro_out behavior:
  163. In previous versions, if the outer class name has not been
  164. given explicitly, javamicro_out would not infer the outer
  165. class name from the file name, and would skip the outer
  166. class generation. This makes the compilation succeed only
  167. if the source file contains a single message and no enums,
  168. and the generated class for that message is placed at the
  169. package level. To re-align with java_out, javamicro_out
  170. will now always generate the outer class, inferring its
  171. name from the file name if not given, as a container of the
  172. message classes and enum constants. To keep any existing
  173. single-message source file from causing the generation of
  174. an unwanted outer class, you can set the option
  175. java_multiple_files to true, either in the file or as a
  176. command line option.
  177. Below are a series of examples for clarification of the
  178. various parameters and options. Assuming this file:
  179. src/proto/simple-data-protos.proto:
  180. package testprotobuf;
  181. message SimpleData {
  182. optional fixed64 id = 1;
  183. optional string description = 2;
  184. optional bool ok = 3 [default = false];
  185. };
  186. and the compiled protoc in the current working directory,
  187. then a simple command line to compile this file would be:
  188. ./protoc --javamicro_out=. src/proto/simple-data-protos.proto
  189. This will create testprotobuf/, which
  190. has the following content (extremely simplified):
  191. package testprotobuf;
  192. public final class SimpleDataProtos {
  193. public static final class SimpleData
  194. extends MessageMicro {
  195. ...
  196. }
  197. }
  198. The message SimpleData is compiled into the SimpleData
  199. class, nested in the file's outer class SimpleDataProtos,
  200. whose name is implicitly defined by the proto file name
  201. "simple-data-protos".
  202. The directory, aka Java package, testprotobuf is created
  203. because on line 1 of simple-data-protos.proto is
  204. "package testprotobuf;". If you wanted a different
  205. package name you could use the java_package option in the
  206. file:
  207. option java_package = "my_package";
  208. or in command line sub-parameter:
  209. ./protoc '--javamicro_out=\
  210. java_package=src/proto/simple-data-protos.proto|my_package:\
  211. .' src/proto/simple-data-protos.proto
  212. Here you see the new java_package sub-parameter which
  213. itself needs two parameters the file name and the
  214. package name, these are separated by "|". The value set
  215. in the command line overrides the value set in the file.
  216. Now you'll find in the my_package/
  217. directory.
  218. If you wanted to also change the optimization for
  219. speed you'd add opt=speed with the comma seperator
  220. as follows:
  221. ./protoc '--javamicro_out=\
  222. opt=speed,\
  223. java_package=src/proto/simple-data-protos.proto|my_package:
  224. .' src/proto/simple-data-protos.proto
  225. If you also wanted a different outer class name you'd
  226. do the following:
  227. ./protoc '--javamicro_out=\
  228. opt=speed,\
  229. java_package=src/proto/simple-data-protos.proto|my_package,\
  230. java_outer_classname=src/proto/simple-data-protos.proto|OuterName:\
  231. .' src/proto/simple-data-protos.proto
  232. Now you'll find my_package/ and the
  233. message class SimpleData nested in it.
  234. As mentioned java_package, java_outer_classname and
  235. java_multiple_files may also be specified in the file.
  236. In the example below we must define
  237. java_outer_classname because otherwise the outer class
  238. and one of the message classes will have the same name,
  239. which is forbidden to prevent name ambiguity:
  240. src/proto/sample-message.proto:
  241. package testmicroruntime;
  242. option java_package = "com.example";
  243. option java_outer_classname = "SampleMessageProtos";
  244. enum MessageType {
  245. SAMPLE = 1;
  246. EXAMPLE = 2;
  247. }
  248. message SampleMessage {
  249. required int32 id = 1;
  250. required MessageType type = 2;
  251. }
  252. message SampleMessageContainer {
  253. required SampleMessage message = 1;
  254. }
  255. This could be compiled using:
  256. ./protoc --javamicro_out=. src/proto/sample-message.proto
  257. and the output will be:
  258. com/example/
  259. package com.example;
  260. public final class SampleMessageProtos {
  261. public static final int SAMPLE = 1;
  262. public static final int EXAMPLE = 2;
  263. public static final class SampleMessage
  264. extends MessageMicro {
  265. ...
  266. }
  267. public static final class SampleMessageContainer
  268. extends MessageMicro {
  269. ...
  270. }
  271. }
  272. As you can see the file-scope enum MessageType is
  273. disassembled into two integer constants in the outer class.
  274. In javamicro_out, all enums are disassembled and compiled
  275. into integer constants in the parent scope (the containing
  276. message's class or the file's (i.e. outer) class).
  277. You may prefer the file-scope messages to be saved in
  278. separate files. You can do this by setting the option
  279. java_multiple_files to true, in either the file like this:
  280. option java_multiple_files = true;
  281. or the command line like this:
  282. ./protoc --javamicro_out=\
  283. java_multiple_files=true:\
  284. . src/proto/sample-message.proto
  285. The java_multiple_files option causes javamicro to use a
  286. separate file for each file-scope message, which resides
  287. directly in the Java package alongside the outer class:
  288. com/example/
  289. package com.example;
  290. public final class SampleMessageProtos {
  291. public static final int SAMPLE = 1;
  292. public static final int EXAMPLE = 2;
  293. }
  294. com/example/
  295. package com.example;
  296. public final class SampleMessage
  297. extends MessageMicro {
  298. ...
  299. }
  300. com/example/
  301. package com.example;
  302. public final class SampleMessageContainer
  303. extends MessageMicro {
  304. ...
  305. }
  306. As you can see, the outer class now contains only the
  307. integer constants, generated from the file-scope enum
  308. "MessageType". Please note that message-scope enums are
  309. still generated as integer constants in the message class.
  310. Nano version
  311. ============================
  312. Nano is a special code generator and runtime library designed specially
  313. for Android, and is very resource-friendly in both the amount of code
  314. and the runtime overhead. An overview of Nano features:
  315. - No descriptors or message builders.
  316. - All messages are mutable; fields are public Java fields.
  317. - For optional fields only, encapsulation behind setter/getter/hazzer/
  318. clearer functions is opt-in, which provide proper 'has' state support.
  319. - If not opted in, has state is not available. Serialization outputs
  320. all fields not equal to their defaults (see important implications
  321. below).
  322. - Required fields are always serialized.
  323. - Enum constants are integers; protection against invalid values only
  324. when parsing from the wire.
  325. - Enum constants can be generated into container interfaces bearing
  326. the enum's name (so the referencing code is in Java style).
  327. - CodedInputByteBufferNano can only take byte[] (not InputStream).
  328. - Similarly CodedOutputByteBufferNano can only write to byte[].
  329. - Repeated fields are in arrays, not ArrayList or Vector. Null array
  330. elements are allowed and silently ignored.
  331. - Full support of serializing/deserializing repeated packed fields.
  332. - Support of extensions.
  333. - Unset messages/groups are null, not an immutable empty default
  334. instance.
  335. - toByteArray(...) and mergeFrom(...) are now static functions of
  336. MessageNano.
  337. - The 'bytes' type translates to the Java type byte[].
  338. The generated messages are not thread-safe for writes, but may be
  339. used simultaneously from multiple threads in a read-only manner.
  340. In other words, an appropriate synchronization mechanism (such as
  341. a ReadWriteLock) must be used to ensure that a message, its
  342. ancestors, and descendants are not accessed by any other threads
  343. while the message is being modified. Field reads, getter methods
  344. (but not getExtension(...)), toByteArray(...), writeTo(...),
  345. getCachedSize(), and getSerializedSize() are all considered read-only
  346. operations.
  347. IMPORTANT: If you have fields with defaults and opt out of accessors
  348. How fields with defaults are serialized has changed. Because we don't
  349. keep "has" state, any field equal to its default is assumed to be not
  350. set and therefore is not serialized. Consider the situation where we
  351. change the default value of a field. Senders compiled against an older
  352. version of the proto continue to match against the old default, and
  353. don't send values to the receiver even though the receiver assumes the
  354. new default value. Therefore, think carefully about the implications
  355. of changing the default value. Alternatively, turn on accessors and
  356. enjoy the benefit of the explicit has() checks.
  357. IMPORTANT: If you have "bytes" fields with non-empty defaults
  358. Because the byte buffer is now of mutable type byte[], the default
  359. static final cannot be exposed through a public field. Each time a
  360. message's constructor or clear() function is called, the default value
  361. (kept in a private byte[]) is cloned. This causes a small memory
  362. penalty. This is not a problem if the field has no default or is an
  363. empty default.
  364. Nano Generator options
  365. java_package -> <file-name>|<package-name>
  366. java_outer_classname -> <file-name>|<package-name>
  367. java_multiple_files -> true or false
  368. java_nano_generate_has -> true or false [DEPRECATED]
  369. optional_field_style -> default or accessors
  370. enum_style -> c or java
  371. ignore_services -> true or false
  372. parcelable_messages -> true or false
  373. java_package:
  374. java_outer_classname:
  375. java_multiple_files:
  376. Same as Micro version.
  377. java_nano_generate_has={true,false} (default: false)
  378. DEPRECATED. Use optional_field_style=accessors.
  379. If true, generates a public boolean variable has<fieldname>
  380. accompanying each optional or required field (not present for
  381. repeated fields, groups or messages). It is set to false initially
  382. and upon clear(). If parseFrom(...) reads the field from the wire,
  383. it is set to true. This is a way for clients to inspect the "has"
  384. value upon parse. If it is set to true, writeTo(...) will ALWAYS
  385. output that field (even if field value is equal to its
  386. default).
  387. IMPORTANT: This option costs an extra 4 bytes per primitive field in
  388. the message. Think carefully about whether you really need this. In
  389. many cases reading the default works and determining whether the
  390. field was received over the wire is irrelevant.
  391. optional_field_style={default,accessors,reftypes} (default: default)
  392. Defines the style of the generated code for fields.
  393. * default *
  394. In the default style, optional fields translate into public mutable
  395. Java fields, and the serialization process is as discussed in the
  396. "IMPORTANT" section above.
  397. * accessors *
  398. When set to 'accessors', each optional field is encapsulated behind
  399. 4 accessors, namely get<fieldname>(), set<fieldname>(), has<fieldname>()
  400. and clear<fieldname>() methods, with the standard semantics. The hazzer's
  401. return value determines whether a field is serialized, so this style is
  402. useful when you need to serialize a field with the default value, or check
  403. if a field has been explicitly set to its default value from the wire.
  404. In the 'accessors' style, required and nested message fields are still
  405. translated to one public mutable Java field each, repeated fields are still
  406. translated to arrays. No accessors are generated for them.
  407. IMPORTANT: When using the 'accessors' style, ProGuard should always
  408. be enabled with optimization (don't use -dontoptimize) and allowing
  409. access modification (use -allowaccessmodification). This removes the
  410. unused accessors and maybe inline the rest at the call sites,
  411. reducing the final code size.
  412. TODO(maxtroy): find ProGuard config that would work the best.
  413. * reftypes *
  414. When set to 'reftypes', each proto field is generated as a public Java
  415. field. For primitive types, these fields use the Java reference types
  416. such as java.lang.Integer instead of primitive types such as int.
  417. In the 'reftypes' style, fields are initialized to null (or empty
  418. arrays for repeated fields), and their default values are not available.
  419. They are serialized over the wire based on equality to null.
  420. The 'reftypes' mode has some additional cost due to autoboxing and usage
  421. of reference types. In practice, many boxed types are cached, and so don't
  422. result in object creation. However, references do take slightly more memory
  423. than primitives.
  424. The 'reftypes' mode is useful when you want to be able to serialize fields
  425. with default values, or check if a field has been explicitly set to the
  426. default over the wire without paying the extra method cost of the
  427. 'accessors' mode.
  428. Note that if you attempt to write null to a required field in the reftypes
  429. mode, serialization of the proto will cause a NullPointerException. This is
  430. an intentional indicator that you must set required fields.
  431. NOTE
  432. optional_field_style=accessors or reftypes cannot be used together with
  433. java_nano_generate_has=true. If you need the 'has' flag for any
  434. required field (you have no reason to), you can only use
  435. java_nano_generate_has=true.
  436. enum_style={c,java} (default: c)
  437. Defines where to put the int constants generated from enum members.
  438. * c *
  439. Use C-style, so the enum constants are available at the scope where
  440. the enum is defined. A file-scope enum's members are referenced like
  441. 'FileOuterClass.ENUM_VALUE'; a message-scope enum's members are
  442. referenced as 'Message.ENUM_VALUE'. The enum name is unavailable.
  443. This complies with the Micro code generator's behavior.
  444. * java *
  445. Use Java-style, so the enum constants are available under the enum
  446. name and referenced like 'EnumName.ENUM_VALUE' (they are still int
  447. constants). The enum name becomes the name of a public interface, at
  448. the scope where the enum is defined. If the enum is file-scope and
  449. the java_multiple_files option is on, the interface will be defined
  450. in its own file. To reduce code size, this interface should not be
  451. implemented and ProGuard shrinking should be used, so after the Java
  452. compiler inlines all referenced enum constants into the call sites,
  453. the interface remains unused and can be removed by ProGuard.
  454. ignore_services={true,false} (default: false)
  455. Skips services definitions.
  456. Nano doesn't support services. By default, if a service is defined
  457. it will generate a compilation error. If this flag is set to true,
  458. services will be silently ignored, instead.
  459. parcelable_messages={true,false} (default: false)
  460. Android-specific option to generate Parcelable messages.
  461. generate_intdefs={true,false} (default: false)
  462. Android-specific option to generate @IntDef annotations for enums.
  463. If turned on, an @IntDef annotation (a public @interface) will be
  464. generated for each enum, and every integer parameter and return
  465. value in the generated code meant for this enum will be annotated
  466. with it. This interface is generated with the same name and at the
  467. same place as the enum members' container interfaces described
  468. above under enum_style=java, regardless of the enum_style option
  469. used. When this is combined with enum_style=java, the interface
  470. will be both the @IntDef annotation and the container of the enum
  471. members; otherwise the interface has an empty body.
  472. Your app must declare a compile-time dependency on the
  473. android-support-annotations library.
  474. For more information on how these @IntDef annotations help with
  475. compile-time type safety, see:
  477. and
  479. To use nano protobufs within the Android repo:
  480. - Set 'LOCAL_PROTOC_OPTIMIZE_TYPE := nano' in your local .mk file.
  481. When building a Java library or an app (package) target, the build
  482. system will add the Java nano runtime library to the
  483. LOCAL_STATIC_JAVA_LIBRARIES variable, so you don't need to.
  484. - Set 'LOCAL_PROTO_JAVA_OUTPUT_PARAMS := ...' in your local .mk file
  485. for any command-line options you need. Use commas to join multiple
  486. options. In the nano flavor only, whitespace surrounding the option
  487. names and values are ignored, so you can use backslash-newline or
  488. '+=' to structure your make files nicely.
  489. - The options will be applied to *all* proto files in LOCAL_SRC_FILES
  490. when you build a Java library or package. In case different options
  491. are needed for different proto files, build separate Java libraries
  492. and reference them in your main target. Note: you should make sure
  493. that, for each separate target, all proto files imported from any
  494. proto file in LOCAL_SRC_FILES are included in LOCAL_SRC_FILES. This
  495. is because the generator has to assume that the imported files are
  496. built using the same options, and will generate code that reference
  497. the fields and enums from the imported files using the same code
  498. style.
  499. - Hint: 'include $(CLEAR_VARS)' resets all LOCAL_ variables, including
  500. the two above.
  501. To use nano protobufs outside of Android repo:
  502. - Link with the generated jar file
  503. <protobuf-root>java/target/protobuf-java-2.6.0-nano.jar.
  504. - Invoke with --javanano_out, e.g.:
  505. ./protoc '--javanano_out=\
  506. java_package=src/proto/simple-data.proto|my_package,\
  507. java_outer_classname=src/proto/simple-data.proto|OuterName\
  508. :.' src/proto/simple-data.proto
  509. Contributing to nano:
  510. Please add/edit tests in
  511. Please run the following steps to test:
  512. - cd external/protobuf
  513. - ./configure
  514. - Run "make -j12 check" and verify all tests pass.
  515. - cd java
  516. - Run "mvn test" and verify all tests pass.
  517. - cd ../../..
  518. - . build/
  519. - lunch 1
  520. - "make -j12 aprotoc libprotobuf-java-nano aprotoc-test-nano-params NanoAndroidTest" and
  521. check for build errors.
  522. - Plug in an Android device or start an emulator.
  523. - adb install -r out/target/product/generic/data/app/NanoAndroidTest/NanoAndroidTest.apk
  524. - Run:
  525. "adb shell am instrument -w"
  526. and verify all tests pass.
  527. - repo sync -c -j256
  528. - "make -j12" and check for build errors
  529. Usage
  530. =====
  531. The complete documentation for Protocol Buffers is available via the
  532. web at: